Exemplary Learning and Evaluation Reports

The National Fund closely monitors its local and national efforts to evaluate the effectiveness and impact of its programs and to identify best practices and innovative ideas.

The following outlines how the National Fund approaches its evaluation activities and provides a resource to its regional collaboratives and other workforce organizations. Resources and evaluations mentioned should be used as a reference for similar national and local activities.

Comprehensive

These are examples of comprehensive evaluation reports for collaboratives, demonstrating scale and impact and lessons learned.

Measuring Questions Sample Reports
Various, using quantitative and qualitative methods What outcomes has the collaborative achieved?

What value does the collaborative provide to the community and various stakeholders?

What has the collaborative learned in the process?

Strengthening Baltimore’s Workforce: Reflections and Lessons Learned (February 2017)

A report on several initiatives of the Baltimore Workforce Funders Collaborative

Workforce Central Evaluation (January 2012) Center on Wisconsin Strategy (COWS)

Independent local evaluation of early stages of Workforce Central

SkillWorks Phase II Evaluation: Overview of Outcomes (2014) Mt. Auburn Associates

Broad overview of results for Phase II encompassing the five-year period 2009-2013

ROI360: How Workforce Partnership Training Benefits Business, Workers and Community (2012) Elyssa Back

Evaluation JOIN-sponsored training efforts showing results for business, workers and community.

Performance Management

These reports demonstrate scale and impact summarizing outcomes and important strategies for a collaborative.

Measuring

Questions

Sample Reports

Funds raised

Job placements

Individuals trained

Employers engaged

Earnings

Industry partnerships

What outcomes has the collaborative achieved?

What approaches and strategies are being used?

CareerEdge Funders Collaborative 2017 Evaluation report (March 2018) Shelley Robertson

A summary report on key outcomes and approach of CareerEdge prepared by a third-party evaluator

Partners for a Competitive Workforce (PCW) 2016 Annual Report

Annual report produced by the Cincinnati collaborative

Summative Evaluation

These studies typically use rigorous methodologies for outcome evaluations to determine value or benefit to specific stakeholders, such as individuals and employers.

Measuring Questions Sample Reports
Value to Individuals:

Job placement

Job retention

Earnings

Job placement in sector (and in comparison to a control group)

How effective is the industry partnership (Sector strategy) approach in improving labor market outcomes for low-wage individuals?

Is it better than alternate approaches?

National Fund Reports

Evidence of Effectiveness of Workforce Partnership Programs in Ohio and Wisconsin, by Marios Michaelides, Peter Mueser, Scott Davis and Kassim Mbwana.

A third-party evaluation report produced by IMPAQ International. This was required as part of the Federal Social Innovation Fund(SIF) grant.

Job Training Programs that Work: Economic Impact of Job Training Programs in Metro Atlanta (2018)

This study for Atlanta CareerRise documents and synthesizes a comprehensive set of outcomes related to job training and assistance over a two-year period

Other Reports:

Tuning in to Local Labor markets: Findings from the Sectoral Employment Impact Study, by Sheila Maguire, Joshua Freely, Carol Clymer, Maureen Conway and Deena Schwartz.

This is a landmark report on the effectiveness of sector strategies or industry partnerships as it was undertaken by Public/Private Ventures(P/PV) using an experimental research design

Value to Employers:

Business impact or return on investment

Is the industry partnership approach effective in addressing the skill needs of employers and providing a high rate of return? JOIN-Philadelphia ROI Report

Case Study of Richards Industries using the Manufacturing Institute’s Return on Investment Calculator (2014)

Investment in training resulting in cost savings and an ROI of 875%

Value to Communities:

Improving overall employment of earnings in a community;

Satisfaction that the approach is addressing a community problem.

Is there value to the community of using the industry partnership approach?

Is there a sufficient social rate of return?

ROI360: How Workforce Partnership Training Benefits Business, Workers and Community (2012) Elyssa BackEvaluation JOIN-sponsored training efforts showing results for business, workers and community.
Measuring Systems Change and policy improvements using qualitative methods. Has the National Fund collaborative achieved its systems change goals?

Is there sufficient, desirable, enduring change in policy and practice?

National Fund Reports:

Systems Change in the National Fund, by Lisa Soricone (2015)

Review of system change in the National Fund network

Boston SkillWorks Phase II Systems change Report (2014), Mt. Auburn Associates

Report on Phase II (2009-2013) Systems Change activities of the Boston SkillWorks Collaborative

Formative Evaluation

These reports are intended to share details of how effective strategies and approaches are implemented within the National Fund network and lessons learned by practitioners.

Measuring Questions Sample Reports
N/A What are effective approaches for gathering and using data?

How to improve program effectiveness?

How to improve retention and advancement of frontline healthcare workers?

How to improve retention of individuals in training, college, jobs or careers?

How did a place-based program effectively address employer skill needs while improving individual training and employment outcomes?

What has the National Fund learned about effective strategies for sustaining a workforce collaborative?

What has the National Fund learned about effective strategies for investing in frontline workers?

Making Data more Useful for the Next Generation of the National Fund (August 2017) Program & Policy Insight

Examples from the network and discussion of factors that ensure high quality data collection and use for program and practice improvement

How to use Qualitative Research in a Workforce Collaborative (August 2017) Business Government Community Connections

Using qualitative research and creating a learning community

Recognizing and Rewarding the Role of Entry-level Healthcare Workers (May 2017) Business Government Community Connections

Report on initiative supporting job retention and career advancement efforts of entry level healthcare workers by the Healthcare Learning Consortium of SkillUp Washington

Guide and Tools for Navigators (November 2013)

Role of navigators in college and career success based on navigators in the College for Working Adults and Skill Link pilot initiative

Boston SkillWorks College Navigator Report (2012)

Key lessons learned from the experience of Boston SkillWorks

Exploration of Coaching Services in SkillWorks (July 2014), Mt. Auburn Associates

Evaluation of investment in career, academic and college navigation coaching by SkillWorks in financial services, healthcare and hospitality partnerships

Innovation in Program Design at the West Philadelphia Skills Initiative (2014)

Highlights of the approach of innovative programs that address real employer needs and provide valuable work experience for training participants

Sustainability Guide for Funder Collaboratives (October 2016) Loh-Sze Leung and Lisa Soricone

CareerSTAT Guide to Investing Frontline Healthcare Workers (October 2016) Randall Wilson and Kelly Aiken