Program Evaluation
IEc works with agencies to measure and evaluate success and strengthen their programs through evaluation. We specialize in evaluating environmental and energy policies and programs designed to spur technology innovation, accelerate technology development, and transform markets. These policies and programs are challenging to evaluate, and IEc has a proven track record of successfully applying innovative, mixed-methods evaluation approaches to capture the economic, environmental, and societal benefits that they can produce.
IEc’s interdisciplinary team of evaluators has deep expertise in areas including building energy efficiency technologies and construction practices, energy code compliance, renewable energy and distributed energy resources, clean transportation and EV-grid infrastructure, energy storage, grid modernization, workforce development, and entrepreneurial support programs.
Our evaluation services include:
- Formative, process, outcome, and impact evaluations.
- Logic modeling and barriers analysis
- Development of performance metrics and measurement frameworks
- Data collection planning and data management infrastructure
- Qualitative data collection methods, including interviews and focus groups
- Survey research
- Social network analysis and supply chain mapping
IEc has developed performance measurement guidance, frameworks, indicators, and data collection forms/tools for both long-term and emerging programs. We work with our clients to clarify program intervention logic, develop benefit taxonomies that reflect program investment plans, identify performance metrics, identify/develop methods, and develop or identify calculation tools.
We recognize that for evaluations to be influential, the results need to be packaged and disseminated to multiple audiences, such as: program managers, program funders, policymakers, consumers and producers of technologies, utilities, and the general public. IEc is skilled in preparing deliverables for different audiences, at different levels of detail. We are expert not only in writing comprehensive evaluation reports for program managers, but also summary reports, fact sheets, and presentations that can synthesize evaluation findings for senior decisionmakers.
IEc is expert at characterizing the baseline structure and functioning of markets and documenting market changes over time. We identify the market actors, determine the stage of commercialization, market saturation of the market as a whole and of the new technology, current sales, characteristics of purchasers, training and technical assistance, competitive analysis, marketing strategies, customer awareness, motivations and barriers for market actors to promote this technology, and current trends. We evaluate a program or policy’s influence on market transformation guided by the program theory of how program interventions reduce or eliminate market barriers. We are well-versed in an array of methods to characterize markets, including case studies, surveys, benchmarking methods, commercialization tracking methods, Delphi studies, expert panel studies, econometrics and historical tracing.
Evaluations of program impact examine the extent to which program activities have been successful in accomplishing stated objectives. IEc uses mixed-method evaluation approaches, grounded in program theory and drawing on layered sources of evidence, to analyze program results. We specialize in using analyses of existing data, interviews, focus groups, case studies, surveys, economic and market data, and relevant literature to test whether observed outcomes are consistent with program theory.
IEc’s approach to impact evaluation is grounded in a firm understanding of the program’s goals and activities, and a logical theory that explains why a causal relationship makes sense. We work with our clients to develop or refine their logic model or theory of change, and we evaluate whether there are outcomes that fit the program theory, while identifying and controlling for any rival explanations for the observed outcomes. IEc is skilled in using experimental and quasi-experimental designs (when random assignment is not possible) to evaluate program impacts.
Programs that save energy have a multitude of non-energy impacts (NEIs); IEc has the expertise to capture and monetize them. Examples of NEIs that our interdisciplinary team of economists, risk assessors, financial analysts, and other experts can quantify include:
- Economic benefits
- Health benefits
- Reliability and resiliency benefits
- Distributional benefits to low and moderate income populations
- Community
IEc often conducts process evaluations that address programmatic results and whether the program activities are carried out as intended. Through process evaluation, we examine issues including: program satisfaction experience, including barriers and motivations to participate; accessibility of programs across various targeted audiences; satisfaction with project outcomes among participants and other involved parties; administrative processes/steps, and streamlining opportunities; and actionable recommendations to improve program effectiveness.
IEc has conducted process evaluations for state and federal agencies across the country. Many of IEc’s process evaluations include a formative component to inform program design, such as ways to more effectively engage disadvantaged communities and low-to-moderate-income (LMI) populations.
Methods that IEc routinely uses in our process evaluations include participants surveys, focus groups, interviews, customer journey mapping, process mapping, benchmarking analysis, review of program materials/data, and social network analysis. We leverage these methods to identify strengths, opportunities, gaps, and needs and to develop practical recommendations to optimize program design and delivery.
Project Examples
Clean Transportation Portfolio Evaluations
New York State Energy Research and Development Authority