Program Evaluation Midterm with complete questions and answers
Program Evaluation - ANS- Using scientific methods to measure program
tt tt tt tt tt tt tt tt tt
implementation/outcomes for decision-making
tt tt tt
- the systematic collection, analysis, and reporting of information about a program to assist in
tt tt tt tt tt tt tt tt tt tt tt tt tt tt
decision-making (CDC)
tt tt
- The systematic collection of information about the activities, characteristics, and outcomes of
tt tt tt tt tt tt tt tt tt tt tt tt
programs to make judgments about the program, improve program effectiveness, and/or
tt tt tt tt tt tt tt tt tt tt tt
inform decisions about future development (Patton, M.Q.)
tt tt tt tt tt tt tt
Evaluation purpose - ANS- Making some value judgments based on research
tt tt tt tt tt tt tt tt tt tt
- Judgments can include judgments about merit, worth, and significance
tt tt tt tt tt tt tt tt tt
- show results or improve program quality
tt tt tt tt tt tt
Types of evaluation - ANS- Formative (Needs Assessment, EA, Process)
tt tt tt tt tt tt tt tt tt
- Summative (Outcome, Impact, Efficiency)
tt tt tt tt
Formative Evaluation - ANSAssessing program inputs to short-term outcomes for improvement;
tt tt tt tt tt tt tt tt tt tt
to improve
tt tt
Summative Evaluation - ANSAssessing program results, impact, and efficiency after
tt tt tt tt tt tt tt tt tt
implementation; to prove
tt tt tt
Process Evaluation - ANS- Process evaluation is the use of empirical data to assess the delivery of
tt tt tt tt tt tt tt tt tt tt tt tt tt tt tt tt
programs
tt
- Purpose: to determine if the right services are getting to the right people, evaluating program
tt tt tt tt tt tt tt tt tt tt tt tt tt tt tt
fidelity (implementation, inputs to outputs)
tt tt tt tt tt
- As a stand-alone, its purpose is to describe and analyze - to improve
tt tt tt tt tt tt tt tt tt tt tt tt tt
Used in combination with impact evaluation, it can target cause-effect questions to help explain
tt tt tt tt tt tt tt tt tt tt tt tt tt
its findings
tt tt
,Program Evaluation Midterm with complete questions and answers
Outcome Evaluation - ANSAssessing the achieved outcomes of a program, effect on participants
tt tt tt tt tt tt tt tt tt tt tt tt
Impact Evaluation - ANSSeeing if the program was the cause of.your results, effect on
tt tt tt tt tt tt tt tt tt tt tt tt tt
participants
tt
5 domains of evaluation methods/questions - ANS- Need for the program
tt tt tt tt tt tt tt tt tt tt
- program design/theory
tt tt
- program process
tt tt
- program outcomes/impact
tt tt
- program efficiency
tt tt
Logic Model - ANS- Illustrates program resources, activities, and outcomes causality
tt tt tt tt tt tt tt tt tt tt
- Inputs--Activities--Outputs--Outcomes (short, medium, long)
tt tt tt tt
- outside of model: goal, context, target population, assumptions, evaluation plan
tt tt tt tt tt tt tt tt tt tt
Stakeholders - ANSIndividuals involved in program operations (management, program staff,
tt tt tt tt tt tt tt tt tt
partners, funding agencies, coalition members)
tt tt tt tt tt
Those served/affected by the program (patients/clients, advocacy groups, community members,
tt tt tt tt tt tt tt tt tt
elected officials)
tt tt
Intended users of the evaluation (decision-makers: funding agencies, program
tt tt tt tt tt tt tt tt
management/staff, general public/taxpayers, coalition members)
tt tt tt tt tt
Theory of change/Program theory - ANSClarifies relationships among program inputs, activities,
tt tt tt tt tt tt tt tt tt tt
outputs, and outcomes
tt tt tt
, Program Evaluation Midterm with complete questions and answers
Rationale behind how & why the program will lead to the goal
tt tt tt tt tt tt tt tt tt tt tt
CDC Framework for Evaluation Steps - ANS1. Identify Stakeholders
tt tt tt tt tt tt tt tt
2. Describe program
tt tt
3. Focus the evaluation design
tt tt tt tt
4. Gather evidence (collect & analyze data)
tt tt tt tt tt tt
5. Justify conclusions
tt tt
6. Ensure use & share lessons learned
tt tt tt tt tt tt
Step 1: Identify stakeholders - ANS- program operations, served/affected, intended users
tt tt tt tt tt tt tt tt tt tt
- engage them to determine the evaluation purpose/focus, will help you define evaluation
tt tt tt tt tt tt tt tt tt tt tt tt
questions based on what their interests are
tt tt tt tt tt tt tt
- participatory evaluation: engage them throughout process, can give them roles (advisory
tt tt tt tt tt tt tt tt tt tt tt
board, data collection, etc.), empowerment
tt tt tt tt tt
Step 3: Focus Evaluation Design - ANS- Define evaluation questions (dependent on type of
tt tt tt tt tt tt tt tt tt tt tt tt tt
program/stakeholder interests, should be measureable & useful)
tt tt tt tt tt tt tt
- Can evaluate need for program, program design/theory, program process, program
tt tt tt tt tt tt tt tt tt tt
outcomes/impact, efficiency
tt tt
- Develop a measurement framework with logic model (for each output/outcome--list the
tt tt tt tt tt tt tt tt tt tt tt
indicator, measure of change, data collection method, source & frequency)
tt tt tt tt tt tt tt tt tt tt
- Then with logic model, eval questions, and measurement framework--build evaluation plan
tt tt tt tt tt tt tt tt tt tt tt
(background info on program, eval questions, eval design, eval planning/management, plan for
tt tt tt tt tt tt tt tt tt tt tt tt
communicating findings)
tt tt
- Choose appropriate eval design & know strengths + weaknesses of it in terms of validity
tt tt tt tt tt tt tt tt tt tt tt tt tt tt tt
(experimental, quasi-experimental)
tt tt
Step 2: Describe program - ANS- Consult stakeholders & determine consensus on logic model (ID
tt tt tt tt tt tt tt tt tt tt tt tt tt tt
program components, performance measures, goals/outcomes, theory of change)
tt tt tt tt tt tt tt tt
Program Evaluation - ANS- Using scientific methods to measure program
tt tt tt tt tt tt tt tt tt
implementation/outcomes for decision-making
tt tt tt
- the systematic collection, analysis, and reporting of information about a program to assist in
tt tt tt tt tt tt tt tt tt tt tt tt tt tt
decision-making (CDC)
tt tt
- The systematic collection of information about the activities, characteristics, and outcomes of
tt tt tt tt tt tt tt tt tt tt tt tt
programs to make judgments about the program, improve program effectiveness, and/or
tt tt tt tt tt tt tt tt tt tt tt
inform decisions about future development (Patton, M.Q.)
tt tt tt tt tt tt tt
Evaluation purpose - ANS- Making some value judgments based on research
tt tt tt tt tt tt tt tt tt tt
- Judgments can include judgments about merit, worth, and significance
tt tt tt tt tt tt tt tt tt
- show results or improve program quality
tt tt tt tt tt tt
Types of evaluation - ANS- Formative (Needs Assessment, EA, Process)
tt tt tt tt tt tt tt tt tt
- Summative (Outcome, Impact, Efficiency)
tt tt tt tt
Formative Evaluation - ANSAssessing program inputs to short-term outcomes for improvement;
tt tt tt tt tt tt tt tt tt tt
to improve
tt tt
Summative Evaluation - ANSAssessing program results, impact, and efficiency after
tt tt tt tt tt tt tt tt tt
implementation; to prove
tt tt tt
Process Evaluation - ANS- Process evaluation is the use of empirical data to assess the delivery of
tt tt tt tt tt tt tt tt tt tt tt tt tt tt tt tt
programs
tt
- Purpose: to determine if the right services are getting to the right people, evaluating program
tt tt tt tt tt tt tt tt tt tt tt tt tt tt tt
fidelity (implementation, inputs to outputs)
tt tt tt tt tt
- As a stand-alone, its purpose is to describe and analyze - to improve
tt tt tt tt tt tt tt tt tt tt tt tt tt
Used in combination with impact evaluation, it can target cause-effect questions to help explain
tt tt tt tt tt tt tt tt tt tt tt tt tt
its findings
tt tt
,Program Evaluation Midterm with complete questions and answers
Outcome Evaluation - ANSAssessing the achieved outcomes of a program, effect on participants
tt tt tt tt tt tt tt tt tt tt tt tt
Impact Evaluation - ANSSeeing if the program was the cause of.your results, effect on
tt tt tt tt tt tt tt tt tt tt tt tt tt
participants
tt
5 domains of evaluation methods/questions - ANS- Need for the program
tt tt tt tt tt tt tt tt tt tt
- program design/theory
tt tt
- program process
tt tt
- program outcomes/impact
tt tt
- program efficiency
tt tt
Logic Model - ANS- Illustrates program resources, activities, and outcomes causality
tt tt tt tt tt tt tt tt tt tt
- Inputs--Activities--Outputs--Outcomes (short, medium, long)
tt tt tt tt
- outside of model: goal, context, target population, assumptions, evaluation plan
tt tt tt tt tt tt tt tt tt tt
Stakeholders - ANSIndividuals involved in program operations (management, program staff,
tt tt tt tt tt tt tt tt tt
partners, funding agencies, coalition members)
tt tt tt tt tt
Those served/affected by the program (patients/clients, advocacy groups, community members,
tt tt tt tt tt tt tt tt tt
elected officials)
tt tt
Intended users of the evaluation (decision-makers: funding agencies, program
tt tt tt tt tt tt tt tt
management/staff, general public/taxpayers, coalition members)
tt tt tt tt tt
Theory of change/Program theory - ANSClarifies relationships among program inputs, activities,
tt tt tt tt tt tt tt tt tt tt
outputs, and outcomes
tt tt tt
, Program Evaluation Midterm with complete questions and answers
Rationale behind how & why the program will lead to the goal
tt tt tt tt tt tt tt tt tt tt tt
CDC Framework for Evaluation Steps - ANS1. Identify Stakeholders
tt tt tt tt tt tt tt tt
2. Describe program
tt tt
3. Focus the evaluation design
tt tt tt tt
4. Gather evidence (collect & analyze data)
tt tt tt tt tt tt
5. Justify conclusions
tt tt
6. Ensure use & share lessons learned
tt tt tt tt tt tt
Step 1: Identify stakeholders - ANS- program operations, served/affected, intended users
tt tt tt tt tt tt tt tt tt tt
- engage them to determine the evaluation purpose/focus, will help you define evaluation
tt tt tt tt tt tt tt tt tt tt tt tt
questions based on what their interests are
tt tt tt tt tt tt tt
- participatory evaluation: engage them throughout process, can give them roles (advisory
tt tt tt tt tt tt tt tt tt tt tt
board, data collection, etc.), empowerment
tt tt tt tt tt
Step 3: Focus Evaluation Design - ANS- Define evaluation questions (dependent on type of
tt tt tt tt tt tt tt tt tt tt tt tt tt
program/stakeholder interests, should be measureable & useful)
tt tt tt tt tt tt tt
- Can evaluate need for program, program design/theory, program process, program
tt tt tt tt tt tt tt tt tt tt
outcomes/impact, efficiency
tt tt
- Develop a measurement framework with logic model (for each output/outcome--list the
tt tt tt tt tt tt tt tt tt tt tt
indicator, measure of change, data collection method, source & frequency)
tt tt tt tt tt tt tt tt tt tt
- Then with logic model, eval questions, and measurement framework--build evaluation plan
tt tt tt tt tt tt tt tt tt tt tt
(background info on program, eval questions, eval design, eval planning/management, plan for
tt tt tt tt tt tt tt tt tt tt tt tt
communicating findings)
tt tt
- Choose appropriate eval design & know strengths + weaknesses of it in terms of validity
tt tt tt tt tt tt tt tt tt tt tt tt tt tt tt
(experimental, quasi-experimental)
tt tt
Step 2: Describe program - ANS- Consult stakeholders & determine consensus on logic model (ID
tt tt tt tt tt tt tt tt tt tt tt tt tt tt
program components, performance measures, goals/outcomes, theory of change)
tt tt tt tt tt tt tt tt
