2017
Educator Preparation Institution (EPI) Performance Score
Calculation
Guide
A Step-by-Step Guide for How EPI Performance Scores are Calculated
OFFICE OF PROFESSIONAL PREPARATION SERVICES (OPPS)
608 WEST ALLEGAN ST, LANSING, MI 48909 | P: 517-373-3310 | F: 517-373-0542
WWW.MICHIGAN.GOV/MDE
!
"
#$%&'%&(
)*'+*,'-./ 0/ 0/ 0/ 0/ 0/ 0/ 0/ 0/ 0/ 0/ 0/ 0/ 0/ 0/ 0/ 0/ 0/ 0/ 0/1
What is the EPI Performance Score? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Calculation Procedures and the Corrective Action System. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
#234532&,$%/6&'7(.// 0/ 0/ 0/ 0/ 0/ 0/ 0/ 0/ 0/ 0/ 0/ 0/ 0/ 0/8
EPI Score Calculation Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
Step 1: Determine Performance Goal Measure Scores
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Step 2: Determine the Rating Factors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Step 3: Compute the Weighting Determinant . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Step 4: Apply the Weighted Multipliers to the Rating Factors
. . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Step 5: Calculate Final EPI Performance Score
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Step 6: Compare the Final EPI Performance Score to the Cut Score
to Determine the Performance Level and Corrective Action Phase . . . . . . . . . . . . . . . . . . . . . . . . . 9
9+$:+'((,*'/#$++'4&,*'//
;4&,$%/6<(&'=./ 0/ 0/ 0/ 0/ 0/ 0/ 0/ 0/ 0/ 0/ 0/ 0/ 0/ 0/ 0/ >>
;77'%?,@./
A$-/B24C/9'+D$+=2%4'//
E'2(5+'/F(/#234532&'?/ 0/ 0/ 0/ 0/ 0/ 0/ 0/ 0/ 0/ 0/ 0/ >1
EPI Score Calculation Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .15
MTTC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .15
Surveys . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .17
Educator Eectiveness Ratings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .17
1
)*'+*,'-.
WHAT IS THE EPI PERFORMANCE SCORE?
Pursuant to Title II of the Higher Education Act, the Michigan Department of Education (MDE), in collaboration
with educator preparation programs, has designed, developed and now administers a system for determining
Educator Preparation Institution (EPI) Performance Scores. The purpose of the EPI Performance Score
system is to identify, assist, and report teacher preparation programs which are not performing at a satisfactory
level.
The EPI Performance Score observes and measures EPI performance relative to three goals aligned to the
Michigan Interstate Teacher Assessment and Support Consortium Standards (MI-InTASC). These are: 1) Effective
classroom teaching through demonstration of content knowledge and methods/pedagogy; 2) Continuous
improvement pursuant to MDE priorities; and support of 3) Educator effectiveness ratings.
CALCULATION PROCEDURES AND THE CORRECTIVE ACTION SYSTEM
The information presented in the remainder of this publication will illustrate how:
• the performance goals discussed above are measured;
• measures of these performance goals (EPI Performance Score calculation component values) are
determined and factored into the calculation steps;
• the calculation steps and procedures are used to determine final EPI Performance Scores and Performance
Levels; and
• the Progressive Corrective Action System functions to assist non-satisfactory programs.
Additionally, the Appendix details the methods used to determine the values for each of the performance goal
components.
8
#234532&,$%/6&'7(./
EPI SCORE CALCULATION COMPONENTS
To observe and measure the three performance goals discussed in the previous section, the EPI Performance
scoring system employs three score components that use data from three sources:
1) the Michigan Test for Teacher Certification (MTTC)
2) Surveys of teacher candidates and candidate supervisors (Surveys); and
3) Educator Effectiveness (Ed. Eff.) ratings.
There are ve steps to calculating each EPI Performance Score and ultimately determining a Performance
Level and a Corrective Action Phase. The table below uses random examples of EPI score component values to
illustrate the rst three steps of the calculation process (factoring in the three data sources discussed above). The
information that follows Table 1 details all ve steps.
Table 1: EPI Score Calculation Steps 1-3: Factoring in the Score Components
Process Function Examples
Step 1
Determine
Performance
Goal Measures
Goal 1 Measure:
(MTTC) Michigan
Test for Teacher
Certication Score =
88.3
Goal 2 Measure:
Teacher Candidates
and Candidate
Supervisor Surveys =
97.0
Goal 3 Measure:
Educator
Eectiveness
Ratings =
83.3
Step 2
Determine
Performance Goal
Rating Factors
(as calculated
from Performance
Goal Measures)
Goal 1 Rating Factor:
70% of MTTC (61.80)
+
30% of Surveys (29.1) =
90.9
Goal 2 Rating Factor:
100% of Surveys (97.0) =
97.0
Goal 3 Rating Factor:
100% of Ed. E. Ratings
(83.3) =
83.3
Step 3
Compute
Weighting
Determinant
(as a quotient
of two Control
Factors)
Control Factor 1: Number of Eectiveness Ratings Received = 50
Control Factor 2: Number of Completers = 81
Weighting Determinant:
Number of Ratings Received (50) ÷ Number of Completers (81) = 62%
G
STEP 1: DETERMINE PERFORMANCE GOAL MEASURE SCORES
Data is gathered and calculated to determine a score for each Performance Goal Measure: MTTC, Survey and
Educator Eectiveness Ratings. (The detailed methods for calculating each Performance Measure are explained
in the Appendix.) Those Performance Goal Measure components are then used to determine the Goal Rating
Factors used in further calculations, as described in Step 2.
STEP 2: DETERMINE THE RATING FACTORS
Goal 1 Rating Factor uses a sum of 70% of the MTTC score and 30% of the Survey score
as shown in this example:
(MTTC score) 88.30 x .70 = 61.81 61.81
(Surveys score) 97.00 x .30 = 29.10 + 29.10
Goal 1 Rating Factor = 90.91
Goal 2 Rating Factor uses 100% of the Survey Score:
(Surveys score) 97.00 x 1.00 = 97.0 Goal 2 Rating Factor = 97.0
Goal 3 Rating Factor uses 100% of the Educator Effectiveness Ratings Score:
(Educator Eectiveness Ratings score) 83.30 x 1.00 = 83.3 Goal 3 Rating Factor = 83.3
STEP 3: COMPUTE THE WEIGHTING DETERMINANT
The EPI Performance Score calculation procedure also employs a Control Factor by computing a Weighting
Determinant. The Weighting Determinant assigns a Weight Category to each EPI. The determined Weight
Category denes a unique Weighted Multiplier for each Rating Factor, as shown in Table 2.
Table 2: Variable Weight Schedule
Goal 1
Rating Factor
Weighted
Multiplier
Goal 2 Goal 3
Weighting
Determinant
Range
Weight
Category
Rating Factor
Weighted
Multiplier
Rating Factor
Weighted
Multiplier
1% – 10% 1 .70 .30 .0
11% – 20% 2 .63 .27 .10
21% – 30% 3 .56 .24 .20
31% or more 4 .50 .20 .30
H
The Weighting Determinant equals the Number of Educational Eectiveness Ratings Received divided
by the Number of Completers:
(Ed. E. Ratings Received) 50 ÷ (Number of Completers) 81 = .62 Weighting Determinant = 62%
In the above example, a program with a calculated Weighting Determinant of 62% falls into Category 4, for
which the Goal Rating Factor Weighted Multipliers are .50, .20, and .30 respectively.
STEP 4: APPLY THE WEIGHTED MULTIPLIERS TO THE RATING FACTORS
As explained in Step 3, the Weight Category denes a unique Weighted Multiplier for each Goal Rating Factor.
The next step in the EPI Score calculation procedure is to apply these Weighted Multipliers:
(Goal 1 Rating Factor) 90.9 x (Category 4 Multiplier) .50 = 45.5 (Weighted Goal 1 Factor)
(Goal 2 Rating Factor) 97.0 x (Category 4 Multiplier) .20 = 19.4 (Weighted Goal 2 Factor)
(Goal 3 Rating Factor) 83.3 x (Category 4 Multiplier) .30 = 25.0 (Weighted Goal 3 Factor)
STEP 5: CALCULATE FINAL EPI PERFORMANCE SCORE
The final EPI Performance Score is the sum of the three Weighted Rating Factor products from Step 4:
(Weighted Goal 1 Factor) 45.5
(Weighted Goal 2 Factor) 19.4
(Weighted Goal 3 Factor) + 25.0
(Final EPI Performance Score) = 89.9
STEP 6: COMPARE THE FINAL EPI PERFORMANCE SCORE TO THE CUT SCORE TO
DETERMINE THE PERFORMANCE LEVEL AND CORRECTIVE ACTION PHASE
In the last step, the final EPI Performance Score — 89.9 — is compared to the cut score of 84.5:
The EPI score exceeded the cut score: (Final EPI Score) 89.9 > 84.5 (Cut Score)
Based upon the EPI Performance Score comparison to the Cut Score, a determination of the appropriate
Performance Level and Corrective Action Phase is made in context of the Progressive Corrective Action System,
as described in the next section.
>>
9+$:+'((,*'/#$++'4&,*'//
;4&,$%/6<(&'=.
The Progressive Corrective Action System consists of three Performance Levels and seven Corrective Action
Phases, as illustrated in Table 3.
The determination of whether or not an EPI has met the Cut Score each year, in relation to their historical
performance, places the EPI into one of three Performance Levels: 1) “Satisfactory”; 2) “At Risk of Low
Performing”; and 3) “Low Performing”; and in one of the seven Corrective Action Phases: “0-6.”
Each year, upon meeting or not meeting the cut score, the EPI Phase number may increase or decrease by
one step, depending on the Phase assigned the previous year and the Performance change (progression or
regression) in the current year, as described in Table 3.
Table 3. EPI Performance Levels, Corrective Action Phases and Goals
Performance
Level
Corrective
Action Phases
EPI Goals
Satisfactory
0
Maintain this Performance Level, while seeking
opportunities to improve.
1
Meet cut score next year to get to Phase 0.
At Risk of Low
Performing
2
Meet cut score for next 2 years in a row to reach Phase 0.
3
Meet cut score for next 3 years in a row to reach Phase 0.
Low
Performing
4
Meet cut score for next 4 years in a row to reach Phase 0.
5
Meet cut score for next 5 years in a row to reach Phase 0.
6
Meet cut score for next 6 years in a row to reach Phase 0.
For example, an EPI that was “At Risk, 2” in the previous year, then meets or exceeds the Cut Score in the current
year, will progress to “Satisfactory, 1.” If the same EPI meets or exceeds the Cut Score the next year, the program
would progress to “Corrective Action Phase, 0.” Alternately, if the same EPI misses the Cut Score in the next
year, it would regress to “At Risk, 2.” (Once at “Phase 0”, an EPI cannot improve numerically.)
>!
Table 4 describes in detail the characteristics of each Performance Level and the specic consequences of each
Corrective Action Phase:
Table 4. Performance Level Characteristics and Corrective Actions
Performance Level Characteristics Corrective Actions
Satisfactory
• high pass rate on MTTC content-based assessments
• teacher candidates report a high level of program ecacy
regarding their teacher preparation program, and clinical
experiences
• supervising faculty rate teacher candidates’ preparation
postively
• graduates almost exclusively earn “Eective” or “Highly
Eective” ratings over the most recent three-year period
0 – No corrective actions required.
EPI collaborates with the MDE; pursues
continuous quality improvement and
may serve as a model.
1 – Awarded conditionally, the year
following the rst year after which no
corrective action was required. Minimal
corrective action required.
At Risk
• low pass rate on MTTC content-based assessments
• teacher candidates report a low level of program ecacy
regarding their teacher preparation program, including
clinical experiences
• supervising faculty rate teacher candidates’ preparation
negatively
• graduates almost exclusively earn “Ineective” or
“Minimally Eective” eectiveness ratings over a
three-year period
2 – Intensive corrective action may be
required. EPI collaborates with the MDE
to design, develop and implement a
corrective action plan.
3 – Intensive corrective action required.
EPI collaborates with the MDE to design,
develop, and implement a corrective
action plan.
>"
Performance Level Characteristics Corrective Actions
Low Performing
• low pass rate on MTTC content-based assessments
• teacher candidates report a low level of program ecacy
regarding their teacher preparation, including clinical
experiences
• supervising faculty consistently rate teacher candidates’
preparation negatively
• graduates almost exclusively earn “Ineective” or
“Minimally Eective” eectiveness ratings over a
three-year period
4, 5, & 6 – Critical corrective action
required at program and institution
levels. EPI collaborates with the MDE and
other resource to design and develop
goal-specic rapid improvement plans.
May necessitate assignment of an
external committee of scholars. May
result in closure of individual programs.
May result in withdrawal of institutional
approval.
>1
;77'%?,@./
A$-/B24C/9'+D$+=2%4'//
E'2(5+'/F(/#234532&'?
EPI SCORE CALCULATION COMPONENTS
1. Michigan Test for Teacher Certification (MTTC) Score
2. Teacher Candidate Survey and Candidate Supervisor Survey Scores
3. Educator Effectiveness Ratings
MTTC
The MTTC is administered by the Evaluation Systems Group of Pearson, Inc. The MTTC contribution to the EPI
Performance Score is the test takers’ best attempt, also known as the Cumulative Pass Percentage. The Pass
Percentage is calculated by dividing the cumulative number of content/subject area tests passed (after a failed
first attempt) by the total number of initial attempts made within a 36-month period
. Test takers may attempt a test an unlimited
number of times. This system accounts for data variability, which occurs year after year.
NOTE: three-year cumulative test data for subject area programs closed in consultation with the MDE as a
result of corrective action activities in the academic year for which the score is calculated are removed from
score calculation.
The basic skills examination component of the MTTC – e.g., professional readiness examination (PRE) or its
alternative pass measures – is not included in the EPI performance score calculation.
Table 5, below, is an example of MTTC Pass Percentage Calculation.
T
able 5: Example MTTC Pass Percentage Calculation
Content Tests
Number
of Initial
Attempts
Number of
Initial Tests
Passed
Percent of
Initial Tests
Passed
Cumulative
Number
of Tests
Passed
Cumulative
% of Tests
Passed
002 English 8 7 87.5 7 87.5
004 Speech 3 3 100.0 3 100.0
009 History 2 1 50.0 1 50.0
>I
Content Tests
Number
of Initial
Attempts
Number of
Initial Tests
Passed
Percent of
Initial Tests
Passed
Cumulative
Number
of Tests
Passed
Cumulative
% of Tests
Passed
010 Political Science 1 0 0 0 0
011 Psychology 1 1 100.0 1 100.0
017 Biology 2 0 0 2 100.0
022 Mathematics (Secondary) 3 2 66.7 3 100.0
023 French 1 1 100.0 1 100.0
028 Spanish 4 4 100.0 4 100.0
043 Health 11 8 72.7 10 90.9
044 Physical Education 13 8 61.5 10 76.9
084 Social Studies (Secondary) 6 1 16.7 4 66.7
089 Mathematics (Elementary) 5 4 80.0 5 100.0
090 Language Arts (Elementary) 11 9 81.8 10 90.9
093 Integrated Science
(Elementary)
4 2 50.0 4 100.0
094 Integrated Science
(Secondary)
2 2 100.0 2 100.0
095 Visual Arts Education 5 4 80.0 4 80.0
099 Music Education 14 13 92.9 13 92.9
103 Elementary Education 21 16 76.2 19 90.5
106 Early Child Ed (Gen & SPED) 3 3 100.0 3 100.0
All Tests (excluding PRE) 120 89 74.2 106 88.3
>8
As illustrated in the example, the MTTC Pass percentage is calculated by dividing the cumulative
number of tests passed by the number of initial attempts:
(cumulative tests passed) 106 / (number of initial attempts) 120 = (Pass Percentage) 88.3
This Pass Percentage, 88.3, is the MTTC component of the EPI Performance Score calculation.
SURVEYS
The survey component of the EPI Performance Score calculation includes data collected twice a year (spring and
fall) from Teacher Candidates (TC), who evaluate their experiences in the teacher preparation programs, and from
Candidate Supervisors (CS) at each EPI, who work with and directly supervise the clinical experiences of teacher
candidates.
The TC and CS surveys are designed, developed, and administered by the MDE. The survey items in both surveys
align with MI-InTASC’s “Model Core Teaching Standards and Learning Progression for Teachers 1.0.” The Teacher
Candidate survey is composed of 31 Likert-type items (scaled 1-4), and distributed across eight categories.
The Candidate Supervisor survey is composed 28 Likert-type items (scaled 1-5), and distributed across seven
categories.
Survey scores are combined to create ecacy scores per item, then category, and then per survey. Ecacy is
dened as the percentage of “Strongly Agree” and “Somewhat Agree” responses.
The overall Survey Ecacy Score is the average of ecacy from the four surveys administered in the year, and
represents the overall percentage of positive responses on the Likert scale across all Items and Categories, in both
sets of surveys (TC or CS).
This Overall Survey Efficacy Score is the Survey component of the EPI Performance Score calculation.
EDUCATOR EFFECTIVENESS RATINGS
Data provided by K-12 administrators is collected through the Registry of Education Personnel (REP) and the
Michigan Online Educator Certication System (MOECS). This data is provided to the MDE by the Center for
Educational Performance and Information (CEPI). The Educator Eectiveness Rating system uses data from the
most recent three years of teaching, within the most recent ve-year span since rst hire with valid certication.
>G
To determine the overall Educator Eectiveness Rating used as the EPI Performance Score component, the data
counts are processed as explained here:
1) The effectiveness level data counts are converted to percentages of the overall count.
2) The category percentages are each then converted to a point value, using a graded system designed
to account for continuous improvement of teaching practices, and to ensure that all classrooms have
effective teachers through ongoing professional development, even when teachers are receiving the
highest scores.
3) The point value for each category is also weighted based on teaching experience, to flatten the
higher learning curve for newer educators. The MDE recognizes factors outside the control of an EPI may
account for differing teacher performance over time and unweighted values would give these factors too
much influence over this component score. Thus, a Weighting Component is applied to the effectiveness
point values, which is based on educator years of experience. As teachers in their first year may face a
steep learning curve, performance in this year receives a slightly lower weight (0.3) than given to more
experienced teachers. Similarly, teachers in their third year of experience may have learned significantly
from their experience rather than from their EPI, so performance in this year receives a still lower weight of
0.2. Performance in the second year of experience receives the remaining 0.5.
Three years of tabulated Effectiveness Ratings (the three most recent years of teaching within the most
recent five-year span) are then totaled, to arrive at a final Ed. Eff. Score. See Tables 6a-6c.
Table 6a. Educator Effectiveness Score Tabulation for Year 1 of Teaching
Tabulations for First
Year of Teaching
Experience
Ineffective
(0 Point
Value)
Minimally
Effective
(0.3 Point
Value)
Effective
(0.8 Point
Value)
Highly
Effective
(1.1 Point
Value in
Year 1 of
Teaching)
Totals
Count 1 15 207 60 283
Percent of Total Ratings .00 .05 .73 .22 100
Rating Level Point Value 0 1.5 58.4 24.2 84.1
Final Weighted Value
(by a factor of .3 for one
year of experience)
0 .45 17.52 7.26 25.23
>H
Table 6b. Educator Effectiveness Score Tabulation for Year 2 of Teaching
Tabulations for
Second Year of
Teaching Experience
Ineffective
(0 Point
Value)
Minimally
Effective
(0.3 Point
Value)
Effective
(0.8 Point
Value)
Highly
Effective
(1 Point
Value after
Year 1 of
Teaching)
Totals
Count 0 9 105 41 155
Percent of Total Ratings .00 .06 .68 .26 100
Rating Level Point Value 0 1.8 54.40 26.0 82.2
Final Weighted Value
(by a factor of .5 for two
years of experience)
0 .90 27.20 13 41.10
Table 6c. Educator Effectiveness Score Tabulation for Year 3 (and no more than five years) of Teaching
Tabulations for Third
Year of Teaching
Experience
Ineffective
(0 Point
Value)
Minimally
Effective
(0.3 Point
Value)
Effective
(0.8 Point
Value)
Highly
Effective
(1 Point
Value after
Year 1 of
Teaching)
Totals
Count 1 3 48 21 73
Percent of Total Ratings .01 .04 .66 .29 100
Rating Level Point Value 0 1.2 54.80 29.0 85.0
Final Weighted Value (by
a factor of .2 for three
years of experience)
0 .24 10.96 5.8 17.00
!J
The final Educator Effectiveness Score is the sum of the Ed. Eff. Scores for 3 years of teaching:
(Year 1 Ed. E. Score) 25.23 + (Year 2 Ed. E. Score) 41.10 + (Year 3 Ed. E. Score) 17.00 = (Overall
Educator Effectiveness Rating) 83.33
This overall rating, 83.33, is the Educator Effectiveness Rating component of the EPI Performance Score
Calculation.
NOTE: In cases where an EPI has teachers who are missing ratings used in the tabulation formula, the “years-of-
experience” weights are scaled proportionately to add up to 1.0 (one). Some possible illustrative scenarios are
presented in Table 7, below.
Table 7: Examples of Alternate Effectiveness Rating Weights
Examples
Rates and
Weights
Years
A
Available Rates Year 1: Yes Year 2: Yes Year 3: Yes
Assigned Weights
.3000 .5000 .2000
B
Available Rates Year 1: Yes Year 2: Yes Year 3: No
Assigned Weights
.3750 .6250 .0000
C
Available Rates Year 1: Yes Year 2: No Year 3: Yes
Assigned Weights
.6000 .0000 .2857
D
Available Rates Year 1: No Year 2: Yes Year 3: Yes
Assigned Weights
.000 .7143 .2857
E
Available Rates Year 1: Yes Year 2: No Year 3: No
Assigned Weights
1.0000 .0000 .0000
F
Available Rates Year 1: No Year 2: Yes Year 3: No
Assigned Weights
.0000 1.000 .0000
G
Available Rates Year 1: No Year 2: No Year 3: Yes
Assigned Weights
.0000 .0000 1.000
OFFICE OF PROFESSIONAL PREPARATION SERVICES (OPPS)
608 WEST ALLEGAN ST, LANSING, MI 48909 | p: 517-373-3310 | f: 517-373-0542
WWW.MICHIGAN.GOV/MDE