Notes from CAEP:  2013 initial preparation -The provider demonstrates that the quality of candidates is a continuing and purposeful part of its responsibility from recruitment [component 3.1], at admission [component 3.2], through the progression of courses and clinical experiences [components 3.3 and 3.4], and to decisions that completers are prepared to teach effectively and are recommended for certification [components 3.5 and 3.6]. The provider demonstrates that development of candidate quality is the goal of educator preparation in all phases of the program. This process is ultimately determined by a program's meeting of Standard 4.

2016 advanced level preparation-- The provider demonstrates that the quality of advanced program candidates [components A.3.1 and A.3.2] is a continuing and purposeful part of its responsibility [component A.3.3] so that completers are prepared to perform effectively and can be recommended for certification where applicable [component A.3.4].

[NOTE: Under CAEP Board policy, component 3.2 (initial) and A.3.2 (advanced level) must be met for full accreditation]

Making a case: In Standard 3, the provider demonstrates that it recruits and selects candidates for initial preparation with potential to have a positive impact on all P-12 students' learning and development, and that its actions contribute to a more diverse and academically able educator workforce. Candidates admitted for advanced preparation represent diverse populations and, over time, should reflect the diversity of P-12 students. The provider's admission requirements, including academic achievement, ensure that candidates successfully complete their chosen specialized preparation program. For all candidates, the EPP provides appropriate support and counseling for candidates whose progress falls behind.

The provider continues to prepare and monitor candidates during their programs to ensure that completers will be effective teachers or effective in leadership or other professional specialties. The programs ensure that there is growth in aspects of preparation that are essential for P-12 student learning.

Similar to Standard 1, evidence for Standard 3 focuses on pre-service preparation. For both initial and advanced preparation, providers should include only what is unique to Standard 3 and not addressed by performance evidence in Standard 1. To demonstrate the link between preparation and effective teaching, the provider may find it necessary to refer to what is included in Standard 4, but it is not necessary to repeat the Standard 4 documentation in Standard 3.

The guiding questions may help focus the selection of evidence and the EPP inquiry of its message:

• What are the provider's plans and goals to recruit and support completion of high-quality candidates from a broad range of backgrounds and diverse populations to accomplish their mission?
• What are the admission requirements?
• What are the criteria for program progression and how does the provider monitor candidates' advancement from admissions through completion?
• How does the provider:
• ensure that the admitted pool of candidates reflects the diversity of America's P-12 students?
• address community, state, national, regional, or local needs for hard-to-staff schools and shortage fields, currently, STEM, English-language learning, and students with disabilities?
• gather data to monitor applicants and the selected pool of candidates?
• establish and monitor attributes and dispositions beyond academic ability that candidates must demonstrate at admissions and during the program?
• select criteria, describe the measures used and evidence of the reliability and validity of those measures, and report data that show how the academic and non-academic factors predict candidate performance in the program and effective teaching?
• What is the provider's evidence that ensures that the average grade point average of its accepted cohort of candidates meets or exceeds the CAEP minimum of 3.0, and the group average performance on nationally normed ability/achievement assessments is in the top 50 percent from 2016-2018?
• What are the criteria for program progression and how does the provider monitor candidates' advancement from admissions through completion?
• How does the provider analyze the evidence to indicate candidates' developing content knowledge, pedagogical content knowledge, pedagogical skills, and the integration of technology in each of these domains?
• How does the provider document that the candidate has reached a high standard for content knowledge in the fields where certification is sought and can teach effectively with positive impacts on P-12 student learning and development?
• How does the provider document that the candidate understands the expectations of the profession, including codes of ethics, professional standards of practice, and relevant laws and policies, before recommending the candidate for licensure?

The EPP should reflect on:

• STRENGTHS AND CHALLENGES-What strengths and areas of challenge have you discovered as you analyzed and compared the results of your disaggregated data on candidate quality, recruitment/admissions, and quality monitoring by program and by demographics? What questions have emerged that need more investigation? How are you using this information for continuous improvement?
• TRENDS-What trends in candidate quality, recruitment and admissions practices, and monitoring of candidate progress have emerged as you compared program and demographic data across evidence sources and programs? What questions have emerged that need more investigation? How are you using this information for continuous improvement?
• IMPLICATIONS-What implications can you draw or conclusions can you reach across evidence sources about candidate quality, recruitment/ admissions, and quality monitoring? What questions have emerged that need more investigation? Improvement? How have data-driven decisions on changes been incorporated into preparation

Summary Statement:

Recruiting high-quality candidates who will complete the program is essential to a strong teacher preparation program.  The Candidate Recruitment and Completion (CRC) packet offers evidence directly related to CAEP Standard 3.  To analyze its recruitment of candidates, the MU EPP considered the following questions related to CAEP Standard 3:

1. Do the criteria for program admission, progression, and completion reflect the MU EPP plans and goals to recruit high-quality candidates?
2. Is the MU EPP’s plan for monitoring candidates' advancement from admissions through completion effective?
3. How does the EPP’s plan for recruitment reflect the diversity of America's P-12 students and address community, state, national, regional, or local needs for hard-to-staff schools and shortage fields, currently, STEM, English-language learning, and students with disabilities?
4. Does the EPP clearly articulate the criteria for program progression, monitor candidates' advancement from admissions through completion, and analyze the evidence to indicate candidates' developing content knowledge, pedagogical content knowledge, pedagogical skills, and the integration of technology in each of these domains?
5. Does the EPP adequately measure that candidates have (A) reached a high standard for content knowledge, (B) can teach effectively with positive impact on P-12 students, and (3) understand the expectations of the profession, including codes of ethics, professional standards of practice, and relevant laws and policies, before recommending the candidate for licensure?

Evidence used to assess the program and answer the above questions related to CAEP Standard 3 is included in the Candidate Recruitment and Completion (CRC) packet.  Evidence includes the institution’s recruitment plan, data regarding Pearson test scores, and dispositional evaluations.

Standard 3.1 CAEP Standard 3.1 is a complex standard.  It involves the recruitment of candidates as well as how the EPP supports candidates in order to reach completion of the program.  To recruit excellent candidates, the institution strives to attract undergraduates who reflect the diversity of P-12 schools.  The EPP worked with the Office of Admissions to create an intentional plan of action for recruiting high quality candidates.  Each fall, the EPP hosts a full-day recruiting event on campus.  The EPP collaborates with all high schools across the state of Indiana, reaching out to all students interested in education and drawing on partnerships with cadet teaching and internship coordinators (CRC packet).  During the Academic Visit Day, potential candidates meet with current candidates, participate in education courses, and meet with EPP faculty; they also explore the campus and have lunch with current candidates.  Table 1 indicates the number of prospective students (18 over 3 years) who attended the visit days and who matriculated to Manchester University. The EPP believes this is a positive in recruitment.  Additionally, two members of the EPP serve as University Ambassadors and receive a list of prospective candidates each week.  The following number of personal contacts made through handwritten letters, phone calls, or emails by EPP’s Ambassadors:  175 (2015-2016, 201 (2016-2017), and 226 (2017-18).   While this is an important way of connecting, the EPP needs better matriculation from initial contact to candidates.

Another aspect of the collaborative recruiting plan involved the DTE traveled with members of the Office of Admissions and football coaching staff to Macon, Georgia, to host a recruiting event in the spring of 2018.  Because of their interest in teaching, football recruits had the opportunity to speak with the DTE about the teaching program and expectations for MU candidates.  Through events like this one and the annual Academic Visit Day, the Office of Admissions and the EPP have made a concerted effort to recruit high-quality teaching candidates from a diverse population (CAC packet). 

Also included in this standard is the intentional recruitment of high need areas.  Despite a state-wide fluctuation in enrollment in teacher preparation programs and in the number of licenses granted in the state of Indiana (IDOE Annual Educator Licensing Summary in Supplemental Evidence folder), enrollment in high-demand areas such as special education is consistent and steady over the past 3 years. According to a study conducted by Indiana State University, school districts in Indiana have reported a teacher shortage in key areas.  Table 3C demonstrates some of the survey data reported by the study.  The MU EPP’s enrollment is identified in Table 3D.  Like other EPPs, MU’s struggles to recruit math and science educators.  Special education is, however, demonstrate steady enrollment.

To attract, admit, and support program completion for the most highly qualified candidates, the Manchester University EPP employs the requirements expected by the Indiana Department of Education.  The requirements include acquiring an overall 2.5 grade point average and 2.5 major grade point average; successfully passing all three parts of the Pearson CORE Academic Skills Assessment (CASA), which indicates a basic skills competency in reading, writing, and mathematics; or having earned a 24 on the ACT (math, reading, grammar, and science), 1110 on the SAT (reading and math combined) if taken prior to March 1, 2016 and then said scores must be converted online if taken after said date, or GRE score of at least 1100 (verbal and quantitative) prior to 8/1/11 or 301 (verbal and quantitative) on or after 8/1/11.  The EPP also requires attendance at the Celebrate Diversity workshop as well as positive disposition reviews from clinical faculty and members of the EPP (CRC packet).

The Field Experience and Assessment Coordinator monitors candidates’ progress towards admission to the program as well as their progress through the program to completion.  As outlined in the Teacher Education Student Handbook, the EPP has established clear checkpoints.  Once a candidate meets the basic skills testing requirements and maintains the minimum grade point average, the candidate meets with the Director of Teacher Education one-on-one for a sophomore interview.  During this interview, the Director reviews field experience evaluations as well as dispositional rubrics completed by members of the EPP.  At this meeting, the Director reviews program expectations, codes of ethics, and the process for matriculating into the field of teaching.  The candidate also provides the Director with the names of four faculty members from within the teaching program and outside the program who serve as references, evaluating the candidate's dispositions and academic performance.

Upon successful completion of the interview, dispositional rubrics completed by references, and confirmation by Dean of Student Experience of satisfactory student conduct, the EPP presents the names of candidates to the Teacher Education Committee which then votes to admit the candidates to the teaching program (CRC packet).  Candidates must maintain the minimum requirements in order to maintain their status in the teaching program.

Standard 3.2 As identified in standard 3.1, to be admitted to the teacher preparation program and continue taking upper level (300 or above) course work within the program, teacher candidates must maintain a 2.5 grade point average and meet the basic skills requirement in reading, writing, and mathematics.  Collectively, the average GPA of candidates completing the program in 2016 was 3.44, in 2017 was 3.04, and 2018 was 3.27.  With a three-year grade point average of 3.25, the EPP is in compliance with the CAEP minimum criteria.   In comparison with other teacher preparation programs across the state of Indiana, completers of the Manchester EPP overall perform above the state average (CRC packet).  Since the 2018 cohort, completers must successfully pass all required Pearson content exams prior to student teaching.  As a result, this inaugural cohort has an unusually low retention rate from entry to the program to completion; however, in terms of comparison to other programs, Manchester has 100% pass rate of content exams for all 2018 completers.    

Collectively, candidates in the program consistently maintain above a 3.0 grade point average.  Within the major and overall, teaching candidates demonstrate academic success at both the point of admissions and upon completion of the program (CRC packet).  This information is shared with important stakeholders such as the Teacher Education Committee and the Teacher Advisory Council.

In accordance with CAEP and Indiana Department of Education Expectations, the Manchester

EPP submits Title II reports.  Reports since 2013 provide a snapshot of the Manchester University teacher preparation program, and as directed, the EPP posts its Title II reports on its web site for external review (https://www.manchester.edu/academics/colleges/college-of-education-social-sciences/academic-programs/education/education-home/accreditation/annual-reports)

Standard 3.3  The EPP articulates, models, and reflects with candidates on appropriate educator attributes and dispositions.  From the first courses in the program, which include EDUC 111: Introduction to Teaching and EDUC 211:  Exceptional Learners, the EPP explicitly discusses the characteristics of effective educators.  As they progress through the program, candidates have opportunities to intentionally investigate and reflect on these attributes and dispositions.  The EPP believes it is important to communicate expectations, offer candidates the opportunity to set goals for themselves in terms of attributes and dispositions, and to grow in their professional stance.  Sharing the EPP-created disposition rubric early with candidates provides clear expectations.  In foundational courses, candidates use the disposition rubric self-evaluate then conference with the instructor (CRC packet).  Additionally, the Field Experience and Assessment Coordinator tracks dispositional feedback for each field experience. 

Upon review of the dispositional evaluations, the DTE meets with candidates who have basic or unsatisfactory ratings to create a plan of action with the candidates’ input. Additionally, the DTE, the Field Experience and Assessment Coordinator, as well as individual members of the EPP send hand-written notes or emails to candidates who demonstrate positive educator attributes and dispositions.  The EPP offers a nurturing environment that allows students to grow positive dispositions and provides support for candidates who choose to investigate and switch to a new major (CRC packet).

During weekly department meetings, the EPP discusses candidates’ progress in the program, and often, this conversation focuses on candidates’ dispositions (Attachment CAEP 2B in CRC packet).  For admission to the program, candidates must have positive disposition checks, have passed the CASA or meet the SAT/ACT scores set by the state of Indiana, and have a 2.5 grade point average.  The TEC then reviews the applications to the program and votes on the candidates’ admission to the teacher preparation program (CRC packet). 

Two extremely important dispositional checkpoints come during the third and fourth years in the program.  To earn permission to enroll in student teaching, candidates must complete a formal interview with the Director of Teacher Education.  Candidates submit letters of interest in student teaching as well as resumes, and they interview for a student teaching position (CRC packet).  During this interview, candidates must support their application to student teach with examples of positive educator attributes and dispositions.  Upon completion of the interview, faculty members within the program as well as faculty in other disciplines complete reference forms which incorporate effective educator dispositions.  Once approved to student teach, the candidates enroll in methods (fall for elementary education majors and January for secondary and all-grade majors).  At the end of the methods experience, which includes extensive clinical experience in classrooms, the clinical faculty complete the disposition rubric, and members of the EPP who teach the methods courses decide together whether the candidates have fully demonstrated the attributes and dispositions of a candidate representing the Manchester University teacher preparation program.

Additionally, the EPP uses the Danielson framework to evaluate candidates during their clinical student teaching experience (Danielson packet).  While each of the domains incorporate expectations for proficient and distinguished candidates in terms of planning and preparation, classroom environment, and instruction, the fourth domain offers specific criteria for professional responsibilities including the dispositions of highly-effective.  For example, the objectives included in Domain 4, examine the ability to interact with others including students, colleagues, and parents (Attachment 3B Danielson Comparison Data).  According to the EPP’s data, for three data cycles, the range for Domain 1 (Planning and Preparation was 2.75 – 3.15 with an average of 2.96/4.0); Domain 2 (The Classroom Environment) had a range of 2.95 – 3.28 and an average of 3.12/4.0; Domain 3 (Instruction) reflected a range of 2.8 – 3.17 with an average of 3.03/4.0; and Domain 4 (Professional Responsibilities) reported a range of 2.96-3.18 and an average score of 3.08/4.0.  Over three cycles of data collected, when taken collectively, candidates earned proficient ratings in all four domains.

Standard 3.4 Over the past few years, the EPP has continued to focus attention on its criteria and monitoring of candidates through the program.  Clear, published checkpoints provide criteria for progression through the program and include academic progress as well as evaluations of candidates’ dispositions.  Academically, the candidates’ GPAs in both the major and the overall academic program as well as the successfully passing of the required Pearson exams is carefully tracked. Clinical faculty as well as EPP faculty complete EPP-created rubric evaluations of candidates’ dispositions. The Field Experience and Assessment Coordinator checks in periodically with the clinical faculty to monitor attendance as well as dispositional performance.  This feedback offers insight for progress in the program.  To be admitted to the program, candidates must meet the minimum GPA and standardized test scores requirements as well satisfactory disposition ratings.  They must also adequately complete an interview with the DTE.

At the one-on-one sophomore and junior interviews, the Director of Teacher Education reviews the criteria and discusses the candidate’s progress in the program.  In the junior year, prior to being approved for student teaching, candidates receive written communication in both an email and a letter mailed to their home addresses (Clinical Partnership packet) reminding them of the student teaching requirements: passing of necessary Pearson content exams prior to December 1 for spring student teaching or June 1 for all fall student teaching clinical experiences.

In the spring of the junior year, all teacher candidates approved for student teaching meet with the Director of Teacher Education and the Field Experience and Assessment Coordinator for a mandatory meeting.  Throughout the evening, the Director and Coordinator outline expectations (CRC packet) and articulate the protocol for contacting the cooperating teachers.  Underscored during this meeting is the required Start of School experience which requires all student teachers to attend the first four days of school regardless of the school system's calendar. The EPP provides clinical faculty and university supervisors with a comprehensive Student Teaching Handbook (Supplemental Evidence Folder); teacher candidates are given a copy, but attention is drawn to the online PDF (https://www.manchester.edu/academics/colleges/college-of-education-social-sciences/academic-programs/education/education-home/resources).

In the fall semester prior to student teaching, the university supervisors, student teachers, and cooperating teachers attend a training session offered by the Director of Teacher Education and the Field Experience and Assessment Coordinator (Clinical Partnership packet).  This meeting articulates the expectations of the stakeholders:  supervisors, clinical faculty, the EPP, and student teachers.

Student teaching is the pinnacle clinical experience, and trained university supervisors and clinical faculty support the candidates in this experience.  The Student Teaching Handbook clearly outlines expectations of the EPP, the candidate, and the clinical faculty.  This handbook specifically articulates the obligations of each party.  Open lines of communication and frequent feedback support the candidates as they complete the program. 

The EPP immediately tracks potential candidates in the EPP’s data collection system; the EPP is currently transitioning from paper files to an electronic software program.  Monitoring candidates’ progress through the program provides data regarding matriculation into the teaching program.  Sometimes, the candidate self-selects out after completing a foundational course, completing field experience observations, or discovering a different major; other times, the EPP denies candidates entry into the program because they have not met the minimum requirements: 2.5 grade point average or competency on basic skills measures (CASA or ACT/SAT scores as previously indicated) (CRC packet). 

In the fall of 2017, the Vice President for Academic Affairs funded the online program CORE. This software allows all students to log their field experience hours, evaluate the teacher and their experience, log incident reports and document what they did during each field encounter. It allows clinical faculty to confirm the field placement hours, evaluate the student, log incident reports, check scheduling of students and communicate with students and the Field Experience and Assessment Coordinator. The Field Experience and Assessment Coordinator run reports regarding evaluations, hours logged, incidents, and other important information to ensure the students are successful in their placements.

Recently, the EPP sent one of the university supervisors for training on the Danielson Framework, the evaluation tool used to evaluate student teachers.  In the fall of 2018, this supervisor will begin training other supervisors and clinical faculty to effectively use the rubric.   During the required literacy courses, teacher candidates use the Danielson Framework to complete a self-evaluation.  Experience with the framework affords opportunities to understand expectations prior to the actual student teaching.

Standard 3.5 The Manchester University EPP complies with the Indiana Department of Education teacher preparation program admission requirements which include a 2.5 grade point average as well as passing the three sections of the Pearson CASA exam (mathematics, reading, and writing).  These tests demonstrate candidates’ proficiency in basic skills required of all educators.  Among several other requirements to earn a license in Indiana, candidates must complete coursework, complete a minimum of ten weeks in student teaching, and pass the content specific Pearson content and pedagogy exams.  With the 2017-2018 senior cohort, the EPP implemented a policy which required candidates to pass content exams prior to student teaching in order to assure clinical faculty and administrators of the teaching candidates’ mastery of content knowledge (CAC packet).

The CASA (or equivalent assessments) standardized test provides a valid measurement of teaching candidates’ basic knowledge, and as they progress through the program, the EPP tracks their academic progress and content knowledge through the monitoring of the grade point average.  Each of the courses is aligned to content standards as well as InTASC standards (EPP Overview packet).  

SPA reports also provide evidence of mastery of content knowledge for specific disciplines including English and history secondary education, elementary generalist, physical education to name a few (see Standard 1 SPA section).

Additionally, clinical faculty and clinical supervisors evaluate candidates’ content knowledge during their student teaching.  The EPP created a content-specific evaluation rubric directly aligned with the content standards (CRC packet).  The EPP uses rubrics like the Danielson framework rubric as well as the content programs directly related to teacher preparation such as history, mathematics, language arts, science, etc. to provide feedback to candidates (CRC packet).

3.6 Within the Teacher Education Student Handbook and the Student Teaching Handbook (both available online for candidates) the EPP has clearly outlined expectations for candidates as they work within the school setting (Teacher Education Student Handbook and Student Teaching Handbook:  Expectations for Student Teachers). Additionally, throughout the student teaching experience, the clinical supervisors and clinical faculty members observe and evaluate student teachers on professional expectations.  Domain 4 of the Danielson framework rubric holds candidates accountable for their understanding of these expectations. 

The state of Indiana also requires certification in Red Cross CPR/AED as well as evidence of training in suicide prevention.  As part of the EPP's progress monitoring, candidates must submit evidence of both trainings, and the EPP keeps copies in the candidates' files.  Candidates must also complete extensive background checks, an issue of which they are aware from the beginning of their program.

During the student teaching clinical practice, candidates must also enroll in and successfully complete the seminar class, EDUC 410.  During this weekly seminar, candidates attend sessions presented by Child and Family Services regarding the most current law for reporting child abuse incidents.  They also attend sessions offered by current administrators and representatives of the Indiana State Teachers Association, sessions in which candidates learn of current school laws by which they must abide. Their participation in these experiences is reflected in the course grade.  The EPP is currently working on an assessment to embed in EDUC 410 which will require candidates to reflect on their professionalism.

Strengths and Challenges

According to the evidence, criteria for program admission, progression, and completion reflects the EPP’s plans and goals to recruit high-quality candidates (Q1).  The EPP has increased its collaboration with Admissions, and the EPP has met several times with the Admissions staff to outline the CAEP expectations adopted by the EPP.  In particular, the EPP has articulated the high needs areas such as ELL and special education.  Admissions counselors are better equipped to talk with prospective candidates about the teacher preparation program.  As reflected in the number of prospective candidates contacted through the Ambassador program as well as the Academic Visit days, the EPP is on the right track for recruiting high-quality candidates.  It also has a structure in place for supporting the completion of the program, focusing on key checkpoints.  The DTE has individual meetings with candidates, focusing on setting goals with the candidates.

The MU EPP continues to explore innovative ways to monitor candidates' advancement from admissions through completion (Q2).  Through discussions with the Teacher Advisory Council and the Vice President of Academic Affairs, the EPP explored ways to electronically monitor candidates’ progress.  Funding from the institution provides access to the CORE software, and the EPP believes this strengthens the EPP’s data collection.  One of the challenges facing the EPP is supporting the candidates in their ownership of the process.  While the EPP believes it is responsible for the tracking, it will also continue to explore ways to engage candidates from the beginning in their own evaluation of their progress.  This will come through more frequent and intentional meetings between cohorts of candidates and the DTE.  Additionally, each course in the EPP should support the progress in the program with explicit outlining of appropriate checkpoints.  Instead of introducing the checkpoints in the first foundational course, the checkpoints and expectations should be revisited with each course, especially in reference to the candidates’ development as professionals.

The EPP’s plans for recruitment reflect the diversity of America's P-12 students and address community, state, national, regional, or local needs for hard-to-staff schools and shortage fields (currently, STEM, English-language learning, and students with disabilities) (Q3).  Due to its location, the EPP struggles, like rest of the institution, to attract candidates to work with diverse populations and in high needs areas; however, it will continue to partner with community partners in different settings to recruit candidates.  It will need to work with the Marketing Office to consider ways to attract high-quality candidates to the teaching program.

The EPP’s Teacher Education Student Handbook reflects the EPP’s criteria for program progression, monitor candidates' advancement from admissions through completion.  The Field Experience and Assessment Coordinator collects data, and together, the EPP analyzes the depth of candidates' content knowledge, pedagogical content knowledge, pedagogical skills, as well as the integration of technology (Q4).  The chart in the evidence packet provides a clear delineation of the data collected at specific points in the program.  Data points indicate checkpoints provide the program with a consistent monitoring of candidates from admission to completion.  The gap between those who enter the program but do not complete the program indicate a level of rigor in the program which gives the EPP confidence that the most qualified candidates are entering the field with a deep level of content knowledge and mastery of pedagogy. 

Implications

While the EPP adequately articulates the expectations and supports candidates through completion, the EPP recognizes it needs more frequent conversations with candidates.  It also needs to intentionally include the candidates in setting professional goals for themselves.  The CORE software program holds potential for increasing the intentionality of the recruitment and completion.

In the fall of 2018, the EPP will collaborate with clinical faculty and other stakeholders to develop an assessment for student teachers which will require them to reflect on their professional development.  Pulling evidence in from their four years in the program will provide completers with an intentional look at their professional journey. 

Because the EPP works closely with clinical, university supervisors, and content faculty, it relies heavily on candid feedback regarding completers’ dispositions.  At a variety of points throughout the program, the EPP asks for evaluations of candidates’ dispositions (CRC packet).  Over the past few years, the evaluation rubric has undergone a few changes.  An important change for the EPP is the move to electronic evaluations of the candidates.  The return rate was faster and more complete than in the past.