(Q1) Based on the analysis of the disaggregated data, how have the results of specialty licensure area or SPA evidence been used to inform decision making and improve instruction and candidate learning outcomes?

Each of the teacher preparation programs with five or more completers for the data collection cycle (academic years 2014-2015, 2015-2016 2016-2017) completed a SPA report; this required each of the program areas to collect evidence related to candidates’ mastery of content specific to the accreditation standards.  Unfortunately, when the current Director of Teacher Education (DTE) and the Field Experience and Assessment Coordinator (FEAC) began their positions in the fall of 2015, they found a haphazard collection of information and no structure to systematically collecting data.  Since then, they have worked to develop a Quality Assurance System (QAS) which outline key checkpoints, the type of assessment collected, and the path for collection.  The EPP requested a shared drive on the institutions internal network, and each program has access to key assessments.  They are able to upload data and work together as a unit to make use of the disaggregated data.  The EPP, as well, has access to each of the shared drives and can make comparisons across programs.  This, however, has come about slowly, and the DTE and FEAC continue to search for more efficient and appropriate ways to help programs use data to drive their instruction.  Additionally, the EPP is working with the Office of Institutional Effectiveness to tie the SPA report data collection to the departments’ annual reports required of that office. Adding additional lines to the current form and asking licensure areas to reflect on key assessments will ensure an even more systematic and institutional ownership of the SPA reports.  The responsibility will no longer rest on the DTE’s and FEAC’s shoulders.

It should be noted the shift between requiring Praxis II to Pearson content exams to evaluate candidates’ content knowledge.  The same occurred for the pedagogy exams as well.  This shift applies to all programs in the EPP applying for SPA recognition.

Elementary Education (ACEI) – Undergraduate/Initial

Status of program regarding SPA report:  National Recognition with Conditions

Resubmitted March 2018

Data was collected from the Pearson Elementary Education Generalist content exams which consist of four separate subtests (Reading and English Language Arts; Mathematics; Science, Health, and Physical Education; and Social Studies and Fine Arts).  Alignment between licensure exams and ACEI standards 1.0, 2.1, 2.2, 2.3, 2.4, 2.5, 2.6, 2.7, 3.1, 3.2, 3.3, 3.4, 3.5, 4.0, 5.1, and 5.2 support the use of the Pearson content exams as a good measure of candidates’ content knowledge.  The EPP reported the following number of completers 17 for 2014-201, 4 or 2015-2016, and 9 for 2016-2017.  Clearly, the EPP has struggled with enrollment in the elementary program.  While the state of Indiana has reflected this overall drop as well, the EPP was concerned by the low numbers.  Despite the low numbers, though, Manchester candidates still tend to score higher than the state average on the Pearson exams.  Additionally, candidates pass the pedagogy portion of the exam typically on the first or second time indicating the EPP is preparing them with skills to teach.

It should be noted only 4 elementary education majors completed the program in 2018 even though the cohort began with 13.  Since the SPA report for ACEI was originally submitted, the EPP implemented the requirement of passing the content exams prior to student teaching.  Currently, the EPP has 12 candidates preparing to student teach and complete in 2019; however, not all have passed their content exams yet.

English/Language Arts (NCTE) – Undergraduate/Initial

Status of program regarding SPA report:  National Recognition

Data points used for the NCTE SPA report include the Praxis II English Language Lit Comp Content Knowledge exam.  While candidates do relatively well, the English education program recognized the need to provide a more intentional alignment between content assessed by teacher licensure exams.  As a result, since the resubmission which earned it national recognition, the department collaboratively aligned required courses and key assessments with the standards assessed on the Pearson content exam (since initial SPA report was submitted, Pearson has replaced Praxis II).  More importantly, the English education program recognized previous assessment of its candidates was unintentional and often simply telling accreditors what they wanted to hear.  During the initial response to conditions, the program created new key assessments to reflect mastery of content knowledge.  The program also collaborated with the faculty in the Education Department to support the key assessment of the unit plan and the SCE Impact on Student Learning capstone project.

Of all of the programs, the mindset shift in English/Language Arts program was most dramatic and transformative.  The faculty in the program reported after pouring over reviewers’ comments and thinking differently about assessment, they finally understood the purpose of collecting data to use for programmatic shifts.  As a result, the English/Language Arts program, despite earning National Recognition, continues to realign courses and key assessments to better prepare English/Language Arts candidates for the classroom.  The EPP uses this program as model for using data and assessment to change programs.

Health Education (AAHPERD/AAHE) Undergraduate/Initial

Status of program regarding SPA report:  National Recognition with Conditions

Resubmitted March 2018

While this program received recognition with conditions, the EPP is quite impressed with the resubmitted report in March 2018, and it anticipates better results next month.  Much like the English/Language Arts program, the faculty in the Health program took ownership of the resubmission, spending hours/days reviewing the reviewers’ comments, examining the collection points and resulting data, and revising the program based on evidence. For the SPA report, the Health program reported a total of 6 completers for the data collection cycle.  During the cycle, Indiana’s requirement of the Praxis II content exam shifted to the Pearson content exam, so the data could not be fully compared. 

The Health program also heavily revised its key assessments, focusing on the written assessment, the health lesson plan, the teaching experience, and the annotated bibliography project.  Each was more carefully aligned with the AAHE standards.  Important to clearly reflecting student performance, many of the revisions focused on articulating expectations in the scoring guides.  Instead of general, broad statements, the program wrote specific, performance-based descriptors. 

This particular program should be complimented on intentionally focusing its program revisions to being better aligned with the required standards.  For several years, candidates in this program struggled to pass the licensure exams.  However, the Health education program has carefully aligned student learning goals with required standards; as a result, the rate of passing the licensure tests on the first attempt reflects this focus.

Gifted Children (NAGC/CEC) – Undergraduate/Initial

Status of program regarding SPA report:  Further Development Required

Not resubmitted according to state status of Gifted Children SPA reports

The EPP received communication from the NAGC which indicated that after the Fall 2017 cycle, NAGC will no longer be reviewing gifted education teacher preparation programs as part of the CAEP accreditation process.  To complete the program review before the onsite accreditation visit, the program may choose program review conducted by Indiana or the national review with feedback option, which is offered by CAEP.  Due to low numbers and confusion of the status of NAGC, the EPP did not submit a response to conditions.

Mathematics (NCTM) – Undergraduate/Initial

Status of program regarding SPA report:  Further Development Required

Not resubmitted – missed deadline for resubmission – few candidates in program

According to the consultant hired by the EPP, the NCTM SPA report was in a bad place.  The EPP had to respond by September 15, 2017 (feedback was received in August). Because the report earned a "further development required" decision it could not respond on March 15, 2018.  The math education program will have to wait a full year to submit a new report.  Unfortunately, the program has too few completers to submit a SPA report; it will have to submit a low enrollment program report to the state of Indiana by September 15, 2018.

Physical Education (NASPE) – Undergraduate/Initial

Status of program regarding SPA report:  National Recognition with Conditions

Resubmitted March 2018

Data sources used for the NASPE SPA report include the revision of assessment tools.  Major revisions were made on the rubrics for the key assessments.  For example, Assessments 3, 4, 5, 6, 7, and 8 were aligned with the NASPE standards/elements on the rubrics.  This provides candidates with understanding of connection and purpose of the assessment.  Other changes included revisions to the teaching lesson evaluation (assessment #8) in order to show alignment between the assignment guide sheet and the rubric used to evaluate the assignment. In this case, the description was improved to align with the standards and elements, specifically highlighting advanced lesson plan preparation prior to candidates teaching the lesson to children and being evaluated in their pedagogical performance.  Assessment 7 was better aligned with the NASPE standards and elements in order to provide more breadth and depth.  Again, expectations were clarified by using performance-based descriptors with varying degrees on the rubric.

The Physical Education program is especially proud of its 100% pass rate on the licensure exams.  Typically, completers of this program pass the standardized tests on the first attempt, indicating the program is directly aligned with NASPE standards.

Social Studies (NCSS) – Undergraduate/Initial

Status of program regarding SPA report:  National Recognition with Conditions

Resubmitted March 2018

Unfortunately, between the time the first response to conditions was submitted in March 2017 and the response from the reviewers, the history faculty member responsible for writing the report died in a car accident.  Subsequently, the history (social studies) program had to work in a different manner to understand what this person had envisioned and submitted.  Without their original input, the program learned the importance of collaboration within the specific licensure area and with the EPP at large.  Since then, a team approach has taken shape within the history education program.  Regardless of what happened within the program, the NCSS response to conditions submitted in march 2018 is a much stronger report than previously submitted. 

Data sources from the history education program, like the other programs, involves content licensure exam, Praxis II and Pearson.  During the submission, the program reported 5 completers (4 in 2014-2015, 0 in 2015-2016, and 1 in 2016-2017).  Data tables were updated to reflect subtests and alignment to specific NCSS standards and elements. Additional documents were added to include the ETS Social Studies Content Study Guide and the blueprint used for the Pearson Historical Perspective Assessment.  Due to the small size of the different licensure programs, all secondary candidates enroll in the same courses such as EDUC 342 Literacy in the Content Area.  The EPP is working to revise the key assessment of the unit plan to differentiate for different content areas. 

Special Education (CEC) – Undergraduate/Initial

Status of program regarding SPA report:  National Recognition with Conditions

Resubmitted March 2018

For the data cycle used for the SPA report, the Special Education program reported the following number of program completers:  11 in 2014-2015, 3 in 2015-2016, and 6 in 2016-2017.  Key data points for the CEC SPA report include Praxis II/Pearson content as well as GPA in required courses.  Additional data came from the following rubric based assessments:  unit plan, student teaching evaluation rubric, reader case study, and the SCE:  Impact on Student Learning capstone project.  The program recognizes the difficulty it created by shifting from one evaluation tool to another to evaluate the student teaching experience.  During this data collection cycle, it shifted from the RISE model to program specific content rubrics and the Danielson Framework.  While the shift does not allow for a clear comparison of data, the EPP believed shifting to content specific rubrics and the Danielson provided candidates and the program with more reliable and valid data to use in program decisions.

In the revisions, the program made major changes to rubrics and collected additional evidence for reviewers.  It has been paramount to the program to carefully align the assessments, particularly the content exams, with the CEC standards.  These standards are also aligned to course content providing candidates with a clear view of where content is covered and the connection to licensure.  In regards to the key assessments such as the unit plan, the EPP is currently exploring ways to make these assessments better aligned with specific programs.  All candidates are enrolled in these courses, such as EDUC 340 Literacy Block, the course in which they write the integrated unit plan.  The EPP, though, must differentiate assignments and rubrics to meet the needs of each program such as those outlined by the CEC.

(Q2) Based on analysis of specialty licensure area data, how have individual areas used data for change?

Based on the process used to write the initial SPA reports at the same time changes in the EPP’s accreditation team [new Director of Teacher Education (DTE) and new Field Experience and Assessment Coordinator (FEAC)], faculty in the EPP and licensure programs have identified specific changes which must take place in the next two years:

1. Prior to the new team, assessment of programs has been unintentional, more of an afterthought simply to write a SPA report.As a result, programs search for data and created alignment with standards after the fact.The DTE and FEAC have made a commitment to the programs to work with them to create a more systematic, intentional way of organizing data.Several key elements are in place: (A) the purchase of the CORE software program will allow the EPP to track key data points and report them to the licensure programs; (B) shared drives on the institution’s internal system have been created so programs can upload assessment data, tables, and track candidates’ progress in licensure areas; (C) the EPP is working with the Office of Institutional Effectiveness to add SPA-specific questions to the forms used for departmental reports.Intentional data collection will result and programs will reflect on the data when it submits their annual reports institutionally.Streamlining data collection and reflecting on the collected data at the end of each year will provide the programs a much more intentional use of data for changes.

    

2. The DTE and FEAC will work with programs to establish regular meeting times throughout the academic year to ensure alignment between required courses and program-specific standards. Each of the programs will need to revisit the identified assessments and follow CAEP guidelines for validity and reliability of the assessment tools.

    

3. The EPP intends on revising its teacher preparation program, and key stakeholders in this revision are content faculty directly involved in licensure programs.The EPP would like to increase the focus on clinical field experiences, making them the core of the program.Course work will revolve around the intentional clinical placements.Input by all stakeholders will be critical to the revisions of the program.The EPP will organize a team including content faculty, practitioners, completers, administrators, and other community partners, and this committee will consider the demands of the 21^st century classroom teacher.Using the CAEP and InTASC standards as well as specialize program standards, the team will redesign the program.The Manchester University EPP believes its current program is too traditional.The process of writing SPA reports has revealed how unintentional much of the data collection seems.Candidates must see a deep connection between their program and the realities of the profession.

    

4. Based on data collection from the SPA reports, the programs believe the EPP believes it must redesign its SCE Impact on Student Learning capstone project so better reflects the realities of teaching.Shifting its focus less from the writing of research paper to a work sample will offer candidates a more realistic experience in planning curriculum and assessments as well as using data to drive instruction.The different licensure programs will be engaged in this revision as well.

(Q4) How are SPA reports that are not nationally Recognized being addressed?

As explored in Q1, all Manchester University licensure programs have submitted the appropriate SPA reports for review.  Currently, only the English Language Arts program has earned National Recognition.  The following programs have earned National Recognition with Conditions:  Elementary Education, Health Education, Physical Education, Social Studies education, and Special Education.  The response to conditions were submitted by March 15, 2018, and the EPP will learn of the status in just a few weeks.  The Gifted Education and Mathematics Education initial responses to conditions earned further development needed.  For a variety of reasons, these SPA reports were not resubmitted.  However, the EPP is working with faculty to establish a timeline and appropriate plan of action for completing these SPA reports in the future.