Hebrew Day School (HDS) Progress Reports
BACKGROUND
BACKGROUND Information
The Hebrew Day School of Ann Arbor (HDS) is a small, private institution and is among fifteen other Jewish private schools in the State of Michigan serving a growing population of 83,115 people (during 2017) who identify as Jewish since 2016 according to the American-Israeli Cooperative Enterprise (AICE). The mission of HDS is to educate youth from kindergarten through the 5th grade in the core subjects of math, English language, science, and social studies, while also providing them with Judaic Studies, with a strong focus on Hebrew language immersion.
PROJECT DESCRIPTION
As part of the SI 501: Contextual Inquiry & Consulting course at the University of Michigan, our graduate student team of four (aka Design for Change) conducted a contextual inquiry for HDS’s student progress report procedure. The project duration was from September 2017 - December 2017
We completed, background research reports, 1:1 interviews, affinity walls, surveys, and data analysis to meet the following objectives:
understand how teachers, parents, and teachers used the current student progress reporting system
inform recommendations for optimizing the student progress reporting system
This case study project includes:
An explanation of the methodology we used in understanding the progress report problem
A detailed account of our findings
Recommended solutions to those findings
problem summary
Hebrew Day School seeks to evaluate and improve its system of communication between school and home as it pertains to the reporting of student progress. The current system needs evaluation with attention toward aligning the goals of the school with the needs of the teachers and the expectations of the parents. The process needs to be improved and streamlined so that it is effective and efficient for all.
Time Consumption: The problem that HDS has identified is the amount of time and effort it takes to produce a single progress report without necessarily knowing the benefits of these efforts.
The following diagram shows five stages that were defined based on the information from interviews with teachers at HDS, as well as, the initial survey data, that shows the average time spent for each stage.
Lack of Inconsistency and Standardized Format: Several inconsistencies lie in teachers’ workflow of producing a progress report. This raises extended problems and renders an ineffective procedure. The lack of a standardized format for the narrative part of the progress report results in teachers worry about the flow of writing, word choices, and the structure of the progress report
The images above are images of a sample progress report pages.
Apprehension to New Technology: Some of the teachers also expressed technological difficulties in documenting and formatting the reports. Moreover, when the school administration considered adapting a new format or platform (e.g. digital platform) in the past, they had concerns about the budget and parents’ and teachers’ literacy in technology.
RESEARCH GOALS
research Goals
Upon investigating the overall process of the progress report procedure, we came up with the following goals to create a set of recommendations based on an analysis of the interview
findings:
To make the progress report less time consuming and reduce the burden of teachers
To make the collaboration between teachers more efficient and effective
To provide recommendations of possible online software for producing progress reports
To make the school-parent communication more informative in terms of all the different channels of communication.
LIMITATIONS
I experienced a family emergency at the beginning of the progress that detracted some time away from the project. As a result:
I was not able to attend the attend the initial client meeting with the main HDS stakeholder, however I listened to a recording of the initial meeting to get caught up
My contribution to the initial surveys sent out to the parents and teachers was minimal, however I collectively analyzed the survey responses/data with my team members.
My contribution to the interview script and protocol was minimal at the beginning.
While we visited the school, we could not perform ethnographic research in which we follow and observe the teachers while they go through the full process of creating the progress reports.
METHOD/PROCESS
bACKGROUND RESEARCH
Each member of the team wrote background research reports to gain familiarity with related topics to our project. This enabled us to have nuanced conversations about the relationship between student achievement, parents and teacher engagement in early childhood education, and student progress reporting systems.
The purpose of the background research reports was to:
To research HDS problem domain
To gain knowledge that would inform interviews and observations
To enrich our data analysis, findings, and solutions
The researched topics included:
Team Member 1 (Me): Alternative Methods for Tracking Student Achievement
Team Member 2: General Background and Insight into Progress Report Processes Procedures
Team Member 3: Using Technology to Support Communication Between Schools and Families
Team Member 4: Communication Between Schools and Families
Surveys
The Design for Change team sent out two separate surveys for teachers and parents at HDS via a free online software called Typeform. The purpose of the online surveys was to help us formulate better questions for the interviews with HDS community members.
Both surveys consisted of about ten questions each that were required in order to submit a response. In total, we received 26 responses coming from 16 parents and 10 teachers.
INTERVIEWS
The team conducted qualitative interviews of eight members of the HDS community, which included teachers, administrators, and parents. Some interviews were conducted in-person at the HDS and others were conducted remotely. The purpose of the interviews was to collect more in-depth information that would help us define the specific source of the problem.
The team (excluding due to a family emergency) created an interview protocol that included
information on the purpose of the interview
the interview procedure
confidentiality agreement
a set of interview questions
future steps in how the interviewee’s answers would be utilized
A primary interview and a notetaker were both present for each interview. Overall, we interviewed a total of 8 people, which included the initial client meeting (an audio recording from the main stakeholder discussing HDS and background information about the progress report system). After having returned from addressing my family emergency I interviewed two participants and served as a notetaker for another interview session.
This image is a breakdown of which interviewees were teachers, administrators, or parents.
AFFINITY MAPPING
Affinity Mapping
Upon completing all of the interviews, we divided the interviews amongst ourselves to create affinity notes. We did affinity mapping because as a team we needed to organize qualitative data points from our interviews in a way that could enable us to pull out overarching themes from our research.
Image of the Design for Change team (me in the green sweater one person from the left) in front of our affinity map after having presented our findings.
Interpretation sessions
Affinity notes consisted of statements that describe each minute of the audio-recorded interviews. For each interview, our team held interpretation sessions to interpret the qualitative data. The team collectively discussed the recordings before moving into the affinity mapping exercise.
1st Round: By using RealTime Board (currently known as Miro) we created an affinity wall bay clustering all of the affinity notes based on themes and wrote a summative statement above each cluster.
2nd Round: After the initial cluster, we aggregated the second set of statements on the theme again and created another statement for those combinations.
3rd Round: We clustered the second set of notes and repeated the processing, merging the third set of statements one more time.
There was a total of 257 affinity notes, with 57 statements produced during the first round of clustering, 17 statements during the second round, and five overarching statements during the last round.
FINDINGS AND SOLUTIONS
About the Findings
We identified two major categories in which to group our findings. Category 1 is the progress report creation process. This set of findings discusses teacher collaboration, data collection, and communication between the teachers and the administration.
Category 2 is the assessment of progress reports. It discusses the parents’ view on the progress reports, the teachers’ view on the progress reports, the teachers’ view on the parents’ readings of the progress reports, and teachers’ skepticism about change.
About the Solutions
There are two sets of proposed solutions, each corresponding to both sets of findings. For the solutions for category I findings the solutions are presented as short-term solutions (i.e., they can be implemented within the following year or two) and long-term solutions (i.e., they should be considered by HDS for the future).
The purpose of these solutions is to ameliorate the efficiency of an aspect of the process to thus decrease the time it takes teachers to assemble progress reports. These proposed solutions are also meant to spark conversations about feasible and specific changes that HDS can implement to the progress report process overall.
FINDINGS I
Category 1: The progress report creation process
There are multiple ways teachers communicate with each other about observations and assessments of students’ academic and social-emotional behaviors. This lack of standardized communication procedures creates a confusing environment in which teachers must parse their way in order to write the progress reports.
There is no standardized way to write the progress reports, causing teachers to figure out other ways to save time.
The back-and-forth nature of the administration’s further evaluation adds another step in the progress report writing process, which is already very time-consuming.
The following is a diagram of the life cycle of a progress report.
SOLUTIONS I
Short-term:
Have a university student intern handle the formatting and data aggregation of the progress reports.
Training teachers in how to develop simple infographics (e.g., infographics, bar graphs, line graphs) to add to the progress reports in place of heavy text would decrease the amount of time that is spent on writing the narrative.
Keep a detailed narrative for students with learning disabilities or special needs but not for students who do not require this type of reporting is the last short-term solution; serving families of students with special needs or learning disabilities.
Create a standardized survey (i.e., Google forms) for all teachers to input observations and assessments that are collected throughout the semester, this will eliminate the need to worry about transition sentences or flow. This is easy to adopt because the teachers already use Google Drive to document observations. (See embedded video below)
The video shows an example of how HDS administration and teachers could adopt Google Forms to both create standardized progress reports and view simple analytic data generated from filling them out.
Long-term:
Investing in software that auto-generates the structure of progress reports is a long-term solution that we recommend HDS seriously consider for their future. Software such as Panorama, Engrade (free), and ThinkWave would enable this transition.
FINDINGS II
Category II: Assessment of progress reports
Parents are only looking for challenges and trends in the progress reports.
There is no standardized way of receiving feedback from parents about the progress reports.
The teachers’ curriculums do not reflect the content of the progress reports; particularly the checklist portion of the reports.
The teachers are disillusioned and do not expect there to be any changes made on the progress report procedure.
SOLUTIONS II
Implement a standardized procedure to determine the composition of the progress reports and to evaluate the performance of the progress reports on a regular basis. Within a year of considering this HDS could complete the following steps to implement this solution:
Establish the goals of the progress reports and discuss and align the goals with the entire faculty.
Once the goals are determined and aligned, keep the goals as the principle to follow when designing the content and structure of the progress reports; the design of the progress reports should follow the goals of it.
Take into consideration parents’ preferences and the design of the curriculum. Sending out a survey to parents is a low cost to know what parents want to see in the progress reports and quantitative data facilitates making decisions.
After the progress reports are finished by teachers and delivered to parents, teachers can spend time to discuss with parents on the format and structure of the progress reports to continue understand parents’ thoughts and needs.
IMPACT AND PERSONAL LEARNINGS
In terms of impact, our main stakeholder/point of contact for HDS wrote to us the following quote below:
“Now the thank you: Thank you very much for your help with the progress report project at Hebrew Day School. We appreciate all of the time and effort you put in to this project. You provided a variety of useful pieces of data, useful suggestions and helped framed this conversation for us. We are trying a variety of new strategies as a result.”
Some of the key personal lessons I gained from this project were:
One cannot assume that multiple stakeholders who are part of the same entity experience the same challenges or hold the same beliefs. Some times stakeholders, even within the same entity, are not always transparent with each other
When working on a small team of researchers, contingency plans are important in the event of an unexpected even that pulls someone away. This plan should be set up at the beginning to avoid any halt in research advancement.
While not all teams will have sufficient time to conduct background research, it’s an extremely important part of initial research. I found that I could ask much more nuance to my questions and observations during the affinity mapping exercise after having researched my topic and having read my other team members’ background research.