Editor’s note: In 2015, Education Pioneers assessed a report by Mathematica Policy Research that discussed the challenges of data use in education, how to address those challenges to harness the power of data to better serve students, and the impact of our Pioneers. Today, education organizations continue to make progress in using data, and challenges still persist. Our “Data Imperative” findings are relevant today, and we’ve updated and streamlined them here.
Data isn’t the new kid in school.
Many education organizations embrace data and have made progress in using it to inform decisions.
At Education Pioneers, we see strong demand for data talent every year, and our Pioneers have lead or contributed to data work with many of our Partners across the sector and the country, including Denver Public Schools, Tennessee’s Achievement School District, and NewSchools Venture Fund.
But challenges persist. If “working with data” sounds simple, in practice, it’s not. For data to be meaningful:
-
The right data must first be collected, both in quality and quantity;
-
The data must be analyzed correctly;
-
It must be applied correctly;
-
Resulting analyses and findings must be accessible to all stakeholders who need it do their jobs better; and
-
Stakeholders need to know what to do with the results for it to make a difference.
There are a lot of steps, and each involves a lot of complex work. No wonder that most organizations – in education and beyond – face similar challenges working with data.
In a report, Mathematica Policy Research (MPR) looked inside 12 education organizations to assess their data capacity and use, challenges encountered, and lessons learned.
(Disclosure: MPR specifically assessed the impact of Education Pioneers Fellows who had been placed with those organizations. The report was funded by the Bill & Melinda Gates Foundation to study their investments in Education Pioneers’ programs to place data leaders in education organizations.)
The 12 organizations vary in type – state agencies, school districts, and one charter management organization – and face numerous challenges around gathering, storing, and using data. But themes emerge across organizations:
-
Data quality. Four organizations reported challenges with data consistency and reliability — including few procedures to ensure data entry by staff is accurate and reliable — poor quality data in general, and lack of reliable data sources.
-
Data infrastructure. Six organizations reported challenges with data systems that were outdated, cumbersome, or not integrated. They experienced difficulty analyzing data from different departments that used separate systems or were slowed down by cumbersome systems that resulted in a small number of experts handling all requests.
-
Staff capacity. Eight organizations reported a lack of staff capacity to work with data, due to varying levels of skills to make data-driven decisions, including knowledge of or training for interpreting data, a lack of enthusiasm about using data, and discomfort with data use.
In response to these obstacles, MPR’s report also suggests two best practices from a majority of the report’s agencies. Fortunately, these practices are also immediately actionable for many organizations:
1 | Create a data culture.
“The first question a data-driven organization asks itself is not ‘What do we think?’ but ‘What do we know?’ This requires a move away from acting solely on hunches and instinct.” – Andrew McAfee and Erik Brynjolfsson, Harvard Business Review
To create a data culture, start at the top. Leaders must embrace data use at all levels. Talk about it in meetings, train staff to become more comfortable with using current and new systems, and approach the cultural change as purposefully as you would any other.
Of the organizations who reported the importance of a data culture in MPR’s report, some newer agencies were able to establish data use from the beginning, while other established organizations took steps to restructure and hire data-savvy staff to make data a priority.
As a result, data became critical to inform many types of work, from instruction, to grants, news reports, school openings and closings, and more.
2 | Invest in data-savvy human capital.
“Big data’s power does not erase the need for vision or human insight.” – Andrew McAfee and Erik Brynjolfsson, Harvard Business Review
Data is only powerful when organizations have people who give it power — people who can ask the right questions to gather the right information, who have the skills to analyze the data they collect, and who then can implement their findings in meaningful ways.
Investing in data-savvy human capital means both developing current staff and bringing in new people who have critical data skills that your organization is lacking.
In MPR’s report, organizations reported training staff on how to use the data warehouse, tapping the expertise of their Fellows to informally teach staff how to create Excel graphs or tables, and also hiring data experts permanently.
Opportunities abound in education to work with data to better serve students. From student attendance — a critical element for at-risk students to succeed in high school and graduate — to instructional data and “summer melt,” data can help us improve outcomes for students growing up in underserved communities.
Education needs strong leaders who set bold visions for their organizations, create thriving organizational cultures that value data to reach ambitious goals, and hire talented people who can interpret significant volumes of data and make meaning of them.
Information is power. Let’s harness it.
April 17, 2017
