Education Science and Culture of Data Use

I've been working on several initiatives that promise to improve the way education agencies use data to drive decisions and improve student outcomes. At a recent event on the topic with some of the brightest minds in the country the topic of organizational culture came up. The idea was that even with the best information systems, best training, and highest level of data quality, the impact of data to drive decisions and actions depends to a great extent on establishing a “culture of data use”.

Alex Horniman, professor at UVA’s Darden School of Business defines organizational culture as “shared habits of believing”, and I would add “of behaving”. In institutions like education there is a great mass of cultural inertia, a strong resistance to change. This is the same kind of inertia that prevents an individual from following through with a New Year’s resolution, but in this Culture is multiplied by the tens of millions of education stakeholders and years of “shared habits of believing”. The good news is that applying the right forces to a culture, like applying a force to an object the physical world, can get the culture, or object, to move. One of those forces for education can be data, or information to the right people at the right time. And here is a great need and opportunity, to better understand not only how to use information to be able to make better decisions in education, but to understand what it will take for people within the education culture to adopt new habits of believing and behaving based on the data...to adopt changes in practice that improve the profession and student outcomes.

There are some untested assumptions about the link between good information and good decisions in education. In the book “Predictably Irrational” author Dan Ariely describes experiments by which he and his behavioral economics colleagues have proven that people will consistently respond to good information with the bad decisions and irrational behavior, based not necessarily on the quality of the information, but on any number of contextual factors. When the contextual factors were changed the test subjects would predictably make different decisions.

Moving to a culture of data use in education may require the same kind of scientific research about the context of information needed to drive decisions and behaviors that will optimize student learning. This context includes what information is presented when, how the information is presented, how it is presented relative to other information, what presentation characteristics result in positive behaviors and what information/presentation/context results in negative behaviors, etc. There are a complex set of factors apart from the data itself that impact a person's willingness to trust and take the best actions based on the data. It is easy for a person with the very best intentions and good data to make less than optimal decisions. I don't claim to be a behavioral scientist, but it seems to me that the kind of experiments needed to better understand the link between good information and good decisions in education can be done in relatively short cycles compared to other kinds of research.

Too Busy to Blog

During the past few months I've had the privilege of being busy with work on the Gates Foundation funded Teacher-Student Data Link Project, New Hampshire's State Longitudinal Data System, Common Data Standards Initiative Adoption and Implementation Task Force, strategic consulting for Minneapolis Public Schools, and promoting cross-state sharing of SLDS resources. With head-down focus on this work, and having released the latest edition of Approaching 100 Percent in September I've had little time to sleep, never mind having time to post to this blog.

My schedule doesn't seem to be letting up any time soon, but I'm resolved to post something at-least monthly to capture what I've learned as I continue in this work that I hope is making a positive impact on student learning in the U.S.

In the past couple of months I've also read a few books including The Checklist Manifesto by Atul Gawande and Who - The A Method for Hiring by Geoff Smart and Randy Street. (I recommend both of these books.) One common theme worth calling out from these books is that "getting things right" doesn't happen by accident or even by skill. Getting things right consistently requires a well defined but simplified process, not a simplistic process, but a process that focuses effort and attention on what is most essential. And in each case a key to successful implementation of a quality improvement process is measurement. Using a scorecard, a checklist, or some other instrument of measurement, is critical to knowing whether or not the process is being implemented with fidelity.

This theme carries over into the work that I've had the privilege to contribute, equipping education agencies with the tools, processes, and information to "get things right". ...Now if only I can define a process and measurement system to ensure that I consistently post to this blog.