Education Science and Culture of Data Use

I've been working on several initiatives that promise to improve the way education agencies use data to drive decisions and improve student outcomes. At a recent event on the topic with some of the brightest minds in the country the topic of organizational culture came up. The idea was that even with the best information systems, best training, and highest level of data quality, the impact of data to drive decisions and actions depends to a great extent on establishing a “culture of data use”.

Alex Horniman, professor at UVA’s Darden School of Business defines organizational culture as “shared habits of believing”, and I would add “of behaving”. In institutions like education there is a great mass of cultural inertia, a strong resistance to change. This is the same kind of inertia that prevents an individual from following through with a New Year’s resolution, but in this Culture is multiplied by the tens of millions of education stakeholders and years of “shared habits of believing”. The good news is that applying the right forces to a culture, like applying a force to an object the physical world, can get the culture, or object, to move. One of those forces for education can be data, or information to the right people at the right time. And here is a great need and opportunity, to better understand not only how to use information to be able to make better decisions in education, but to understand what it will take for people within the education culture to adopt new habits of believing and behaving based on the data...to adopt changes in practice that improve the profession and student outcomes.

There are some untested assumptions about the link between good information and good decisions in education. In the book “Predictably Irrational” author Dan Ariely describes experiments by which he and his behavioral economics colleagues have proven that people will consistently respond to good information with the bad decisions and irrational behavior, based not necessarily on the quality of the information, but on any number of contextual factors. When the contextual factors were changed the test subjects would predictably make different decisions.

Moving to a culture of data use in education may require the same kind of scientific research about the context of information needed to drive decisions and behaviors that will optimize student learning. This context includes what information is presented when, how the information is presented, how it is presented relative to other information, what presentation characteristics result in positive behaviors and what information/presentation/context results in negative behaviors, etc. There are a complex set of factors apart from the data itself that impact a person's willingness to trust and take the best actions based on the data. It is easy for a person with the very best intentions and good data to make less than optimal decisions. I don't claim to be a behavioral scientist, but it seems to me that the kind of experiments needed to better understand the link between good information and good decisions in education can be done in relatively short cycles compared to other kinds of research.

Too Busy to Blog

During the past few months I've had the privilege of being busy with work on the Gates Foundation funded Teacher-Student Data Link Project, New Hampshire's State Longitudinal Data System, Common Data Standards Initiative Adoption and Implementation Task Force, strategic consulting for Minneapolis Public Schools, and promoting cross-state sharing of SLDS resources. With head-down focus on this work, and having released the latest edition of Approaching 100 Percent in September I've had little time to sleep, never mind having time to post to this blog.

My schedule doesn't seem to be letting up any time soon, but I'm resolved to post something at-least monthly to capture what I've learned as I continue in this work that I hope is making a positive impact on student learning in the U.S.

In the past couple of months I've also read a few books including The Checklist Manifesto by Atul Gawande and Who - The A Method for Hiring by Geoff Smart and Randy Street. (I recommend both of these books.) One common theme worth calling out from these books is that "getting things right" doesn't happen by accident or even by skill. Getting things right consistently requires a well defined but simplified process, not a simplistic process, but a process that focuses effort and attention on what is most essential. And in each case a key to successful implementation of a quality improvement process is measurement. Using a scorecard, a checklist, or some other instrument of measurement, is critical to knowing whether or not the process is being implemented with fidelity.

This theme carries over into the work that I've had the privilege to contribute, equipping education agencies with the tools, processes, and information to "get things right". ...Now if only I can define a process and measurement system to ensure that I consistently post to this blog.

Now Available "Approaching 100%" 2010-2011 Edition

My new book is available here: https://www.createspace.com/3464615

Innovators Continued

(I am long overdue in completing my thoughts from the July data conference. I've had some worth while distractions.)

...One very encouraging session at the DC-STATS conference, which I had the priviledge to help facilitate, brought together State Education Agency participants around sharing of best practices and intelectual property assests developed as part of each state's SLDS work. There are some great public domain resources out there. It is encouraging to see attitudes of "we are all in this together", "we don't need to reinvent the wheel", and "I want my work to have value beyond my state". This attitude has not always been prevalent. Changes in the environment such as addoption of Common-Core accademic standards have helped and will hep to align the needs of data systems accross states. We are now working with key leaders of states that have something to share to establish a formal organization and governance structure for cross-state sharing.

Up Close with Education Secretary Duncan and Other Innovators

Last week was an interesting week for this education reformer. I was at DC-STATS Data Conference hosted by IES, the research arm of the U.S. Department of Education. I had a front row seat for the opening key note by U.S. Secretary of Education Arne Duncan and IES Director John Easton. One positive element of this administration that was evident in the address and follow-up questions is an understanding that innovation in education cannot be a top-down mandate. Bueracracy is not the best incubator of innovation. However, Federal and state agencies can improve its support of local and regional innovation...

It is being done.

A school in Oregon is implementing proficiency-based student-centric learning where time becomes the variable and learning becomes the constant. We've known for a long time that this is the right thing to do. It's a shame, the main thing stopping this from happening everywhere is "cultural inertia", and in some cases a sense of security for some adults taking precedent over the best interest of future generations of learners.

Here is the article:

http://www.statesmanjournal.com/print/article/20100208/NEWS/2080332/Students-in-charge-of-own-education

We Don't Need Anesthesia, It's Painful This Way and That's How We Like It!

If you needed surgery to save your life and you had a choice to have the surgery with an anesthetic or without, which would you choose? Which option do you think your doctor would recommend?

Today a surgeon wouldn’t think of performing painful surgery without anesthesia, but there was a time when surgeons couldn’t imagine surgery without pain...

Robert Krulwich interviewed Richard Holmes on National Public Radio's Morning Edition yesterday about his book Age of Wonder. Holmes recounted that in 1799, Sir Humphrey Davy, then a 21 year old scientist and apprentice to a surgeon discovered that nitrous-oxide (laughing gas) could stop pain.
In his book, Holmes describes Humphrey Davy’s discovery and his suggestion in an article to the medical community that nitrous-oxide could helpful for patients undergoing surgery. Unfortunately, it took 40 or more years for surgeons to see past preconceived notions about pain and accept that pain was not necessarily a good thing. They just couldn't imagine surgery without pain.

Here is an excerpt from the NPR interview:

"The idea that you could have pain-free surgery was completely radical, novel," says Richard Holmes. Doctors at the time had a different view: that pain was a good thing! The doctors thought pain helped them polish their surgical skills (screaming being a prompt to cut fast and accurately), and it was believed that pain would help the patient rebound. "It was proof that the body was fighting back and healing itself," Holmes says.

The education profession can be like the medical profession of the early 19th century. We know that the way things have always been done in schools is not working for many students, but we "can't image" it working any other way. We have new discoveries from brain science research, new methodologies and innovations that could stop the "suffering" and meet individual student needs. We have a better way, but lack the collective vision and will to put such a radical change into practice. It is a crisis of belief, what I like to call "cultural inertia", that prevents individuals and groups from seeing the answer that is right before them.

Maggie Dugan and Tim Dunne at InstantBrainStorm said it well in this video clip about the "aha moment". (They graciously produced this clip for me to use in a presentation to the National School Boards Association annual conference back in 2009.):


I hope it doesn't take 40+ years for the education profession to have an aha moment, to see that learning can be less "painful", to imagine a system where all failure to learn is not the norm.

How can students reading significantly below grade level catch up?

Students reading below grade level are not only behind in reading, it is also harder for them to keep up in other classes when text books and other content are written at grade level. Part of the solution is providing reading materials across the curriculum tailored for the student’s reading level. Richard Allington, one of the country's most recognized experts on early literacy, addresses this in a recent EdWeek article (http://www.edweek.org/tsb/articles/2010/04/12/02allington.h03.html?cmp=clp-edweek):

“…reading instruction has to take place all day long. In other words, if he’s reading at a 2nd grade level in 4th grade, this child would need texts in social studies, science, and math that are written at the 2nd grade level but cover the 4th grade curriculum, so he has a book in his hands all day long that he can actually read.”

Of course the reading instruction itself must also be adaptive and effective. Students reading below grade level must have more time reading with the right materials, and the reading instruction must be high quality and adaptive to individual needs and learning style. Allington asserts that the teachers working with these students must be trained to use proven instructional practices, that far too many teacher don’t know how to teach kids to read. This includes teachers providing pull-out instruction in small groups or one-on-one and teachers in the typical classroom setting.

Allington is critical of packaged reading programs. “Well, the problem is that the concept of a packaged reading program doesn’t have any scientific validity to start with, because we know that if you take 100 kids or even 10 kids, there are no prescribed programs that will work with all of them,” he said. I think that might be true for one-size-fits-all classroom-centric packaged programs. However, I would argue that computer-adaptive "packaged reading programs" can work with most students if they are created by experts in reading instruction and apply the same diagnostic and prescriptive logic that would be applied by a reading specialist when working on-on-one with a student.

I don't claim to have expertise in early reading instruction, but I would argue that the factors that make good computer-based reading instruction effective when compared to static reading programs, are some of the same factors that set apart good reading teachers from those who don't know how to teach reading. The teacher (or program-logic) must continually assess how well and how the student is reading (using a variety of indicators). More importantly the teacher (or program logic) must recognize WHY the student is struggling with a particular aspect of reading, and know the appropriate response(s) to overcome the block. The response must fit the need, be it strategies for coping with dyslexia, more time reading engaging content matched to the student's interest and ability, or activities to strengthen phonemic awareness.

It is the diagnostic and prescriptive ability of the teacher, or similar logic within the "packaged reading program" that makes it work for the individual learner. Research is now being done on biometric feedback that can give the interactive program some of the cues that a good teacher would use when observing a struggling reader (e.g. vocal inflection when reading out loud, eye position when reading on screen or device, posture). While traditional outcome-based assessments can diagnose competency, these in-process cues can help diagnose process problems, and point to appropriate remediation.

Personal one-on-one instruction for all struggling readers with highly proficient reading specialists would be the ideal, but would be cost prohibitive. An effective alternative for many students could be an interactive one-on-one program, designed with the knowledge of top experts on early literacy and with the diagnostic and prescriptive logic of proven-practice reading instruction. I predict that such programs will increasingly be accepted and proven scientifically valid as our education system shifts from a teacher-centric model to a student-centric model.

Presenting the Public Domain Education Data Warehouse at MIS Conference

Last week I presented with colleagues from New Hampshire Department of Education at the NCES Winter Forum and 23rd Annual Management Information Systems (MIS) Conference in Phoenix. The presentation introduced a "Public Domain Education Data Warehouse—Including Student-Teacher Connections That Inform Instructional Change".

At a time when national education data standards are a priority, NH has taken a leadership role in developing a public domain data warehouse model, that is a practical implementation for a data warehouse based on national standards. It has been a privilege to help shape that public domain architecture and work with selfless group of people working hard to make a difference for students in New Hampshire, and in other places that can benefit from the public domain model.

Out of School Time Matters

The time that students spend learning outside of the traditional classroom/school is increasingly important. Student's not meeting minimum learning objectives inside the box of the fixed-time/fixed-space classroom can be successful when in-school time is supported by the right learning experiences and support during out-of-school time. And for students succeeding inside the classroom, the out-of-school time help them reach their potential as contributing citizens.

A recent article, Not just and add on, in CatalystChicago makes the point that "Forty years of research has proven that after-school and summer programs for low-income students can help to close the achievement gap."

A good resource on out-of-school-time is the Harvard Family Research Project.

It Can't Be Done!

Yesterday, NASA Administrator Charles Bolden, testifying before a congressional committee, said that NASA's ultimate goal is "eventually Mars", but it could not be done in 10 years or more "because there are some things we just don't know."

We would have never landed a man on the moon if NASA leadership in the 1960s took that position. We didn't know in 1961 how to land a man on the moon and return him safley to the earth. And there are some things we don't know when it comes to the goal of quality education for all. (That is, an education system from which 100% of PK-12 students are equiped for college or the workforce, for life-long learning as a foundation for a productive/fulfilling life.)

When President Kennedy set the "man on the moon" goal everyone in the United States understood why we wanted to do that. The "how" was not 100% clear, but the "what" and "why" was concrete.

It occured to me that our education system operates like the NASA of today. We don't have a common understanding of "what" and "why". We need a concrete vision, understood by all, that fosters a willingness to do whatever it takes. An understanding that part of the pursuit is a pursuit of knowledge, to overcome the "things we just don't know".

Please let me know your thoughts on what should be our "man on the moon" goal for education by the end of this decade.