Wednesday, February 10, 2016

How Data Works to Support DIY Learning

The following is post I guest-authored for, and originally appeared on Getting Smart...

Noah is a 17-year-old multi-lingual student. He can speak six languages even though his family speaks only English and his public high school offers classes in only two non-English languages. Noah didn’t have formal opportunities to pursue his linguistic interests, so he took matters into his own hands, discovering online tools and social networks for self-directed learning.

Noah’s story provides some good examples of the kinds of data and technology enabling do-it-yourself learning.

Generation Do-It-Yourself (GenDIY) has unprecedented opportunities to chart their own course for lifelong learning as part of a career pathway, to reach a personal academic goal, or just to satisfy a curiosity.

The data used to match learning experiences with personal needs, preferences, and ability levels, and data within online learning applications to provide continuous feedback, are empowering learners like Noah to move beyond the constraints of traditional education.
Do-it-yourself learning is taking place on two levels:
  1. Formal systems of education are adopting student-centered options, giving students voice and choice, and visibility into how short-term choices support longer term career goals, and
  2. Learners of all ages are acting on their own, discovering and using technology enabled tools to reach their own learning goals.
Prior to high school Noah took an online course in Latin. He worked through a book and viewed videos at his own pace. At the time Noah was home schooled, but schools across the U.S. and around the world are also leveraging a rich set of online options to offer courses that they cannot staff. Course choice opens doors for students, especially in communities that cannot attract teachers with specialized subject matter expertise, or cannot fill a class with enough students to justify the course.

After he discovered his interest in language learning, a friend told Noah about a free language-learning tool that he happened to read about in a technology blog. That tool was Duolingo, the award winning free website and app. 

Data for Discovery
Noah was fortunate to have a friend point him toward Duolingo, but data is also helping the GenDIY self-discover the right DIY learning tools and opportunities.
Linked data on the Web supports discovery of learning resources (courses, apps, learning experiences, and social learning opportunities). Metadata (data about data) is being used by the major search engines to better filter search results to meet learner needs and preferences. Publishers of learning resources tag web pages with metadata attributes, such as specific competencies addressed and intended audience, in a format that the search engines can read. Metadata may include tags about accessibility of the resource, such as if a video is closed captioned for the hearing impaired. This helps the self-directed learner find resources to fit personal needs and preferences. is a standard for tagging web content developed through collaboration of the major search engines such as Google, Yahoo, Bing, and Yandex.

Paradata” gives DIY learners indicators of learning resource usefulness, for example, many Facebook “likes” for a language learners group increases the visibility of the group and becomes a paradata assertion about its usefulness. Likewise social media posts with links to a page describing a learning resource say something about its popularity, or a formal endorsement of the resource by an organization (such as a state education agency) may be captured in a public repository, such as the Learning Registry.

Gamification and Intelligent Tutoring Data
With the help of Duolingo Noah learned Spanish, Portuguese, French, and Irish well enough to engage in conversations, and a bit of 11 other languages.  Apart from Duolingo, he is also learning Haitian/Creole using other web resources and with a friend at school who speaks the language.
Factors that make Duolingo an effective tool include its bite-sized assessment-as-learning lessons and continuous game-like feedback. This is competency-based tutoring at its best. Learners advance only after demonstrating mastery on granularly defined competencies, such as translating a specific word or phrase. Feedback is instantaneous and focused on correcting specific weaknesses. I see a lot of similarities between principles within gamification and learning sciences, both draw from an expanding knowledge of how the human brain develops and adapts to new challenges.  Game mechanics address learner motivation, providing the right level of challenge at the right time (zone of proximal development), building new knowledge/skills on existing knowledge/skills (constructivism), goal setting and visibility into thinking and progress (learner agency).
To deliver this kind of experience for the learner requires a rich set of data behind each assessment item (the granular competency being assessed, what a correct or incorrect answer means and what remedial feedback to give, etc.), detailed data collected every time the learner attempts to answer to guide feedback and progress, and data about the competencies and competency-based pathway.

“Big Data” and a Warning about Learning Styles Data
The theory of learning styles has been intensely reviewed, tested and debunked,” but well meaning organizations still offer learning style assessments and attempt to use the data to personalize learning.

Yes, big data sets can be used by recommendation engines to help filter all possible learning activities down to a few that are a good fit, just like Google targets advertising and Amazon suggests products “you also might like.” However, the notion that a person is a fixed type of learner that can be classified using a one-time assessment is oversimplified. Preferences change over time, the “best” instructional/study methods will vary based on context, and students may need to try multiple modes of instruction (see a concept in different ways) before mastering some learning objectives. It may be helpful for a learner to think about what kind of learning mode they generally prefer, but multiple options for each lesson allow the learner to choose how they right now. Even Google search results give a list of options and let the user pick…I don’t know anyone that regularly uses the “I’m feeling lucky” option.

The mode of presentation (visual, auditory, kinetic, etc.) is just one of many variables factor into selecting a learning activity. Being precise about the granular competency that the learning activity addresses, and the quality of the resource, is more important than the mode of presentation.
Analytics engines, informed by big data, can do more than predict how well a learning activity will work for a student.  They can help create conditions for motivation and engagement to help the learner reach personal goals.

Social Learning
Noah learns with friends on social media including Google hangouts and Facebook language learners groups. He also seeks out native speakers of the languages he is learning. When visiting the city where a relative lives, he made it a point to walk into a Portuguese bakery and start a conversation with the people working there.

Through school choice, he is attending a high school outside of his home district and enrolled in a French class just to get required credit for graduation, but he doesn’t think he’s learning anything there that he has or could learn on his own initiative. And his friends on social media are more at his level for conversations in French. So next semester his high school teacher will create a special “French 5” independent study option in which Noah will help teach French to freshmen.
Peer assessment can be an effective part of DIY learning. For some subjects data may be collected with online rubric-based peer assessment tools. Assessment-for-learning data is informs feedback.

Data for Feedback
There are three levels of feedback to support student-centered learning:
  1. Immediate feedback given during the learning activity after each click/response,
  2. Feedback at the end of a lesson that answers the question “What next?”
  3. Dashboards and progress maps that answer the question “How am I doing in reaching short and long-term goals?”
The 3rd kind of feedback allows learners to carry out personal learning plans as a kind of GPS guiding them to longer-term goals.

Data for Planning and Decision-Making
DIY learners are motivated by a purpose. Noah‘s fascination with linguistics motivated him to take ownership of his own learning. That interest is leading to decisions about college and career.  Often the purpose for learning is to gain abilities needed to support a cause, calling, or career goal.  Noah sees himself pursuing a career as a translator, but realizes that his interests and goals may change in the future.

Emerging sources of data will help DIY learners map backwards to identify credentials needed to support cause or career, and the competencies required to attain each credential. There is a trend in higher education and workforce training to offer stackable credentials such as a certificate that counts toward a degree. Projects such as the Credential Registry plan to provide data to help DIY learners make informed decisions about long-term learning goals and alternative pathways to reaching those goals.

The DIY learner then can track progress toward goals with the right data about achievements. Most of the time progress data is not in control of the learner and constrained to a specific context, such as language learning data within Duolingo, mathematics data in Khan Academy, course transcript data in a high school or college information system. However, several initiatives are working to give students control of their data. Initiatives like the Badge Alliance have published standards for the data representing achievements, and other organizations are building on previous work toward student-centered, secure, verifiable claims and credentials.
Data about pathways, plans, and progress can be combined and presented in a dashboard for the DIY learner. This is already available within silos, but someday learners will be able to get a more complete picture.

Finally, the same kind of “paradata” used to rate quality and fit of individual learning resources can also be used to inform bigger decisions, such as quality, fit, and cost-effectiveness of college programs. 

Now, Noah is considering a college that has a large language department with a good reputation, but that doesn’t tell him if the program is better than other options at preparing people to do what he wants to do after college. It also doesn’t tell him if the program is the most cost effective way of reaching his long-term goals. Some of this information can be discovered/collected from unstructured data, e.g. within social media and surveys. Other data might be generated through “big data” analytics. (Existing “college recommendation engines” tend to be more about evaluating the student’s chances of being accepted, rather than evaluating the value that a college program offers its graduates.)

A Vocabulary for Talking about GenDIY Education Data
The Common Education Data Standards (CEDS) defines the meaning of data elements used to support DIY learning including data for discovery of learning resources/opportunities, data used in assessment-as-learning and intelligent tutoring systems, data for planning and decision-making (including competency and credentials definitions, and achievement tracking). includes a searchable glossary of data “vocabulary” that is aligned to many of the other standards mentioned in this article. Other standards address the protocols and technical details for interoperability of systems and content for each of the kinds of data.

Tuesday, January 12, 2016

Grade Level What?

This post originally appeared on Getting Smart.  The theme came up in my discussions with Smart Parents author Tom Vander Ark and Getting Smart Managing Editor at the iNACOL Symposium. The factory model of schooling is giving way to more flexible options for personalized lifelong learning pathways. This is one close-to-home story about that transition...

At a recent holiday gathering I witnessed an interesting exchange between my son Benjamin and a relative who hadn’t seen him in a while. My relative asked, “What grade are you in now?” There was a long pause… I smiled. I could see the wheels turning as he thought about how to respond. What would have been an easy answer for me when I was Benjamin’s age is more complicated now. He could have given several different correct answers.

My son is an example of how “grade level” is becoming an outdated concept. In his public virtual high school he was enrolled this year as a sophomore, but his classes include a dual-enrollment writing class at a local college, and high schools classes usually taken by “9th” and “11th” graders. He also self-enrolled in self-paced guitar lessons via an iOS app and he is supplementing his French class with the DuoLingo app. From middle school to high school he jumped ahead and back in “grade-level” when changing schools, first after 7th grade and then changing high schools between “9th grade” and “10th grade”. So, he was never technically enrolled in 8th grade, but he took classes that sufficiently covered 8th grade learning standards.

Benjamin’s case is not unusual. He has friends that are home schooled and taking college classes as 15 years olds, and others enrolled in WPI’s Mass Academy, a public school in Massachusetts whose students attend a private university full-time as seniors in high school. According to ECS:
Forty-seven states and the District of Columbia have statute and/or regulations governing one or more common statewide dual enrollment policies,” and “three states leave dual enrollment policies to the discretion of local districts and postsecondary institutions/systems.
It’s not new that high schools determine a student’s grade level based on credits earned rather than age or cohort. What is new is the growth in options that allow student to advance at their own pace and earn college credit while in high school. Students that used to depend on the capacity of the local high school to offer an advanced placement class can now take advantage of AP or college-level online courses.

With dual-enrollment the lines are blurring between K-12 and postsecondary education, even as the institutions and public policy remains deeply rooted in the cultural inertia of separate domains.
With school choice and course choice, lines are also blurring between school districts. A student’s education is no longer fated to be on the same course and pace as everyone else that happens to be in the same zip code and age grouping. Students are benefiting as the factory model of education erodes and more student-centered options emerge. Benefits include greater potential for success and reduced costs for college and career training.

As a parent, I’m encouraged that my children have and will have options for lifelong learning that were not available when I was their age. I also see that they have new responsibilities as 21st century learners. They will need to take more ownership of their own learning. I grew up in an age of spoon-fed, one-size-fits all, everyone-moves-at-the-same-pace schooling. To take full advantage of emerging student-centered options, students today need to learn a new set of mindsets and dispositions. 

As a parent, I also have a responsibility to encourage and guide my kids in those attitudes and dispositions. So I need to keep learning. Resources like Smart Parents and Mindset: The New Psychology of Success are helpful. I can also help by learning about course choice options as they become available and learning how to evaluate what is a good fit for each of my children.  (I also have a child that is thriving in a traditional brick-and-mortar school.) By doing all this learning myself, I am modeling what it means to be a lifelong learner. Someone once told me that children learn more from what their parents do than what they say.

So, my son had multiple right answers when asked, “What grade are you in?” He could have said “I haven’t yet completed 8th grade,” or “I’m in 10th grade,” or “I’m in college.” I think he ended up saying something like “based on the courses I’m taking, I’m mostly in 11th grade.” Someday we will stop asking, “What grade are you in?” With the shift to lifelong student-centered learning, a more relevant and more interesting question (for all ages) can be “what have you been learning?”

Tuesday, September 15, 2015

Stackable Portable Digital Credentials

I've been participating in work groups from several key organizations that are developing standards for digital credentials. The following brief summarizes what's happening in the credentials space, particularly with stackable and digital credentials.

Stackable Portable Digital Credentials in Education and Industry

There is a growing interest in “stackable credentials” as a solution to problems faced by students, higher education institutions, workforce training programs, schools, and employers.  A report by the Center for Postsecondary and Economic Success at CLASP defines credentials to include “degrees; diplomas; credit-bearing, noncredit, and work readiness certificates; badges; professional/ industry certifications; apprenticeships; and licenses—all of which in different ways testify to people’s skills, knowledge, and abilities.”

The U.S. Department of Labor defines a credential as stackablewhen it is part of a sequence of credentials that can be accumulated over time and move an individual along a career pathway or up a career ladder.” The same concept might apply to a pre-career sequence of educational achievements such as credentials that qualify a secondary student to enter higher education.

An example “stackable” credential is a job-specific certificate earned in the short-term while counting toward the longer-term goal of a degree. Stackable credentials provide value to both the student and potential employers by showing short-term value  (what can a person can do now) and as a milestone toward a larger educational achievement. This is especially valuable for people who enter the workforce while continuing to pursue a degree. The Department of Labor recommends that higher education and workforce training providers “modularize curricula into smaller portions, or chunks, enhancing the ability of individuals to earn interim credentials and combine part-time study with full-time employment and/or supporting a family.”

Many organizations including the U.S. Department of Labor, U.S. Department of Education, community colleges, four-year colleges, workforce training programs, and industry groups are investigating how stackable credentials might address problems such as:

·      students giving up before completing high school and college,
·      the overwhelming cost of an all-or-nothing college credential,
·      unemployment persists while employers have trouble filling positions, and
·      training programs having trouble keeping up with changing needs in the global and local economies.

Stackable credentials also include certifications and licenses earned after receiving a degree. For example, medical professionals with multiple specialties may be more likely to be hired because they can fill more than one role (e.g. phlebotomist and EKG technician). Digital Promise is developing a micro-credential system that provides teachers with the opportunity to gain recognition for skills they master throughout their careers.

Portable credentials are credentials that are accepted across institutions, and across domains. One issue of portability has to do with a common understanding of the student competencies that the credential represents. When a student receives a baccalaureate degree in accounting, potential employers expect that the credential means the student has certain skills that qualify her for an entry level position in the accounting field. If the student is earning a credential with the intent of using it to qualify for a job then the competency model used by the issuing institution should be industry recognized.  If a K-12 student is earning a high school diploma with the intent of going on to college, the diploma should be acceptable evidence for the postsecondary institution to know that the student is college ready.

Another issue of portability is the acceptance of the credential in another jurisdiction, for example, if an associate degree or certificate earned at a community college in one state is accepted at a 4-year institution in another state as credit towards a bachelor’s degree.

Digital Credentials

Digital credentials are verifiable electronic records of a person’s achievements or qualifications. Digital credentials take different forms depending on how they are used. Technical implementations include electronic transcripts, digital certificates, and digital badges. For portability, the digital credentials must use widely adopted technical standards for interoperability between issuing and consuming data systems.

Technical Standards for Stackable Credentials

Government agencies, industry groups, standards bodies and education providers are developing approaches to the data collection and use related to stackable credentials. For example:
  • The Badge Alliance and related Open Badges Initiative have developed an open standard and free software for digital badges (an image file with embedded metadata representing a personal achievement) that links back to the issuer, criteria and verifying evidence.
  • The Common Education Data Standards (CEDS) include standard vocabulary for data used to recognize student achievements linked to evidence.
  • Credential Transparency Initiative is creating a credential registry that will allow users to see what various credentials — from college degrees to industry certifications and micro-credentials — represented in terms of competencies, transfer value, assessment rigor, and third-party endorsement.
  • IMS Global is working with college registrars on an extended electronic transcript standard that would include record of competencies and non-course activities.
  • PESC has formed an Academic Credentialing & Experiential Learning Task Force to build on its previous eTranscript standard
  • W3C Credentials Workgroup plans to publish a standard for encoding personal credentials in a way that can be authenticated and verified using technology similar to bitcoin and technologies addressing personal identity and privacy.

Some of the implementation challenges that these organizations are wrestling with include:
·      Should a persons digital credentials from multiple institutions be kept in a “locker” or “backpack” under the stewardship of a third party hosting organization, held privately by the recipient, or exist in a distributed network?
·      What technology should be use to certify the validity of a credential and protect against counterfeit credentials?
·      What method and data standards should be used to standardize the information about what a credential represents?
·      How to digitally link the identity of a person to a credential they have received?

Key terms related to this topic include:

career pathway – a series of achievements and that qualify a person for a career
career latter – a path of achievements that allow a person to move into increasingly more advanced jobs within a single industry or career path
career lattice – a connected sequence of achievements that allow a person to move up in a career pathway or over to a new career using transferable qualifications
digital credential – a verifiable electronic record of a person’s achievement or qualification
portable credential – credentials that are accepted across institutions and/or domains
stackable credential – part of a sequence of credentials that can be accumulated over time
micro-credential – a credential that recognized mastery of a single competency

Wednesday, August 6, 2014

Data Standards for Competency Education

This is a video capture of the slide deck that Liz Glowa, Maria Worthen, and I used in our session "Data Standards for Competency Education" at the STATS-DC conference on July 31, 2014.  (Slides from the one hour presentation compressed into two minutes of video...use pause as needed.)

Thursday, June 26, 2014

Open Education Resources (OER) Digital Ecosystem

I recently participated in the "OER Annotation Summit" in Berkley, California.  The event was  supported in large part by an OER technology grant from the William and Flora Hewlett Foundation.  The Summit explored opportunities and barriers to fostering greater collaboration in solving shared technology challenges for open education resources initiatives.

At the event I worked with a break-out group to map the OER Digital Ecosystem. The following infographic is the QIP visualization of the "ecosystem" derived from a picture of the whiteboard and Felix Tscheulin's gliffy diagram of the same...

Monday, June 23, 2014

EdFacts Community Video

Here is a video that I produced with QIP and AEM teams...