2020 Updates

 2020 Publications:

 

Science of Remote Learning

Jim Goodell & Aaron Kessler

May 22, 2020  publication descriptionMIT Open Learning

This document was created as a resource for those working on developing, implementing, and engaging in remote teaching and learning. One goal of this work is to provide a starting point for transitioning away from emergency online instruction in response to COVID-19 toward increasingly effective remote learning. This resource is written in plain language to be broadly applicable to professional educators and non-professionals; teams designing remote learning experiences, people supporting at-home learning (mostly parents) and learners. While we do not believe this document provides a single best process for creating and supporting online learning - learning is a deeply contextual process that requires attention to the needs of learners in your context- our hope is that in considering the strategies contained within, grounded in research based principles, you can iteratively improve and support remote learners and learning.

Design Recommendations for Intelligent Tutoring Systems, Volume 8 - Data Visualization (Book) - Chapter 2 - A Learning Engineering Approach to Data Visualization

Jim Goodell & Bridget E. Thomas

December 29, 2020  publication description

US Army Combat Capabilities Development Command - Soldier Center

This chapter  presents  the development  of  data visualizations as  a learning  engineering  exercise.  The  approach  is based on  the  theory  that  a viewer’s  interaction with  a data visualization is a  learning  experience intended  to  address  specific  learning  objectives.  The process  starts with questions like:  What  do you want the viewer/learner  to know, be able  to do, or  do  after  experiencing  the  data visualization? Is it  a call  to  action? Do you want  to change the viewer’s behavior? Do you want  the viewer  to be  able  to  apply  the new knowledge in some way?  The approach applies  human-centered design, incorporates  key  elements of  cognitive science, uses  data  to  inform  design decisions, and iteratively  develops and tests  aspects of  the visualization to  optimize it  for  the desired outcomes.

A Learning Engineering Model for Learner-Centered Adaptive Systems

Jim Goodell & K.P. Thai

International Conference on Human-Computer Interaction
HCII 2020: HCI International 2020 – Late Breaking Papers: Cognition, Learning and Games pp 557-573

Goodell J., Thai KP.. (2020) A Learning Engineering Model for Learner-Centered Adaptive Systems. In: Stephanidis C. et al. (eds) HCI International 2020 – Late Breaking Papers: Cognition, Learning and Games. HCII 2020. Lecture Notes in Computer Science, vol 12425. Springer, Cham. https://doi.org/10.1007/978-3-030-60128-7_41

Applying Self-Sovereign Identity Principles to Interoperable Learner Records

(Technical Consultant)

Jun 16, 2020 - U.S. Chamber of Commerce Foundation

Learning and Employment Record (LER) Wrapper and Wallet -- A Universal Cross-Standard Digital Container for Self-Sovereign Management of Learning and Employment Records with Cross-Standard LER Wrappers

publication date(Editors) Jim Goodell, Alex Jackl, Joe Andrieu, Jim Kelly

Jul 10, 2020  publication descriptionU.S. Chamber of Commerce Foundation

An LER is a digital record of learning and work that can be linked to an individual and combined with other digital records for use in pursuing educational and employment opportunities.  An LER can document learning wherever it occurs, including at the workplace or through an education experience, credentialing, or military training. It can also include information about employment history and earnings. LERs are similar to electronic health records (EHRs) and have the potential to improve education and hiring outcomes in the same way that EHRs have improved healthcare delivery. What makes LERs unique is their ability to be fully transferable and recognized across student information, learning management, employer HR, and military systems. 

LERs go by many names and are also referred to as an interoperable learning record (ILR).This draft specification is being developed within the T3 Innovation Network (T3 Network) with assistance from project teams to leverage existing LER standards, not replace them. This draft specification was developed in the public domain and will be offered for recognition as a standard by relevant standards organizations concerned with Learning and Employment Records (LERs). The editors facilitated cooperation with a large group of standards and stakeholders including industry, academia, and standards organizations. Contributors included representatives from Access 4 Learning (A4L), Common Education Data Standards (CEDS), IMS Global Learning Consortium, Postsecondary Electronics Standards Council (PESC), HR Open Standards Consortium, and World Wide Web Consortium (W3C), among others. In addition, IEEE approved a workgroup for a new guide to interoperable learner records that will be informed by this document. This draft specification will be reviewed and pilot-tested by LER pilot teams to improve and update the specification over time.

publication descriptionThis draft specification is being developed in the public domain and will be offered for recognition as a standard by relevant standards organizations concerned with Learning and Employment Records (LERs). An LER is a digital record of learning and work that can be linked to an individual and combined with other digital records for use in pursuing educational and employment opportunities. An LER can document learning wherever it occurs, including at the workplace or through an education experience, credentialing, or military training. It can also include information about employment history and earnings. LERs are similar to electronic health records (EHRs) and have the potential to improve education and hiring outcomes in the same way that EHRs have improved healthcare delivery. What makes LERs unique is their ability to be fully transferable and recognized across student information, learning management, employer HR, and military systems. LERs go by many names and are also referred to as an interoperable learning record (ILR).This draft specification is being developed within the T3 Innovation Network (T3 Network) with assistance from project teams to leverage existing LER standards, not replace them. It will be offered for adoption by various standards organizations. This work has been done in cooperation with a large group of standards and stakeholders including Access 4 Learning (A4L), Common Education Data Standards (CEDS), IMS Global Learning Consortium, Postsecondary Electronics Standards Council (PESC), HR Open Standards Consortium, and World Wide Web Consortium (W3C), among others. In addition, IEEE approved a workgroup for a new guide to interoperable learner records that will be informed by this document. This draft specification will be reviewed and pilot-tested by LER pilot teams to improve and update the specification over time.

IEEE IC INDUSTRY CONSORTIUM ON LEARNING ENGINEERING, Proceedings of the 2019 Conference on Learning Engineering, IEEE SA INDUSTRY CONNECTIONS

Blake‐Plock, S., Goodell, J., Kurzweil, D., Kessler, A., Olsen, J. (Editors)

publication dateJul 24, 2020  publication descriptionIEEE SA INDUSTRY CONNECTIONS

 

Proceedings of the 2019 Conference on Learning Engineering

What we Discovered at the Roots of Learning Engineering

Jim Goodell, Mark Lee, & Jodi Lis

This article examines the practice and process of learning engineering. It is based on site visits and interviews we conducted in Pittsburgh, PA in November 2018 with representatives from several departments at Carnegie Mellon University, Duolingo, Carnegie Learning, and Acrobatiq.

CEDS Data Model Guide - Version 8 

Jim Goodell

May 6, 2020  

ceds.ed.gov 

The CEDS Data Model Guide describes how to use the Common Education Data Standards Data Models published on the CEDS website (https://ceds.ed.gov) and Open Source Community (https://github.com/CEDStandards). CEDS includes a broad scope of data elements definitions spanning much of the P-20W spectrum (pre-kindergarten through workforce education). The CEDS Domain Entity Schema (DES) includes a hierarchical schema of domains and entities—as a nontechnical reference showing CEDS elements in context. The Integrated Data Store (IDS) is a fully normalized logical, or physical, model. The Guide also includes examples showing CEDS elements in the context of other types of data models, such as the star schema typically used in dimensional data warehouse design.

CEDS Conceptual Model Guide - Version 8

Jim Goodell/QIP/AEM

May 6, 2020  

ceds.ed.gov 

This overview of the CEDS Conceptual Model contains general information that can be applied broadly across the four domains of P-20W education and training:

1. Early Childhood

2. K-12

3. Higher Education

4. Workforce (including military)

The CEDS Conceptual Data Model is built on four key concepts: Person, Organization, Resource, and Relationship. These key concepts are modeled as CEDS Entity super classes. They also serve as broad categories for understanding and organizing P-20W longitudinal data. The model supports longitudinal data, recognizing that data and relationships change over time:

● People have roles in Organizations for specific periods of time.

● The status of a Person, Organization, Resource, or Relationship may be different at different points in time.

● Events involving one or more Persons, Organizations, Resources, or Relationships occur at a point in time and over periods of time.

 

2019 Publications

Are you Doing Learning Engineering--Or Instructional Design?

This article for Learning Solutions examines the IEEE ICICLE definition of learning engineering and its overlaps with the practices of instructional design.

Education and Career Pathways Navigation with Data

This piece originally appeared in two parts on the Quality Information Partners site:

Maps for Learning and Job Success

Recent statistics show a mismatch between the skills secondary and postsecondary students are acquiring and the rapidly changing needs of industry. In June 2018, the Bureau of Labor Statistics reported that U.S. job openings had increased to 6.6 million, while the number of unemployed people was down to 6.3 million. According to the 2017 ExcelinEd white paper Putting Career and Technical Education to Work for Students, “Many of these open positions offer middle- and higher-wage salaries, as well as opportunities for continued training and advancement by employers, but they go unfilled due to a lack of appropriately skilled workers who have completed aligned programs of study.” Pathways data—data that help students navigate through different points in their education and career trajectories—can help solve this problem. These data define not just the routes to success (i.e., to the desired destination), but also the milestones along the way.

It is clear from these reports that current students and education providers could use better alignments to the most promising opportunities in higher education and the workforce. At the macro level, we see gaps between what students are learning and what they need to learn to transition into the college programs of study and work positions that are available. At the micro level, a student’s skill gap in any area (e.g., proportional reasoning) becomes a roadblock for learning further skills that depend on that prerequisite understanding or ability (e.g., operations with fractions, word problems, and physical science applications). The lack of well-defined education pathways data—and the failure to use the information that is currently available—is limiting opportunities for students, employees, and employers.

Four kinds of education and career pathways

There are four kinds of pathways that serve different purposes:

• Competency pathways define recommended sequences of learning. They show prerequisite and post-requisite relationships between competencies. Competencies can include skills, knowledge, dispositions, or practices.
• Content pathways define sequences of learning resources or learning experiences.
• Credential pathways define sequences of credentials that build an individual's qualifications. These pathways often include “stackable” credentials that can help a person qualify for a different and potentially higher-paying job, by adding qualifications to those he/she already has. (See also this explanation of stackable credentials from the U.S. Department of Labor.)
• Career pathways define a series of structured and connected education programs and support services that enable students, often while working, to advance over time to better jobs with higher levels of education and training. (See also this explanation of career pathways from the Career Ladders Project and this definition from ExelinEd.)

Visualizing pathways as a map

Although the four kinds of pathways have different purposes, their structure looks the same. In each case, the information can be visualized as a map. Points of interest on the map, called milestones, can represent

• a competency (e.g., a skill, piece of knowledge, disposition, or practice);
• content (e.g., a learning resource or program);
• a credential (e.g., a qualification or degree); or
• a career opportunity (e.g., an internship or job).

Figure 1. A pathways map has milestones (which are like points of interest on a street map) connected by paths (which are like road segments on a street map).

While these different types of milestones can all be points in a pathways map, the metadata for each will be different, depending on type. For instance, a credential milestone will have different metadata properties than a competency milestone.

A path is a connector between two milestones. Paths, similar to road segments on a street map, represent recommended ways someone can navigate from point A to point B. On a pathways map, a path shows how to get to a slightly more advanced milestone via its prerequisite milestone. Figure 1 shows the relationship between two milestones and a path.

Figure 2. A pathways map can have multiple routes (which are also called routes on a street map). The route in blue represents one of many education/career possibilities in nursing.

A pathways map can be formed by connecting many milestones and paths. People can then select routes based on interests and needs. A career pathways map in nursing, for instance, may have several possible routes. There could be an entry-point milestone of a high school diploma, with two paths leading from there, one to a Licensed Practical Nurse (LPN) qualification and another to an Associate Degree in Nursing (ADN) to qualify as a Registered Nurse (RN). Another path could lead from the LPN to the RN. The LPN and RN could each have a path to a Bachelor of Science in Nursing (BSN). All of this creates many possible routes and destinations (illustrated in figure 2). Additional routes could be created, thus expanding the map, by adding paths from the BSN to graduate degree qualifications for other positions in health care.

Note that, unlike a street map, a pathways map is unidirectional. While people commonly travel from point A to point B and then back to point A, they do not travel from a more advanced milestone to its prerequisite. Of course, people may need to relearn a prerequisite they either missed or forgot in order to advance; they may also decide to double back and change routes. But they will never begin at a master-level job and move from there to a basic internship in the same field, or start by learning differential equations before moving on to addition and subtraction.

Data Standards for Pathways

Education and career pathways are maps. Students, educators, employees, and employers can use them to navigate through the various stages of attending school and participating in the workforce. As I explained in my previous blog post on education and career pathways , just as people use regular maps to travel from point A to point B, they can use education and career pathways to advance from one milestone to another in their education and careers.

In order to create education and career pathways maps, we need data and metadata. We also need standards to make the data interoperable. These data collection and standards efforts must be open and created with input from various stakeholders.

Moving toward a Google Maps model

Google Maps is a good metaphor for education and career pathways maps. In both types of maps, people can choose among possible routes based on needs and interests.

Data attached to each milestone (like a credential or job) help people determine where they are and what their goal or destination is. Data allow the technology to show different ways to reach each destination and to suggest the fastest or best route, given internal and external circumstances.

On Google Maps, the internal circumstances may be that a person is riding a bike, or a driver can’t take toll roads. The external circumstances may be construction or traffic congestion on some roads. In education and career pathways, the internal circumstances may be that a person has a job, is a single parent, and lives 50 miles from the nearest college. The external circumstances may be that a state law passed that will change certification requirements in three years’ time. Like Google maps, a data-driven pathways navigator would suggest personalized routes based on the circumstances. It would recommend different career pathways to people in different circumstances, even if both share the same goal.

We have not yet gathered the large amount of data and metadata needed to create education and career pathways maps. We also don’t have a complete set of standards that can make data operable between systems. Although several promising initiatives aim to address these problems, we are still in the beginning stages of creating rich and open pathways maps that have the power and utility that Google Maps brings to street navigation.

Data needed for the four kinds of pathways

Education and career pathways come in four varieties. Each kind of map serves different purposes and requires different kinds of data and metadata.

In a competency pathways map, routes are defined based on expert recommendations for sequencing learning. Each milestone contains data defining a competency (a skill, piece of knowledge, disposition, or practice). For example, mathematics teachers recognize that proportional reasoning skills are prerequisite to success in algebra (see this Doing What Works presentation on developing proportional reasoning). A competency pathways map may indicate that students must reach a defined level of mastery in proportional reasoning before learning about linear equations.

A content pathways map serves the needs of curriculum developers who are building coherent sequences of learning activities. Each milestone contains data defining a learning resource (for example, a video or discussion guide). Digital resources are alternatives to static resources such as printed textbooks. Data linking specific lessons and activities may define prerequisite and post-requisite relationships to maintain a coherent sequence while allowing for personalized learning. The data of each content milestone may also link to competency definitions (milestones in a competency pathways map) that define what the learning resource is intended to teach or assess.

In a credential pathways map, routes indicate means of achieving each credential. This kind of map shows how “stacking” credentials in different ways could lead to the same outcome. A credential pathways map could show, for example, that a series of micro-credentials add up to the same qualifications as a certificate program.

A career pathways map may include milestones for career options as well as for job qualifications. Many professions require education credentials, licensure tests, entry-level experience (for example, working as an apprentice), and/or achieving full certification. Additional conditions might be required before becoming a master of the trade or profession. Data on a career pathways map must be attached to the destination milestone (the job itself, linked to the competencies required for the job and other metadata), as well as to milestones that indicate how one can qualify for the job.

The future of education and career data systems

Pathways maps can help bridge traditional institutional boundaries—such as between K-12 and higher education and between education and employers. When education and training programs are better aligned to what lies ahead, they can prepare students for long-term opportunities. Moreover, students are able to make more informed choices when they understand the full range of options available to them.

Furthermore, as new careers are invented, learners will be able to see how to train for emerging, high-demand, higher paying jobs. If learners have trouble acquiring new competencies, they can explore other modalities of learning and practicing to achieve the same milestone.

Learning pathways data, combined with experience data, can be improved using artificial intelligence (AI) technology to optimize route recommendations. The full potential of this kind of optimization will depend on pathways data being open on the web and fully interoperable, and with comprehensive coverage connecting competencies, credentials, and careers.

Making education and career pathways a reality

Without access to robust learner navigation systems, students are not fully informed about routes to prosperous and fulfilling careers. Educators and students often make guesses about which routes are best, or make random choices due to uncertainty. Education institutions assume they are helping students acquire the competencies they need for their futures, but data show a mismatch between workforce needs and job seekers' skills.

I invite you to join in the effort to work toward robust education and career navigation systems, and to create the data standards needed to make systems interoperable. With dedication and collaboration among a variety of experts, organizations, and agencies, we can make standardized, open-data pathways maps a reality.

Turning ‘Google Maps for Education’ From Metaphor to Reality

This piece originally appeared on EdSurge on Jul 14, 2018.

By Jim Goodell

In his latest EdSurge column, Michael Horn laid out how Google Maps offers an aspirational metaphor for what the future of educational tools could look like. But as he also noted, locating where people are geographically is one thing; pinpointing where they are educationally is another.

Today, Google Maps is an open ecosystem for accurate, real-time geospatial and navigation data. Unfortunately, current learner navigation systems more closely resemble the early, self-contained GPS devices with incomplete and inaccurate maps.

To bridge the gulf, it will take a similar open-data ecosystem to support learner navigation. But in the field of education, we don’t even have a complete set of static competency frameworks for digital data that are openly accessible and interoperable—to say nothing of dynamic data that support real-time pathway optimization.

Yet there are several initiatives, some of which I’ve had the privilege of working on, that aim to support the educational data ecosystem necessary for learner navigation.

Alignment of Data Standards for Describing Learning Objectives

When applying the Google Maps metaphor to learner navigation, the points of interest on the map are competencies—the things that a person can learn, such as skills, knowledge, dispositions, and habits of practice. This data must be in a machine-readable format and interoperable to work in all apps and systems.

The existing standards for this kind of data were like a Babylon of different languages, understood only in their own domain (such as medical training, human resources, K-12 or postsecondary). To connect these standards, there are efforts such as the Credential Ecosystem Mapping Project, where participants are working together to understand how data elements within various standard formats can be converted to other formats. This project maps across existing data standards, making it possible to translate data at all levels and sectors of education and training, such as the MedBiquitousstandards for health care and HROpen for human resources.

The IEEE Learning Technology Standards Committee (LTSC) plans to update its existing standards based on this work. Last updated in 2007, the international standards body has defined a data model for describing, referencing, and sharing competency definitions, primarily in the context of online and distributed learning. The LTSC is also working on related standards for mobile learning platforms, adaptive instructional systems and augmented reality learning environments.

Open Registry of K-12 Learning Standards

State academic standards help define the learning objectives for U.S. K-12 learners. These standards have been traditionally published only as human-readable documents, such as PDFs, that can’t be used directly by education technology tools. In other words, statements within PDF documents cannot be reliably referenced in information systems and digital content.

Something is usually lost in translation when content publishers and software developers try to put state standards into their own databases. Also, mapping is problematic with 50 sets of state standards for each grade level and subject, plus many more derived versions of those standards used locally and for other specific purposes.

In an effort to solve this problem, IMS Global recently announced it will host a 50-state registry of academic standards. This registry aims to provide a definitive set of machine-readable statements and a freely available set of global identifiers for use in digital content. If it achieves that goal, it will also support crosswalks for systems to discover whether one state’s standard is an exact match to a standard from another state. Equally important, it will allow states themselves to maintain the digital and human-readable standards so that nothing is lost in translation.

Linked Data Defining Competencies and Credential Pathways

Data about credentials that are available for a person to earn are just as valuable as data about the things a person must learn to get them. To this end, the Credential Registry, hosted by nonprofit Credential Engine, has created an open catalog of data about postsecondary degrees and other credentials available in the United States.

Dozens of credentialing institutions and quality assurance bodies are already posting information on the registry, which includes different kinds of credentials, from degrees, certificates and certifications to licenses, badges, and micro-credentials. Credentials in the registry include linked data for the competencies that each credential represents.

Today, micro-credentials (sometimes issued as digital badges) and micro-master’s degrees have emerged as a more dynamic model for credentialing than the traditional 2- or 4-year degree, offering an alternative reflecting the ever-changing world we live in. Digital micro-credentials, such as those offered by Digital Promise for educators, provide recognition for the skills people learn throughout their careers.

Pathways data can help people navigate opportunities to earn “stackable” credentials, or a sequence of credentials that can be accumulated over time to build up an individual’s qualifications for a different and potentially higher-paying job. (See this explanation of stackable credentials from the U.S. Department of Labor.) These more flexible pathways could also be used in K-12 education, such as a student earning a micro-credential as a step toward licensure in a trade.

Translation Competency Definitions Between Data Formats

Organizations in different domains use different formats for the same kind of data. For example, a K-12 state education agency could use tools based on the CEDS and IMS CASE data standards for academic standards. In the same state there may be a district with a career and technical-education program in health science and medical technology, but the medical industry uses the Medbiquitous standard to encode competency definitions.

To address this issue, the CASS system, developed by Eduworks with funding from Advanced Distributed Learning Initiative, is being used with the Credential Registry as a translator to move competency framework data between different serialization formats of technical standards. That could allow for better alignment between a university’s digital competency frameworks, and what is used by the profession or industry that a student will enter.

Linked Data on the Web

Billions are spent developing digital educational content and trying to develop systems to better recommend what digital content a student should experience next. To date, defining where digital content belongs on a learning map has only happened in closed systems with very limited maps of learning progressions. With openly available learning map data, links to those data can give digital content a point of reference. This is as simple as providing URLs that “locate” at what learning milestones the digital content may be used.

As the learning map references become available on the open web, dynamic learning content will also be able to link to specific activities and assessments. Technology standards such as Experience API (xAPI) and IMS Caliper link specific learner experiences to points on the learner navigation map and add useful contextual data. This is similar to how up-to-the-minute traffic data helps Google Maps find the fastest routes.

Aligning Learning, Workforce, and Credentialing Data with the Needs of the New Economy

It’s often discussed that there is a mismatch between the skills secondary students are acquiring and those needed for post-secondary coursework, as well as a mismatch between the skills needed for current jobs and the skills that college graduates have. One of the challenges is to build a learner navigation map that has coherent pathways between K-12 academic standards, postsecondary programs, and occupational competencies.

The T3 Innovation Network, funded by the Lumina Foundation and the U.S. Chamber of Commerce Foundation, is investigating, among other things, whether artificial intelligence algorithms and resources can be used to discover information used in learner navigation. For example, they are studying how to turn unstructured information about job skills on the web into structured data that conform to data interoperability standards and can link to learning opportunities and credentials. This can add value to existing information sources like the U.S. Department of Labor’s O*NETcareer exploration and job analysis resource, by linking information about job skills and the education credentials that best represent those skills.

More to Be Done

Beyond these initiatives, an open ecosystem for learner navigation will need additional research-driven data, such as data about contexts and conditions in which learning takes place, available learning experiences, how to measure mastery levels, the cognitive and metacognitive gaps and barriers that learners face, and which kinds of practice or experiences can lead past those barriers to mastery.