Rapid research progress in science and technology (S&T) and continuously shifting workforce needs exert pressure on each other and on the educational and training systems that link them. Higher education institutions aim to equip new generations of students with skills and expertise relevant to workforce participation for decades to come, but their offerings sometimes misalign with commercial needs and new techniques forged at the frontiers of research. Here, we analyze and visualize the dynamic skill (mis-)alignment between academic push, industry pull, and educational offerings, paying special attention to the rapidly emerging areas of data science and data engineering (DS/DE). The visualizations and computational models presented here can help key decision makers understand the evolving structure of skills so that they can craft educational programs that serve workforce needs. Our study uses millions of publications, course syllabi, and job advertisements published between 2010 and 2016. We show how courses mediate between research and jobs. We also discover responsiveness in the academic, educational, and industrial system in how skill demands from industry are as likely to drive skill attention in research as the converse. Finally, we reveal the increasing importance of uniquely human skills, such as communication, negotiation, and persuasion. These skills are currently underexamined in research and undersupplied through education for the labor market. In an increasingly data-driven economy, the demand for “soft” social skills, like teamwork and communication, increase with greater demand for “hard” technical skills and tools.
121 million advertisements from
Burning Glass Technologies
2 million syllabi from
Open Syllabus Project
1 million article abstract from
Web of Science
How has the emergence of big data, artificial intelligence, and automation realign skills that are researched, taught and demanded by industry?
To answer this question, I examined the centrality of skills as they co-occur across 120 million job advertisements, 3 million course syllabi, and 1 million research publications over 7 years from 2010 to 2016. To model the hierarchy of skills, I used an embedding technique operationalized by the Facebook AI group to represent tree-like hierarchy without distortion using 2-D hyperbolic geometry or Poincaré disks. On each disk, the radius quantifies position in the hierarchy - skills of small radius hold a central position in the network of co-occurrences, and the angle between two skills quantifies their proximity.
I found that soft skill, which involve interpersonal interaction and language facility, were most central than hard skills, which involve quantitative and operational skills in jobs. Nevertheless, they were somewhat less central in course syllabi, and much less central in research publications. This highlights a substantial discrepancy between how central soft skills, like communication, are to the workplace, including in technical jobs as main driving forces of an increasingly data-driven economy, but how peripheral they are to technical courses and publications.
Poincaré disks, or 2-D hyperbolic geometry, is an efficient way to represent tree-like hierarchy of co-occurrences networks of words, ideas, people, and other objects without distortion. On each disk, the radius quantifies position in the hierarchy - nodes of small radius hold a central position in the network of co-occurrences, and the angle between two skills quantifies their similarity.