A Crisis of Perception? Viewing the Skills Crisis Anew

Posted September 5, 2019 in Business Agility & Software Engineering Excellence
The Skills Crisis

Despite the impression that software is rapidly changing, with wave after wave of new ideas and technology, the truth is that it is very static. Modern ideas driving the technical focus of Industry 4.0 are perceived as revolutionary, even though ideas such as the actor model were conceived in 1972. Machine learning is the application of algorithms to statistics, which has theoretically been established for several generations. Such is the disconnect between academia and industry that after 20 years of practical software engineering, most professionals’ daily work is far removed from the lectures they attended, so when these subjects resurface, they appear brand new! The theoretical basis for Industry 4.0 already exists and can be made available to anyone through MOOCs (massive open online courses) and online platforms; only the experience of practical application in changing contexts is missing.

For this reason, talk of a skills crisis in Internet of Things a few years ago was plainly ridiculous — the market or engineering foundations are still not established enough for a more formal emergence of expertise to exist so trying to hire that expertise by searching a skills database will lead to a perpetual sense of crisis. Instead of seeking expertise from the beginning, accepting emergence of expertise over time may be the best way to combat a crisis that possibly exists more in our perception than in reality. Perhaps the skills gap is not a gap in knowledge of platforms and products, but a gap in the ability to navigate the unknown without the comfort blanket of product or platform expertise. Many see the solution as a workforce trained in the latest and greatest technologies and call the gap between the solution and reality a crisis, but perhaps the assumption that such a workforce can exist in this environment is the cause of the problem?

The ability to navigate complex situations and problems is the major issue, not knowledge of specific languages, frameworks, or vendor tooling. Government initiatives to teach coding miss the point, as successful software engineering in complex envi­ronments is going to need skills outside of coding to be successful. Current thinking and policy focus on producing more people who can code, not who can think in a way that allows coding to be used properly.

The basics of computer science are still important, but these are easy to master if taught in context and shown to be relevant to the everyday work of software engineering, rather than a separate rite of passage that seems to bear little relation to the working world graduates are released into — a world that leaves 13% of them unemployed six months after graduating!

If we are to abandon the simplistic idea that the skills crisis can be solved by increasing the number of people entering the field or who know the platforms we are working with, we need to propose new solutions.

However, employers are keen to see graduates who are work-ready, and in many cases this means already knowing the latest trends and tools. This is an impos­sible task for universities to meet, given the never-ending pace of change; even if they succeeded, students graduating with relevant skills today will still need to retrain in a few months as new patterns and tools emerge. Regardless of what universities do or how accessible government or industry programs make IT education, every single graduate will eventually face a choice between self-sustaining renewal or career stagnation.

Alternative Solutions

The tech industry also suffers from another, remarkably well-hidden problem: age discrimination. Over the age of 45, many in the industry feel dispensable and struggle to find work if they should find themselves unemployed. Having been through many waves of technology before, this group of people undoubtably contains the critical-thinking skills needed by Industry 4.0. However, these resources possess only out-of-date skills, with little weight being given to the critical faculties developed over a career. Allowing for easy mid-career transitions to different areas of specializations would make this group a powerful remedy to skills shortages but requires a shift in thinking from employers.

A huge number of software projects still fail and multiplying the number of people who do the same old things will not change this failure rate. Today’s skills gap is probably smaller than the number of talented developers currently wasting their time on failing projects or dedicating their time to overhyped trends with no basis in economic reality. Teaching programmers and their extended teams to think critically would allow for much faster abandonment of failing projects and acceptance of this as a natural way of doing business with technology will free up resources and ease the workload.

Industry 4.0’s predilection for hype is also an issue. For example, many technologies form their own skills crisis as businesses seek resources who know this specific toolset — only for that toolset to be retired after a few years with few tangible benefits. These resources are also wasted, with specialist knowledge gained now useless.

If we could leverage the huge pool of older resources, waste less time on failed projects and hyped projects, and encourage the acceptance of emergence, we would at least reduce the number of resources needed. This solution, however, requires organizations to be more critical in their navigation of Industry 4.0.

Teaching Critical Thinking

Some theories on critical thinking view the skills of critical thinking as separate from contextual knowledge, while others see the two as inseparable. There is little consensus. Regardless, Industry 4.0 is pushing the fields of business and technology so close that they are becoming a single context and thinking critically about one is impossible without knowledge of the other. This is a defining characteristic of Industry 4.0, and one that creates problems but also provides oppor­tunities. This alignment of the two fields demands that many engineers and associated business roles on Industry 4.0 projects need to broaden their context to cover both business and technology specializations, another step toward becoming generalists. By exposing more computing graduates to business studies, and more business people to computing, we increase the size of the pool that can engage in critical thinking around delivering technology into business environments and make more effective decisions that reduce issues of resource wastage on hype-driven projects or ill-defined initiatives.

Research shows that there are four techniques that have specific impact on a student’s critical thinking, and while we leave the debate about how to better produce critical thinking in K-12 education to those that work in that area, one important point was the impact of mentoring in combination with dialogue and real-world problem solving. Encouraging this kind of mentoring within a company could have a significant impact on how critical thinking is spread through an organization, making it easier for technologists to tackle shifting technology landscapes with confidence.


We currently live in a world where we need the knowledge and experience built up over a career to navigate the complexity involved in most fields. In the technology industry, we have come to expect this experience to simply appear despite constant technological change. The seemingly obvious solution of educating faster and cheaper and wider has been shown to be ineffective — and the obvious next step is to question how we can better navigate this crisis in the future.

Assuming that we must learn in the same way is a mistake, and we limit ourselves by consigning solutions to copies of what has worked in the past. Rather than learning faster, one option is simply to learn better. A field of software engineering where knowledge of a platform or technology is not a prerequisite to working on a project would solve the perception of a skills crisis overnight. Equipping engineers with the necessary skills to handle complexity, transition, and breadth, with a real and solid grounding in practical computer science that stays with them throughout their careers, is essential.

[For more from the author on this topic, see “The Skills Crisis 4.0: Accepting New Realities.”]

About The Author
Barry M. O'Reilly
Barry M. O’Reilly is the founder of Black Tulip Technology and creator of Antifragile System Design. Previously, he held positions as Chief Architect for Microsoft's Western Europe practice and IDesign, IOT TAP Lead for Microsoft’s Western Europe practice, Worldwide Lead for Microsoft’s Solution Architecture Community, and startup CTO. He can be reached at