The theory of innovation, part 1: The structure

How innovation works, and how you should design your business around it

This is a weekly-ish newsletter from Al Cottrill and Class35. It’s mostly an informal outlet for the theory and esoterica that underpins our practice. We hope you find it interesting.

This is the first in a series of notes on the dynamics of innovation.

In the work we do, I talk a lot about ‘designing systems’, because in the end that’s what business design is about. And well, most organisations don’t design their innovation system particularly well - be that a lab, an accelerator, a ‘factory’, or a programme. I know this because over the last decade or so I’ve designed a few… poorly. But also because I’ve spent most of this time trying to understand how they should be designed.

Designing a good innovation system requires knowing what a good innovation system looks like and why. And doing this requires understanding the underlying structure and dynamics of innovation. That is, how it actually works, and it's universal principles and patterns, so that you can align your approach with these realities.

Otherwise you’re just doing cargo cult innovation - all the labs and post-its and product teams and sprints and Dragon’s Dens, and none of the leverage. In essence, relying on something close to blind luck.

So here goes, innovation theory 101, or ‘how to build an innovation lab even the CFO will love’.

Part 1: The structure of innovation

“What the pupil must learn, if he learns anything at all, is that the world will do most of the work for you, provided you cooperate with it by identifying how it really works and aligning with those realities. If we do not let the world teach us, it teaches us a lesson.” — Joseph Tussman

The most important thing to understand is that innovation has a common structure to it. A structure that is common across markets and domains, from capitalism to nature to the synapses in your brain.

And the most important thing to understand when encountering structural dynamics like this is that it’s best not to fight them.

We tend to think about innovation in our own little corner of the world as like a corporate ‘doing innovation’ and trying to deliver new products and services. But it’s obviously a much broader idea. And the best place to start is probably here, with the the emergence of the bicycle over time.

Source: Utterback, J., Mastering the Dynamics of Innovation

You can see how it started out with the hobby horse, which in turn gave way to some attempts at adding pedals to it. From there, we saw a proliferation of imitators and innovators, changing wheel sizes, moving the crank shaft, adding third wheels, as we gradually got closer to what we know no as a standard ‘bicycle. (From there it branched off into a whole range of racers and mountain bikes and BMXs and fold-up bikes etc, but I’ll get to that later). You can see a nice animated version here - being linear it doesn't quite capture the point, but it does show the ‘evolution’ beautifully.

But what’s really going on here is ‘innovation as a search process’. The point is that no one knew what a bicycle should look like. They just had the basic technology of ‘a pedal-powered, single-person vehicle’. And over time, through trial and error, gradually the market worked out what a ‘bike’ should look like. That is, it made a bunch of guesses at what would work, and gradually a standard bicycle emerged.

A common structure

The interesting part to all of this is not the bicycle itself, but the underlying structure of innovation we see at play. The pattern we see above is common at the start of any new market. Here are four more examples: automobiles, TV, disk drives, PCs (from Utterback’s Mastering the Dynamics of Innovation, 1975; and Michael Mauboussin’s More Than You Know: Finding Financial Wisdom in Unconventional Places). On the left axis the the number of firms in an industry, and years along the bottom.

Utterback’s theory is that whenever a new technology (defined broadly, be it technology or business model or legal structure) comes along, we see entrepreneurs pile into the market in search of profits, each with a slightly different version of the design. Over time, under pressure from competition and feedback from the market, the design improves until what Utterback calls a ‘dominant design’ emerges.

The reason this exists is because of the prediction challenge described above. No one really knows what the market wants ahead of time, but there’s a pretty good profit motivation to find out. So the result is a proliferation of guesses – marginally different combinations of features and designs and business models.

Each of these entrepreneurs iterates their solution based on feedback from the market and observing competitors (in parallel to improving technology). And over time the ‘fitness’ of solutions improve as they work out what works and doesn’t work. Unfit solutions die off, fit solutions thrive, until a ‘dominant design’ emerges. The ‘dominant’ design becomes the de facto industry standard because it is the ‘fittest’ – or most viable – solution, and all the others competing designs are killed off. (From there the industry moves into a ‘scaling’ or ‘optimisation’ or ‘deployment’ phase as the design is rolled out across the market, before it starts to re-proliferate into more niche designs).

So we see this common pattern: of a range of diverse solutions attempted, gradual improvement as the iterate based on feedback from the market, the emergence of a ‘viable’ design, and the failure of all the ‘wrong’ answers.

We can map this same pattern to the bicycle above. Emergence of a new technology, proliferation of designs as entrepreneurs enter the market – each with a different variation on a bicycle – before the emergence of a dominant design and the failure of the competing designs.

This pattern of proliferation of designs, then pruning back to a viable solution is the common pattern of any innovation system.

Some modern day (and less modern day) examples

I’ve collected a number of these over the years. Here’s the emergence of software the industry, as reported in BusinessWeek in 1984. Note the reference to the ‘3,000 hopefuls had jumped into the fray’ and the headline about the shake-out that everyone sees coming.

And again in 1999/2000 software bust: proliferate and prune

Source: More Than You Know: Finding Financial Wisdom in Unconventional Places

And across today’s startup landscape we see the same phenomenon, whether it is food delivery, ‘Uber for X’, or DTC and fintech and their many niches, a proliferation of startups on the back of the combination of new technology and business models.

Google Trends: Uber for X

The UK DTC landscape: Proliferation phase

In every early-stage industry – be that a bike, television, open banking, a platform or DTC business model etc. – we see the same pattern: Mass entry from entrepreneurs searching for a viable business model that can survive in the market - before an industry shake-out and emergence of a few sustainable firms.

So what?

If it’s not obvious, this is a fundamental structure of innovation: exploration through a diversity of designs; iteration and improvement over time; the mass failure of most variants; and the emergence of a viable model.

And we can start to extract a few key principles:

Proliferation is fundamental. It’s this proliferation – broad exploration and diversity of design variants - that means you search broadly enough and wide enough to find something that works. Even though they fail, they are serving an essential purpose to the overall system.

The system innovates. The hardest idea to grok here is that it’s the system that finds a solution. The odds of any one design variant being right is relatively low, but the odds of overall success across the market is high. It’s the portfolio that generates the return, not any one bet.

It’s an effective system, not an efficient one. Most designs don’t work, and most start-ups fail. And across these there is generally enormous investment and effort that went nowhere. It's 'inefficient' when viewed through a normal economic lens. But at a system-level, it is effective in that it finds viable solutions.

It’s a universal pattern. If your innovation system doesn’t look like this, or if working in conflict with any of these patterns, it’s likely less effective than it could be.

But this is only the first piece of the puzzle. Next up, we’ll get more into this pattern and why it exists, before turning to the implications for designing your innovation function effectively.

Hope you stick around for the next 10 or so of these… (and hit subscribe at the top if you’d like to read them).


And read part 2 here.


Utterback, J: Mastering the Dynamics of Innovation

Geroski, P: The Evolution of New Markets

Mauboussin, M: More Than You Know: Finding Financial Wisdom in Unconventional Places