The theory of innovation, part 4: Product Design by Mad Libs

How innovation works, and how you should design your business around it

This is a fortnightly-ish newsletter from Al Cottrill and Class35. It’s mostly an informal outlet for the theory and esoterica that underpins our practice. We hope you find it interesting.

Thanks to everyone who got in touch after the note on Schumpeterian Waste with questions and comments. And thanks to Phil for harassing Tom Peters on Twitter about it…

As mentioned in the title, this is issue 4. You can find back-issues here:

So for Issue 4, I was going to start talking about how to design a good innovation system in an organisation. But as I started writing, I realised I needed to double-down on the idea of Schumpeterian Waste.

Because I genuinely believe it’s the missing piece in most corporate innovation.

Again, my rough thesis is that it’s not the innovating bit that is necessarily hard, it’s creating the right system around it to enable it. That is, you have to create the pre-conditions for successful innovation, as much as you have to focus on the ‘innovating’ bit itself. Unfortunately the latter is not only more tangible (‘we’re doing stuff’), it’s also the more interesting stuff. Certainly more than governance and funding mechanisms and policies.

So I’m going to keep on banging on about ‘Waste’ a bit more. So here we go…

Roam, if you want to

So I’m going to start today’s note with a Twitter conversation I read involving the founder of Roam Research. For those not familiar, Roam Research is a cult note-taking application (or just a straight-up cult). I started using it November last year and it immediately became my default note-taking app.

Their difference is bi-directional linking. Imagine that every paragraph or dot point you write is its own ‘node’, and these nodes can be linked, referenced or embedded in other pages. And every word you denote as a ‘page’ creates as a page that also includes other references to that term. It’s hard to explain in a single paragraph, but in essence, as you use it, it creates a huge web of connected ideas that you can continue to re-use and build on. With no filing into notebooks.

Now, two months from launching their paid version, they’re apparently doing $1M in annualised recurring revenue. Clearly they have amazing product-market fit.

But these things don’t just come into the world fully-formed.

As Conor points out here, it took two years and 20 versions to iterate to a viable design. And a lot of ‘wasted’ effort on ugly bicycles before they got there. But as interesting here is the idea that “we knew when we got it right [it] would look obvious”.

This is a fundamental dynamic of innovation, and I can give you three tenets we draw from it:

  1. There is no act of genius to invent a viable design, you can only feel around in the dark and iterate to something that does

  2. In retrospect, the final design will look obvious

    And importantly:

  3. Once you know what works, the other attempts will look ridiculous

Product design by Mad Libs

One of the things that’s hardest about doing innovation is the challenge in recognising you probably don’t know.

Per John Loeber’s question above, ‘all the parts existed, why did it take until now to do this?’. Because innovation is combinatorial, taking different ‘technologies’ and piecing them together to create something that effectively solves a problem. The iPod is a famous example of this. Apple didn’t invent anything to create it, it just configured a set of pre-existing technologies the right way.

But ‘knowing’ the right way to piece the parts together is hard.

If we go back to our startup examples, Eugene Wei puts it well here:

“With the rise of Instagram, with its focus on photos and filters, and Snapchat, with its ephemeral messaging, and Vine, with its 6-second video limit for a while there was a thought that new social networks would be built on some new modality of communications […] which is why we have seen a whole bunch of strange failed experiments in just about every odd combinations of features and filters and artificial constraints in how we communicate with each other through our phones. Remember Facebook's Snapchat competitor Slingshot, in which you had to unlock any messages you received by responding with a message? It felt like product design by mad libs.”

It’s a great quote if we break it apart. We’ve got a new technology and clear problem statement: “how do we communicate with each other through our phones”, and then this proliferation of ‘strange experiments’ in trying to solve this question.

To the casual observer, this probably looks something like ‘product design by mad libs’. Apparently random different combinations of features (i.e. mutations) with seemingly no real rhyme or reason – like people just rolling the dice and seeing what works.

But this is the key point: these "strange failed experiments" only look strange in retrospect. Or more accurately, they all looked strange at the time, the obvious ones are only obvious in retrospect. But at the time, all of them were plausible guesses at what would work in the market (or what a ‘fit’ design was).

It's no different from Conor White-Sullivan’s description of Roam, above. “We knew it would look obvious when we found it”.

We only have to look at all the legitimately ridiculous (in retrospect) attempts at bicycles to understand this:


And this holds across industries. Every time we look backwards at an industry – be it bicycles or messaging apps or (soon) fintechs – the right answer is ‘obvious’, and non-viable designs look like ‘strange failed experiments’. Like, “what were they thinking, IDIOTS”.

But looking forward into uncertainty, all we have is the ability to make plausible guesses, and iterate based on what works. And tolerate the Schumpeterian Waste associated with it.

More Morses

This dynamic holds for any creative pursuit.

After last week’s newsletter, I had a conversation with a mate, (Emmy Nominated 👀) Stephen Robert Morse (who has a new documentary out on ABC that you should all watch).

In essence, Morse needs to shoot a LOT of film to find a viable ‘design’. As he said, he shot over 1,000 hours of film to get to a final cut that was 1.3 hours long (a 99.9% waste ratio!).

In innovation, this is equivalent to allowing projects the space and breadth to explore different avenues, have competing ideas and different versions of designs, and not converge too quickly. You need waste within a project like this.

Zooming out slightly, Morse will make a number of films over time and some won’t make it. Some, like Amanda Knox, will be nominated for an Emmy. Some will make it to TV. Some will be sold off to secondary distribution for cheap. Some will be made and never go anywhere. And some will be killed earlier - at trailer or concept stage.

This can be thought of as the project-level exploration in business. There’s going to be a range of bets made, some will pay off big, some will pay off a little, some will lose money. Some will be killed early, some will be killed late. But to find the ‘hit’ we need to invest in a range of initiatives, we need multiple bets.

Zooming out one step further, the industry is full of Morses (a scary thought if you know Morse). Each of them running the same process, trying to find viable designs that generate significant returns from customers. This is the industry-level cycle of innovation – the proliferation of bicycles we saw here, or of messaging apps, fintechs or DTC brands.

Put another way, there is no lone act of genius that can shoot a few hours of film to create an hour-long film, just as there is no lone act of genius that can build a cult note-taking app from scratch. There is no innovation system that can consistently produce acts of genius without a lot that fail. It’s one big, distributed, parallel search going on that is on one-hand effective, and on the other, wildly (and necessarily) inefficient.

Quantity over quality

The most interesting thing about all of this is just how unavoidable it is.

UC-Davis academic, Dean Simonton, has spent his career studying creativity, talent, and greatness. One of his better-known theories is what is the ‘equal-odds’ principle. Broadly, the Equal Odds principle says “that the average publication of any particular scientist does not have any statistically different chance of having more of an impact than any other scientist's average publication”, or as Michael Martinez puts in his 2010 book Learning and Cognition: The Design of the Mind:

Simonton found that the probability of producing a highly recognized work product, such as an influential research article, is roughly the same for all contributors, whether eminent or not. This is what Simonton called the equal-odds principle. What distinguishes highly eminent scholars is the overall volume of works they produce. By sheer dint of productivity, those who reach professional eminence stack the odds in their favor of producing another masterpiece."

In Creative Expertise: A Life-Span Development Perspective (available inThe Road to Excellence: the Acquisition of Expert Performance in the Arts and Sciences, Sports, and Games, Simonton himself expands:

The equal-odds rule is compatible with our overall Darwinian perspective on creativity. It is simply the variation-selection principles operating at yet another level of analysis. Just as ideas within a creator’s head must undergo selection before they are ready to present to the world, those ideas that the creator decides to offer the world will often endure ruthless selection in the minds of others. Only a subset of a creator’s work will actually have an impact. In the sciences, for example, a large proportion of journal articles receive no citations at all by colleagues in the scientific community. What is especially fascinating is that creative individuals are not apparently capable of improving their success rates with experience or enhanced expertise. This longitudinal continuity is consistent with the notion that the variational procedure is ultimately “blind”. Creative persons, even so-called geniuses, cannot ever foresee which of the aesthetic creations will win acclaim. All they can do is maximise their “productive success” by maintaining prolific output across the life span.”

It’s really remarkable research if you stop and think about it. Even the ‘creative genius’ can’t foresee which will work. Often all they are doing is engaging in enough proliferation and waste to create ‘genius’.

This is what I mean when I say the waste is point. The waste is investment in quantity, in bet-making, that in turn delivers quality.

Surviving bias

The problem with all of this is that the waste is essentially invisible. As Bill Janeway, whom I referenced last week, says in Doing Capitalism in the Innovation Economy:

Schumpeter’s process of creative destruction can only proceed by trial and error. We see that which is created through the lens of survivors’ bias and ignore the “hopeful monsters*” that economic evolution has spawned and left behind in metaphorical emulation of Darwin’s process of natural selection

Similarly, as Samir Rath and Teodora Georgieva write in No Startup Hipsters: Build Scalable Technology Companies

After any process that picks winners, the non-survivors are often destroyed or hidden or removed from public view. The huge failure rate for start-ups is a classic example; if failures become invisible, not only do we fail to recognise that missing instances hold important information, but we may also fail to acknowledge that there is any missing information at all.

We have a tendency to fall trap to what is known as ‘survivorship bias’.

Survivorship bias or survival bias is the logical error of concentrating on the people or things that made it past some selection process and overlooking those that did not, typically because of their lack of visibility. This can lead to false conclusions in several different ways. 

Essentially our bias is to look only at the successes, which ‘look obvious in retrospect’, and infer learnings from that.

So when we see a final, working design that looks ‘obvious’, we see it as an act of ‘insight’, putting it down to entrepreneurial genius rather than an ‘evolved’ design that was built on the hard grind of trial and error, experimentation and iteration.

We don't see the ‘waste’ that goes into the final product.

I didn’t know that Roam took 2 years and 20 versions to get to what was a very rough beta when it launched, I assumed it was roughly an idea that came into being fully-formed. And that ignores that it was one of the ‘lucky’ ones, and the number of failed note-taking apps from other entrepreneurs that the market rejected in the process.

To view that design 'correctly', we should actually look at the entire system that created it. If we did this, we'd see the same dynamic of the bicycle: all the iterations and attempts and prototypes tried by the successful entrepreneur. And looking broader, we would see all the other failed attempts by other entrepreneurs (or documentary makers). And we would see the sheer amount of waste invested in finding this viable design.

But because we mostly don’t see these failures, we fail to see just how many attempts were required to get to a viable design. It makes us dramatically underestimate the fuel required for our innovation efforts, i.e. the amount of Schumpeterian Waste required.

In turn, this leads us to over-estimate the probabilities of success of an individual effort, and so we mostly dramatically underinvest in the innovation efforts required to produce success. We bias towards trying to bet on the successful, rather than allowing for the grind to get there.

Put another way, we think we need to explore a little bit. Like, we acknowledge some things will fail. Buuuut not much.

And this leaves us in this sub-optimal equilibrium I described in the article on Schumpeterian Waste – of under-investing in exploration, and therefore not generating enough failure to generate a high probability of adequate returns.

Most organisations live in the valley of death between not investing and keeping the money, and investing enough to generate a good probability of success.

As Bezos said, simply “most organisations are rarely willing to suffer the string of failures required to get there”.

Making it real

I’ll finish with a short example, just to make this real and maybe harden the learnings.

Take ANY organisation’s innovation lab (and trust me I’ve built a few, and worked with a few more) and they systematically fail to tolerate enough waste.

I was working with one of the UK’s big four retail banks on an innovation project, and there were 2-3 quite different proposition and business model directions a concept could take. My answer, obviously, was this was to be expected, and they should take three versions through to prototype in the next phase, and find out what works. (And maybe, given this is evolutionary behaviour, we’d actually find out later that was a different combination of two of them we hadn’t even thought of). The optionality that this would provide would increase the probability of success enormously.

But what were the chances of the bank funding two versions of the same innovation? In essence asking them to invest probably 1.8x as much, knowing that one would definitely fail, and the other may too. Obviously, even with good validation of the underlying demand, they were unwilling to tolerate the ‘waste’ inherent in the redundancy of multiple variants.

In failing to conceptualise the ‘waste’ as investment in success, they asked us to choose one to build out and take into a beta phase. And in doing so, undermined the probability of success.

But here’s the thing. What the bank also did was break the evolutionary dynamic. In essence, they turned us from an innovation system that was working across a population, to an entrepreneur essentially backing a single design.

And this is where most organisations end up: making random bets on individual projects, and then forcing those projects to function as entrepreneurs unable to iterate. All in the name of ‘avoiding waste’.

Next week

Hopefully you can see why I keep prattling on about waste. Because we tend not to see it. Because our organisations can’t tolerate it. Because we systematically underestimate it, and under-invest in it. And because innovation lives off it.

And so, it lies at the very heart of the challenge of building effective corporate innovation.

I never know quite what I’m going to write in the next week’s newsletter (to use my own analogy, to write this one I wrote portions of about 6 different ones before I worked out what the right one was). But I think it will starting to build the foundation of how to build corporate innovation functions the right way. Continuing on this sort-of first principles bent.

As always, if you have a question, ask it. If you liked it, please share it with anyone you think might be interested in it. And if you didn’t, let me know your thoughts.



You can find the rest of the series here:, just click ‘Let me read it first’. (And also subscribe!).

*Hopeful monsters

I just wanted to add a bit about Janeway’s reference to ‘hopeful monsters’.

The term comes from an evolutionary theory by Richard Goldschmidt. Goldschmidt’s hypothesis is that new species develop suddenly through discontinuous variation, or macromutation, rather than just the small changes normally proposed in Darwinian evolution.

The thing about large mutations is, relative to the normal population, the animal, well, tends to look like a mutants. In reality, these sort-of mutant looking variations will more than likely die, but they’re nature’s ‘hopeful’ bets – extreme variations that may create a leap forward in fitness. And thus their wonderful title ‘hopeful monsters’.

In my innovation practice I like to encourage clients to keep their hopeful monsters visible, like a rogues gallery of failures, and to track the familial lineage between them as we would an evolutionary tree.  It helps make visible the waste, but also makes real the evolutionary metaphor, the learnings and what was broken, and the fact that all we can really do is make plausible guesses and iterate.