The last decade has seen a growing interest and uptake in government of innovation methods that focus on becoming more agile, bringing iteration into development processes and accelerating organisational learning. These methods aim to use public money more efficiently by finding out what works and what doesn't as early as possible.

You may have come across a range of different approaches for testing and developing solutions, including proof of concept, prototypes, pilots or even Minimal Viable Products (MVPs). They all have something in common and aim to:

• Identify and challenge assumptions by “making things real” and testing them
• Accelerate learning by finding out what works and what doesn’t without investing large amounts of time or resources

Confusion around terms

Often these terms are used interchangeably. In meetings or workshops, we often notice that ‘prototyping’ is used synonymously for ‘testing an idea’, even if that's actually testing a component (proof of concept) or validating a solution before implementing or scaling (pilot).

These different terms can be confusing to anyone not familiar with innovation jargon. To add to matters, these terms are used in different places with different meanings; whether that's engineering, product development, software development, design or (public) innovation.

This blog is our attempt to explain what they mean. We have developed some descriptions and diagrams to add clarity and help you understand the differences. These are not necessarily universal definitions that everyone should use, but how we would explain them for the work we do in public innovation.

Proof of concept

A proof of concept often involves a small exercise to test the real-world potential of an incomplete idea. This isn’t about delivering the idea, but demonstrating whether it is feasible. It should be used in the early stages when you first have an instinct about an idea. A proof of concept shows if a product, feature or system can be developed, whilst a prototype shows how it will be developed. For example, a proof of concept might be used to test a technical feature of an online service by quickly building a working model.

Prototype

A prototype is the visible, tangible or functional manifestation of an idea, which you test with others and learn from at an early stage of the development process. For example, a prototype might be used to test an idea for a service touchpoint – the point of interaction between a citizen and public service. This might be a mockup of a website or a role playing exercise between a citizen and frontline employee to test out a service script.

Prototypes should be used when you have a hypothesis about a solution, but there is still uncertainty about how it looks, feels and works. Insights from testing can then be used to improve the idea. By developing and improving the prototype, you can maximise what you learn and refine your idea. This helps you move from a version with little detail or functionality (like a rough draft that illustrates the idea) to a version with much more detail and functionality (giving test-users a better sense of how it works).

Prototypes are also a way to engage your stakeholders to develop a shared vision or common ground for a solution. With the uptake of design-driven innovation in governments, we’ve seen prototyping become a popular method to test out and develop ideas with citizens.

Pilots

Pilots are often used as the first stage of a new policy or service rollout and are the more accepted norm in government. Rather than a test or experiment, they are a ‘live’ activity, usually with a small group of real users or citizens receiving the new service. For example, the UK government is currently testing a mobile app with EU nationals working in higher education who want to register for settled status. This helps the government to identify and fix any teething problems before the settlement scheme is opened for full operation in 2019.

Pilots should be used when you believe you have an effective solution and are looking to iron out the creases and understand how it works in reality. By offering a partially implemented concept to a limited population, it is possible to see what actually happens. This is useful when preparing to scale a solution to a wider group. Pilots, however, are ultimately measured by success or failure, and there is usually only room to make minor tweaks.

MVP (Minimum Viable Product)

An MVP allows you to accelerate your learning about a possible solution whilst using minimal resources. It does this by testing only the essential core of your concept (rather than the full solution) with real users in practice. This means that you can find out early on if there is an actual need or demand for the solution, what is working and what isn’t, and make any adjustments accordingly (this is called pivoting in the lean-startup scene). MVPs are often associated with technology, and aren’t currently common in public innovation, but may have great potential for situations that deal with a fast-paced political development cycle or require ongoing improvement of public services and public policies. MVPs are about using fewer resources and minimal effort to gather insights and obtain feedback on potential changes.

The differences between these methods are illustrated in the diagram and table below. In general:

• They are used at different stages in the development process
• They need more or less resources.
• They vary in fidelity, level of detail or completion
• They may have different scope (i.e. testing parts or the whole)

In the diagram below we have mapped out the four different methods across two axis. The horizontal axis indicates how “complete” it is, ranging from idea to implemented solution. And the vertical axis indicates the “unit of testing”, whether that be parts (elements of the solution) or the whole.

Proof of concepts and prototypes are usually developed and tested within a controlled environment – such as in a workshop or lab workplace – although some prototypes are tested out in the field. Pilots and MVPs, on the other hand, are usually tested in the real world. The table below should make these differences clear.

Although the diagram above suggests there is a certain sequencing to how these methods can be used, there isn’t a specific order. These methods are usually part of a wider innovation process that may include only one or more of them.

Where to learn more

These methods are still emerging in government, so there are no comprehensive ‘how to’ guides yet. Nevertheless, there are a few on prototyping, such as ‘Designing for Public Services’ a guide we developed with IDEO, and the Open Policy Making toolkit by the UK’s Policy Lab. It’s also worth looking at ‘Lean Impact’ by Ann Mei Chang, who has adapted the principles of a lean startup and MVPs from the private sector to the public sector.

Found this a useful explainer?

Hopefully it was! But as with all the work we publish, please get in touch with us on twitter, or email us if you have any thoughts or feedback on how to make this more useful.