In October 2014, I coached a (very) cross functional team at Spotify, tasked with discovering what the ideal running experience would be for Spotify’s customers. In less than 3 months, we had defined the minimum product experience needed to differentiate our product in the eyes of our customers and potential customers. The project was a success and in May 2015, we launched our feature to the world, along with a bunch of other new updates like Podcasts & Video shows. If you want to learn more about the Spotify Running feature, take a look at our website: Spotify Running
This blog post outlines how we did it, in a way that is (hopefully) easy to follow. I call it a framework as I relate it to any other framework, that is, a series of tools & rules that you can choose to follow and eventually adapt as you discover what works well for you.
Who is this for?
This framework (1 – What do I mean by “framework”) is for any team that wishes to innovate on their idea(s) in a short time span. It’s super relevant to the product discovery phase of the product development cycle or any group that is looking to find the best path (out of many) to solve a business problem.
What is it?
This is a step-by-step framework that you can choose to follow, or choose to cherry pick from, depending on where you are in your project. I use the word ‘framework’ as I see it being no different to Scrum, Kanban or any other framework that one uses when trying to find an effective way of working, it is not meant to be dogma nor should you need to follow it to the T. I borrowed inspiration from many different books, workshops, techniques and practices.
A massive thanks to the authors & contributors to these books / articles, the most of the practices & inspiration for how we worked in our team:
- Lean UX (Jeff Gothelf)
- Lean Canvas (Ash Maurya, also author of Running Lean)
- The Lean Startup (Eric Ries)
- Google Ventures Design Sprints
- Lean Startup Machine (Workshop Guide, thanks Henrique!)
Setting the team up for success
Before we get carried away, I think its important to consider the make-up of your team and the environment in which they will operate. Here are some tips from my experience working in the Running team, various innovation teams and startups:
Clear your calendars folks!
In my opinion, the most effective teams and startups have most people, doing most of the things, most of the time. If you’re stuck in meeting hell, your ability to execute, adapt, discuss, be innovative and creative will be severely diminished. This has been by far our biggest challenge and in terms of competition with startups, our greatest disadvantage. Your greatest ideas will come from the people you’d least expect, so try to involve all of your team in the work that you’re doing. Your ability to execute will be so much faster when everyone is on the same page.
Have a diverse team, start small & don’t overstaff one role competency
I’m going to over-generalise here, but if you have a team comprised mostly of engineers, then you’re going to argue technical solutions too much, if you’re UX heavy, then you might be worried about crafting the ultimate experience. You risk creating useless work (waste) just to keep people happy.
I’ve found working in the Running team to be such a great experience, as there were generally only 1 of each role. We had a PO, designer, user researcher, marketer, developer and me, the coach. Many of the most successful companies in the world started with a diverse group of people. (Jobs & Woz for example)
Surround yourself with data & insights about the customer(s)
Your other biggest challenge is to make decisions based on evidence and customer feedback, not in your comfort zone in your cosy office environment. My advice is to create an environment where your data, learnings and expectations are clear for all to see. You’re going to be totally confronted when your assumptions and ideas that you hold dear are invalidated, you have to be prepared to swallow that pill and move on. If you’re not reminded of your past learnings, its easy to forget them and continue down a rabbit hole that you once disproved, because subconsciously, you’re still attached to the idea.
Your team area and team activities need to focus and inspire you
Do whatever it takes to have an environment conducive to being inspired. Watch videos, go outside, put up posters, have a nap, break from the status quo. For our team, this was a challenge, working in one of the Spotify building’s areas called “Call Centres” (they aren’t actually call centres), which are hardly inspiring. We eventually got our own team room, but initially this was tough.
The process has 4 major steps, which you’ll start by following sequentially, but will almost certainly return to again and again.
- Team formation & building shared understanding
- Visualising your business plan
- Hypothesis (backlog) creation
- Ideation and Validation (aka Design Sprints)
Team formation & building shared understanding
As a team, we have to constantly agree on which of the MANY ideas we have that we want to validate, we need to do this quickly because we (should) believe that our opinions aren’t as valuable as real user feedback, but we agree that as a team, we’ll choose a better idea to validate with end users than if it were up to one person (see: Lost at sea exercise, basically group think > individual think). This constant state of agreement and disagreement can operate very efficiently (and in a healthy fashion) with a high degree of alignment and shared understanding. This is true for any team, but particularly vital when dealing with such a creative, open-ended endeavour such as ‘innovation’.
Start by building shared understanding of:
- People – Each other
- Product – Why we’re here
- Process – How we (want to) work
Thankfully, I’ve written some blog articles on each of these areas, with examples of activities you can run with your team, many of which I ran with our Running team:
- Bootstrapping an Agile Team [Part 1 – People]
- Bootstrapping an Agile Team [Part 2 – Product]
(although the next step, visualising the business plan, will go a long way in this area.)
- Bootstrapping an Agile Team [Part 3 – Process]
Having done this, you now have a firm foundation of shared knowledge in your team and hopefully you’re speaking the same language. It’s good to disagree on stuff, but your core reasons for being and goals should be the aligned.
2. Visualising your business plan
You’re about to navigate into a huge ocean of uncertainty. You have assumptions about your product that you think will impact the business. These assumptions underpin how you make decisions as you go and you might reach a point where these assumptions are in themselves, invalidated. So, what I’ve found valuable is to put up a lightweight canvas that represents your business assumptions so that you can point to them, reason with them and ultimately change them if necessary.
Here is the canvas that we used (called the Lean Canvas), with the numbering representing the easiest order in which to fill it out:
Its important to mention here that you might not know the answers to everything here and thats ok. At least you can visibly see on the wall that you don’t know the answer to a key part of your strategy, its a constant reminder in fact! Also, you won’t get this nail this straight away, you’ll continually improve it over time as you begin to learn more about your customers, the solution and other aspects of your business plan.
We used the filling out of this canvas as a way to surface our own assumptions about the project. We took one column at a time, then each individually wrote on sticky notes what we thought the answer was, compared our answers and eventually agreed on what it should be (for now). I’d highly recommend involving your key stakeholders in this activity. This was also a great way to surface differences in our thinking and to align on the crucial elements of our plan.
For more information on the Lean Canvas, check out the following links:
So by now, you should have had some lively discussions and a group of individuals that are coming together as a team and have a shared understanding of the problem that they are trying to solve. The next step is to start trying to solve it!
3. Hypothesis (backlog) creation
You might already have ideas, features, prototypes and solutions so far, but the key discipline to apply here is to formulate these ideas as experiments that you can learn from. Remember, our primary measurement of progress in this think-it stage is learning and the framework we’ll use to create these learnings is hypothesis driven. This sounds very scientific but in the early stages your feedback will not be definite, that is, you will receive feedback and signals on which direction to move in, rather than definite proved/disproved results.
Step 1: Create some personas (aka, learn about your customer)
Personas are a great tool to help you empathise with and identify your customers. You may only need one, you may need several. My advice when you’re starting out is: less is more. Start with a very basic persona(s) and use them as a tool to identify and talk to your potential customers. In the early stages of your project, they are your best guess as to who is using (or will use) your product and why. They are meant to change, so don’t spend too much time writing them! Here’s a simple format:
Prior to creating our personas, we had conducted User Research into how our customers run to music. We used this data to form our personas, which are a little more detailed than the above example, although I’d argue you don’t necessarily need to have this level of detail, but it certainly is very helpful. Here is an example of one of our customer personas:
Depending on how much you know in the beginning, you may wish to leave this exercise blank & fill it out as you go, or start with just the demographic that you’re going after then fill in the blanks. I want to stress that even though we had conducted a fairly large scale user research piece (using surveys), we still used it as a compass in our conversations with the customer, not as the be-all-end-all data to base our decisions. Whilst the User Research was helpful, what enabled us to move fast was the team having a shared understanding of the user, which was only possible, in my opinion, by building empathy for the customer with real conversations.
Step 2: Outcomes (2 – What is an outcome?)
Next, you want to discover what you think your persona’s want in the form of outcomes. Individually as a team, write down the outcomes you think Persona X wants to achieve. Present to each other, cluster & theme them.
Step 3: Features
Now do the same thing again, except for features this time. Try to avoid implementation details like platform (mobile vs web), technology (push notification vs sms) etc.
You should have a backlog of hypotheses now. The hard part is to figure out which one you want to test first. Some questions to ask yourself:
- Which persona represents the biggest market opportunity?
- What do we want to learn first?
- What is our riskiest assumption? (that is, if this fails, the whole thing falls on its head)
- What are we the most uncertain about, where do we lack the most knowledge?
Another way to prioritise is to map your hypotheses on a 4 x 4 grid:
You should test your riskiest assumptions first as they provide the biggest opportunity for learning.
And finally, although we call these hypotheses, in the early stages they won’t be super scientific. You’ll be learning and gaining confidence more than proving / disproving a scientific experiment. Your data is likely to be more qualitative than quantitative and thats OK, as a team, you’ll use this information to make decisions on what to test next and why, rather than necessarily be 100% confident in an MVP feature set.
4. Ideation and Validation
The last step is to start running experiments with your hypotheses. To do this, we chose the design sprints framework, inspired by Google Ventures.
Design Sprint Format*:
Day 1 – Understand
Day 2 – Diverge
Day 3 – Converge
Day 4 – Prototype
Day 5 – Validate (and we included a retro here too)
*Note: At the time of writing this article, the above link was what we used. It seems like Google have updated their approach to the 5 day design sprints:
Day 1 – Understand
Take the time to understand the problem you are trying to solve, the people you want to meet and the learnings you hope to gain. You may also need to take this time to book user interviews for the Friday. This day will look very different depending on where along in the project you are, but for us we used it to look at our hypotheses, decide what we want to test, what we want to learn and plan out the week. You can also take this time to review competitors and any existing user research you have.
Day 2 – Diverge
Individually, go away and came up with ideas on how to build a feature(s) around the customer’s desired outcome. Come back together and present your ideas and put them on the wall. Either vote on the one you like the most, then go away, diverge again on a more understood solution, or simply move on to the next day.
Day 3 – Converge
Label the screens, theme and group them, then have a discussion around the ideas. Quickly agree on which ones you like the most and start to flesh out the spec for the prototype. Here you will make decisions like:
- How functional does the prototype need to be?
- Hi-fi? Lo-fi?
Basically this day is for deciding on what to build and fleshing this out in more detail.
Day 4 – Prototype
This is your day to actually build what you’re going to put in front of users. You might ask, “a day!?! that’s not enough time!”. Exactly. This focuses you on doing the least amount of work necessary to get the learnings you desire. You’ll also spend this day designing & building your interview script. Don’t get too detailed with the scripted, think of it more as a logic flow rather than very specific questions you’re going to ask. Conversations aren’t scripted and you want the user to tell you a story. Such questions as “Tell me about the last time you _________” work really well.
Day 5 – Validate (aka, get out of the building!!)
On your last day, you’re going to get out of the building and speak to real people that fit your persona. You’ll first ask them a bunch of questions around them, their behaviour and their needs. After your initial interview, you’ll show them your prototype and get feedback. This is not easy, you’ll need to fight off the urge to tell the customer what its meant to do, or ask leading questions. Conducting user interviews is a skill that you can learn but only really get good at with practice. I’d recommend starting out with some help from someone in user research, but if you don’t have that luxury, read up on some tips for doing this and commit to getting better at it.
Some tips for interviewing customers:
Try to meet up half way through your day to compare notes and share experiences. You might have some really good tips for your colleagues that will help them conduct better interviews.
At the end of the day, go through your learnings. You might wish to put them up on the wall on sticky notes. A recommendation is to answer the follow questions:
- Whats the most important thing we confirmed today?
- Whats the most important thing we disproved today?
- What was the most surprising thing we learned?
You should now have some learnings and new confidences on what to do next. This isn’t easy and you’ll only get better at it with practice and time.
Over time, your prototypes will evolve from simple concepts, to semi-functional lo-fi mockups, to functional prototypes that aren’t integrated, to fully working software. The key is to only build as much as you need to validate your assumptions. Once your assumptions are validated, only then does it make sense to build your features for real.
Good luck and have fun! 🙂
Final Reflections / Learnings
I wrote this article not too long after we delivered our pitch for the MVP of the product. Here are my biggest learnings from working this way:
- Fail fast at your way of working. This stuff is really difficult and frustrating at times, so you’ll gain confidence in yourself by simply getting out there, trying things and talking to customers. The goal for our first sprint? Just do it. That’s it. Figure out what that feels like and go from there. (less tension, less arguments, more/some confidence)
- Visualise everything. You’ll soon forget the learnings you had and slip back into arguing over each other’s opinions. Take the time to physically put up your learnings on the walls.
- Confidence > Validation. The Lean Startup talks a lot about validation of assumptions & so did we. We quickly learned that in the early days, you never really, truly ‘validate’ anything, you simply increase or decrease your confidence in your ideas. It’s both an art and a science.
Finally, a massive thanks to the amazing colleagues that turned these post-it note ideas into reality & put up with my incessant requests to get together and talk about stuff. This feature is by far my proudest accomplishment since joining Spotify and it was an absolute pleasure working with you.
Much <3 to Dariusz, Mateo, Matilda, Rahul, Nick for welcoming me into the team in order to make this happen in the early days.
Much <3 to Will, Pawel, Mattias, Caroline, Dage, Kajsa, Tristan, Minwei (and many more) for making these ideas into reality and shipping an amazing product.
Here’s a picture from my proudest moment throughout the project, it’s members from the team demonstrating the feature to the whole company at one of our company Town Hall presentations.
As this is a mixture of techniques, processes and philosophies, I take no credit for their invention, I am merely mashing them together in a practical way that I feel can help you and your team. In the Pre Squad, we have implemented most of these techniques with varying degrees of success. The reason I felt compelled to share this is because I felt that no “one technique” or process was sufficient for us to be effective at product discovery. I felt that each of these steps represented an important angle to consider when ideating around a business problem.
Here’s a mental model:
Persona – your customer
Output – code, wireframes, questionnaire, concepts etc (easy to measure)
Outcome – a change in behaviour or a result you can see from delivering an output (harder to measure and attribute, can be qualitative or quantitative)
Impact / Metrics – the change we hope to see over time, like a conversion rate (easy to measure, hard to attribute, typically quantitative)
You want to first discover if your customer desires the intended outcome, then discover if your solution is appropriate to achieve that outcome, eventually you’ll discover if it had the intended impact to your business. This sequence of events starts off as cheap to validate, to eventually expensive to build. In other words, why build something properly without first understanding whether or not your customer has the problem (or desired outcome) that you think they have?
For example, losing weight is an outcome that one of our personas wishes to achieve.