I’m an Agile Coach in Spotify’s Infrastructure & Operations department (IO tribe). Our mission is to allow our development teams to iterate quickly on our product, we look after everything from bare metal provisioning of hardware to networking, build servers, source code repositories, big data infrastructure and monitoring systems to name just a few. The IO tribe is quite large and subsequently we have divided our organisation into logical product focuses, for example, ‘data infrastructure’ (the area I work with) is one example.
Our organisation (IO) has been setup to allow teams to go about executing on their missions with relative ease, the teams are autonomous and there is little friction in them getting feedback from our customers (mostly feature developers). What I had noticed recently was that we weren’t doing much in the way of asking our customers which of their broad use-cases they felt were the most in need of prioritisation. In other words, at an organisation level, where should our focus be? What is the priority for our org in the eyes of our customers? I then enlisted the help of our head of user research to try something new – use their user research techniques internally on our feature engineers, to help us make informed prioritisation decisions.
The result of the workshop was a prioritised (ordered) list of use-cases and a list of the biggest problems that these engineers had with our top 2 use-cases. The following is a facilitation guide on the prioritisation workshop that we conducted on a group of back-end engineers.
Use case & problem prioritisation workshop
What is the purpose of this workshop?
This workshop uses the KJ technique (well, most of it) to derive consensus around which is the most important problem we should be solving. Specifically, it aims to determine what the most important use-cases are that we should prioritize, from the perspective of our engineers.
Once we are aware of the priority of the use-cases, it then aims to determine what the most important problem(s) are with the top 1,2 or 3 use-cases.
What it does not do, is aim to provide solutions to these problems or understand the true nature of them. (e.g. is it really a problem or just perceived that way?)
What will I need?
- Sharpies (at least 1 per person)
- A large room, go for the biggest one you have at hand as you’ll need a lot of room to do this effectively.
- Post-it notes (square ones for problems, wide ones for groups/clusters/voting)
- Square – All square notes should be the same colour (so you’ll need quite a lot)
- Rectangular – You’ll need 2 different colours of the rectangular (wide) notes for group names & voting (example)
- Dot stickers for voting (5 per person per use case)
- Willing participants that have volunteered to be there. Diversity of tribe / squad / tenure helps. 10 – 20 participants to ensure there is a big enough sample size. We chose to segment by competency group, e.g. Backend Engineers, Client Engineers, Front-End, QA, etc
- Use cases prepared beforehand. Here’s some we used, for example:
- Create a test-and-deploy process for my backend service
- Add alerting and monitoring to my service
- Create data from a backend service and visualise a graph from that data
How long will it take?
Our workshop took 2 hours, including approximately 30m for lunch at the beginning before we got started and 45m per use-case.
Introduction & Use-Case voting (15 – 20m)
- Intro: Introduce yourself. Thank everyone for coming. Explain the purpose of the workshop. You may need to mention that you are ignorant to the details that will be mentioned and may come across as ignorant. That’s ok because the aim is not to understand the details but to find out which are the most important.
- Explain use-cases: Present the group with what you believe are the main use-cases that they use on a daily basis. Try to keep this number low, we had 6.
- Opportunity to add anything we’ve missed: Have we missed any use-cases? If they come up with a use-case, ensure its 1 short sentence (like the others) and is an actual use-case, not a specific tool/system or similar to another use-case. Less is more.
- Vote on use cases: Ask everyone to take a post-it note & pen/sharpie and go up to the board where the use-cases are. Ask them to write down on their sticky the 3 most important use-cases that they believe we should be working on, no order of priority this time. Collect up their responses and place a dot next to each use-case corresponding to their responses. You should be left with a prioritised list of use-cases that they believe we should work on. (the reason we ask them to write on the post-it first is so that they don’t influence each other) In the case of a tie, do a re-vote for the tied ones. The reason we choose the top 3 is that we hope to get through 2 – 3 iterations of this exercise and want to focus on the top ones.
- Write up the #1 voted use-case on the whiteboard for all to see. Re-iterate what it is.
Problem Creation & Clustering
- Write stickies (10 – 15m): Ask the participants to write down on stickies all the problems associated with this use case, with no discussion allowed at this point. Some points to mention while they are doing it, our facilitator mentioned these things multiple times:
- “The more detailed / granular the better”
- “Don’t write solutions, just focus on the problems”
- “Don’t worry about whether or not you have duplicated something that someone else wrote.”
- “Don’t worry about prioritising these problems just yet, quantity of post-its is more important.”
- “Can you go into more detail? Can you break it down into more stickies? As many as you can.”
- “Try to keep it to 1 idea / problem per note. (e.g. avoid bullet points, seperate them out)”
- Put them up on the wall: After 10 – 15 minutes, ask them to put up their stickies on the wall.
- “If you’re done, put them up on the wall”
- Read the notes: While they are putting them up, advise them to read other people’s notes and ask each other for clarification, if they don’t understand it. It is vital to make sure that people are reading each other’s notes throughout the workshop for consensus to be built.
- “If you’re inspired, feel free to add more”
- Cluster the stickies 1 by 1: Here are the guidelines / important things to mention:
- Don’t gather them up in your hand, move 1 sticky at a time.
- Move them to the other side of the room / whiteboard (1 at a time)
- Don’t put stickies on top of each other, put them side by side, there is lots of space on the wall, use it!
- Don’t be precious about other people’s stickies, just move them.
- Name the clusters:
- I’d like you to give each group a descriptive title, you can disagree thats fine.
- If you have an alternative title, just put it up. (unless its exactly the same)
- Try not to write sentences, try to stick to a noun(s).
- It’s ok to speak now!
- Note: this step has a hidden agenda, it ensures that each of the participants have read the problem clusters and have agreed what the problem actually is (in terms of naming the problem)
- Combine if necessary, split if necessary
- “We have small buckets, is it possible to combine some?”
- In our workshop, 2 clusters seemed similar. Some weren’t sure whether or not to group them together. So…
- For example, if people worked on fixing xyz but they didn’t improve this other cluster, would that improvement be broken?
- If a cluster has 2 themes, is it possible to split this?
- It’s also possible that there will be some stickies that don’t have a group that they belong to, that’s completely fine.
- “We have small buckets, is it possible to combine some?”
- Prioritise the top 3 problems: Ask each person in silence, without sharing their results, to write down their top 3 most important problems (clusters) that they believe we should fix / prioritise. This is in order of priority #1 = blah, #2 = blah and so on…
- Make the results visible: Give everyone 5 dot voting stickers & ask them to put them up on the clusters that they voted for on their stickies. (3 dots for #1, 2 dots for #2, 1 dot for #3) Place the dots on voting stickies that are next to the cluster names (see photos for example)
- Finally, ask if people feel this is a fair representation…
- Are there any “WTFs?”
This will give you your #1 prioritised use-case with a set of prioritised problems to solve regarding that use-case. It has been prioritised in a non-bias, objective and non-emotional way. When we did this exercise, we had time to repeat the exercise for the 2nd highest use-case in the 2 hours we had set aside, we also had 30 mins for lunch.
Each use-case took approximately 45mins to run through, but it went very efficiently with very little resistance / blockers. I’d allow for some fat here.
Overall, the exercise was successful. It gave us insight into what our customers want us to prioritise in general, regardless of how our organisation is structured (focus on the customer!). We’ve used this feedback to better staff up one of our product areas in IO and to set our own priorities for the quarter. We now plan to continue this workshop with other competency groups and other cities (e.g. NYC).
Hope you enjoyed this article and please leave any feedback you have in the comments below.