I was recently undertook some trend analysis on the Spotify health check with the goal of identifying areas which are affecting multiple squads which we may want to provide help with. This was a really interesting exercise as with qualitative data it doesn’t easily lend itself to trend analysis.

If you aren’t au fais with the Spotify health check model start here then come back. As suggested in the Spotify model the health check is currently being run every quarter.

 

The analysis took two forms, one looking at the improvements made across teams and the other graphing the health.

Improvements

The second question in the Spotfiy model for each category is to ask the squad whether things are improving, staying the same or getting better. With this data I started each improvement from 0 and each squad positive vote was +1 so if we improved 3 quarters in a row the score got to 3. Below is a completely fake example Teamwork graph showing generally the 3 teams are all improving, probably teamwork not something to worry about.

An example of a something which may need a little deeper debate and discussion with the squads looked like this. Straight lines with the odd dip as things got worse.  Each squad negative vote was -1.

 

Health

For health I plotted Good as a 1, somewhere in the middle as 0, and -1 as bad. Rather than making this a cumulative. An example of something that appeared healthy looked like this with lines consistently at good.

An example of something that may need a deeper debate looked something like below. These were characterised more by static lines typically in the ok and dipping into Bad on occasion.

Learnings:

More data would be useful, maybe not every sprint but more than quarterly

Make it very obvious to everyone that this data is not for comparison of teams, this is not a maturity model and we are not comparing. Removing the squad names from the graphs helped as this meant squads couldn’t compare each other.

I did consider graphing the two items together but thought that could produce potentially strange results. When we get more data we will try this. I didn’t want to confuse what already wasn’t an immediately obvious set of stats.

Things to Ponder:

Gamification of the totals. You could sum up the totals for health and improvement for each squad and then put out a league table to promote competition to be better. Would this promote intrinsic motivation and push the teams on to improve or would this mean that the teams gamed the test and simply answered good when things really weren’t so. I definitely wouldn’t attach any sort of reward to the game!

 

Update April 2017:

We have changed the questions we ask, broadly the categories are similar but there are a few things which were not as relevant for us. We run this monthly and use Google forms to make the data as anonymous as possible. We also include a comments section to allow anyone to ask for help. The value of the survey to those attempting to help the team appears to have increased. It takes less time for the team to complete and the groupthink element appears to be removed with people filling in at their own convenience. Learning: Optimise the health check for your context