The Cost of Retrospectives at Large Animal Games

Well, that December agenda was a bust; who’d have thought that the birth of a child could so heavily affect your schedule?

Here’s a presentation I originally conducted at the NYC Scrum User Group, hosted by Rob Purdie of the Economist, and took place on 12/16/10. I hope you enjoy it and I appreciate any and all feedback, so please leave a comment.

The Cost of Retrospectives at Large Animal Games (PPTX)
The Cost of Retrospectives at Large Animal Games (PPT)

If you can’t download those files, here are the notes from the PPT (no slides).

The Cost of Retrospectives at Large Animal.

VI. This is the Roman numeral for 6, which is the number of days it took my team to create a game for a client. The six days spans from the moment the client approved the project through to the game being live on the client’s website, and includes all game design work. For reference, you can play the game here: http://www.hgtv.com/house-hunters-international/show/index.html

This slide isn’t just for us to brag about how effective we are at Large Animal, nor is it to encourage you to contact andrew@largeanimal.com about client work (although please don’t let me stop you…); it is, instead, here to illustrate an improvement in our team.

Previous projects of a similar size would take somewhere between 10 to 14 days, with the variance usually being due to extra time for client feedback. Through improvement to our team and our processes – improvement derived from frequent and effective retrospectives – we have cut the time for one of these projects down from 10 days (best case) to 6.

I’d like to segue into a short description of my personal history with retrospectives.

Prior to working at Large Animal I plied my trade for a decade in the UK games industry, working for companies such as Warthog, Travellers Tales and, unfortunately, Jester Interactive. At each of these great companies (and Jester Interactive) I was at some point responsible for the creation of a post-mortem for our most recently completed project. Generally, this project had lasted somewhere between 12 to 18 months and was invariably “waterfall”; by that, I mean we were taking various aspects of waterfall development to facilitate the production of the project and calling it “waterfall” much in the same way teams do now with “Agile”.

Even between companies, these retrospectives would follow the same formula: roughly a page on what went right, roughly a page on what went wrong and then roughly a page of how to do things better next time. And the more of these documents that I wrote, the clearer it became that I could save a lot of my time by just changing the title of the document to reflect the latest game, do a search-and-replace for the name of the publisher and just save a copy.

The same mistakes happened time and time again; after all, how quickly can you iterate an improvement when your development cycle lasts at least 12 months and changes style so radically throughout that period (prototyping, grey boxing, bug fixing, etc…)

In hindsight, when I sat down to write these retrospectives we weren’t actually producing a useful document with which to ameliorate future projects; we were just performing an autopsy.

Do you know what my favorite animal is? The octopus. As a kid I was fascinated by anything that lived at the bottom of the sea, and anything that lived at the bottom of the sea and had eight arms was always going to command my attention.

I’m sure you’ve seen the many videos of Octopi doing incredible things? (*cough* http://www.youtube.com/results?search_query=octopus&aq=f *cough*) From using coconut shells as protective armor, changing color to blend into the scenery, working their way through man-made assault courses, opening jars to get to the contents and picking the winner of the world cup, their intelligence is clearly unmatched in the underwater kingdom.

What’s amazing, though, is that for an octopus, reproduction is fatal. Most male octopi live only a couple of months after mating and the mother octopi die soon after their eggs hatch. What this means is that each octopus learns everything for itself. An octopus was not taught how to open a jar, or use a coconut shell as armor or how to pick the winner of the world cup; every generation of octopi has had to learn these tricks for themselves.

This is great for mankind – if each generation of octopi could teach the next generation we would probably be answering to our octopi overlords right now, but, clearly, it sucks for the octopi; doomed to blend in with their ocean surroundings for eternity…

When we started using scrum at Large Animal we were very much like the octopus; each sprint we would make the same mistakes and seem to have to learn everything again, from scratch, with no guiding hand or residual generational intelligence. Sure, we know that doing retrospectives was the right thing to do, but clearly we weren’t doing them properly:

1 – They were unselective; those with the loudest voices tended to be heard the most.
2 – They were uncomfortable. Often the blame would be applied to individuals.
3 – They were shallow. We could identify the surface areas but the underpinning elements were left untouched.
4 – They were broad; we were trying to fix everything immediately.
5 – They were expensive; especially so, as they weren’t working

So where do we go from here?

Between 1985 and 2005, the number of adult North Americans who accepted evolution declined from 45% to 40% (http://en.wikipedia.org/wiki/Creationism#United_States). The justification of that decline is not a debate that I want to get into here, but as I’m about to use a pig-science interpretation of evolution as a metaphor I’d appreciate those of you who don’t believe in evolution to go with the flow, so to speak…

During the Devonian period, which occurred during the Paleozoic Era spanning from 416 to 359.2 million years ago, the pectoral and pelvic fins of lobe-finned fish evolved into legs as they began to walk on land. (http://en.wikipedia.org/wiki/File:Fishapods.jpg)

If our early scrum teams had been lobe-finned fish during the Devonian period I feel we would have been smart enough to:

1 – Notice our budding limbs, and…
2 – Look out of the ocean to the land and to say, “Gee, we’re here in this water surrounded by predators and there’s a whole untapped land of forests and mountains and rolling hills… we need to get there first!”

We’d then all jump out of land only to realize… crap, we can’t breathe. Let’s have a retrospective!

Our retrospective would have gone something like this… “Okay, what went right? Yup, we managed to correctly gauge our acceleration and launch angle to leap majestically out of the water and successfully reach land. Okay, what went wrong? Yes, breathing was definitely an obstacle for us. Okay, how do we improve? Yes, we can remove that obstacle by not jumping out of the water, or by creating magical breathing devices to enable us to survive on land, or by removing the predators in the ocean… man, there’s so many things we could do… not being able to breathe kind of sucks, do you have all of those ideas in the “how to improve” section? Okay, good, let’s sprint plan! The PO has prioritized streamlined fins for swift swimming.”

Okay… if you’re still reading this you’re probably good at dealing with sudden 180-degree shifts in topic, so here’s another one; the above metaphor is a vehicle for me to declare just how much I hate the term “retrospective”.

I mean, here’s the definition of retrospective from dictionary.com, “looking or directed backwards, esp in time; characterized by retrospection.” How is that going to help? You know who is able to get good value out of that kind of retrospective? Bands who want to release another album so they can get out of their recording contracts. Looking backwards at what we did and musing on it is not going to help us at Large Animal.

The fish in the water metaphor I used correlates well to the environment that Large Animal survives in. We’re a small social gaming developer in a large, deep ocean of big, hungry fish; Zynga, Playfish/EA, Playdom/Disney, etc… in order to survive, we need to keep evolving as a company, to keep looking for the step ahead or the next great opportunity, and we need to strike effectively and with purpose when the opportunity presents itself.

Here’s the first step in that process. No more retrospectives. Instead, let’s ask ourselves one simple question…

How can we be even more awesome?

This is the name of the meeting that replaced our retrospectives, henceforth referred to as “HCWBEMA meetings”.

It has a clear aim; everyone who enters the meetings knows the point and what needs to be achieved.
It is more than just a meeting; the aim isn’t to discuss our last sprint, but to demonstrate actual improvement during the next sprint.
We’re not berating ourselves; we understand we’re already pretty damn awesome so we know we’re going to have to work very hard in order to improve and we’re cool with that.
The team decided on the name; everyone understood the drawbacks of what we had been doing and were committed to change. Yes, I am blessed with an awesome team.

Our HCWBEMA meeting agendas change regularly, but we always focus on the following fundamental elements…

Part 1. Among other functions, the parietal lobe assimilates information from its surroundings. The first core step is to engage our parietal lobes.

Here’s how we’re currently doing it at Large Animal. The team will form a semi-circle around a whiteboard and the facilitator will say, “okay, what topics are there that you want us to discuss today?”. We’ll then snake from one end of the semi-circle to the other (i.e. first person to last person, then back again to the first person), with each team member required to throw out one topic each time they’re up; so, if you have a team of 6 you’ll end up with twelve topics on the board, as you’re not allowed to duplicate an existing topic.

What you’ll find is that the first few topics that get thrown out are generally easy surface topics, such as “my computer kept breaking”. However, pretty soon the team will start to dig deeper to find unique topics and you’ll tend to uncover more specific topics for discussion, such as “I didn’t have enough time to properly maintain my computer”.

Once all the topics are on the board, we prioritize with dots (each team member gets three dots) and we keep prioritizing until there is one winning topic. If your team is like mine then they’ll be upset the first few times they do this, especially if their topic is not the one chosen; just remind the team that improvement is iterative (a marathon, not a race) and they’re welcome to raise the topic again during the next retrospective. You will not fix everything at once.

If you know that certain team members have some influence over others, use gentle facilitation (such as conversation, “What do you think of the topics?”) so these people do their prioritization last.

For part 2 we’re going extract some brain juice directly from the temporal lobe, which contains the hippocampus and plays a key role in the formation of long-term memory.

The collation of information relating to the topic of discussion is, in my opinion, the most fragile part of this process. In retrospectives of old it is the time where people can be too intimidated to speak up, may start to blame themselves instead of digging into problems and are likely to start endlessly debating surface problems instead of the deeper, fundamental issues.

For these reasons, I find this part of the retrospective too fragile to be done via group discussion. Instead, everyone on the team gets eight index cards and a sharpie; their mission is to write down eight cards that contain their thoughts on the topic. We’re not looking for solutions here, although they’re welcome; it’s fine to write down questions, emotions, anything relating to the topic.

What we’re trying to do here is to encourage the team to identify what they feel are the most important cards on the topic, which is usually the first two cards people write, and then enable the team to think deeper and identify more fundamental data relating to the topic, which is usually what the remaining six cards consist of.

Then each team member selects six of their cards… and destroys them. The data is not shared with the team; no one knows what was written on those cards. The two cards that remain for each team member are then stuck to the white board. The aim here? A fresh prioritization of the data in front of each team member.

Once everything is up on the white board, give the team time to read and digest everyone else’s thoughts. Don’t go immediately into discussion or people will only talk about their own ideas… give the team time to soak everything up. If they want to group similar cards, let them… let them do whatever they want to help themselves figure it out, just make sure it’s a group decision and not the decision of natural leaders.

The final part of the triad is represented by the frontal lobe. The frontal lobe contains most of the dopamine-sensitive neurons in the cerebral cortex, which are associated with reward, attention, planning, and drive.

One of the mistakes I’ve seen facilitators make is to get “white board fever”; whatever anyone says gets written down on the whiteboard, whether it’s a well-formed comment, a random brain-fart, an off-topic rant, whatever… At this point in our meetings, there’s nothing that’s been written on the white board except for the contents of the Off-Topic Parking Lot. That’s about to change…

…but not that quickly. Let’s understand the rules first. Anything written down on the white board is a binding commitment that the team has made in order to improve; that by our next meeting, we can demonstrate how we have improved in the area that we are about to notate. That the team is agreeing to work hard to meet the demands of our action points.

After explaining those rules to the team, I give them the white board marker and turn it over for group discussion. The team will discuss the topics and the data on the board, and formulate specific, attainable and demonstrate-able ways to improve upon that topic. And these are binding agreements; we write them out, put them next to our scrum board and hold each other accountable for satisfying them at every scrum we do.

Here are some examples of real action points that our team successfully committed to:

No communication over email or Google Wave
Listening to time management podcasts and discussing them as a group
Two scrums a day
Taking personal responsibility for moving verification cards out of verification before the next sprint
Bugs on the scrum board on red cards
Stop breaking stories down into tasks and use definition of done instead
Team Beer every Thursday
Better definition of done for stories
Kill all meetings
Using agendas (sent out ahead of time) and coming to meetings informed

The typical image of evolution shows a primate evolving into man. However, who says you have to stop there? When you’ve evolved to man, surely you’re just a primate on the next rung of the evolution ladder? Rung to rung, sprint to sprint.

Iterative Evolution.

After seeing how our team was improving through the use of this form of retrospective, I wanted other teams to start incorporating these methods to see if it also helped them. However, I felt the worst way of getting my message across the company would be to stand in front of everyone and deliver a presentation like this one.

Instead, I want anyone in the company to be able to facilitate a meeting and understand the fundamentals of what we’re trying to achieve.

On every team there are people who are willing to stand up and facilitate and those who would never, ever put themselves forwards. To enable a completely democratic method of selecting the facilitator, I opened each meeting by revealing a number of playing cards face down on a table, one for each team member. Each person selects a card and whoever selects the ace of spades facilitates the meeting. And they’re dropped in at the deep end, so even if they’ve never facilitated a meeting before it’s their responsibility to ensure the meeting goes well. (And in my experience this has only ever led to fantastic, highly-positive outcomes. Just try it.)

One more important part: The first thing I ask the new facilitator to do is to pick someone who will be their Parking Lot Underling, and will keep the Off-Topic Parking Lot updated with items the team wants to discuss that don’t fit within the current meeting. The point of doing this is that you’re immediately placing the new facilitator in a position of authority; they’ve made a decision that has affected the team. This enables even the shyest, greenest facilitator to take control of the meeting.

A retrospective is a breath of fresh air. In with the good, out with the bad. End product… a team that’s more cohesive, better teamwork, more communicative, more productive, more cost effective and more efficient. A meeting that is better value. There’s no one way of tracking if you’re doing it right. Will your team sprint velocity increase? Maybe… if one of your action points is“don’t be too optimistic with story points?” then your sprint velocity will probably decrease. Over time, however, you will be able to take a snapshot and see an effect in the following areas:

Customer/PO satisfaction
Team harmony
Product quality
Team Work

E-mail me: andrew@largeanimal.com
Follow me: @readytoscrumble
Read me: letsgetreadytoscrumble.wordpress.com

This entry was posted in Uncategorized. Bookmark the permalink.

1 Response to The Cost of Retrospectives at Large Animal Games

  1. Jon Archer says:

    Didn’t click through to the PPTs, just read the text above. Entertaining, enjoyable, useful and masterfully written. Well done.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s