Awesome Acceptance Criteria

I have a problem with acceptance criteria, for two reasons.

First of all, their definition in the literature is hazy, which frequently means they're awful. They're either too vague and have gaps, or they're too detailed and still have gaps!

Second of all, user stories were always meant to be a token for a conversation.

A token. For a conversation.

What I see people do, time and time again, is use acceptance criteria as a substitute for a conversation. They add the story into their Jira backlog, and they add some acceptance criteria, and then they throw it over the wall to the developers to code it up, no questions asked.

This is not how it's supposed to work.

A token for a conversation

Let's start by going back to 2001, when Ron Jeffries, one of the originators of eXtreme Programming (XP) wrote a seminal article describing the “three critical aspects” or “three Cs” of User Stories: Card, Conversation, Confirmation. Since User Stories originated in XP, and Ron is one of the three people who invented that methodology, it seems important to listen to what Ron has to say here.

Bear in mind we're talking about the late 1990s, the pre-Jira times. Everyone sat in the same room, and the planning tools were analogue.

User stories are written on cards.

I spent most of the 1990s in nightclubs and ski-towns, but I'm old enough to remember a time when we kept our agile team’s plans on physical index cards on the wall of our team room. One advantage of an index card is that there's only so much room to write on. No comment fields. You can't go into too much detail.

The card does not contain all the information that makes up the requirement. Instead, the card has just enough text to identify the requirement, and to remind everyone what the story is.

And, crucially,

The card is a token

So the card represents the need for a future conversation. It's a way to defer work, to allow you to focus on the detail of what you're doing now and next, without forgetting what you'll need to be doing after that.

Even though we rarely use physical cards to represent user stories these days, this metaphor is still important. There is no point in going into too much detail too soon. It's wasteful and potentially even counter-productive.

The second “C” comes when it's almost time to do that work, when you do need those details:

The requirement itself is communicated from customer to programmers through conversation: an exchange of thoughts, opinions, and feelings.

I can't emphasise this enough, but I'll try: The requirement is communicated from customer to programmers through conversation.

Why conversation? What's wrong with just writing down the details of the requirement, telling the user's story, in the Jira ticket? We're no longer limited by physical cards, so what's the problem?

Gaps

Let's assume, for a moment, that the project you're working on is actually significant. That it adds something fresh, does something innovative. Something that's never been done before.

If that's the case, then I would argue that the most important part of your work is discovering all the things you don't know yet. The gaps. The unknown unknowns.

The rest is easy.

Yet time and again, we see product owners spending inordinate amounts of time on the easy bit: writing down everything they already know about the problem as acceptance criteria in their Jira tickets, and paying scant attention to trying to deliberately discover the unknown unknowns lurking in their backlog.

This is one huge reason a conversation is so important. Although the product owner naturally has the best grasp of the user's problem, there's no way they can have thought through everything. Even if they could, how can they know by simply writing some words in a Jira ticket, that their understanding has been perfectly transmitted to the minds of the people who will actually be writing the code.

Waste

By having a direct conversation with the developers, the product owner gets two benefits: firstly, many minds scrutinize their emerging mental model of the problem to be solved, giving the team the best chance of discovering as many of those unknown unknowns as possible. Secondly, they get an opportunity to check for understanding, to make sure the developers see the same thing as they do. Software is made of thoughts, and to build anything useful we need to make sure those thoughts are at least roughly aligned.

But let's be clear: working software is the goal; everything else is waste.

The game here is to try and do just enough planning, just in time. The beauty of a well-timed conversation is that we barely need to write anything down at all. The programmers carry the understanding that emerged from that conversation straight into the codebase.

Often it does make sense to take down some notes so that we can remember the details of what we decided in the conversation. So what's the best way to do that?

How can we make our acceptance criteria awesome?

Rules and Examples

Not to pick on anyone in particular, but a significant detail that was missing from the descriptions of acceptance criteria in books I read as an impressionable young scrum-master was the distinction between rules and examples.

Acceptance criteria, or conditions of satisfaction as Mike Cohn calls them, would usually look something like this:

Story: As a new user, I want to sign up for an account using my email address and password, so that the system will remember my details for my next visit.

Acceptance criteria:

These look fine, right? And that's precisely the problem. They are hiding the gaps.

That's because they are rules. The problem with rules is that they lull you into a false sense of security, give you a feeling that everything is OK. Yet our experience should tells us that it's probably not: What exactly do we mean by a strong password? How exactly will the email address be confirmed?

This didn't become clear to me until I read Liz Keogh's blog post, acceptance criteria vs scenarios. Here's how Liz describes the importance of examples:

Talking through scenarios is, for me, the most important aspect of BDD. That’s how we discover whether we have a shared understanding or not; by using specific examples to illustrate our understanding or discover our ignorance.

Examples are great. They are vivid and real and they fire people's imaginations. This is exactly how you give yourselves the best chance of seeing the gaps.

But on their own, examples are like a photograph without a caption. They lack context. One of the worst examples of acceptance criteria I've seen is when a product owner takes it upon themselves to write reams of Given-When-Then scenarios into a Jira ticket.

What you need is rules and examples. This is where Example Mapping comes in.

Example Mapping

One problem with the conversation is that it can often lack structure. People generally hate long rambling meetings, and for good reason.

The beauty of example mapping is that it gives you a clear structure to the conversation, and visual feedback on what you're learning and deciding in real time as the conversation progresses.

Neither the rules or the examples have to be exhaustive, but they should reflect the conversation, and your shared understanding, back at you.

Example mapping adds one more element to our awesome acceptance criteria checklist: Questions.

Many people have told me that the explicit capturing of questions in example mapping has had a profound effect on their team's behaviour. This is where we capture the gaps, the things we are just starting to realise we still need to figure out.

I won't go into detail about example mapping in this post, so let's end with a recap about what we've learned so far about what makes for awesome acceptance criteria:

Coming back full circle to Ron's original post, the third “C” is Confirmation, and specifically the acceptance tests that the team will automate on the customer's behalf in order to know when the story is done. Examples are a great basis for these tests. Perhaps we'll write more on that another time.

—

If you want to learn more about example mapping, we’re teaching an online, cohort-based course. Sign up here.