Matt Wynne

Tea-driven development 🫖

I have a problem with acceptance criteria, for two reasons.

First of all, their definition in the literature is hazy, which frequently means they're awful. They're either too vague and have gaps, or they're too detailed and still have gaps!

Second of all, user stories were always meant to be a token for a conversation.

A token. For a conversation.

What I see people do, time and time again, is use acceptance criteria as a substitute for a conversation. They add the story into their Jira backlog, and they add some acceptance criteria, and then they throw it over the wall to the developers to code it up, no questions asked.

This is not how it's supposed to work.

A token for a conversation

Let's start by going back to 2001, when Ron Jeffries, one of the originators of eXtreme Programming (XP) wrote a seminal article describing the “three critical aspects” or “three Cs” of User Stories: Card, Conversation, Confirmation. Since User Stories originated in XP, and Ron is one of the three people who invented that methodology, it seems important to listen to what Ron has to say here.

Bear in mind we're talking about the late 1990s, the pre-Jira times. Everyone sat in the same room, and the planning tools were analogue.

User stories are written on cards.

I spent most of the 1990s in nightclubs and ski-towns, but I'm old enough to remember a time when we kept our agile team’s plans on physical index cards on the wall of our team room. One advantage of an index card is that there's only so much room to write on. No comment fields. You can't go into too much detail.

The card does not contain all the information that makes up the requirement. Instead, the card has just enough text to identify the requirement, and to remind everyone what the story is.

And, crucially,

The card is a token

So the card represents the need for a future conversation. It's a way to defer work, to allow you to focus on the detail of what you're doing now and next, without forgetting what you'll need to be doing after that.

Even though we rarely use physical cards to represent user stories these days, this metaphor is still important. There is no point in going into too much detail too soon. It's wasteful and potentially even counter-productive.

The second “C” comes when it's almost time to do that work, when you do need those details:

The requirement itself is communicated from customer to programmers through conversation: an exchange of thoughts, opinions, and feelings.

I can't emphasise this enough, but I'll try: The requirement is communicated from customer to programmers through conversation.

Why conversation? What's wrong with just writing down the details of the requirement, telling the user's story, in the Jira ticket? We're no longer limited by physical cards, so what's the problem?


Let's assume, for a moment, that the project you're working on is actually significant. That it adds something fresh, does something innovative. Something that's never been done before.

If that's the case, then I would argue that the most important part of your work is discovering all the things you don't know yet. The gaps. The unknown unknowns.

The rest is easy.

Yet time and again, we see product owners spending inordinate amounts of time on the easy bit: writing down everything they already know about the problem as acceptance criteria in their Jira tickets, and paying scant attention to trying to deliberately discover the unknown unknowns lurking in their backlog.

This is one huge reason a conversation is so important. Although the product owner naturally has the best grasp of the user's problem, there's no way they can have thought through everything. Even if they could, how can they know by simply writing some words in a Jira ticket, that their understanding has been perfectly transmitted to the minds of the people who will actually be writing the code.


By having a direct conversation with the developers, the product owner gets two benefits: firstly, many minds scrutinize their emerging mental model of the problem to be solved, giving the team the best chance of discovering as many of those unknown unknowns as possible. Secondly, they get an opportunity to check for understanding, to make sure the developers see the same thing as they do. Software is made of thoughts, and to build anything useful we need to make sure those thoughts are at least roughly aligned.

But let's be clear: working software is the goal; everything else is waste.

The game here is to try and do just enough planning, just in time. The beauty of a well-timed conversation is that we barely need to write anything down at all. The programmers carry the understanding that emerged from that conversation straight into the codebase.

Often it does make sense to take down some notes so that we can remember the details of what we decided in the conversation. So what's the best way to do that?

How can we make our acceptance criteria awesome?

Rules and Examples

Not to pick on anyone in particular, but a significant detail that was missing from the descriptions of acceptance criteria in books I read as an impressionable young scrum-master was the distinction between rules and examples.

Acceptance criteria, or conditions of satisfaction as Mike Cohn calls them, would usually look something like this:

Story: As a new user, I want to sign up for an account using my email address and password, so that the system will remember my details for my next visit.

Acceptance criteria:

  • Password must be a strong password
  • Can't create two accounts with the same email address
  • Email address must be confirmed

These look fine, right? And that's precisely the problem. They are hiding the gaps.

That's because they are rules. The problem with rules is that they lull you into a false sense of security, give you a feeling that everything is OK. Yet our experience should tells us that it's probably not: What exactly do we mean by a strong password? How exactly will the email address be confirmed?

This didn't become clear to me until I read Liz Keogh's blog post, acceptance criteria vs scenarios. Here's how Liz describes the importance of examples:

Talking through scenarios is, for me, the most important aspect of BDD. That’s how we discover whether we have a shared understanding or not; by using specific examples to illustrate our understanding or discover our ignorance.

Examples are great. They are vivid and real and they fire people's imaginations. This is exactly how you give yourselves the best chance of seeing the gaps.

But on their own, examples are like a photograph without a caption. They lack context. One of the worst examples of acceptance criteria I've seen is when a product owner takes it upon themselves to write reams of Given-When-Then scenarios into a Jira ticket.

What you need is rules and examples. This is where Example Mapping comes in.

Example Mapping

One problem with the conversation is that it can often lack structure. People generally hate long rambling meetings, and for good reason.

The beauty of example mapping is that it gives you a clear structure to the conversation, and visual feedback on what you're learning and deciding in real time as the conversation progresses.

Neither the rules or the examples have to be exhaustive, but they should reflect the conversation, and your shared understanding, back at you.

Example mapping adds one more element to our awesome acceptance criteria checklist: Questions.

Many people have told me that the explicit capturing of questions in example mapping has had a profound effect on their team's behaviour. This is where we capture the gaps, the things we are just starting to realise we still need to figure out.

I won't go into detail about example mapping in this post, so let's end with a recap about what we've learned so far about what makes for awesome acceptance criteria:

  • Derived through conversation
  • Include both rules and examples
  • Include questions we couldn't answer yet

Coming back full circle to Ron's original post, the third “C” is Confirmation, and specifically the acceptance tests that the team will automate on the customer's behalf in order to know when the story is done. Examples are a great basis for these tests. Perhaps we'll write more on that another time.

If you want to learn more about example mapping, we’re teaching an online, cohort-based course. Sign up here.

As I consider my future, I’ve been thinking about what I want the next chapter of my working life to look like.

Here are the things that really matter to me, in rough priority order:

  1. Provide a sustainable livelihood for my family

  2. Enjoy a fulfilling, inclusive, creative and fun work environment

  3. Solve an important problem for society

  4. Contribute to the commons

  5. Teach others what I know

#1 is a given. I realise I’m lucky to be able to expect this, but I do.

I’ll put up without #2 for the right price, but I won’t stick around for long unless I get the influence to create that kind of environment for me and everyone else.

I am really puzzled about #3. It seems like so much of the work available to me is simply making the engine of capitalism slightly more efficient. I’m OK with that – it’s the paradigm we live in, like it or not – but I deeply yearn to do something more political, to use the skills I have to actually improve society.

#4 matters to me a lot, but probably not quite as much as #3. I’m not really sure I believe in proprietary code at all anymore, and at the same time I’m really curious about how we can make open source software pay, to make it more sustainable instead of just relying on corporate philanthropy.

#5 gives me a great deal of pleasure, and I guess is really a part of #2.

If I could find a role with all five, I’d be very happy!

Back in 2010, David Chelimsky was re-writing RSpec, and had the idea to use Cucumber scenarios to drive the development. The RSpec website was not much at the time, and David’s intention was to generate user documentation automatically from those scenarios.

I was invested in the Cucumber open source project as a core team member, working as an independent consultant, and just starting to think about how we might be able to build revenue-generating tools around it.

So, together with an RSpec team member Justin Ko, I started work on Relish. Over time, Justin drifted away from the project, and my friend John Hardcastle got involved to help with the UX.

For a while, I was really into it. The service was growing, we had users who wanted to install it on-prem, and there was an active UserVoice forum (remember them?!) where people left their feedback on what else it should do.

In 2013 I started Cucumber Ltd with Aslak Hellesøy and Julien Biezmans, and Relish became part of the IP of that company. We did some lightweight maintenance on it, but our attention became focused elsewhere on building shiny new things – a collaborative editor for Gherkin based on CRDT, which we called Cucumber Pro – and building the training business that was bringing in the money.

To my frustration, each new iteration where we’ve tried to build a commercial tool for BDD practitioners (Cucumber Pro, Jam, CucumberStudio, Cucumber Reports) has never matched up to Relish for the basic use-case of publishing read-only documentation. We never quite got there before the project was shelved and we moved on to the next thing. So the RSpec project have remained its most loyal and biggest users.

Fast-forward twelve years, and the pages are still visited by hundreds of Ruby programmers every day.

Relish's IP and hosting is now owned by SmartBear, who acquired Cucumber Ltd in 2019.

Mostly, Relish has “just worked” and hasn’t caused us too many problems. There has been the occasional glitch that needed a touch of heroku run rails console but we’ve mostly been able to get away with paying it very little attention.

Until this week.

Unfortunately, I’m no longer in a position to be able to help. We have started some early conversations with SmartBear about releasing the IP for Cucumber, which I assume would include Relish’s codebase, but it’s early days.

My hope would be that we can open-source the code for Relish (in retrospect I wish we’d done this ages ago), find some sponsors for the hosting, and let it continue to serve as a platform for the RSpec team. At the very least they could salvage the rendering code to generate their own static documentation.

For the time being though, I’m afraid we’re a bit stuck.

My friend Jeff on a skin-track climbing towards the John Carter peak here in BC in January 2023 with the Kokanee Glacier's peaks in the background

Well, this is new.

After ten years building a company around the Cucumber open source project, I suddenly find myself out of a job.

I was laid off last Friday by SmartBear, who in June 2019 acquired the company I co-founded back in August 2013. It’s no secret that I had recently been involved in a research project to determine Cucumber’s future direction at SmartBear. As the last remaining employee lucky enough to be paid to work on the open source project full-time, I guess the powers that be had already made their minds up about what the outcome of that research would be.

I figured we might be in the end-game, but I hadn’t expected it to come around so suddenly.


I was actually on vacation, with no internet, while all this happened last week, so I was not prepared. I’ve lost access to my email address, meaning there are various services (like Cucumber’s community Slack) that I can’t sign into with my normal account. My GitHub notifications for the Cucumber project were also going into that inbox too.

So if you’ve written to me at that email address, or you’ve pinged me in GitHub or Slack and haven’t got a response, that’s the reason why.

Wow. Are you OK?

Emotionally, I’m OK; thanks for asking 💖

While it’s been interesting, from an anthropological point of view, to spend so much time on the inside of a private-equity backed US company, the culture didn’t suit me well I am pretty relieved—even excited—to be out. I also did a lot of grieving around the time of the acquisition, and I have a lot less attachment to the Cucumber project or my role in it than I used to.

Still, I feel a sense of responsibility towards it, and I hold a great deal of respect and affection for the volunteer members of the community who put so much of their free time into nurturing the project.

I’m not taking this personally. SmartBear has grown significantly since we were acquired in 2019, and the scale of their ambitions has grown too. Although our research showed there is a market for commercial tools built for the Cucumber audience, that market probably just isn’t big enough for a company like SmartBear. There’s no compelling reason for them to continue sponsoring an open source project that doesn’t directly contribute to the bottom line.

I’m optimistic that SmartBear will have the good sense to avoid doing a Hudson, and I’ll do what I can to try and ensure that assets such as the domain, trademarks, training videos, reports service and so on will be transferred into community ownership in due course. I’ve had some positive conversations with senior SmartBear executives about this before I left, and have had some reassuring signals since as well. So I’m hopeful, but we’ll see.

Still, without a significant commercial sponsor, the future of the project is uncertain right now. Would your company like to step in and help?

What are you going to do now?

Almost fifteen years since my first commit to Cucumber, I want to take some space to consider what the next chapter of my work-life will look like; to figure out what’s important.

If you’ll indulge me, I may be using this new blog to think out loud about that a little. It uses the service, which I rather like so far.

I’m going to try and keep the laptop closed as much as possible for the next few weeks, but I’d love to hear from you, whether you think you have an interesting work opportunity for me, or just want to say hi.

In the short term, I’m interested in providing remote coaching to teams in and around the Pacific timezone (I’m based on BC, Canada) who want to apply skills like ensemble programming, test-driven development (TDD), domain-driven design and continuous delivery to interesting and worthwhile problems.

If you want to chat, you can reach me at or grab some time in my calendar.

You can find my old tea-driven development blog here.

Enter your email to subscribe to updates.