Defining acceptance test criteria
Posted by andy in : Agile, Testing on December 8, 2006The best part of XPDay is talking to people about what they do. I managed to catch Joe Walnes and Dan North’s talk on Awesome Acceptance Testing.
For as long as I can remember, Chris Matts and Dan North have been saying A user story’s behaviour is simply its acceptance criteria. If the system fulfils all the acceptance criteria, it’s behaving correctly; otherwise, it isn’t.
What I had missed is their really nice acceptance criteria template.
Given some initial context,
When an event occurs,
then ensure some outcomes.
For example
Given Joe has a current account with a balance of £1 and an overdraft limit of £100,
When Joe withdraws £50.00,
then the balance will be £49.00.
This is so expressive and so wonderfully simple that anyone in the business can write them. Thanks to Joe for showing me this.
I like this rigour and yet I am also wary that this approach can lead to people emphasising specification over empiricism. Is a system behaving correctly because all the acceptance criteria pass? Well, not if the acceptance criteria is wrong. How do we know the acceptance criteria was wrong? Well when we demonstrated it, we received negative feedback. When we tested it with users, they were not able to accomplish their goals.
I agree with you.
I also use lots of “Show and Tells” and encourage people to play with what we have built as much as possible.
It’s just that I like to have the conversation “how do I know this story has been completed to your satisfaction” so I have a sporting chance of catching the essence of what they care about. This is one technique that helps me do it.
I would caution that these be presented with a good dose of regression tests and standard test cases, lest users and managers become convinced that acceptance tests like these are a substitute for rigorous system testing.
I guess I did not explain that these are converted into executable regression tests.