The Perfect Customer
Posted by andy in : Agile, Teams, Testing on March 25, 2008I enjoyed reading Keith Braithwaite’s post on tests and gauges. I like the way Keith uses the the metaphor of gauges in metalwork to describe Fit style tests:
You don’t work anything out from a gauge. What you do is apply it to see if the workpiece is within tolerance or not. And then you trim off, or build a bit up, or bend it a bit more, or whatever, a re-apply the gauge. And repeat.
I agree with Keith. The fit tests I write are business facing and for the benefit of the business customer.
On a recent project our customer got so enthusiastic about our fit tests he got extremely upset when I implemented a story without a fit test. He refused to let the system go live until we had the fit test in place.
The story in question was very technical and involved sending a particular xml message to an external system. We just could not work out what a fit test would look like for this type of requirement. Placing the expected xml message, with all it’s gory detail, in the fit test would not have been helpful as this is a technical artefact and of no interest to the business. We could not work out what to do. The customer was not around to discuss this so I just went ahead and implemented the story (very naughty!).
What the Customer wanted was to be sure that we were sending the correct product information in the xml message. To resolve the issue, I suggested that we have a Fit test that shows how the product attributes get mapped onto the xml message using xpath; although I still thought this was too technical for a business user.
We gave the Customer a couple of links to explain what xpath was so he could explore if this was a good solution for him. To my amazement, he was delighted with xpath (I now know who to turn to when I have an problem with xpath) and filled in the fit test.
The interesting bit for me (and why I’m posting it on the blog) is that as soon as he knew what the message looked like and how it was structured, he realised that it did not really support the business - we were sending information that was outside our scope of our work and should have been supplied by another system. He was also skeptical about the speed the external team could add new products due to the complex nature of the xml.
Most agile people tell this story to think we have the “perfect customer”!
[…] Andy Pols: the customer who wouldn’t deploy without a test (via Keith […]
This application of the information obtained from Fit tests is different from what we usually hear about. Usually we hear about how swell it is that there are hundreds, thousands, of confirmatory examples that do more or less the same thing every time we run them, over and over. It’s unusual to hear that the customer learned something and used the information obtained from creating the test. (That’s not to say that it never happens. In my experience it happens quite a lot. It’s just unusual to hear about it.)
This is much closer to my view of what is really important about testing: discovering and revealing information so that people can make informed decisions. Most of the time, we hear about something different: confirming and validating information so that people can feel reassured knowing that last week’s tests are still passing this week. That might be reassuring, but it has enormous potential for self-deception. We need always to ask if our tests are helping us to learn, not just helping us to sleep.
Nice story.
—Michael B.
The main benefit our team gets from FitNesse is this communication - we can express tests in the language of the business/customer. This is a great story. We have never gotten our customers to write FitNesse tests directly, although they will write tests in spreadsheets and we can convert them.
@Lisa: If they are happy to write tests in spreadsheets, then why not use FitLibrary to parse them directly? I’ve had some success in stealthily getting customers to write tests themselves without realizing it this way.
I hear a lot of, “That’s too technical for business folks.” Interestingly enough, these objections rarely come from the business folks themselves, who seem fine with whatever technical stuff I care to throw at them, but from technical people.
cjs@cynic.net