In the ongoing series of posts on Agile Test Case Management (which is pronounced ‘figure out things for my talk on Thursday’) I am now heading onto slightly shakier ground with Session Sheets. Session Sheets are the tracking method of proposed in Jon Bach’s paper Session-Based Test Management. SBTM itself is a much larger discussion and one that is beyond scope here. What is in scope is the sheets themselves as they are, according to Jon, the magic ingredient of SBTM.
Sheets themselves are…
- plain text (not .doc — that is a binary format)
- tagged for categorization and organization
- machine readable
- stored in revision control
- highly customized to the team and context
That could describe, well, almost anything. And it is supposed to; you are doing it wrong if you don’t tailor it to your environment. But what distinguishes a ‘good’ sheet and a ‘bad’ sheet. Again, beauty is thin the eye of the beholder and the devil is in the details – or rather in the charter. A session’s charter is what guides the testing and provides the mission. If you get the charter ‘wrong’, then everything else is as well.
The key thing, to me, is the detail of the charter. A good charter guides the tester, but does not direct. The Agile community is used to thinking in this stepped-back manner from dealing with user stories but it is mind-bending to new people or those still in a waterfall-ish environment.
According to Rob Sabourin at Star East 2010 a charter is…
- under 160 character
- statement of mission
- ties to purpose
- focuses work
- confirms understanding
- delineates scope
Also from Rob and Jon are a list of ‘types’ of charters.
- Discovery – searching for broad knowledge
- Capability – typically starts with ‘Confirm…’
- Failure Modes – what could break
- Quality Factors – the ‘ilities’
- Usage Scenarios – happy, sad, evil
- Creative Ideas – soap opera testing
- States
- Data
- Environments
- White Box
- And a myriad of different domain specific ideas
This list shouldn’t be considered definitive or complete, but should help guide and mould your thinking of charters. A well SBTM’d (yes, using it as a verb) product will have a mix of each. But again, the mix will change from group-to-group and project-to-project.
As an activity for this, I’m planning on ‘borrowing’ Rob’s idea of a charter contest — but without the contest part. Basically it will be a create-and-share each type of charter. Lots of practice — I have lots of chart paper and crayons.
Nice start. Tip of the iceberg.
There are different points/topics/threads that all converge onto your blog post here.
1) SBTM is a good way (IMO) to manage exploratory testing (ET). The idea of timeboxed, chartered sessions provides a nice way for agile developers and testers to frame their (informal or structured) testing activities.
As you know, not all agile test efforts are documented or tracked (i.e. in a formal way), so some discussion can be had as to *why* it might be important to track and document these test sessions in ‘sheets’ somewhere. That is, what is the value-added to document the testing using these sheets?
I know why *I* use this SBTM framework. Some reasons include:
* to aid in communicating testing coverage between testers;
* to have a sense of the productivity of the testing effort (i.e. the metrics that can be run against the sheets);
* to help the testers learn and develop their testing skills by having a reviewable result that we can use as a basis for coaching during ‘debrief’ sessions; and
* to have a reference of the things we learned during testing (i.e. captured in the Test Notes) that we can mine at a later date for more efficient regression testing and automation
2) Charter development really gets to the heart of testing by forcing you to _think_ before you _act_. (Egad! Blasphemy! ;))
I have observed many people jump into testing something without really thinking about it first. The way I test, I like to have an idea of *why* I’m testing before I sit down and test. Thinking about the Charter/mission/purpose/objective of testing is a good practice IMO that will help improve any testing effort.
Developing good charters takes time and practice. Understanding that there are different ‘types’ of charters is important. For instance, when you sit down to test, what do you want to learn/know/observe?
* are you trying to figure out what the system does and where you think some of the important risks are?
* do you want to generate specific Performance metrics?
* do you want to check the error/exception handling for a particular unit/module/feature?
* and so on.
Exploratory (or other) Testing is *not* about just sitting at a computer and letting your mind wander wherever it goes. That’s pretty much a waste of everyone’s time and is what a monkey would do.
The “value added” proposition here is to focus the precious time spent testing to generate information on something that someone cares about.
Charters are NOT test cases. Test cases/ideas/checklists/matrices/etc. may be generated from Charters.
I see a “charter contest” (with or without the ‘contest’ part) as a VERY useful activity for anyone who works in Software Development. Hope you have fun with this. Sorry I’ll miss it. Please blog about it afterwards so we can find out how it went.
Cheers! Paul.