Posted on August 10, 2010 in Uncategorized by adam1 Comment »

In the ongoing series of posts on Agile Test Case Management (which is pronounced ‘figure out things for my talk on Thursday’) I am now heading onto slightly shakier ground with Session Sheets. Session Sheets are the tracking method of proposed in Jon Bach’s paper Session-Based Test Management. SBTM itself is a much larger discussion and one that is beyond scope here. What is in scope is the sheets themselves as they are, according to Jon, the magic ingredient of SBTM.

Sheets themselves are…

  • plain text (not .doc — that is a binary format)
  • tagged for categorization and organization
  • machine readable
  • stored in revision control
  • highly customized to the team and context

That could describe, well, almost anything. And it is supposed to; you are doing it wrong if you don’t tailor it to your environment. But what distinguishes a ‘good’ sheet and a ‘bad’ sheet. Again, beauty is thin the eye of the beholder and the devil is in the details – or rather in the charter. A session’s charter is what guides the testing and provides the mission. If you get the charter ‘wrong’, then everything else is as well.

The key thing, to me, is the detail of the charter. A good charter guides the tester, but does not direct. The Agile community is used to thinking in this stepped-back manner from dealing with user stories but it is mind-bending to new people or those still in a waterfall-ish environment.

According to Rob Sabourin at Star East 2010 a charter is…

  • under 160 character
  • statement of mission
  • ties to purpose
  • focuses work
  • confirms understanding
  • delineates scope

Also from Rob and Jon are a list of ‘types’ of charters.

  • Discovery – searching for broad knowledge
  • Capability – typically starts with ‘Confirm…’
  • Failure Modes – what could break
  • Quality Factors – the ‘ilities’
  • Usage Scenarios – happy, sad, evil
  • Creative Ideas – soap opera testing
  • States
  • Data
  • Environments
  • White Box
  • And a myriad of different domain specific ideas

This list shouldn’t be considered definitive or complete, but should help guide and mould your thinking of charters. A well SBTM’d (yes, using it as a verb) product will have a mix of each. But again, the mix will change from group-to-group and project-to-project.

As an activity for this, I’m planning on ‘borrowing’ Rob’s idea of a charter contest — but without the contest part. Basically it will be a create-and-share each type of charter. Lots of practice — I have lots of chart paper and crayons.

Posted on August 9, 2010 in Uncategorized by adam3 Comments »

This is second post in the series that will eventually turn into a my Agile talk. It also the easiest one to cheat on as I’ve more or less already written it. You see, mindmaps as a test case management system was 1/3 of my chapter in Beautiful Testing and since it the chapters are under Creative commons I can link to it directly.

But for the lazy…

  • Provide a concise, visual means of tracking test ideas
  • Are the secret way the testing cabal keeps track of things; not bullets or essays.
  • Show only the what of testing, not the how
  • As you think of new things, add them
  • Standard ‘brainstorming’ rules apply: anything can be added, those some will be pruned later
  • One strategy that has worked for me is each person does a different ‘feature’ in isolation, then the group expands it
  • Projecting mindmaps onto a whiteboard is pure win
  • Unfortunately, the formats are usually binary or formatting which means diff’ing versions is near impossible
  • If auditability is really a concern you can print out the map and initial beside the end node with the date saying the result

The exercise for this one is going to be, unsurprisingly, to create a testing mindmap. And then we’ll compare them to see which ideas different groups come up with.

Posted on August 8, 2010 in Uncategorized by adamNo Comments »

My session this year at Agile is on ‘Agile Test Case Management’ which looks at the various ways we can manage test cases without needing to have the ridicuouls amount of never read, took too long to think of, highly constraining documentation that is found on too many non-Agile projects. (And I’m sure on some ‘Agile’ ones as well.) This is the first of a series of posts as I explore my thoughts on them. (Yes, I know it is only five days from the talk time and I’m just starting…)

  • Most testers use checklists to organize their thoughts. Don’t believe me? Look at their desk. It is covered with notes to themselves about current and future test ideas. Calling those notes a checklist is just a giving it a nicer name
  • Checklists are not new; other professions have been using them for years. And years. (And years.)
  • Like say ‘law’ and ‘psychology’ (all taken from Cem’s paper below)
    • Use them as aids to critical thinking in the moment, rather than as directives to be followed.
    • Myth: A script acts as “training wheels” for the new tester to learn the application domain, the program itself and how to test it. (This is a hard myth to accept and rebuff — I still have trouble with it.)
    • Students learn (and transfer) better when they discover concepts, rather than by being told them
    • Adults benefit from activity-based and discovery-based styles
    • A script specifies
      • test entry conditions
      • test operations
      • expected results
      • comparisons the human or machine should make
    • It is not about just what to ask, it is why
    • Data Collection checklists..
      • help you prepare
      • are broader than needed
      • are used in the context relevant order (not in preparation order)
    • The smart checklist designer(s):
      • captures test-relevant information
      • and organizes it into lists
      • list items are often annotated
    • Lawyers use checklists and predesigned forms, but customization is seen as a fundamental requirement of professionalism
    • lists are reminders, not scripts
  • Atul Gawande turned an article in the New Yorker on checklists in medicine into a whole book on the subject.
    • The average patient required a hundred and seventy-eight individual actions per day
    • Generalist vs. super-specialist
    • But what if experience is not enough? Enter the checklist
    • Nurses have always had their ways of nudging a doctor into doing the right thing, ranging from the gentle reminder (“Um, did you forget to put on your mask, doctor?”) to more forceful methods (I’ve had a nurse bodycheck me when she thought I hadn’t put enough drapes on a patient). – See also my post on Mitigated Speech
    • The researchers found that simply having the doctors and nurses in the I.C.U. make their own checklists for what they thought should be done each day improved the consistency of care to the point that, within a few weeks, the average length of patient stay in intensive care dropped by half.
    • Two main benefits
      • Help with memory recall, especially with mundane matters that are easily overlooked in patients undergoing more drastic events
      • make explicit the minimum, expected steps in complex processes
    • Checklists can also bring to light organizational problems that only management can address properly
  • In my experience, the best checklists (for testing)
    • Focus on a specific idea
    • Contain heuristics and mnemonics
    • Prod, but don’t bully towards the desired action
    • Can be used by anyone — and will have slightly different results (which is a Good Thing)
    • Can contain very specific items if warranted
    • Are tailored to the environment / task

Since this is at Agile and the prevailing way of learning is action, there will be two activities associated with checklists. The first is to create a checklist on how to test something (tbd) before — and then revist it after I ramble a bit. The second will be to create a testing mnemonic (because I really like them).


« Previous Page