I’m busy drinking from the Rails firehose at work to both figure out the technology itself but about how the application is wired together as well. Today’s discovery/exploration was regarding plugins.

Rails is itself actually a gem, and it can be modified via a plugin system. From what I can tell, the proper way to install a plugin (when using Subversion) is to teach Rails where the plugin’s repository it. That way when you do an repo sync you get not only your updated app code, but you get the current revision of your plugins too.

This is pretty trick, but also very scary. When you are in pre-release or development mode this makes sure you don’t fall too behind as far as versions go. I’ve seen (okay, worked for) companies that are so far behind technology wise they are at the point where upgrading part of their infrastructure requires upgrading it all. However, when as soon as you are thinking about stabilizing / releasing the code you need to freeze the revision. Otherwise you can get into the situation where you have changed a CSS file with an isolated change but a change in a plugin you are using broke the system completely. This page explains how to freeze to a specific revision. Once you have things frozen you can systemically manage the risk of updating Rails’ plugins.

This is heuristic of course.

Some plugins are there as development tools though; rspec and rspec_on_rails for instance. They can pretty safely be run off of the edge code since they need to be manually invoked (an empty init.rb indicates that they do not get loaded into rails automagically).

(Or so I currently understand things.)

One of the key concepts from my grade-12 science fiction class was that of archetypes. For those not familiar with archetypes, wikipedia defines it as a generic, idealized model of a person, object, or concept from which similar instances are derived, copied, patterned, or emulated

As a person in a QA/Tester role there are a number of archetypal roles that can be adopted.

• Sheriff – Bringing order to the wild west. Chaotic development processes are roped in and consistency and discipline instilled in the vacuum
• Cheerleader – Pumping the team up to be more than they are. ‘Better unit tests? Way to go!’
• Cop – More or less the opposite of the Cheerleader. While typically you get more flies with honey, sometimes you do have to bring out the vinegar
• Negotiator – An active role in figuring out the trade-offs between development and test
• Marketer – A longer-term variation of the Negotiator which uses more subtle techniques for furthering agenda
• Coach – Strategist and morale support

That’s a quick list generated on the train this morning. The trick with all these is to know which one(s) you are both expected to perform and which one(s) you are good at. And recognize that those might be two very different lists.

Which ones am I missing?

In the era of ‘Automate Everything!’ fanaticism coming from the Agile community, and with more and more vendors releasing products which will automate you towards zero shipped defects it is near blasphemy to suggest a testing strategy without automation. What is even worse is that you suggest that time be invested in creating scripts only to toss them aside once they are complete. But that is exactly what I am suggesting.

Static analysis tools provide high value with little maintenance once the initial tuning is complete and usefulness of unit tests has been well demonstrated by the TDD crowd so we want keep those around. Classical end-user functionality automation (think QTP and Selenium) however provide little long term value, and as such should be discarded.

If that wasn’t controversial enough a statement, this one will certainly top it. You should still go through the exercise of writing these tests though. Why? By doing so, you learn about how the feature that is being automated is wired together. If you are going to do good automation you will inevitably answer these questions (and many more):

• What pages are involved?
• What are the field validations?
• Where is information stored in the database?
• What manipulations happen to the data?
• Are the business rules that govern the process?
• Are the developers providing sane id’s of objects?

All the answers to the above help you form other, better, more targeted, questions to ask the software. This deep learning of the product is the true value of the automated tests you have written. The value is not in catching a regression bug, as that was hopefully found by the static or unit tests. Once an automated test is written it also starts to become a victim of the Pesticide Paradox where bugs are learn to “hide” from the tests or are likewise immune to them.

I’ve been doing automation for a number of years on a variety of platforms. During that time I would guess that running of the scripts resulted in likely fewer than 10 bugs. I’ve found many times over that number he when writing the tests.

So why throw out the scripts when they have already been created? Shouldn’t you just keep running them? I suppose, if you commit to not spending any time if a test ‘breaks’. The reason for this comes down to maintenance. Any time you spend reidentifying gui objects in the screen or otherwise tinkering with a script is time that you have lost from testing. Time is almost always is constrained resource and we would be best served to use it as effectively as possible.

What we need to do then is change how we look at test automation. Instead of having it as a safety blanket against regressions that provides management with a nice warm-and-fuzzy feeling, test automation needs to be reclassified as a technique which aides in the exploratory testing of an application at a technological level.

Put another way, automated functional testing is not about the end result, it is about the journey.

I can only accept partial blame for this idea. The seed of this was planted by Michael back in December though I suspect I might have run much further with it than was discussed then.

This month’s Financial Post Business Magazine has a small article inspired by Dan Ariely’s new book Predictably Irrational. Much like How Doctors Think, this book appears to look at how humans think, and the more we know about those processes the better we can test software by countering built-in human blind spots.

Here are some of the points that make this book seem interesting.

• When you want to believe something, you will shape reality to fit very much what you want to see.
• There is something in the human mind that overpowers logic with the immutable desire to believe something true, despite evidence to the contrary.
• It would be naive to assume such pleasing effects (from placebos) are limited to the medical realm.
• They say a con artist, after all, is not in the business of convincing skeptics, but rather, to let people believe what they already wish to. (okay, that is from the review, not the book itself — but is still a good line)
True to modern book marketing, he has gone multimedia with a blog and podcast.

Now to get myself an actual copy of the book. I suspect there are a tonne of nuggets that apply to testing.

Well, it’s the beginning of the month, so might as well have a quick housekeeping-ish post for some updates.

• CAST – The track sessions have been announced for CAST. Now that they have been announced spots are starting to get filled up so if you are hoping to go you need to register soon.
• Agile – It looks like I am slated to speak at this year’s Agile conference. (The Tuesday afternoon specifically)
• Teaching – I’m thinking of revamping the courses I have been teaching the last couple years for delivery over something like Skype. Is there interest out there for a ‘Python for Testers’ course? Or ideas on how one would promote that?
• Book Reviews – I’m going to start publishing my book reviews in the currently-searching-for-a-new-name magazine that the AST publishes. I have 3 books on the queue, but if you have something testing / quality related feel free to send it my way.
• Employer – Starting May 15th I’ll become the Quality Czar (that’s not actually the title, but I like it) for Zerofootprint after a year and a half at Jonah.