I’ve mentioned our crazy SOA architecture here a couple times in the past, but essentially, the platform we are moving towards is a whole bunch of shared services which drive a diverse set of front-ends. In any setup like this, the big risk is a change to an underlying service which manifests itself in one or many of the front-ends. To try and mitigate the risk, and testing effort involved I am creating Selenium scripts around the bits that are service related. This post is a how-to on using Selenium with Rails in a sane, extensible manner. Well, as sane as anything that is in Rails-land at any rate.

• Get Selenium Running – Since this is Rails we’re talking about, we’re going to be scripting in Ruby, so you need to install the Ruby client.
  

  sudo gem install selenium-client

  You will also need to get the Selenium Server; of which release 1.0 was finally finished after 5 years. Start the server.

• Start your application – This might seem to an obvious step, but there is a trick that makes it worth mentioning and that is you need to start it in ‘test’ mode. If you do not, then your manipulations in the Selenium script will go against one database and the browser will go to another. Thats a bit of brain pain you don’t want. Trust me. (Care to guess how I spent my day?)
  

  script/server -etest

• Get your Rake task ready – To be fully Rails-ish, we will want to run our scripts using Rake. So create in lib/tasks a file called selenium.rake with the following task
  

  namespace :test do
    Rake::TestTask.new(:selenium) do |t|
      t.libs << "test"
      t.pattern = 'test/selenium/**/*_test.rb'
      t.verbose = true
    end
    Rake::Task['test:selenium'].comment = "Run the selenium tests in test/selenium"
  end

  There are variations of this out there that include a pre-requisite task of db:test:prepare, but we don't want that as our server is already running in test mode and wiping out tables from underneath it is not a good idea.

• Give yourself a helper - Rails comes with a file called tests/test_helper.rb; create a copy of it and call it selenium_helper.rb. And while you are at it, set self.use_transactional_fixtures to false. This results in us leaving stuff in the database after a run, but we are smart enough to clean up after ourselves when necessary and to use unique values whenever possible.
• Record and start hacking - I detailed the steps an automated script should go through in this post, but I'm jumping straight to the end of having your scripts be intelligent. Here is the script, which needs to be named something_test.rb for the Rake task to work.
  

  # include our helper
  require File.expand_path(File.dirname(__FILE__) + "/../selenium_helper")
  
  # selenium
  require 'rubygems'
  gem "selenium-client", ">=1.2.15"
  require "selenium/client"
  
  class LoginTest < Test::Unit::TestCase
    def setup
      @verification_errors = []
      @browser = Selenium::Client::Driver.new "localhost", 4444, "*firefox", "http://localhost:3000", 10000
      @browser.start_new_browser_session
      
      @authentication_service ||= OurCustomAuthServerClientConnectionStuff()
    end
    
    def teardown
      @browser.close_current_browser_session
      assert_equal [], @verification_errors
    end
    
    def test_login_success
      @user, password = create_active_basic_sop_user
  
      @browser.open "/"
      @browser.type "email", @user.email
      @browser.type "password", password
      @browser.click "//input[@name='commit' and @value='Login']", :wait_for => :page
      begin
        assert @browser.is_element_present("logout")
      rescue Test::Unit::AssertionFailedError
        @verification_errors << $!
      end
    end
  end

  Here is the helper, minus all the stuff that come with it originally since just did a copy/paste originally. That stuff is still in the file, just not here.
  

  require 'random_data'
  
  def create_active_basic_sop_user
    z_user, password = create_basic_sop_user
    z_user = activate_user(z_user)
    return z_user, password
  end
  
  def create_basic_sop_user
    # create local user
    my_password = generate_password
    my_attrs = {"email" => generate_email, "password" => my_password, "password_confirmation" => my_password}
    @user = User.new(my_attrs)
    if not @user.save
      assert_false
    end
    
    # give it some user_types
    user_type = UserType.new
    user_type.user = @user
    user_type.role = Role.find_by_rolename('user')
    # the role might not actually exist so create it if not
    if not user_type.role
      u = Role.new
      u.rolename = 'user'
      u.save
      user_type.role = Role.find_by_rolename('user')
    end
    user_type.save
    
    return @user, my_attrs["password"]
  end
  
  def activate_user(z_user)
    z_user = @authentication_service.lookup_user_by_email(FRONTEND, z_user.email)
    z_user = @authentication_service.activate_user(FRONTEND, z_user)
    return z_user
  end
  
  def generate_email
    Random.email
  end
  
  def generate_password
    Random.alphanumeric
  end

  Notice how all the environment logic is in the helper. A common pattern of Selenium success is to wrap it in a DSL. This isn't really at this point, but you get all the benefits of code reuse, etc.. Note as well how small the functions are so you can build even more useful abstractions of as the effort continues.
  
  Another thing that is important here is the two generate functions. These come from the random_data gem and allow for unique test data every time through. (Or at least gives you a much greater chance at uniqueness.) These help us negate the Mine Field Problem and Pesticide Paradox and allows us to be slightly sloppy about cleaning up after ourselves. This slop is a design choice which will mean more database (detailed) interaction down the road when it comes to validation, but that is likely a good thing in and of itself.

Last night I did an Ignite talk at DemoCamp Toronto 20. It was my second time presenting, but first time doing an Ignite. Some lessons learned from doing it…

• Ummm, practice. No, really.
• You will get tripped up. Accept it and recover
• 15 seconds a slide is a long time
• 15 seconds a slide is a short time
• If you get ahead of yourself, just keep going. You will catch up. I should have talked about slide 2 during slide 1 which caused me to get behind on slide 3
• Have I mentioned practicing?
• If you write cheat sheets, make sure you can read them
• Having a fuzzy idea of what you want to say is more effective than having a specific thing you want to say
• Not everything works on evenly spaced auto-advancing slides. Some of my slides could have used slower advance, others longer
• Cheat as much as possible. I didn’t, but should have
• Remember to have fun
• Practice seems like a good idea

Anyways, my talk was right before the pizza and was based on The Legend of Zealot Anarchist Robot. You might want to open that in another browser before going through the slides; they don’t make much sense otherwise.

Here are the credits for where I lifted all the photos from. In order.

• Slide 2
• Slide 3
• Slide 4
• Slide 5
• Slide 6
• Slide 7
• Slide 7
• Slide 8
• Slide 9
• Slide 9
• Slide 10
• Slide 11
• Slide 12
• Slide 12
• Slide 13
• Slide 14
• Slide 15
• Slide 16
• Slide 17
• Slide 18
• Slide 19
• Slide 20

Going through my bag this morning I found my notes from Mark Meninger’s talk on Considerations for an Enterprise Test Automation Implementation from KWSQA Targeting Quality 2009. I had a bit of deja-vu with the metaframework stuff from two years ago which makes me think that there really is something to this pattern that could be monetized. He had way too much content for his time block, but that seemed to be a pattern with the other talks I peeked my head into as well. That’s okay though. I don’t claim to be a master speaker yet and these regional conferences are where you are supposed to make your mistakes, erm, learning opportunities.

• enterprise – serves all departments of the organization, across silos
• typically, every group does something on their own to achieve similar results
• benefits of enterprise automation are: shared components, shared development and a consistent philosophical approach
• the notion of ‘working with developers to improve their test process’ seemed to be a bit of a shocking one. Ummm, this is kinda the definition of agile-testing and that’s almost mainstream now so it certainly shouldn’t be shocking to anyone.
• when merging different efforts you have to decide what will stay the same and what will be different. (politics)
• framework – manages the process, SUT, pulls strings
• don’t glue your framework to a single type of testing
• metaframework + dsl = enterprise automation (as far as I could tell)
• Mark’s group doesn’t own testing; individual product test leads do. His group provides test tooling services (enterprise tool smiths)
• manual tests -> automated tests -> automated automation
• the barrier is coming down between development and qa; there is still a division but it is now more of a fence than a wall

Business Week had an article talking about Alan Mulally’s stint at Ford recently. What was interesting was the cultural insights and some of the tools that are being employed to try and keep the ship off the shoals. Here is a collection of out-of-context quotes and my commentary on them.

• Under Mulally, decision-making is more transparent, once-fractious divisions are working together, and cars of better quality are moving faster from design studio to showroom. So, more agile
• But he was criticized for leaving too much of his team out of the loop. Mulally, they said, needed to check in more often to let his reports know if they were headed in the right directions. Internal communication. Shocking.
• Managers commonly held ‘pre-meetings,’ where they schemed how to get their stories straight for higher-ups
• Information should never be used as a weapon on a team
• Mulally was determined to have a constant stream of data that would give his team a weekly snapshot of Ford’s global operations. Seems a lot like burn-down and/or big visible charts
• Mulally has imposed discipline on a company that veered from one strategy to the next. Discipline at an organizational level is needed if there is any hope at organizational quality

My Dakar when 40 plan is currently in (likely) permanent non-start, but that can’t stop me from reading dirt bike magazines. In this month’s Dirt Bike, the From the Saddle column is about Heros. Here is the (first) important part.

Here’s the problem with heroes: They’re people. Very few are able live up to the honor. In fact, most of them don’t.

This is important to life in general, but also in the testing field. Too often the people put others up on a pedestal based on their articles in magazines, books they have published, mailing list participation, etc.. I think it is far healthier to respect someone rather than hero-ize them. I’ve met a lot of people in the testing and agile-testing community and as a whole they are all human (so far as I can tell). They can be arrogant, boring, childish and petty. But they can also be brilliant, insightful, compassionate and respectful. It is less of a blow when someone you respect takes a blow than someone you consider a hero.

He ends the column with an equally paragraph.

Along the way, I’ve met lots of other former heroes. Like I said, most didn’t deserve it. But having heroes is the point; meeting them isn’t. The bottom line is, they make you want to be something better.

I should add, to offset something that often comes with heroes; better does not mean exactly like your hero. The testing world already has one of them, we need you to be yourself.

Yes, I know that Beautiful Testing is barely into the technical review and I’ve got a tonne of other things on my plate, but I’ve been thinking about ‘the next book’. Here is one of the ideas.

How much interest do we think there would be in a how-to-ish e-book on Continuous Quality/Checking/Integration? The flow of it would look a bit like the CI Maturity Model based on my experiences of doing similar things in a number of organizations over the years. Potential structure would look something like below with technologies explored for the Java and Rails community.

• Version Control (subversion, git)
  • Workflow
  • Branching / Tagging
  • Pre / Post commit scripts
• Groundwork for ‘Smart’ Builds
  • Maven
  • Rake
• Continuous Integration (unit level)
  • On-commit
  • Nightly
  • Unit testing basics
  • Test doubles (no network, no disk)
• Static Analysis
  • Running
  • Tuning
  • Failing builds
• Continuous Integration (Selenium)
  • Stages of automation maturity
  • Distributed execution
  • Parallel execution
• Automated Deployment
  • To testing environments

It wouldn’t go so far as some of the ’50 deploys in a day’ chaos of full-on Continuous Deployment bouncing around, but we can automate the process of getting a build into a testing environment for humans to interact with with the bar raised higher in terms of initial quality than just the manual production of builds.

Baseball season is now 1/4 done for the year and the Jays are in first place so things are good up here in Toronto. (Though more important, the Brooklin Redmen start their season this week.) And as usual, Jays’ pitcher Roy ‘Doc’ Halladay is doing an outstanding job; first pitcher in the majors to reach 8 wins an American League player of the week last week.

In the postgame for AJ Burnett‘s return to Toronto as a Yankee for the first time to pitch against him (Doc mentored AJ during his time here), two statements were made about Doc that I thought were great.

• Everyone has a number one pitcher, but not everyone has an ace
• One can mentor by sitting down and talking to a person, or by action

Think about the team you work with, or even your company as a whole.

• Do you have an ace, or just a number one?
• Are you an ace? What do you have to do to become one?
• Do you have a mentor? How do they do the mentoring? Officially with weekly meetings or just by being an outstanding role model?
• Are you a mentor? How are you doing it?

Ready, Aim … Fail is an article I’ve had in my bag for awhile; in fact, even pointed it out in a conversation in twitter. It was in the ‘Executive’ section of the paper, but there is a bunch of cautionary information in it that apply to testers as well. Especially if you are in a metrics-oriented environment.

• …industry analysts argue that one of most damaging things he company did was set that goal.
• As a result, GM … fell victim to a goal
• Among psychologists, the link between setting goals and achievement is one of the clearest there is, with studies on everyone from woodworkers to CEOs showing that we concentrate better, work longer, and do more if we set specific, measurable goals for ourselves.
• But a few management scholars are now looking deeper into the effects of goals, and finding that goals have a dangerous side.
• The argument is not that goal setting doesn’t work — it does, just not always in the way we intend. “It can focus attention too much, or on the wrong things…”
• Goal setting has been treated like an over-the-counter medication when it should really be treated with more care, as a prescription-strength medication
• Goals, they feared, might actually be taking the place of independent thinking and personal initiative
• A 2004 paper on what people do when they fall just short of their goals. .. what they do is lie to make up the difference.
• Narrow corporate goals can keep employees from asking important questions that they otherwise might.
• … stretch goals are most likely t be pursued by desperate, embattled companies — the sort least equipped to deal with the costs of ambitious failures.
• Most concentrate so hard on the goal that they become blind to other information…
• … goals with rewards, if not carefully calibrated, can short-circuit our intrinsic enthusiasm for a task — or even interrupt our learning process.
• … goals can’t protect us from ourselves.
• … sometimes nothing works like a goal. But ensuring that it doesn’t backfire requires care.
• Although simple numerical goals can lead to bursts of intense effort in the short term, they ca also subvert the longer-term interests of a person or company
• In work requiring a certain amount of creativity and judgement, the greatest risk appears t lie in overly simplified goals. Reducing complex activities to a bundle of numbers can end up regarding the wrong behaviour …
• … goals need to be flexible when circumstances change.
• …we might be better off creating workplaces and schools that foster our own inherent interest in the work.

I’m speaking in July at CAST 2009 on how to market yourself as a tester. One of my beliefs in this regard is that everyone should be writing, if for no other reason than it forces you to really think about your ideas. For most people, this is achieved easiest by blogging.

A couple months ago I stumbled on a review of Blog Blazers by Stephane Grenier and asked for a copy in order to do my own review. Stephane kindly agreed and I’ve been hauling it around ever since (and using its content for my own work; properly attributed of course).

Blog Blazers follows the pattern of ‘X questions with’ that can be seen here or here with the targets of the questioning being successful bloggers. How do you define success? Well, that is part of the interesting aspects of the book. Overwhelmingly it is ‘I could quit my day job’ but there are also the ideas of ‘influence’, ‘respect’ and ‘marketing’ get some space as well. This for-profit bias is not that unsurprising as the sub-title of the book is ’40 top bloggers share their secrets to creating high-profile, high-traffic, and high-profit blog.’

As the sub-title suggests, the book is broken into 40 chapters with one blogger answering questions per chapter. I really liked this format as it made for easy reading while waiting for the bus or whatnot. I also really liked the layout with a picture of who is answering questions right in the masthead of each chapter. (I kinda want to explore this layout in Beautiful Testing…).

To succeed, a book like this needs two things: good questions, good answerers.

First, the questions. Each chapter is based on the same set of questions. Here are my favourite ones:

• What makes a blog successful according to you? Is it traffic, reach, revenue, etc.?
• What is your most successful blog post ever?
• What is your best advice in regards to content and writing for bloggers?
• What are your quick and short five best tips for blogging?
• What is the most common pitfall new bloggers generally fall into?

The fact that I can have favourites hints at one of the downsides of this book. By halfway I found myself skimming chapters, skipping over questions I wasn’t interested in.

The respondents are also equally strong, though again, heavily weighted to people who could be defined as ‘professional’ bloggers. My copy has the most answers underlined in David Armano, David Seah and Derek Semmler, likely because they under-emphasis the monetary aspect of success.

On the whole, if you starting to blog, especially if you think you want to do it for a living, then Blog Blazers is a great book to pick up. It won’t teach you about how to use WordPress or other blogging platforms, but it will give you a peek inside the heads of people who have done this for awhile and have spent some time thinking about it. It has already helped refine some of my thoughts on blogging and is referenced a couple times in my CAST paper.

With a cover price of under $20 (in both USD and CDN), I think it is correctly priced, but I suspect it could be found for even less online. Say like from Amazon.

I was in a meeting last week where part of the discussion veered into ‘what corners can we cut’ territory. The carpenter’s mantra of measure twice, cut once came to mind.

Cutting corners is a quick way to accrue technical debt. But unlike some forms of technical debt, cut corner debt tends to be the toxic type. You know, the kind that pulls down economies.

So what corners get cut? Well, in my experience these are the easiest for management to cut…

• Testing time allotment
• Developer oriented testing
• Upgrading tools and/or infrastructure
• Internationalization
• Fine-grained security
• Testability

The first two are pretty self-explanatory I think. The next one is about improving the scaffolding your application is built on and the last three are really hard to shoe-horn back into a product.

Now, every time I’ve seen someone cutting a corner it is not out of some malicious desire to make a crappy product. It is often a time constraint being imposed by external factors, and having been in management roles before I appreciate and understand this. But you have to be sure, really, really sure that you have a good measurement of what you are cutting off before the saw comes out.