I dove back into Selenium on a week or so ago, and part of that meant getting the nightly build. And much to my dismay, the Ruby client is not packaged as part of it.

Bug 1: The gem binary is not part of the nightly build

So I had to check the tree out of subversion to make it myself.

Bug 2: The repository url is wrong

On the OpenQA site, the url specified is for openqa.org, but seems like things are migrating to a new domain, so you need to use

https://svn.seleniumhq.org/svn/selenium-rc/trunk/

instead.

Once I got the code I wandered over to the ruby client directory and tried to figure out which Rake task to run. rake -T is the standard way to have rake parse it’s Rakefile and tell you the tasks. Except, you need to have built the system before it would tell you that.

Bug 3: rake -T doesn’t work

Okay. Fine. Build the whole thing (mvn install) then go back to the ruby client directory and build it (rake gem). Finally you need to install the gem. Just make sure that you specify the correct version of your gem in the script

This is more of a feature request, but if you are building a gem over-and-over I can see your gem store getting pretty cluttered. To partially address this versioning issue, I propose

Bug 4: version of gem should be timestamped in addition to version number in non-release builds

For awhile now I have been advocating that building an automated test is a four phase process. The phases are:

1. Record
2. Add checks
3. Data drive
4. Make it smart

In this post I’ll illustrate these steps in automating our One Minute Calculators using Selenium. Because this is tutorial in nature, not all aspects are automated; just enough to illustrate. It is also a huge post, so it is behind the break rather than make the main page huge.

(more…)

We have the pretty standard server configuration of a load balancer (apache) in front of two machines which serve the actual content. Through the process of trying to upgrade a machine I realized they were not configured to easily bring up or down on their own. In commercial production (Websphere and WebLogic for instance) you just go to the admin console and take a node out of the pool. Not so with our rig. Here is the solution I came up with. I think it is pretty elegant (though I am sure it is just reinventing the wheel).

1. Directory structure – I created 2 directories in the apache config directory (/etc/apache2 in this case): nodes-available and nodes-enabled
2. Creating the nodes – We had all our load balanced ports in a single file. Those got broken in to separate files depending on which machine they were on. (Lets call them zfp1 and zfp2 for this example.) These files get placed in the nodes-available directory.
   

   $ cat nodes-available/zfp1 
   BalancerMember http://x.y.z.a:8000
   ...
   BalancerMember http://x.y.z.a:8019

3. Enabling the nodes – The nodes are at this point both disabled which is not really an ideal situation. To enable them we create a symlink from the nodes-enabled directory into the nodes-available one for each node. For example, nodes-enabled/zfp1 links to nodes-available/zfp1.
   

   $ ls -l nodes-enabled/
   total 0
   lrwxrwxrwx 1 root root 33 Sep 16 14:12 zfp1 -> /etc/apache2/nodes-available/zfp1
   lrwxrwxrwx 1 root root 33 Sep 16 14:36 zfp2 -> /etc/apache2/nodes-available/zfp2

4. Use the new system – Until this point we were just playing with files in a way that was completely transparent to apache. To start using this you need to make use of the Include functionality of apache’s configuration. I changed the reference to the single file (that we split in step 2) to be Include nodes-enabled/zfp*. By using a wildcard we can add more nodes without having to do anything to the actual server configuration. It also means we can use the load balancer for multiple clusters by just having differently named files.
   

   <Proxy balancer://mongrel_cluster>
     Include nodes-enabled/zfp*
   </Proxy>

5. Reload the config – Apache will happily run forever without re-reading its config file, so once you are comfortable with your new configuration, so you have to remember to tell it to reload.

All the above is still really only half the solution though. Actually, it is what makes controlling individual nodes possible, but it is a pretty manual process still. For this particular property we are using Vlad the Deployer to manage things, though I suspect you could use Capistrano or Puppet just as easily.

desc 'add a node to the server'
remote_task :add_node, :roles => :load_balancer do
  if ENV['node']
    run %{[ -f #{ apache_root }/nodes-available/#{ ENV['node'] } ] && [ ! -f #{ apache_root }/nodes-enabled/#{ ENV['node'] } ] && sudo ln -s #{ apache_root }/nodes-available/#{ ENV['node'] } #{ apache_root }/nodes-enabled/#{ ENV['node'] } && sudo /etc/init.d/apache2 reload || echo "Node does not exist"}
  else
    p 'You need to specify a node to be able to add it. node=foo'
  end
end

desc 'remove a node from the server'
remote_task :remove_node, :roles => :load_balancer do
  if ENV['node']
    run %{[ -f #{ apache_root }/nodes-enabled/#{ ENV['node'] } ] && sudo rm #{ apache_root }/nodes-enabled/#{ ENV['node'] } && sudo /etc/init.d/apache2 reload || echo "Node does not exist"}
  else
    p 'You need to specify a node to be able to remove it. node=foo'
  end
end

(Yes, I know that the error messages are not very nice, or even accurate in a lot of situations, but it does the job which is all I require of it right now.)

To enable a node you simply do a ‘rake lb:add_node node=zfpN’ where lb is the namespace you created for your load balancer machine and N in this case is the node number. In this example there is only zfp1 and zfp2 but it should scale linearly.

I have been looking for a Ruby/Rails equivalent of checkstyle since I started working with it, but have only found a single orphaned project. Until yesterday that is. Yesterday’s Double Shot had a link to the Roodi project. Roodi stands for ‘Ruby Object Oriented Design Inferometer’, but essentially it is checkstyle for Ruby.

The Java community has a rich set of guidelines to build static checks from, but the Ruby community seems to lack that cohesion. That is going to make for an interesting collection of checks over time and no doubt some discussion over what should be and checked and how. Here are the checks it currently does:

• AssignmentInConditionalCheck – Check for an assignment inside a conditional. It‘s probably a mistaken equality comparison.
• CaseMissingElseCheck – Check that case statements have an else statement so that all cases are covered.
• ClassLineCountCheck – Check that the number of lines in a class is below the threshold.
• ClassNameCheck – Check that class names match convention.
• CyclomaticComplexityBlockCheck – Check that the cyclomatic complexity of all blocks is below the threshold.
• CyclomaticComplexityMethodCheck – Check that the cyclomatic complexity of all methods is below the threshold.
• EmptyRescueBodyCheck – Check that there are no empty rescue blocks.
• ForLoopCheck – Check that for loops aren‘t used (Use Enumerable.each instead)
• MethodLineCountCheck – Check that the number of lines in a method is below the threshold.
• MethodNameCheck – Check that method names match convention.
• ModuleLineCountCheck – Check that the number of lines in a module is below the threshold.
• ModuleNameCheck – Check that module names match convention.
• ParameterNumberCheck – Check that the number of parameters on a method is below the threshold.

Thats all well and good, but a tool that is not run is just a clever piece of code. And relying on people to remember to run a tool is not a plan I would bet the house on. But that is why we have things like CruiseControl.

Here is a pretty heavily edited version of our Rakefile.

desc "Cruise Control"
namespace :cruise do
  output_dir = ENV["CC_BUILD_ARTIFACTS"]

  task :build do
    ...
    CruiseControl::invoke_rake_task 'cruise:roodi'
  end
  
  desc "Run the roodi tests"
  task :roodi do
    `roodi -config="#{RAILS_ROOT}/config/roodi.yml" "#{RAILS_ROOT}/app/**/*.rb" > #{output_dir}/roodi.txt`
  end
end

There is not (yet) a roodi_on_rails plugin, though I expect it is only a matter of time, so I just run the tool on the commandline and put the output in the place CruiseControl needs it to display it on the report page. The output is, to say the least, un-pretty but if the community backs this tool I can see it growing nice rspec like reports with links to the actual offending line.

Oh, and here is config/roodi.yml, which is just the default configs for all the rules, but explicit is better than implicit.

AssignmentInConditionalCheck:    { }
CaseMissingElseCheck:            { }
ClassLineCountCheck:             { line_count: 300 }
ClassNameCheck:                  { pattern: !ruby/regexp /^[A-Z][a-zA-Z0-9]*$/ }
CyclomaticComplexityBlockCheck:  { complexity: 4 }
CyclomaticComplexityMethodCheck: { complexity: 8 }
EmptyRescueBodyCheck:            { }
ForLoopCheck:                    { }
MethodLineCountCheck:            { line_count: 20 }
MethodNameCheck:                 { pattern: !ruby/regexp /^[_a-z<>=\[\]|+-\/\*`]+[_a-z0-9_<>=~@\[\]]*[=!\?]?$/ }
ModuleLineCountCheck:            { line_count: 300 }
ModuleNameCheck:                 { pattern: !ruby/regexp /^[A-Z][a-zA-Z0-9]*$/ }
ParameterNumberCheck:            { parameter_count: 5 }

My first blog post defined Quality as Value (which originated from Jerry Weinberg but I heard it from James Bach). I still really like that definition but have been thinking about the expanded definition that I’ve been seeing more and more; Quality is Value to some Person who Matters. (I’m attributing the expanded definition also to James though I don’t have a direct reference to it. Instead I have a definition of a bug from a deck of Cem Kaner’s which if you work backwards give you the definition of a Quality. Actually, he uses stakeholder instead of person, but I like person better).

So why am I thinking about this now? I’m trying to concoct the testing strategies I’m going to try and one of the things I need to figure out is the ratio to test with each web browser. My fallback position on this is usually to look at the browser market share and use that as a pretty good slice.

But.

I’ve figured out that if you are going to use such a heuristic though that you really should know the situations where it might fail. For instance, if you are testing a site that is for mac power users you likely care more about Safari and Firefox than you do Internet Explorer. But even that is rather gut-feely which may or may not be acceptable [to you / management / the customer]. What you (hopefully) have though is the server’s access logs which is as close to am empirical source as you are likely going to get. Access logs are usually pretty safe as far as privacy leakage so you likely can get them if you asked.

Here is the browser percent breakdown from 10h 20m of access log from one of our properties:

	Internet Explorer 6: 31
	Internet Explorer 7: 14
	Internet Explorer 8: 0
	Firefox 2.x:         8
	Firefox 3.x:         1
	Safari 2.x:          1
	Safari 3.x:          6
	Opera:               0

Interesting.

What this shows me is that even though IE7 has the global lead in market share, our customer base is (currently) overwhelmingly using IE6 so any testing should be skewed towards that browser. Yes, in an office full of Macs with near 100% Firefox saturation making it work on non-IE browsers might give us a warm-and-fuzzy feeling in that technologically cool and standards compliant manner but it is (nearly) completely irrelevant; we don’t matter. In our environment, at this snapshot in time, the people who matter are IE users, 2:1 in favor of version 6.

(The script I used to parse the log is below the cut)
(more…)

I’m busy drinking from the Rails firehose at work to both figure out the technology itself but about how the application is wired together as well. Today’s discovery/exploration was regarding plugins.

Rails is itself actually a gem, and it can be modified via a plugin system. From what I can tell, the proper way to install a plugin (when using Subversion) is to teach Rails where the plugin’s repository it. That way when you do an repo sync you get not only your updated app code, but you get the current revision of your plugins too.

This is pretty trick, but also very scary. When you are in pre-release or development mode this makes sure you don’t fall too behind as far as versions go. I’ve seen (okay, worked for) companies that are so far behind technology wise they are at the point where upgrading part of their infrastructure requires upgrading it all. However, when as soon as you are thinking about stabilizing / releasing the code you need to freeze the revision. Otherwise you can get into the situation where you have changed a CSS file with an isolated change but a change in a plugin you are using broke the system completely. This page explains how to freeze to a specific revision. Once you have things frozen you can systemically manage the risk of updating Rails’ plugins.

This is heuristic of course.

Some plugins are there as development tools though; rspec and rspec_on_rails for instance. They can pretty safely be run off of the edge code since they need to be manually invoked (an empty init.rb indicates that they do not get loaded into rails automagically).

(Or so I currently understand things.)

Kevin Skoglund is (according to his site) a web developer and instructor specializing in Ruby on Rails, PHP, SQL, HTML, and CSS. He is also running a series of articles on ‘Testing in Rails’. Currently he has the introduction and the first 5 of X parts posted. This search will give you all the articles in the series though I suggest you just subscribe to his rss as there are other good things that come out from him too it seems.

For those too lazy to click one of the above links, here are direct links to the articles out at time of writing.

• Introduction
• Unit Testing in Ruby
• Unit Testing Ruby Classes (con’t)
• Unit Testing in Rails
• Unit Testing using ActiveRecord Models
• Fixtures
• ActiveRecord Relationships
• Validations
• Attributes and Callbacks
• Running Unit Tests

(found via Mike Gunderloy)