Category Archives: Ruby and Rails

All posts about Ruby or Ruby on Rails.

Using SagePay in your Ruby Projects

Posted on by

Once upon a time, in a galaxy far, far away … Ok, I’ll stop now. A few years back, I was working on a client project and they needed to integrate with a billing platform. They’d already picked Protx (now SagePay) as their platform of choice, and in particular, the Server variant. Wait, I’ll backtrack. SagePay has three variants:

  • Form which is basically just sticking a random form/button on your web page that redirects to SagePay’s servers and redirects back afterwards. Useful if your budget is low, you don’t have a web developer on hand and your requirements are utterly trivial.

  • Server which is more full featured, and allows much finer control of the billing process, including delayed billing, recurring subscriptions, that kind of thing. It’s a little more involved, but you still don’t capture the credit card details themselves (that is still a redirect step to SagePay’s servers) so you get to sidestep the painful parts of PCI-DSS compliance. Which you want to do, unless you have a bucketload of money and want to be labelled clinically insane.

  • Direct which involves you doing the store/forward of the credit card details. Its the ultimate in flexibility and doesn’t tie you to SagePay in the long term (say, because they have all the CC numbers for your recurring payments!). Choosing this option, however, is one of the criteria for free entry to the PCI DSS loony bin.

I’d previously built an integration for a client with Protx Form, and had modified ActiveMerchant to support it (forked here, not that the patch ever got merged, but n’mind). So, naturally, the second time, I reached for ActiveMerchant again and started hacking away. Unfortunately, SagePay Server’s model doesn’t really work with ActiveMerchant because it’s half way between an integration and a gateway. We struggled on and got that working (see this fork here). (That patch was never merged back upstream either, but I’m less surprised by that.)

Fast forward a couple of years, and another client wants an integration with SagePay. Their requirements are more complex in that its a SaaS offering which requires flexible recurring billing, so the Server version definitely fits well. But this time I’ve decided for sure that AM is not a good fit.

So we start from scratch and model SagePay Server’s API natively in Ruby. The sage_pay gem is the result of our efforts. There’s also an example Rails application using most of it’s features just so you can see what’s happening. Its all available under a liberal license, so you’re free to do what you like with it. I’d love to here from you if you use it. I’d also be utterly delighted to accept patches and improvements.

Documentation is a little sparse so far, so your best bet is to check out the sample app and work from there. I’d be particularly happy to accept pull requests that contain documentation. ;-)

(Oh, and while I’m on the subject of ActiveMerchant forks that never made it back upstream, here’s one for Barclays ePDQ too.)

Converging your Home Directory with Chef

Posted on by

In this tutorial, I’ll take you through using Chef and Homebrew to manage your home directory in Mac OS X. I’ve also included a neat cookbook which will allow you to use Homebrew as your native packaging system in Chef.

TL;DR

  • Chef is awesome.

  • You can grab the cookbook to use Homebrew as the native packaging system for Chef on GitHub at chef-homebrew.

  • Managing your configuration isn’t just for servers.

  • If you spend any time at a command line, you owe it to yourself to use tmux (or screen).

  • The formatting of the code samples on this post is a work in progress… Sorry.

Getting Started with Chef

Mac OS X, and the current breed of open source developer tools (notably Homebrew and RVM) introduce an interesting possibility when it comes to managing one’s home directory. These mechanisms allow you to install arbitrary packages, owned and run by your own user.

So, you’ve got a package manager which installs software and allows you to start/stop services. You’ve got a bunch of configuration files, both for those services, and dotfiles in your home directory. What’s the best way to manage these? With some sort of configuration management tool, of course! There’s plenty to choose from (including lcfg, cfengine and Puppet) but the one I’ve been working with on and off for the past couple of years is Chef.

How do you get started? Well, personally, I’d just choose one piece of configuration to manage. Let’s say, for example, that you want to manage your tmux configuration. Let’s create a cookbook for that. Start with a repository:

mkdir personal-chef
cd personal-chef
touch README
git init
git add .
git commit -m "First post."

I like to use bundler to manage gem dependencies. There’s only one dependency right now — chef — but we might as well start out sensibly. Create a Gemfile in the root of your repository and give it the following content:

source :rubygems
gem 'chef'

Save that and run:

bundle

which will install chef & its dependencies. You might want to noodle around with a .rvmrc to get an isolated gemset for the project too (I do), but that’s very much a personal taste thing (that I’ll cover in another post).

Now let’s create a really basic configuration for chef-solo to allow it to put things in the right place, and to find its roles & cookbooks. Create a config folder in your new repository and add the following to config/solo.rb:

root_path = File.expand_path(File.join(File.dirname(__FILE__), '..'))
 
cookbook_path   File.join(root_path, 'cookbooks')
role_path       File.join(root_path, 'roles')
 
# Move all the state stuff from /var/chef. I wish there was a single config
# variable for this!
state_root_path = File.expand_path('~/.chef/state')
file_cache_path  "#{state_root_path}/cache"
checksum_path    "#{state_root_path}/checksums"
sandbox_path     "#{state_root_path}/sandbox"
file_backup_path "#{state_root_path}/backup"
cache_options[:path] = file_cache_path

I haven’t caught all the paths pointing to /var/chef but it has been enough for chef to stop whining at me so far. ;-)

Now let’s create ourselves some data that tells chef what we want to do with ourselves. As the configuration grows larger, I’m probably going to want to introduce roles, but let’s keep it simple for now. Create config/mathie.json (adjust to taste!) and fill it in with the following:

{
  "run_list": [ "recipe[tmux]" ]
}

Very simple for now. The final bit of context we’re going to want before we get to the interesting bits is a quick script to run chef. Here’s one I prepared earlier, called converge.sh:

#!/bin/bash
chef-solo -c config/solo.rb -j config/mathie.json $*

(That should probably be bundle exec chef-solo but it failed the time I tried it, and I haven’t yet tracked down why.) Make it executable:

chmod +x ./converge.sh

Add that lot to your git repository and commit if you haven’t been doing so already (little and often!). The last thing we need to do before we can verify the machinery is working is create a skeleton for the cookbook itself:

mkdir -p cookbooks/tmux/recipes
touch cookbooks/tmux/recipes/default.rb

All being well, the next thing we can do is test that it’s all wired up correctly:

./converge.sh

which should spit out ~6 lines from chef saying that it’s starting, finishing and tidying up. If that doesn’t work so well, try:

./converge.sh -l debug

(which is why we stuck the $* on the end of the chef-solo invocation in there!) and see if you can see what’s going wrong. If you can’t, drop me a comment and I’ll see if I can help!

Converging on tmux

So, we’ve got the infrastructure in place. Next thing we need to figure out is installing packages. Chef has some neat separation between what it refers to as ‘resources’ and ‘providers’ so that, in your recipes, you can say:

package 'tmux'

and it will do the right thing, no matter what platform you’re on. Unfortunately, Chef doesn’t ship with the ability to manage packages with Homebrew. That’s why I come in. I’ve put together a cookbook which will install homebrew if it doesn’t already exist, keep it up to date and, most importantly, hook into Chef to use Homebrew as the native package manager. Let’s pull that wonderful cookbook in using git’s subtree merge strategy (which, since we’re DevOps Ninjas, in this situation, is pretty much the way forward):

git remote add -f homebrew git://github.com/mathie/chef-homebrew.git
git merge -s ours --no-commit homebrew/master
git read-tree --prefix cookbooks/homebrew -u homebrew/master
git commit -m "Pull in mathie's awesome Homebrew cookbook."

(If I’ve just lost you, or you’re not using Git, head to https://github.com/mathie/chef-homebrew and stick the contents of that repository in cookbooks/homebrew.)

Now we should be good to go with getting tmux installed through Homebrew. Edit cookbooks/tmux/recipes/default.rb so that it contains:

include_recipe 'homebrew'
package 'tmux'

Save and commit (early and often!). Now run ./converge.sh. It should bumble around for a bit, installing Homebrew if you haven’t got it already, and installing tmux using Homebrew. Run it again, and it shouldn’t do a whole lot. Win.

Converging a tmux configuration

Last thing we’re going to do here, just to demonstrate managing configuration too, is installing a managed ~/.tmux.conf file. Add the following to cookbooks/tmux/recipes/default.rb:

template "#{ENV['HOME']}/.tmux.conf" do
  source "tmux.conf.erb"
end

and create cookbooks/tmux/templates/default/tmux.conf with your favourite tmux configuration. Let’s say, for example, that you’re on Mac OS X and you want to know the current battery level in your status bar:

set -g status-right "#[fg=green]#(pmset -g ps |awk 'BEGIN { FS=\"\t\" } /InternalBattery/ { print $2 }')"

(kinda useful, huh?) Now run:

    ./converge.sh

and your tmux configuration will magically be installed into ~/.tmux.conf. Neat, huh?

Conclusion

Chef isn’t just for managing your massively scaled production infrastructure. You can use it to manage lots of other things too, including your home directory. The win here is magnified with each computer you work with (home desktop, work desktop and a laptop while you’re out and about?).

So, have you tried using Chef to manage your home directory? How did you find it? Good stuff? Bad stuff? What other strategies have you tried?

Update In addition to publishing the Cookbook on Github as chef-homebrew I’ve also uploaded it to the Opscode Cookbooks. Get it here: homebrew.

Equality, Comparison and Uniqueness in Ruby

Posted on by

Ruby has the Comparable module, which, if you implement the spaceship operator <=> (winner of “Best Named Operator” 10 years running!) then it will give you a bunch of comparator operators for free (<, <=, ==, >= and >). Win. Enumerable’s #sort method uses the spaceship operator to do sorting too, so implementing the spaceship gives you a whole bunch of interesting behaviour pretty much for free.

However, while you might think that Array#uniq used ==, or even <=> directly, to implement its uniqueness semantics, you’d be wrong.

First, let’s take a step back. There are a few ways to determine different qualities of equality in Ruby:

  • == is value equality where you consider two values to be equal. So, 1 == 1 or 1 == 1.0.

  • eql? is stronger than == so that in addition to value equality they must have type equality. The above example demonstrates the difference: 1.eql?(1) is true but 1.eql?(1.0) is false.

  • equal? is even stronger still; they must be the same object id. To be honest, the only time I’ve had call to use this is when testing that a DataMapper-style implementation was doing the right thing.

And it turns out that Ruby uses .eql? to determine whether an object is uniq or not.

Here’s the first thing I found slightly surprising: eql? is not implemented in terms of ==. To my mind, it’s an obvious implementation:

def eql?(other)
  self.class == other.class && self == other
end

and there would be faster implementation in C, of course, but I guess there’s a reason it doesn’t do that. Then again, I wonder if that should be:

def eql?(other)
  is_a?(other) && self == other
end

It’s not really obvious from the Pickaxe, and I can’t think of an example to test it with. Anyway, back on track. So, if you’re wanting to customise the way that #uniq works, you’re going to have to implement #eql? on your own class.

And you’re still going to be surprised that doing so doesn’t make #uniq work. See, there’s a bit of a performance optimisation going on here. First of all, it uses #hash to group together potential duplicates, then it uses #eql? to verify that the duplicates are definitely are the same.

So, if you want to customise the way #uniq works for your particular class, you also have to implement #hash. Here’s what I came up with in the end (imagine that nominal_code fully encompasses the object’s identity and ordering if you will):

class NominalAccount
  include Comparable
  def <=>(other)
    nominal_code <=> other.nominal_code
  end

  def eql?(other)
    self.class == other.class && self <=> other
  end

  def hash
    nominal_code.hash
  end
end

Now I can sort and uniqueify collections of these objects through their natural primary key. Win.

Why Ruby (and Rails) is Awesome

Posted on by

I was invited to give a short introduction to Ruby on Rails at Tech Meetup in Edinburgh a couple of days ago. I’d been racking my brain for days on what to talk about — 15 minutes is too short for me to give a meaningful introduction to Rails — and eventually settled on telling a few stories.



The slides don’t make much sense on their own, so I’ve included the “script” of what I talked about too. I deviated quite a bit from the script as I got into it, so hopefully I should be able to provide audio (or, dread the thought, maybe even video) of the talk in due course.

Intro



I’m Graeme. I’m the Managing Director of Rubaidh Ltd, and have been developing Ruby on Rails applications professionally for 3 years now.

Telling Stories



To be honest, I didn’t know what my audience this evening was going to be like. I wasn’t sure if you’d all be business folks in suits, or whether you’d all be hardcore geeks that know more about web development than I could ever hope to. So I didn’t want to have slides full of code showing the technical prowess of Ruby. And equally, I didn’t want a bunch of pie charts showing the return on investment you can make from choosing Ruby on Rails as your strategic development platform.

Instead I’m going to tell you a few stories. These are stories that shaped my decisions and represent some of the things that make me realise the power of Ruby on Rails. They are all true stories. They all happened to me. I may have changed the odd detail to protect the guilty parties, though.

I hope that, in imparting these stories to you all, not only will you appreciate my choice, but you will also reflect upon the choices you could make to improve your current (and future) business.

Migrations



I used to have the pleasure of working full time on an open source email response management system. It was the first open source business in Scotland to receive VC funding and was a fantastic product with huge potential. (It still is, I’m sure, but I don’t keep up with development these days.)

As is always the case, there were a few skeletons in the development cupboard. One was migration. The first prototype of the product used an object database back end. This database did not scale well. In order to meet the scalability requirements of the hosted service for the business, we shifted to a SQL-based backing store. We also made the decision to support multiple SQL databases – primarily MySQL and PostgreSQL, though there was demand in the open source community for MS SQL Server support too.

And there’s where we hit our major problem. There was no database-agnostic layer between the application and the back end database. It was a problem to maintain the slightly different SQL queries for different databases, particularly since a major part of the application was full text searching and we delegated that task to the database.

However, the biggest trouble we ran into was migration. Migrating people from the object database back end to either PostgreSQL or MySQL. Migrating people from one SQL schema to another. Or, in the case of one client, migrating from the PostgeSQL back end to a MySQL one. Our home grown solution was flawed and wound up with a number of client installations having subtly different database schemas (my fault!). So, in addition to having to cope with different database back ends, the problem had grown exponentially into having to deal with different schemas. Yeuch.

When I first looked at Ruby on Rails, one of the features that caught my eye was its built in support for database migrations. In a back end-agnostic manner, it will allow you to plot the course of a database schema through the versions of an application, gracefully (and consistently!) moving from one version to the next. This totally blew my mind after a day of dealing with a customer who’s database schema didn’t quite match my expectations…

Deployment



This is another horror story about the very same application. What can I say? It’s what eventually made me jump to Rails.

The hosted service, at this point, was spread across two servers. We had sharded the data — free accounts on one server and paid accounts on the other. This at least meant that when the free trial server got overloaded, at least the paid accounts didn’t suffer.

Deploying a new version of the application was a manual process. We would ssh into each of the servers, stop the front end Apache server (so that nobody could mess with the application while it was in an inconsistent state), stop the application servers, update the code from the subversion repository, perform a number of other tidy-up tasks that I dread to recall, start the application servers, and finally restart the Apache servers. Deployments would take up to an hour.

One nightmare involved a number of complex database migrations because the two servers had wound up with an inconsistent schema. They needed to be made consistent because a free trial customer had just upgraded to a paid account. It took us about 2 days to sort the production platform out. Downtime was about 5 hours in total.

Shortly after that incident, I ran into Capistrano, the deployment tool of choice for Ruby on Rails applications. It gracefully allows you to manage front end web servers, application servers and database servers, running on 1 machine or spread across 20. With Capistrano, deployments are simple, fast and, mostly importantly, reversible. In some of our applications, we’re doing in the region of 10 deployments a day. This means that we are delivering production-ready features to the business users who are requesting them up to 10 times a day. Can you offer that kind of turnaround?

Of course, it’s not just Capistrano’s deployment assistance that allows us that level of efficiency, but it is a key factor.

Velocity



This last story is one from since I started working full time with Ruby on Rails, but it often reminds me why I still feel we made the correct business decision. At the tail end of 2006, the Chief Operating Officer of a start up approached me for a bit of consultancy. They had put together an awesome business plan for an online training platform which needed a large web application to be developed. The COO had done his research and decided that Ruby on Rails was the right development platform and, in particular, that Rubaidh was the right company to deliver the platform.

In order to secure the funding required for the project, the company had put together a detailed specification for the application, including cost estimates and staffing requirements. I believe those estimates showed that it would take 4 developers around 9 months to build the application.

We came in as a full time team of three developers, and we delivered working software to the business every two weeks. After 3 months, the system was complete enough to allow the editorial team to start entering the training courses. And after 5 months, we had a production-ready application which the sales team could use as a demonstration of the system’s potential.

Just to underline the point here. A project that was estimated (by an experienced software engineer and project manager) should have taken 36 person-months to deliver was, thanks to Ruby on Rails (and Agile development practices), delivered in 15 person-months.

JumpStart



With apologies to Sun for stealing their name (and to Thoughtbot for apparently stealing their idea — we thought of it independently, honest!), what presentation would be complete without pimping our own company a little? We are offering a service to get your next project off the ground fast. We will:

* Start building your dream web application, working as an experienced in-house team, immediately.

* Help you hire, and train, your in-house team.

* Gradually step away and let your in-house team take over, when they are ready.

We’ve already done this once, and it was a major success for both us and client, so we’ve decided to turn it into a service we offer.

Scotland on Rails



Scotland on Rails Conference 2009 is running on 26th-28th March in Pollock Halls in Edinburgh. Keynote speaker, and Rails Core member, Michael Koziarski, recently described last year’s conference as “the best small conference” he’d ever attended. That’s high praise, because he attends a lot of conferences!

This year, we’re featuring keynotes from Marcel Molina Jr, long time Rails Core team member, and Michael Feathers, an early adopter of Extreme Programming and Agile development practices, who is probably best known for his book, “Working Effectively with Legacy Code.”

We also have presentations from big names in the Ruby community, like Yehuda Katz (Merb), Dave Thomas (Pragmatic Programmers), Scott Chacon (GitHub) and Jim Weirich (definitely, without a doubt, a werewolf!).

Tickets are £175 for the conference, though there’s an early bird offer of £150 available until Feb 8th.

Prior to the two day conference, Chad Fowler and Marcel are running an all day charity tutorial. All the proceeds go to the Children’s Hospice Association Scotland. It’s a £75 minimum donation, but considering these folks are both Pragmatic Studio teachers (where the courses are nearly £300/day), it’s an utter steal.

Run, don’t walk, to scotlandonrails.com to sign up.

To close, I’d like to share a story from Jim Weirich, one of our speakers at Scotland on Rails. He was talking a couple of years ago at RailsConf Europe about a financial application he and his team had developed in Java. It took a team of 4 of them about 6 months. As an experiment, they tried a spike of it in Ruby. It took them 4 evenings.

That’s not the impressive part, though. After all, since they now had a deep understanding of the problem domain, naturally they would be able to translate the solution into another language quicker than they could have written it in the first place.

The impressive part? The total number of lines of Ruby code required to implement the application was less than the number of lines of the Java version …’s XML configuration.

I can haz questions?



So, questions?

Ruby with an Accent

Posted on by

Alan Francis and I had the pleasure of speaking at RubyConf 2008 in Florida. We did a slightly newer take on Alan and Paul’s talk from RailsConf Europe 2008 entitled, “Ruby with an Accent: Organising a Regional Conference”, all about our experiences of organising and running Scotland on Rails.

Anyway, the lovely folks at Confreaks recorded the entire conference and have now uploaded the talks. You can find our talk here: Ruby with an Accent: Organising a regional conference. It’s definitely not my best performance ever (where did that accent come from?!), but the content is pretty useful.

In other news, registration for Scotland on Rails is now open. You can pick up an early bird ticket for just £150 (which is about $230) until 8th February. So, run, don’t walk, to the registration page to book your place now! I mean, can you afford to miss out on seeing Jim Weirich, Michael Feathers, Dave Thomas, Yehuda Katz and Marcel Molina Jr in Edinburgh this March? It’s going to be an awesome weekend!

What do I most love about git?

Posted on by

Scott Chacon just asked on Twitter:

poll: gitsters – what do you love most about git and why it over other scm/dscms?

— chacon

I’m only going to answer the first part, really, as I haven’t really evaluated any other distributed SCM since Bitkeeper in 2002/3.

So, what do I love most about git? Well, it’s the branching, but maybe not in quite the same way as everybody else. My brain can be somewhat easily distracted (I have been compared to an over-caffeinated butterfly in the past). In addition to that, I have clients who’s number one OMG super-critical the-world-is-going-to-end bug fixes and feature requests can change on a daily basis.

So I have multiple lines of development running at once. On my own. In my own development tree on my laptop. I can switch from one line of development to another with a single command. I can combine features from one line of development into another. When I’ve finished one of the things I started, I can merge it back in. And I can rebase all my other lines of development to coincide with the changes.

Since I’ve got a great test suite, I can be sure that all of this branching and merging (which, in my case anyway, shows up quite a few conflicts) isn’t breaking code, either.

Anyway, that’s what I most love about git: not just short-lived branches, but an easy way to have multiple lines of development for arbitrary lengths of time and not have any of those lines rot over time.

Now, if only I had git stashed what I was doing before I context switched to write this post, I could apply it again and remember what I was supposed to be doing!

Dropping Daily Rates

Posted on by

I had a thought while I was heading out to RubyConf 2008. I’m currently looking for more contract Ruby work and so I was planning to noise up a few folks and see if there was anything going.

What I hadn’t realised is that I (and other UK developers) are particularly attractive to US companies right now since, due to the exchange rate fluctuation, my daily rate has dropped by 25% in the last 6 months!

Back in May 2008, when I was at RailsConf, the exchange rate was almost $2 to the pound sterling. Let’s say that, hypothetically, I charge £100/hour (I don’t, though I wish I could!) since it’s a nice round number. That would mean I would have to charge $220/hour to a US company just to maintain the same level of income (adding on a somewhat arbitrary 10% to cover the extra costs involved in getting money from a US company to my UK bank account).

Now, the exchange rate has dropped to about $1.50 to the pound. That in turn means that what I have to charge has dropped by 25%. Let’s continue the example from above. In order to maintain the same level of income, I would now only have to charge $165/hour. Doesn’t that sound a whole lot more hospitable?

So, US companies: if you’ve been toying with the idea of making use of the awesome pool of UK Ruby talent that’s available, now is the time to try it out!

And, to the UK Ruby talent; if you’ve always fancied the idea of remote working for a US company, now is a good time to convince them it’s worth a try. But only once I’ve got my books full again. :-)

Oh, and the main, subtle, take-home point of this post: Rubaidh Ltd is looking for new work and is happy working for anybody, anywhere! Get in touch! :-)