andHapp

A while back, I was working with an application that used custom headers. Applications use custom HTTP header to transfer domain specific information, either in request or response. Earlier the trend was to prepend “X-” to the custom header, but since June 2012, this approach has been deprecated.

I used rack-test to write some tests and to my surprise it appends HTTP_ in front of custom headers. I think it may do that in front of other headers too, and I did read this somewhere, but I can’t seem to find that link anymore. So, the first lesson of this post is to make others aware that HTTP_ is appended to custom headers with rack.

All this worked smoothly in local environment, and the moment I pushed it to Nginx+Passenger, it just blew up. Debugging revealed that it was actually just the way custom headers are treated in Nginx. The custom header had underscores in it, and Nginx is known to ignore custom headers with underscores, and therefore, this custom header never made it to the actual application. For example: X_API_KEY will be ignored by Nginx, where as X-API-KEY will be translated into X_API_KEY. I had to change the custom header to X-API-KEY and remove the underscores.

To summarise, these are the stages a custom header goes through:

1. Starts it’s life as X-API-KEY in the client request

2. Reaches Nginx, and is changed from X-API-KEY to X_API_KEY

3. Reaches Rack, and is changed from X_API_KEY to HTTP_X_API_KEY

4. Should be interpreted as HTTP_X_API_KEY in the actual application code.

Hope it helps anyone facing a similar issue.

The first think we learn as programmers is DRY i.e. Don’t Repeat yourself yet we programmers break that principle all the time, sometimes unknowingly, and sometimes on purpose. Writing a gem is easy, but why write one from scratch when one can extend and improve an existing gem. Anyways, just my opinion.

I never understood the reason for rbenv. I love rvm far too much to move to rbenv. My work colleague suggested that rbenv is much faster than rvm. I just ignored him then, but later on decided to investigate it further. I decided to write a small Dtrace script.

It’s a very trivial DTrace script. It collects and displays the process count called in the background. So, if it just lists the process (or scripts) with the count when I execute bash. In my bash script, I had the following line which loads rvm and allows for selecting gemsets and rubies.

 if [[ -s ~/.rvm/scripts/rvm ]] ; then source ~/.rvm/scripts/rvm ; fi

Here are the results:

In short, rvm script makes quite a few extra calls and that makes it different from rbenv. But, it’s not a significant difference for me to switch or write another library that does exactly the same.

If you think about it, RVM is trying to solve two problems:
1. Store gems based on projects, so one can easily separate a Rails 2.3 application’s gems from Rails 3′s gems.
2. Easily switch between different rubies.

Does it need to do that? In my opinion, Bundler is quite good at managing gems. Therefore, my solution to this issue is to not create gemsets. That’s exactly what rbenv does. In the meantime, I have created a simple bash function to load rvm shell on demand.

I wrote a quick script to extract contents from my gmail account. Here’s a gist. It might be of some help to someone.

For non-programmers, Spring symbolises sunshine, warm breeze and holiday season. However, the story is a little bit different for Programmers. Yes, Spring brings timezone issues for Programmes. In other words, the moment of truth is upon us now and it’s time to check your timezone code. I spotted two obvious issues, and both are somewhat related to github. My pull request on rails are showing an hour behind the actual time, and the resque web admin retries the failed jobs an hour before the actual time.

Anyways, just a little Spring warning.

Testing has always been part of the Rails framework, right from the beginning. The whole TDD concept became hugely popular after Rails was adopted in the mainstream. It has been around but Web applications were tested in a different manner 10 years ago. On my first day as a Java Developer, I was told by a Senior Developer that you can’t really test a Java web app by writing unit tests. I took his advice and frankly speaking, I didn’t know better.

Anyways, I am not here to talk about my career. I have been doing Rails for quite a while now and on one project, test suite took about 30 minutes to run. I know, 30 minutes. Because, we didn’t fake the calls to external APIs. How silly was that? So, recently I started working on the project and every time I ran the test suite, it took 2 minutes for them to run. That kind of put me off the whole testing thing. I mean what’s the point. I can just write the code and use the good old presentation driven development, where if it works it works. If it doesn’t, then we’ll fix it. This strategy kind of works, but becomes a nightmare when one bug fix results in another regression bug.

I ventured out looking for solutions and I found that instead of fixing the issues around testing, community has moved to the whole client/server architecture towards testing. I started on Spork. It’s a very nice idea but I don’t want to muck about with my configs and do the setting up. The downside is that now I have to get the entire team to use Spork and set it up and all. I know it works and a lot of developers use it but I don’t want to do it. Then, accidentally, I found spin. Spin is the same client/server idea but instead of a drb server, like Spork, it uses UNIX sockets for communication. There is absolutely no setup required. I have been using it on two different projects and it has worked nicely.

Some other solutions are to load your database in memory, easy to do with sqlite but with postgres you have to create a ram disk and load the database from there. Stop abusing machinist, factory_girl and the like as they make db calls. Instead, use mocks and stubs.

But, I still think these are just workarounds for the existing problem with Rails load times and testing. Is there a better solution to this problem? How can I make my application testable, follow TDD and yet never have to worry about loading rails? I guess I need to objectify my application. I will probably talk about it in my next post.

If you haven’t already installed the release candidate of superfast bundler, then I feel sorry for you. It’s hard to put your faith in me but here’s an article that might change your heart. It’s super simple, just do:

gem install bundler --pre

Last year, I released a gem called is_bot (which is just a trivial way to fool the spammers). I have been using it in several projects and keep updating it with new releases of the Rails. Few months back when rubygems was having trouble with Syck yaml parser, I released a version (0.3.3) of my is_bot gem. As a result, this version of is_bot suffered from the same issue and it gave me error messages on subsequent installs. Since then, I have released three more versions of the gem and the current version stands at 0.3.6.

Now, the earlier version of bundler was quite forgiving and I could hack the cached gemspec file to install it. But the new version of bundler, simply rejects it. I was quite surprised to see the error message I get from bundler when I tried installing the current version (0.3.6). Because, it doesn’t say anything about the 0.3.6 version but complains about 0.3.3 version. I thought I was going crazy. I even specified the version of the gem to 0.3.6 in my Gemfile but still it complained about 0.3.3 version. I know, I know that bundler needs to download the ‘Big Index’ for dependency resolution but I never had this problem with the earlier versions of bundler, then why now. Why my gem?

So, I did more investigation and ran the following command to see what exactly does bundler get back when it hits the rubygems’s endpoint:

ruby -ropen-uri -rpp -e \
  'pp Marshal.load(open("https://rubygems.org/api/v1/dependencies?gems=is_bot"))'

and this returns something like this:

[{:platform=>"ruby",
  :name=>"is_bot",
  :dependencies=>[["rspec", ">= 0"], ["rails", "= 3.1.1"]],
  :number=>"0.3.6"},
 {:platform=>"ruby",
  :name=>"is_bot",
  :dependencies=>[["rspec", ">= 0"], ["rails", "= 3.1.1"]],
  :number=>"0.3.5"},
 {:platform=>"ruby",
  :name=>"is_bot",
  :dependencies=>
   [["rspec", ">= 0"],
    ["rails", "# 3.1.0.rc6"]],
  :number=>"0.3.3"},
 {:platform=>"ruby",
  :name=>"is_bot",
  :dependencies=>[["rspec", "= 2.6.0"], ["rails", "= 3.1.0.rc5"]],
  :number=>"0.3.2"},
 {:platform=>"ruby",
  :name=>"is_bot",
  :dependencies=>[["rspec", "= 2.6.0"], ["rails", "= 3.1.0.rc4"]],
  :number=>"0.3.1"},
 {:platform=>"ruby",
  :name=>"is_bot",
  :dependencies=>
   [["sqlite3", "= 1.3.3"], ["rspec", "= 2.6.0"], ["rails", "= 3.1.0.rc4"]],
  :number=>"0.3.0"},
 {:platform=>"ruby",
  :name=>"is_bot",
  :dependencies=>
   [["sqlite3", "= 1.3.3"], ["rspec", "= 2.4.0"], ["rails", "= 3.0.3"]],
  :number=>"0.2.0"},
 {:platform=>"ruby", :name=>"is_bot", :dependencies=>[], :number=>"0.1.0"}]

And you can see “YAML::Syck::DefaultKey:0xbd36f20″ bit that results in illformed gemspec error. I guess the earlier versions of Bundler had a fix for this issue, but the latest rc version loads up this information and straight-away complains about the illformed gemspec. That’s all good but what’s the solution for this issue. Simple! Yank the 0.3.3 version of the gem and you are good to go. Another solution would be to regenerate the gemspec for 0.3.3 again, which I hope rubygems stores uncorrupted.

Phew.

The title could be slightly misleading but I am not talking about installing clearance in your Rails application. This post is just a small note to myself since I had considerable difficulties in getting the clearance gem’s test suite working in my local environment. It’s a very trivial detail that I missed or rather overlooked whilst trying to get them working. For readers who don’t know, Clearance is a Rails authentication engine and is developed by ThoughtBot and let me tell you that they love Cucumber. For testing the engine, Clearance uses Aruba, i.e. CLI steps for Cucumber.

The background story is that I have been quite interested in Clearance for quite a while now and am quite active in the ML and do chime in with my opinions and I just updated my local fork after a while and ran the specs and features. My preferred approach when bug fixing or adding any features or just code browsing any library is to install the dependencies in the vendor directory via bundler. I could use rvm gemsets and install the dependencies there but I usually don’t go for that approach.

So, I did exactly the same for clearance and installed the dependencies to vendor directory and ran the specs and features. Specs ran without any issues but with features I got a weird “bundler can’t be found” error. I was stumbled because I do have bundler installed on my machine. Anyways, after hours of debugging I figured the root of the cause. Below is the cucumber feature that does the background work in testing the engine(*taken from Clearance repository):

When I successfully run "rails new testapp"
    And I cd to "testapp"
    And I remove the file "public/index.html"
    And I remove the file "app/views/layouts/application.html.erb"
    And I configure ActionMailer to use "localhost" as a host
    And I configure a root route
    And I add the "cucumber-rails" gem
    And I add the "capybara" gem
    And I add the "rspec-rails" gem
    And I add the "factory_girl_rails" gem
    And I add the "dynamic_form" gem
    And I add the "database_cleaner" gem
    And I add the "clearance" gem from this project
    And I add the "diesel" gem
    And I run "bundle install --local"
    And I successfully run "rails generate cucumber:install"
    And I disable Capybara Javascript emulation
    And I successfully run "rails generate clearance:features"

and you can see it creates a new rails app and installs the dependencies as part of the feature.

To run the test suite I used the following command:

bundle exec rake cucumber

because I installed all my dependencies in vendor and for them to be in the LOAD_PATH ran rake scoped via bundler. Now, when it came to running the CLI steps it complained about bundler’s absence because it was not installed to vendor as it was not in the Gemfile.

I did not use:

rake cucumber

because then bundler path would not kick in and the features would fail straight-away. So, how can we get this to work. Simple. Just create a new gemset for all the clearance development dependencies and get bundler to install it to the gemset as opposed to vendor and then run the features using:

rake cucumber

And they will be green. Phew. It took me a while to figure this out but I am so relieved I did.

About 3 months ago, I was working on this piece of code which had some “virtual models”. Virtual models are the entities that are well defined models except they are never persisted to the database. They are just kept in memory and discarded. You could by all means persist them but I didn’t have to do it at the time.

The application had a search interface (that allowed users to search across the application). We all know any user interaction needs robust validations and it was same for this application as well. The interface allowed users to select 4 (or 5) different filters to search across. So. the task was to capture the user supplied search filters and validate them and if valid search the and display the results based on it. There are several ways to solve this problem but I think with ActiveModel API the task at hand becomes really really easy.

ActiveModel API makes a ruby object feel like activerecord. This essentially means creating one “Search” virtual model (object) and use ActiveModel API and add validations to it. And all the search filters are defined on the Search model itself which is in line with Martin Fowler’s Parameter Object design pattern. Well, better designed code promotes better practices.

Sorry, for absence of any code in this post as I did this ages ago and was for a client project. Please leave comments if you need any clarifications.

I always wanted to do a technical presentation and it finally happened on 29th March 2011 at Woking. I just think putting yourself in front of 20-30 people and to respond to their queries is a courageous task and I am quite pleased to have experienced it now. Well, it all started off with a simple email from the SRUG organiser and I dived head first. Since it was my first ever presentation, I was more inclined to pick an easy topic. I am thankful there that there is a Surrey Ruby User Group’s Uservoice page with several preferred topics and I decided to go for “Creating Gems”. You can find my presentation “Creating Gems” on github. I used showoff gem along with a simple screencast.

So, now here’s what I learnt from the whole experience. This is my personal feedback on my performance:

• Always, always and always carry all the equipment you might need on the day. For example: I did not have a Mini DV to VGA adapter to hook up with the projector which meant I had to go second.
• Check your presentation on different resolutions. Projector’s resolution can spoil the whole presentation. It works on my machine would just not cut it.
• I am still quite new to showoff gem and I should have spent more time exploring it. It’s quite amazing and you can do a ton of stuff with it but I still need to delve into it a bit more.
• Use an iPad to do your presentations. Jon (the fellow presenter on the night) was using an iPad and the whole experience was just amazing.

and now on the funny side:

There’s a twitter user called killallclients. Hilarious stories in that stream and ofcourse you know that such presentations are amazing for networking and meeting people who could offer you work in the future. I was quite disappointed by my presentation and I am working on improving it next time.

Yahoo announced quite a while ago that delicious (their bookmarking tool) will be shut down and I started looking for alternatives. There are few things I am looking for in a bookmarking tool:

1. I don’t want to pay for it.
2. It should have a nice interface to work with.
3. It should have a browser extension for example: delicious Firefox extension was a pleasure to use.
4. It should have a nice API to I can build my own extensions (provided none exists) or may be a desktop app.
5. and I don’t want to pay for it. )

Luckily, there is another similar bookmarking tool known as Trunkly which ticks most of my requirements except that I could not find a browser extension for the same. So, as I said, I scratched my own itch and built one for Chrome. One thing that must be said is that writing an extension for Chrome is the easiest thing I have done in a while. The documentation is amazing and if you can read the documentation and follow the samples then writing an extension will be a walk in the park.

At the moment, the code is very young and it uses HTML5 localStorage to save the API key which might not be secure due to all sorts of reason but this is early days and I plan to change it in the later iterations or perhasp use the web database. The extension is not hosted in the Chrome store yet or anywhere else and if you do want to try it out just download the zip and load it in Chrome. The code is hosted on github and if anyone wants to improve it or hack it, please go ahead.