Load Rails Fixtures in Production Mode

Another installment in the “it probably exists elsewhere on the web, but my Google searches weren’t finding it”… how does one take their development database with them into production mode (for performance testing purposes, in our case).

Option 1 – Export Fixtures, database independent. Complicated.
First, run “rake db:extract_fixtures” to make your existing database into a set of YAML files that are saved in your /test/fixtures directory.

Next, copy the .yml files in your /test/fixtures directory into a subdirectory of your /lib folder. We have a folder for rake tasks, which is /lib/tasks, so I copied the fixtures into /lib/tasks/production_fixtures.

Finally, create a task that will import all those fixtures. Part of my reason for writing this blog is in hopes that someday someone will stumble upon it and suggest a better way to accomplish this importing of a directory of fixtures. But until then, you can create a file called “db.rake” in your /lib/tasks folder, then add the following lines to that file:

task :load_fixtures => :environment do
require 'active_record/fixtures'
ActiveRecord::Base.establish_connection(RAILS_ENV.to_sym)
Fixtures.create_fixtures('lib/tasks/production_fixtures', 'table_name_pluralized')
Fixtures.create_fixtures('lib/tasks/production_fixtures', 'table_name_2_pluralized')
...

Where “/lib/tasks/production_fixtures” is the directory you put your .yml files in. With this method you need one of these lines for every table, but it works (I just created a list of the files by outputting my directory list into a text file and copying it into the text file).

Finally, run “rake db:load_fixtures RAILS_ENV=production” and you got yourself a populated production database.

TODO: How does this compare to “rake db:fixtures:load”?

Option 2 – Copy MySQL database. Requires MySQL. Simple.
This option is far simpler, but requires MySQL for it to work in the way I’m about to describe. These examples assume that your mysql user is “root”, password is “secret”, and db names are “development_db_name” and “production_db_name”.

To dump your existing database to a file,

mysqldump -u root -psecret development_db_name > db_dump.sql

And to import it into a new database

mysql -u root -psecret production_db_name < db_dump.sql

Voila.

NOTE: Make sure your development and production databases are on the same migration before trying this stunt.

Compare Rails IDEs for Windows

Jordan said it, and I couldn’t agree more: one of the real weaknesses of Rails development in Windows is the lack of a killer IDE. I never realized how good I had it when I was developing C++ with Visual Studio and Workspace Whiz, but after a few months of developing in Rails, where I’ve gone through about an IDE per month, I have sussed out my baseline requirements for a usable IDE. Most of these things are not what I’d consider “fancy” features, but if they aren’t in my IDE I get very annoyed very quickly:

  1. IDE keeps a project workspace that can be updated to include new files with somewhere between “little” and “no” effort
  2. When I press “CTRL-F” on a word, that word has automatically populated the find dialog box
  3. There exists the means to define a key combination for invoking a “find in files” dialog box,
  4. And that find in files dialog can search for regular expressions
  5. I can open any file in the project in a couple keystrokes without using my mouse
  6. Syntax highlighting exists for HTML, CSS, JS and Ruby
  7. The IDE does not fight with me about tabbing
  8. The IDE doesn’t crash
  9. The IDE doesn’t pause and stop me from what I’m doing. Ever.
  10. Intellisense (Not essential, but highly desirable)
  11. Built-in debugging (Not essential, but highly desirable)

I guess they aren’t trivial requirements, but it doesn’t seem to me like I’m asking for a ton here.

So far, I have over the last six months spent some time using (in chronological order): EasyEclipse, Ultraedit, PSPad, NetBeans, and Ruby in Steel.

I don’t have the time to do a wordy comparison of the bunch of them at present, but the following spreadsheet details my experience with them in regard to my 11 essential (and almost-essential) attributes:

As you can see, some IDEs got evaluated more than others, because some of them got on my nerves almost instantly (UltraEdit, Netbeans), and some stayed in good favor for long enough to evaluate them thoroughly (PSPad, Ruby in Steel). My biggest beef with Ruby in Steel had been that I could not figure out a way to get a file opened with a single keystroke. But after exhaustive Googling, I learned today about the magic of CTRL-ALT-A, which brings up a command window where I can type “open [filename]” to immediately open any file in the project. Eureka! At $50, Ruby in Steel is currently my decisive winner in the contest for least-annoying Rails IDE.

— UPDATE —

After some cajoling, I persuaded Josh Jensen, creator of Workspace Whiz, to create a version of his world-class Visual Studio plugin that is compatible with Ruby in Steel. What does it mean? It means that you can get instantaneous one-key lookup of any file or tag (=method or class name in Javascript or Rails) in your entire project. That is, you can hit CTRL-D, type “message” and, as you are typing in “message”, Workspace Whiz will be interactively telling you every Rails or Javascript function/method in your entire project that begin with the word “message”, similar to Google Desktop searching, if any of you use that. Or you can just press CTRL-space on any method and have Workspace Whiz jump to the method (again, Javascript or Rails). CTRL-left arrow will then return you to where you started.

Pretty damn cool stuff. Curious if RadRails has anything like it?

The only question is when Josh will release this version to the public. If you post a comment that you want it below, I can find a way to make it happen.

Comparing Rails Geolocator Plugins

James Stewart maintains a terrific blog that enumerates the particular differences between the many Rails plugins that interface with Google Maps and Yahoo! Maps (both for showing the maps, and for doing distance calculations, etc.). On his blog he refers to an “updated comparison” of all the plugins that exists on the “foss4r wiki.” However, the link he gives has been dead for awhile. I found the cached version on Google and thought others might find it helpful to put it online here on a site that exists. It ain’t the most beautiful table you’ll ever see, but if you’re doing geolocating with your Rails app, it is one of the most useful ones (click to see the full-size version).

I’ve been using Geokit and YM4R, but have been disappointed that as yet I haven’t been able to find a way to get Geokit to look up distance using a through clause, that is, it looks like any model that you want to do a distance lookup on needs to be “acts_as_mappable” itself. This is a bummer, since we have at least three models that will all need to store the same [lat, lng] coordinates if I’m not wrong about this. Not very DRY, but for the most part, it is a very functional, easy to setup, and extremely well documented plugin that I would recommend.

Savage Beast 2.0 – A Rails 2.0 Message Forum Plugin

Update: If you’re running Rails 2.2 or 2.3, see the newer version of Savage Beast.

I’ve been working the last couple days on creating a new version of the original Savage Beast plugin that is Rails 2.0 compliant and integrates the changes to the Beast source code that have been added over the last year. The result has been an interesting trek through the ins and outs of Rails plugin writing, that has given birth to a new version of the Savage Beast plugin, re-ported from scratch from the Beast trunk.

Installation

Currently, the following is necessary to use the Savage Beast plugin:

  1. The Savage Beast 2.0 plugin. Go to your application root directory and:
    svn export http://savage-beast-2.googlecode.com/svn/trunk/ vendor/plugins/savage_beast
  2. Most of the stuff you need to run Beast
    • Redcloth: gem install Redcloth
    • A bunch of plugins (white_list, white_list_formatted_content, acts_as_list, gibberish, will_paginate, engines). The easiest way to install these en masse is just to copy the contents of savage_beast/tested_plugins to your standard Rails plugin directory (/vendor/plugins). If you already have versions of these plugins, you can just choose not to overwrite those versions
    • For the engines plugin to work, add this line to the top of your environment.rb, right after the require of boot: require File.join(File.dirname(__FILE__), '../vendor/plugins/engines/boot')
  3. Copy the migration in /vendor/plugins/savage_beast/db/migrate into your own migration directory (and run it)
  4. Implement in your User model the four methods in plugins/savage_beast/lib/savage_beast/user_init that are marked as "#implement in your user model
  5. Add the line “map.from_plugin :savage_beast” to your routes.rb. Location shouldn’t matter unless you intend to override it.
  6. Add the line “include SavageBeast::UserInit” to your User model. Location shouldn’t matter unless you intend to override it.
  7. Implement versions of the methods in SavageBeast::AuthenticationSystem (located in /plugins/savage_beast/lib) in your application controller if they aren’t already there (note: technically, I believe only “login_required” and “current_user” are necessary, the others give you more functionality). Helpful commenter Adam says that if you have the “helper :all” line in your application controller, be sure to add the “SavageBeast::AuthenticationSystem” line after that.

And off you go! When you visit your_site/forums something should happen. I’ve been creating new forums by visiting /forums/new. There’s probably a hidden admin view somewhere.

I’d like to remove some of these steps during the plugin install process with subsequent releases (is it possible to install a dependent plugin during your plugin install process?), but given that this is my first plugin project (and not a small one at that), I’m just trying to “get it done” before I “get it done beautifully.” I think DHH said somewhere that I should do that.

Implementing Your Own Views and Controllers

The engines plugin makes it eminently easy to mix in your own stuff as you see fit. Just create a new file in your /controllers or /views directories with the same name as the file you want to override in Savage Beast. If you just want to override a particular method in a controller, you can do that piecemeal if you just leave your XController empty except for the method you wanted to override.

If you’re integrating this into an existing site, I’d recommend you start by creating a forums layout page (/app/views/layouts/forums.html.erb). This will give you a taste of how easy it is to selectively override files from the plugin.

Demo

You can check out a (slightly-but-not-too-modified) version of Savage Beast online at Bonanzle. The differences between our version and the version checked into Subversion are 1) addition of topic tagging (users can tag topics to get them removed, etc) 2) recent post list shows posts in unique topics, rather than showing posts from the same topic repeatedly (there’s another blog on here about the SQL I used to do that) and 3) skinning. None of those changes feel intrinsic to what SB is “supposed to do,” which is why they aren’t checked in.

Differences from Savage Beast 1.0

The main difference is that this incorporates about a year’s worth of progress to the Beast source, and it actually takes that code a step further by being Rails 2.0.x compliant.

One thing Jodi seemed pretty excited about in the first Savage Beast was the ability to create forums off of models. Bonanzle doesn’t need that functionality, so I haven’t tested it, but I imagine it will probably work if you follow the steps that did it in the first Savage Beast.

TODO

Would be nice to have some help writing tests. Would also be great to figure out how to install this through the standard “ruby script/plugin install”.

Without Engines?

There are a couple means by which you can use this plugin without using the Engines plugin. I originally had intended to do this, suspecting that Engines would be a performance detriment. Turns out it’s not. But maybe you want to roll without Engines for some other reason.

The most dumb-easy way to get by without Engines would just be to copy the views, helpers, controllers, and models into your own project. Yes, it means having a lot more of all the above in your project, but it’s no worse than if you were coding the plugin from scratch yourself (actually, I’d argue it’s a lot better, since you didn’t have to do the work yourself :)).

Another way is to uncomment the code at the bottom of Savage Beast’s init.rb. Before I knew about the Engines plugin I created the code in there that simulates Engines without having Engines installed. The caveat is that you won’t get Engines’ ability to selectively override particular methods/views, and you need to copy all your helpers into the SB/lib directory, where they become global (yuck). If anyone out there has cycles to hone this Engines’-less approach, please email me and we can talk about getting those changes into the project.

Conclusion

Comments are most welcome. I’ll be checking in changes to the project as I find bugs and improvements in using it, but this is admittedly something I don’t have a lot of spare time to closely follow (see my other entries on the wonders of entrepreneurship). Hopefully others can contribute patches as they find time. If you like the plugin, feel free to stop by Agile Development and give it a rating so that others can find it in the future.

Rails Track Online Users through Session

We recently needed to implement tracking which users are online at our site. Matt Beedle posted a good starting guide to how to do this, but I found that there were a possible improvements from his tutorial.

Like Matt, we’re using an ActiveRecord session store to track who’s online when. Unlike Matt, we wanted to keep track of when the user last accessed a page, and what page they accessed. The most SQL-efficient way I could find to do this was to add fields directly to the session model and update those. To add the fields to the model, I just created a migration that added the stuff I wanted to track:

add_column :sessions, :user_id, :integer
add_column :sessions, :last_url_visited, :string

Next, I created a before_filter in our application to update these as the user moved through the site, like so:

session.model.update_attribute(:user_id, session[:user_id])
session.model.update_attribute(:last_url_visited, request.url)

With these properties getting updated, all that remained was to write a query to figure out who’s been moving around the site recently. This ended up being as simple as:

online_sessions = CGI::Session::ActiveRecordStore::Session.find( :all,
:select => "user_id, last_url_visited",
:conditions => [ "updated_at > ? and user_id is not null", Time.now() - 30.minutes ],
:limit => 50 )

And that was about it. The tracking is just one SQL UPDATE a pop, and the session find doesn’t need any fancy joins and such, so it ought to scale pretty well.

Rails Performance: A Brief History of the Universe

After much ado, the time has finally come to get Bonanzle moved to real servers, with Capistrano and all the tricks the big boy Rails apps use. But as I started to look into the logistics of this, I ran into a quick interview with Alex Payne regarding Rails performance on Twitter that made me raise an eyebrow. A couple minutes later, I came to the realization that this “quick interview” actually represents the center of an impassioned web of controversy that has enveloped the Rails community regarding the scalability of Rails.

On one side of the argument are DHH and the programming purists, whose solution to the performance problem is to say any number of things other than, “yes, performance is an important issue to us, and we intend to make it better.” On the other side of the argument is people like Rob Conery and Joel Spolsky who write that maybe RoR developers should give some weight to these performance concerns, as they seem to be the only argument of substance against mainstream RoR love.

Judging by the responses to Rob’s article, methinks there is still quite a ways to go before the community will come to acknowledge the problem at hand. What makes me think it’s a problem? Um, how about objective data from reliable sources? Ruby is slow. Rails is slow and getting slower as time goes on. To be fair, I tend to agree with Joel that Rails can never be fast until Ruby is, so this isn’t all on DHH, as some have implied. At the same time, when some crazy Japanese guy (developer of YARV) with this web page is the supposed prophet of Ruby optimization, I am concerned. Even if the guy is crazy like a fox, the extent of what he will divulge about the future of YARV is that “YARV development is too HOT!” Better HOT! than cold, but something like an expected date of arrival would be a comforting piece of data to add to the development page.

Others data bits I’ve seen around:

* Some suggest JRuby is the answer to Ruby’s problems. But by all accounts, it isn’t the solution to Ruby’s performance problems. At least, not yet.

* A common argument I see being used to dilute the significance of performance is that “performance isn’t scalability.” What people mean when they say this is that poor performance doesn’t always mean you’re unequivocally screwed. And to the extent that you have developers that like to spend their time setting up caching and multi-database transactions, I suppose that is true. But as a programmer who knows a lot of other programmers, I can pretty confidently say that “how to implement a load balancer” is not the sort of problem that many programmers exalt to face. Esoteric details about how to implement Pound or a memcache or HTTP connections between multiple Mongrel processes and an Apache frontend — this is the punishment that Ruby on Rails developers bear for the joy that they experience programming their apps.

* Another common “solution” to the problem I see is to use something similar to RoR that makes an effort at better performance. Entries from this side of the arena include Grails and Merb. The question I’d be asking myself if I were a Rails evangelist that wanted to see the language thrive is, “why do these similar applications exist?” Partly because people are comfortable with legacy languages (in the case of Grails), but moreso because there are many practical people who love RoR as a language, but can’t stomach the performance trade-offs that go along with it.

I suspect that many Rails-evangelists may be quick to point out that if I don’t like it, I can go to hell or fix it myself. And if I won’t fix it, I am in no position to complain, blah blah blah. Whatever. The first step is admitting we have a problem. If there is any blemish upon Rails name, this is the one. Can we agree on that?

Manually Set Rails Migration Version

Here’s another piece of info that wasn’t easy to discover using the Google query I expected. If you should ever find yourself needing to explicitly set the migration version that Rails thinks you are at, perform the following steps:

1. Open your Database (using MySQL, this should be something like “mysql -h localhost -u root -p”, followed by “use [database name];” in the MySQL console)

2. Run “Update schema_info SET version=[version number];”

And you’re done!

Automatically Reload Modules On Change In Rails 2.0.x

This problem had me stumped longer than any other single problem I’ve run into so far, and no group or search query I could come up with was able to help me; ultimately I had to figure this out by diving into the actual Rails source code and digging until I found the right concoction. Perhaps this solution was not more findable because we’re running edge Rails? All data I could find on the forums seemed to say that Rails 1.2 did not have these sorts of problems, but I can neither accept nor refute that.

The problem I’d been having was that I had created a Rails Module in a file that was in a subdirectory of my /lib directory. The point of this Module was to allow me to share some methods between a select few Rails controllers without putting the methods into the global Application.rb (which seems sloppy to me…methods should only be available in the scope they’re needed says I). Actually, figuring out that I could create a Module in the lib directory to DRY up my controllers was also surprisingly difficult to find Google information on… you’d think people would object to putting stuff in Application.rb more often. Anyway, I digress.

This Module in the lib subdirectory had one annoying problem: when I made changes to it, I had to stop and re-start my Rails server for those changes to take effect. That got old fast. After the aforementioned digging, here were the TWO LINES OF CODE that took me way too many hours to figure out.

In Environment.rb: Add the path for the lib subdirectory where my module was stored, e.g.

config.load_paths += %W( #{RAILS_ROOT}/lib/item_setup )

There is a commented line in environment that has an example of the syntax for this.

In Development.rb: Add the name of my Module to the explicitly_unloadable_constants list, e.g.,

Dependencies.explicitly_unloadable_constants = 'NameOfMyModule'

And voila! It works. If you have other modules you want to automatically reload, use the << operator to add them to the explicitly_unloadable_constants array, e.g.,

Dependencies.explicitly_unloadable_constants << 'NameOfAnotherModule'

The other possible problem that I read about on the forums that didn’t come about for me was that if my lib directory had somehow gotten listed in Dependencies.load_once_paths, then it would not reload on being changed. You can Google around to figure out how to fix that (hint: delete your directory from the load_once_paths array), but it probably won’t be an issue.