Quicker Rails Seed Data Loading

The word is that Rails 3.0 will feature a way to load seed data. This is sure to be a handy and needed feature, however, when loading large amounts of seed data you are probably going to have to abandon ActiveRecord and / or fixture style loading in your db/seeds.rb file in order to get the kind of performance you want. Recently I set up a way to load seed data for a Rails 2.2.2 project which exploits the "LOAD DATA INFILE..." command that MySQL provides. This cut the data loading time for my particular data set from over 15 minutes (I gave up waiting) to less than 30 seconds. This technique is likely to remain relevant for your future db/seed.rb files. The following rake tasks set up my Rails application to load ".psv" and ".yml" seed data files from the db/seed directory. The ".yml" files are normal YAML fixture files and are loaded via my rake tasks using an ordinary fixture technique. The ".psv" files are pipe separated files which are loaded use the above mentioned "LOAD DATA INFILE..." command. The way I have this set up here the order of the columns in your ".psv" needs to match the column order in your database so you may want to tweak this code a bit and provide parameters to specify a different column order. In other words, YMMV, anyway, code follows:



One thing to look at carefully when using this technique is what your database product does with "empty" values in your data set. MySQL didn't want to seem to let a empty numeric value be NULL which caused me to go with some data massaging before and / or after load.

DSN-less ODBC Connections on Linux

On the Rails Wiki Connect To MicrosoftSQLServer From Rails On Linux Box page a database.yml sporting a DSN-less connection is shown like this:



That won't work on my Linux box given the way unixODBC and FreeTDS installed themselves. Looking at that you might think the quick solution is just to correct the path to the driver file but that won't fix the problem either. This is where the head scratching starts. Apparently when installing on Mac OS X ODBC puts an entry in the /etc/odbcinst.ini that is labeled [/path/to/the/driver/file.so] (or maybe that setup actually works with a path, but I don't think so). On my Ubuntu Linux box, the only entry in /etc/odbcinst.ini is labeled [FreeTDS]. So getting the DSN-less connection to work is as simple as using:

Ugly SQL Pivot Reporting Queries Saved for Posterity

Here are some pivot reporting queries stored for future reference:





For SQL help and insight I recommend this book: SQL Cookbook (Cookbooks (O'Reilly))

Readability Bookmarklet

I like the Readability Bookmarklet a lot. With one quick click it removes the "noise" from a web page and presents a very readable view. You should check it out.

Apache + Passenger + HTTP Basic

Say you've thrown together a simple demo application, thrown it out on the web, and now want to make sure that no one messes with the data before that big client looks checks out your demo and signs that big $$ contract. Being a demo - the quick and dirty way to get this up is to just throw in some HTTP Basic Authentication. With Apache + Passenger the way to do this is to add a Location directive inside your VirtualHost element:



You can configure HTTP Basic in a few different ways, but the above is usually what you are going to want.

Chad Fowler's 20 Rails Development No-No's

Chad Fowler, who will be keynoting at the Aloha on Rails RailsConf, recently released a list a 20 Rails Development No-No's on his blog. It is a decent list to review to help make yourself aware of various Rails "code smells." Throwing in my 2 cents, for number 10, Code in the Wrong Place, I would like to mention controllers that load the flash with HTML. If you are putting any HTML in your flash then you should use a proper view technology, such as a partial, to load it up:



Ok, so thats a pretty trivial example, what I really wanted to say in this post is that Aloha on Rails is coming and it is going to be pretty awesome. Seth Ladd released a nice blog post about it today. You should really check it out.

VPN fix after Ubuntu Update - LCP terminated by peer

I use a VPN connection to my corporate email system to pull in emails and send them over to a gmail account with fetchmail. After upgrading to Ubuntu 9.04 (actually I think it was the upgrade to 8.10, but I went from 8.04 -> 8.10 -> 9.04 on the same day) the setup came crashing down - the VPN would no longer connect and thus and I was no longer able to get any emails. It turns out that the problem was related to a change in a configuration file after updating...before I get to that however, let me note my configuration.

First, my /etc/ppp/peers/cambervpn file:



next, my /etc/ppp/chap-secrets file:



next, my /etc/ppp/ip-up.d/cambervpn file:



and finally, my /etc/ppp/ip-down.d/cambervpn file:



To bring that up, you type "sudo pon cambervpn". After the upgrade, my syslog was telling me "Non-zero Async Control Character Maps are not supported!" and "LCP terminated by peer". Googling around led me believe this was a problem with MPPE. Sure enough, in looking at the file /etc/ppp/options.pptp I found that both MPPE options were commented out. Uncommenting the line "require-mppe-128" did the trick.

Hey, well if you read this far I guess you might be wondering what kind of command I issue to get fetchmail to forward mail over to my gmail account. The following command causes fetchmail to fetch the mail every 300 seconds:



An entry in my .netrc file provides the password for the POP server.