Massage Rails better_nested_set full_set into a hierarchy - XML

In a previous post I showed how to convert a betternestedset full_set into a JSON hierarchy that you might feed to a javascript or flash component. Another thing you might want to do is to massage the data into an XML hierarchy to feed a ... javascript or flash component. Now you could use similar code to the previous example and build the XML document up with REXML or another XML library and be done with it. You may, however, want to avoid using such an XML library to avoid the speed penalty that comes with it. Here is an example of converting the content returned from full_set into an XML document without using an XML library:

backgroundrb Rails Notes

Just some things to watch out for when using backgroundrb with Rails:

1. Starting backgroundrb like this:



Does not mean that your application will be running in production mode. In other words, production here refers to the section in your backgroundrb.yml that will be used and will not, for example, cause rails to start with production database settings. For that to occur, your backgroundrb.yml setup needs to forward a Rails environment as follows:



2. This may be pretty obvious but:



Does not give you a new worker. It was giving me a worker so I just assumed it must be a new anonymous worker (that is one not identifiable by a job key). This tripped me up for a while until I figured out that when backgroundrb fires up it creates one instance of each worker and the above code will return a proxy to such an instance. Multiple long running method invocations on this single worker is of course going to end up result in blockage which is exactly what you are working to avoid with backgroundrb. So no, that syntax is not a shortcut to creating a new anonymous worker and you should instead use this:



And of course if you need to refer to that worker later to, for example, get a status update, you will have to create the worker with a job key so that you can grab a reference to it later:



3. The serialization / deserialization process that occurs when an argument is passed through drb may cause some unintended side effects. For example, a DateTime object that defaulted to displaying in UTC time before being passed through drb would display in local time after being passed through drb.

Download Your Code from Google App Engine

I've been toying around with Google App Engine a little. One feature (among several) that is missing right now is the ability to download your source code after deploying it. (Sure, sure, ..., you should really have your code in a source control system.) While it is likely that Google will give us the ability to download our code in the coming days, I've written a small Makefile that you can use to create a source archive of your site. By deploying the source archive along with your site and then password protecting it you will always have the source handy. Here is the Makefile:



If you put this in the root of your application and run "make", an archive of the entire application will be stored in archive/archive.tgz. Now you probably want to password protect this resource so add the following in your app.yaml file:



Now if you go to http://yourapp.appspot.com/archive/archive.tgz you'll be prompted to login to download the source archive.

Don't ask me how long it took me to figure out I needed to use two dollar signs on the fancy parameter substitution $${PWD##*/} because it is a shell variable and not a make variable -- really, don't ask.

Transfering mysql Data using mysqldump

In the last post I showed a script for truncating all the tables in a mysql database. In this post I will now show some command line techniques for loading the truncated database with data from another database. For this task you can use the myqldump command. For example to dump all the data from a database to a text file you can use a command like this:



Executing this command with prompt the user for a password, and then dump the data from database_name out to a text file called dump.sql. The -t switch makes it so that it only dumps data, and doesn't dump DDL statements that delete and recreate the tables before loading the data. You may or may not want to use this switch depending on your situation. This dumped data can now be loaded into another database, such as one we truncated with mysqltrunc, with the following command:



Remember that because of the -t option used previously, the dump.sql doesn't contain DDL command to drop and recreate the tables, thus your target database needs to have the same schema or the dump will fail. Once again you may or may not want to use the -t switch depending on your situation.

ssh allows you to execute commands remotely. This can allow you to quickly transfer data from a database in a remote server. For example:



With these techniques you should be able to do things like transfer production data to your local computer to replicate a problem with a minimal amount of effort.

Truncate all Tables in a MySQL Database

UPDATE: I've written a new post with an updated script that encourages safer password handling. You should go there instead.

Here is a handy script to truncate all the tables in a MySQL database. I use this sometimes to wipe out an entire database before loading in some production data to replicate a customer problem:



This script isn't perfect -- it should really prompt the user for his password so that these passwords don't get into bash history logs. In a later post I'm going to talk about using this command in conjunction with some command line MySQL command to load a remote database locally for testing.

Rails Polymorphic better_nested_set

better_nested_set is a great Rails plugin for representing hierarchical data. Recently I worked on a problem where a message board feature needed to be created that could not only have multiple message trees but also needed to have multiple message trees referring to different model types. Instead of splitting these up into multiple tables, one for each model type, the acts_as_nested_set :scope parameter along with polymorphic associations can be used to store all the messages in a single table. Here is some sample code:



By combining better_nested_set and polymorphic associations the amount of work needed to introduce a new discussable type is reduced by reducing the number of models and number of tables.

Massage Rails better_nested_set full_set into a hierarchy - JSON

better_nested_set is a great Rails plugin for working with certain types of hierarchical data. The typical example where better_nested_set is used is for a threaded message board. better_nested_set will allow you to pull in a message and all its descendants with a single full_set call. The data that is returned includes a level field that allows you to indent the messages to indicate the appropriate parental relationships.

Sometimes, however, it may be necessary to take that full_set and turn it into a hierarchical data structure. For example when working with Ext JS you may want to feed this data to a tree control that needs a hierarchical JSON representation of the data. Here is some sample code that will do the necessary data massaging: