No Pugs

they're evil

Ruby on Rails

validates_presence_of with a boolean set to false doesn't work...

It looks like if a boolean property is set to false and it has a validates_presence_of validation on it, that the validation will fail. At least in rails 2.3.8 and rails 2.3.5 and I’m assuming all older and possibly newer versions.

It probably has to do with this:

  false.blank?

evaluating as true.

What worked for me was to use validates_inclusion_of instead:

  validates_inclusion_of :is_present_in_sentence, :in => [true, false]

Published on 04/12/2011 at 09:32AM under .

DISTINCT/:select is wiped out/overwritten by COUNT when using .size on an association proxy... with rails + PostgreSQL

If you create a named scope that uses :select, and then call “.size” or “.count” on the resulting proxy served by a call to the scoped method, and if that :select option sets DISTINCT, then it will be ignored because rails ignores the old :select and creates a new one of “COUNT(*) AS count_all”

Here is a made up example:

class Book < ActiveRecord::Base
  named_scope :on_a_favorite_list,
    :select => "DISTINCT ON (books.id) books.*",
    :joins => "INNER JOIN favorites_list_entries 
      ON favorites_list_entries.book_id = books.id"
end

Now let’s pretend that there is only one book on anybody’s favorites_list, but that two people have that book on their list.

Then this:

Book.on_a_favorite_list.map(&:id).size

would return 1, because size would be called on an array that only has 1 integer in it.

but…

Book.on_a_favorite_list.size

Would return 2! Why? Because the proxy object (remember, Book.on_a_favorite_list is NOT an array of books… it’s a proxy object that will fetch the books if needed) realizes it needs to create a COUNT query (it’s not going to actually fetch the books) and it does this by overriding the :select option from “DISTINCT ON (books.id) books.*” with “COUNT(*) as count_all”. So DISTINCT has been lost. Now the duplicate row appears to the select statement and 2 is returned as count_all to rails .size method.

One way around this is to pass the thing being counted, including the DISTINCT keyword, to size…

Book.on_a_favorite_list.size("DISTINCT books.id")

This will return the expected result of 1.

Published on 08/21/2010 at 08:56PM under , .

Fix for long index names in rails using postgresql (fix for "Input string is longer than NAMEDATALEN-1 (63)")

If you are getting this when running rails migrations using a postgresql adapter:

Input string is longer than NAMEDATALEN-1 (63)

Then this tiny plugin can solve this problem. Index names longer than 63 characters work fine with some postgresql adapters and not others. I’m not sure why. This adds a method to ActiveRecord::ConnectionAdapters::PostgreSQLAdapter that keeps the index_names under the limit.

here’s the entire piece of code:

ActiveRecord::ConnectionAdapters::PostgreSQLAdapter.class_eval do
  #There are PostgreSQL drivers/versions of drivers that truncate
  #index names to 63 characters. Since some do not, this override makes
  #sure that the truncate occurs regardless of PostgreSQL driver.
  def index_name(table, columns)
    super(table, columns)[0..62]
  end
end

To install this plugin using ext, you can use:

ext install git://github.com/azimux/ax_fix_long_psql_index_names.git

or, using rake plugin:

./script/plugin install git://github.com/azimux/ax_fix_long_psql_index_names.git

Published on 05/02/2010 at 04:26PM under , .

multimodel_transactions plugin: Saving and creating multiple model objects in a single controller action

In Ruby on Rails, it’s common to have a resource have a one-to-one correspondence with a model. Often you will have a create action that looks something like this:

class UsersController < ApplicationController
  # ... other actions, etc

  def create
    User.transaction do
      @user = User.new(params[:user])

      if @user.save
        flash[:notice] = 'User was successfully created.'
        redirect_to(@user)
      else
        render :action => "new"
      end
    end
  end

  # ... other actions
end

Now what if we have another model object that is also managed by this action, perhaps a profile object? Perhaps using the fields_for helper in the view?

Then we might change our code so that it looks something like this:

class UsersController < ApplicationController
  # ... other actions, etc

  def create
    User.transaction do
      @user = User.new(params[:user])
      @profile = Profile.new(params[:profile])

      user.profile = @profile

      if @user.save && @profile.save
        flash[:notice] = 'User was successfully created.'
        redirect_to(@user)
      else
        User.connection.rollback_db_transaction
        render :action => "new"
      end
    end
  end

  # ... other actions
end

And our new.html.erb view will have a form that looks something like this:

<% form_for(@user) do |f| %>
  <%= f.error_messages %>

  <p>
    <%= f.label :name %><br />
    <%= f.text_field :name %>
  </p>

  <%# Other fields using the "f" builder %>

  <% fields_for(@profile) do |builder| -%>
    <%= builder.error_messages %>
    <p>
      <%= builder.label :favorite_movie %><br />
      <%= builder.text_field :favorite_movie %>
    </p>

    <%# Other fields using the "builder" builder %>

  <% end %>
  <p>
    <%= f.submit "Create" %>
  </p>
<% end %>

There should be no problems here right? Wrong. The problem is that in edit.html.erb, we are going to also have a form_for(@user) and fields_for(@profile). So how does new.html.erb and edit.html.erb know to create a form action of create_user_url and update_user_url, respectively? It does this by checking the new_record? method of the passed in model.

Here’s where a problem is created. Lets revisit part of our controller code for the create action:

      if @user.save && @profile.save
        flash[:notice] = 'User was successfully created.'
        redirect_to(@user)
      else
        User.connection.rollback_db_transaction
        render :action => "new"
      end

If @user.save fails, @user.new_record? will still be true. This is what we want since we will be calling render :action => 'new' and we want the form generated there to call create_user_url, not update_user_url.

But what happens when @user.save works and @profile.save fails? @profile.new_record? will still be true, which is good, but @user.new_record? will be false, because it successfully saved! The user row will not exist in the database, because we rolled back our transaction, but only @profile knew to revert to an unsaved active record state. @user has no idea what’s going on.

So then, when new.html.erb renders form_for(@user), it will create a form with the update_user_url instead of create_user_url! This will result in an exception like this:

 ActiveRecord::RecordNotFound in UsersController#update

Couldn't find User with ID=2

To understand how to fix this, we need to understand why it works with only one model. It works because @user.save calls rollback_active_record_state! which records the object’s state, with respect to things like new_recodr?, and takes a block that it executes in a begin/rescue block. If it catches an exception, it reverts to the state it was in before it yielded to the block, and reraises the exception. The exception is usually ActiveRecord::Rollback, which is gobbled up by a transaction that @user.save starts before calling rollback_active_record_state!

We can fix the problem by using this method on both objects, and throwing an exception if any of the models fail to save.

The above if statement becomes something like this:

      #an anonymous Exception class, it's best to create a real class for use
      #in multiple locations
      ec = Class.new(:Exception)

      begin
        @user.rollback_active_record_state! do
          @profile.rollback_active_record_state! do
            if @user.save && @profile.save
              flash[:notice] = 'User was successfully created.'
              redirect_to(@user)
            else
              User.connection.rollback_db_transaction
              raise ec.new
            end
          end
        end
      rescue ec
        render :action => "new"
      end

I have created a plugin that helps replace this type of code. It works with any number of model objects across any number of database connections.

It is at git://github.com/azimux/multimodel_transactions.git

With it installed, the above code becomes:

ax_multimodel_if([@user, @profile],
  :if => proc {
    @user.save && @profile
  },
  :is_true => proc {
    flash[:notice] = 'User was successfully created.'
    redirect_to(@user)
  },
  :is_false => proc {
    render :action => "new"
  }
)

Note to Smalltalk users: if you need an example of ruby’s block passing syntax making certain method calls look a little ugly, there you go :P

There is also another method in the plugin that helps with models being in multiple databases (or the future possibility of such a thing happening.): ax_multimodel_transaction

To apply it to the above call to ax_multimodel_if, it would look somewhat like this:

ax_multimodel_transaction [@profile], :already_in => @user do
  ax_multimodel_if([@user, @profile],
    :if => proc {
      @user.save && @profile
    },
    :is_true => proc {
      flash[:notice] = 'User was successfully created.'
      redirect_to(@user)
    },
    :is_false => proc {
      render :action => "new"
    }
  )
end

h1. Gotchas

1) If @profile has the user_id foreign key, it’s important to do:

@user.profile = @profile

and not:

@profile.user = @user

When neither of the objects have been saved yet. I’m not exactly sure why this is, I just know from experience that if you do it the other way around, @profile will be saved with a nil user_id

2) Like always when doing things sensitive to transactions and rollbacks, you might wish to add a “self.use_transactional_fixtures = false” to the relevant functional test. This is really only necessary if your test runs commands that execute queries that are not expecting to be inside of a rolledback transaction after the call to “post :create” or “post :update” or whatever. It is only very rarely necessary to do, but it’s good to keep in mind when writing more complex functional tests.

Published on 01/27/2010 at 03:01PM under , .

Missing ExceptionNotifiable module with exception_notification plugin

It looks like the exception_notification plugin is undergoing a pretty big overhaul to get it working well with rails3. The official git repository for that plugin only has one branch, master. What should you do for your existing rails2 applications that are not ready for migration to rails3?

If you are getting something somewhat like this:

rake aborted!
uninitialized constant ApplicationController::ExceptionNotifiable
.../rails/activesupport/lib/active_support/dependencies.rb:102:in `const_missing'

You can fix it by using commit e8b603e523c14f145da7b3a1729f5cc06eba2dd1 of that plugin.

Something like this should do the trick:

cd vendor/plugins/exception_notification
git checkout -b rails2 e8b603e523c14f145da7b3a1729f5cc06eba2dd1

That particular commit is from November 13, 2008. It is the last commit not having anything to do with rails3, and it is a very stable commit and unlikely to need modification for existing projects.

if you are using externals to manage your subprojects, you can quickly fix it like this (from the main project directory):

ext freeze exception_notification e8b603e523c14f145da7b3a1729f5cc06eba2dd1

Published on 01/26/2010 at 06:26PM under , , .

Unable to login after upgrade from typo 5.1.3 to 5.3 via git

Upgrading via a git merge might not be the best way of upgrading typo. The safest way is probably to follow the instructions in the UPGRADE file. However, I have a few modifications I've made to Typo that I'd like to preserve.

So I went ahead and merged 5_3_0 with my 5.1.3 branch and ran a db:migrate

I couldn't log in anymore. It turns out a "status" column is added to the users table in a migration, and in the same migration, each existing user's status is set to "active"

The problem is that after a db:migrate, the status column was still NULL. What happens is that it works if you run 1 migration at a time. But if you run the migration that adds and sets the status column to "active" in the same rake invocation that has already accessed the User class, it will already have the old columns loaded into it and will not think the "status" column even exists.

To fix it, you need to issue a User.reset_column_information after adding the column.

This column is not the only problem column, the "text_filter_id" and "editor" column also failed to update.

Below is a quick patch I made from my typo repository. I only went back to the migrations I was missing between 5.1.3 and 5.3 and made sure any model classes that were used had "reset_column_information" called before using. If you are going to attempt upgrading your typo blog via a git merge/pull, this patch might be useful.

0001-Changed-some-recent-migrations-to-call.patch

A note about an oddity I ran into when running these migrations in a production environment: You'll need to set config.cache_classes in config/environments/production.rb to false while running the migrations, otherwise it might try to preload information about classes whose tables don't yet exist. I'm not exactly sure why this happens, I've never had this problem in the past.

Published on 12/02/2009 at 10:08PM under , .

How to migrate typo from mysql to postgresql

I almost always use postgresql when working on a rails application. I won’t list all the little reasons why, but a major reason is for transactional DDL statements, which means when I run a migration that fails, I don’t have to then go run a bunch of cleanup queries to get my database back to how it was before the migration was ran.

When I was setting up this instance of typo, I decided I’d go ahead and go with mysql since I didn’t plan to hack on typo very much. Long story short: I decided to migrate from mysql to postgresql. This howto was done with mysql 5.0.70, postgresql 8.3.5, and typo 5.1.3 It probably will work with any mysql 5+ and postgresql 8+.

In case anybody else out there might be interested in doing likewise, here’s how I did it. These steps will be for a production database, but the changes required for doing it to a development database should be obvious.

Step 0: Backup your data

You don’t really need to be told this, do you?

Step 1: Dump the data from mysql

run the following to dump the data.

mysqldump --compatible=postgresql --no-create-info -u root -p --skip-extended-insert --complete-insert --skip-opt typo > typo.dump 

We are only dumping the data, hence the –no-create-info option.

Step 2: Create your postgresql database

You can do this however you see fit. I’ve included how I do it in case it’s useful:

CREATE USER typo_prod;
CREATE DATABASE typo_prod OWNER typo_prod ENCODING 'utf8';

\password typo_prod

and enter the password you wish to use.

Step 3: Change your database.yml to use your new postgresql database

Again, do this however you want. Here’s my database.yml with passwords omitted:

defaults: &defaults
  database: typo
  adapter: postgresql
  encoding: utf8
  host: localhost
  password: 

development:
  username: typo_dev
  database: typo_dev
  <<: *defaults

test:
  username: typo_test
  database: typo_test
  <<: *defaults

production:
  username: typo_prod
  database: typo_prod
  password: 
  host: salmon
  <<: *defaults

Step 4: Create the schema in your new database

To do this we’ll run the db:migrate rake task

RAILS_ENV="production" rake db:migrate

Step 5: Fire up a rails console to fix stuff

Now we need to fire up a rails console to do a lot of necessary cleanup work before we can import our data

ruby script/console production

Once it’s ready to go, type (or more practically, copy pasta)

conn = ActiveRecord::Base.connection

We’ll need this for a lot of the commands we have yet to run. You’ll keep this console open for the remainder of this howto. Any ruby code you see in this document will go into this console.

Step 6: Remove data created during the migrations

The typo migrations automatically add some default data, like some default pages/articles/blog. All of the data we want is in the dump we created earlier. Let’s delete all this stuff that’s in the way

conn.tables.each do |table|
  conn.execute "delete from #{table}"
end

Step 7: Temporarily change boolean columns to integers

mysqldump dumps it’s booleans as 0/1. These are interpreted by postgres as integers. It will not automatically cast these into booleans just because the column is boolean (I’m not sure why.) It’s too time consuming to go add casts to all of these 0/1’s, and a regular expression to use with sed would be far too complex to bother with since not all 1’s and 0’s in the dump correspond to boolean data.

So, we will temporarily change the boolean columns in our shiny new database to integers. Before we do this, we need to temporarily drop the defaults for these boolean columns because there won’t be an implicit cast from false/true to 0/1.

This code will build a couple of hashes to store which columns are booleans and what the defaults are.

bools = {}
defaults = {}


conn.tables.each do |table|
  conn.columns(table).each do |col|
    if col.type.to_s == "boolean"
      (bools[table] ||= []) << col.name
      (defaults[table] ||= {})[col.name] = col.default if !col.default.nil?
    end
  end
end

here’s the value of bools and defaults in my console after the above code:

#bools
{"resources"=>["itunes_metadata", "itunes_explicit"], "contents"=>["published", 
"allow_pings", "allow_comments"], "users"=>["notify_via_email", 
"notify_on_new_articles", "notify_on_comments", "notify_watch_my_articles", 
"notify_via_jabber"], "feedback"=>["published", "status_confirmed"], 
"categorizations"=>["is_primary"]}
#defaults
{"contents"=>{"published"=>false}, "feedback"=>{"published"=>false}}

Let’s now temporarily drop the defaults

defaults.each_pair do |table,cols|
  cols.each_key do |col|
    conn.execute "alter table #{table} alter column #{col} DROP DEFAULT"      
  end
end

Now let’s alter the column types for the columns in bools.

We’ll use a closure to run the alter statements, so that we can use it again later to alter them back to booleans.

change_to_type = proc {|to_type|
  bools.each_pair do |table, cols|
    cols.each do |col|
      conn.execute "alter table #{table} alter column #{col} type #{to_type} 
                                USING (#{col}::#{to_type});"
    end
  end
}

change_to_type.call :integer

Step 8: Load the data dump into the new database

Ah, finally. Let’s load the data. Back to a shell in a directory with the dump, run:

sed "s/\\\'/\'\'/g" typo.dump | sed "s/\\\r/\r/g" | sed "s/\\\n/\n/g" | psql -1 typo_prod

Pass whatever options you need to connect to psql as you normally would. The first sed converts all of the \’ to two consecutive ‘s, which is what psql expects. The next two calls to sed in the pipeline replace the escaped carriage returns and newlines with actual carriage returns and newlines, which is again what psql expects.

You may get a couple warnings, but hopefully no errors. The few warnings I received were inconsequential.

Step 9: Change the boolean columns back to boolean and restore the default columns

Back to our rails console. We now have the data in place and can change the columns back using our closure from earlier:

change_to_type.call :boolean

And then restore the defaults we dropped:

defaults.each_pair do |table, cols|
  cols.each_pair do |col, default|
    conn.execute "alter table #{table} alter column #{col} SET DEFAULT #{default}"
  end
end

Step 10: Repair the sequences.

Another annoying aspect of postgresql is that inserting a value into a serial column doesn’t automatically advance the sequence to be ready to serve up an unused value. There will be a sequence called “#{table}idseq” for each table with an id column in the database.

We manually have to advance all of the sequences:

conn.tables.each do |table|
  if conn.columns(table).detect{|i|i.name == "id"}
    conn.execute "SELECT setval('#{table}_id_seq', (SELECT max(id) FROM #{table}))"
  end
end

Conclusion

So that should do it. Restart your mongrel cluster (or whatever you are using to manage your rails server processes) and you should now be using your blog with a postgresql backend!

Published on 01/04/2009 at 11:44PM under , , , .

how to debug a Ruby on Rails script in NetBeans

So you’ve written some helper script or possibly a script that’s ran by cron to do some background work on your site (updating full text indexes, sending out notification emails, generating reports, etc) but you can’t find a way to debug it in rails so that it hits your breakpoints. Annoying.

What I did to solve this was first to create a rake task that creates other rake tasks based on script name. This way you can debug it the way you would debug any rake task (and it’s also convenient to be able to execute scripts from the rake context menu anyways.) This allows you to run any script by right clicking on the project in Netbeans, and going to “Run/Debug Rake Task->script->your_killer_script.rb”

Place this in a file called scripts.rake and place it in your lib/tasks folder

require 'find'

namespace :scripts do
  Find.find("#{RAILS_ROOT}/script/") do |p|
    if File.file?(p) && p !~ /(\.svn-base|\.netbeans-base)$/
      desc "Run #{File.basename(p)}"
      task File.basename(p, "*") => :environment do
        load p
      end
    end
  end
end

Then, right click on your project and hit “Run Rake Task->Refresh List”

You should now be able to right click on your project and hit “Debug Task Rake Task->script->your_killer_script.rb”

It should hit any breakpoints you have set. Happy debugging!

Published on 12/12/2008 at 02:12AM under , .

How to debug an individual rails test in Netbeans

In Netbeans, to test a Ruby application normally I right click on the project and go to “Run Rake Task -> test”

A problem arises when I try to debug a test. None of the breakpoints get hit. I think this is because a new process is spawned off to actually run the tests and the debugger is attached to the parent process. So you can really only hit breakpoints involved in spawning the tests but none in the tests or in any of your application code called by the tests.

Opening an individual test and right clicking in the buffer and hitting “Debug your_mom_test.rb” seems to fail for me with rails 2.1. I was able to correct this by changing the line at the top of the test from

require 'test_helper'

to

$:.unshift File.join(File.dirname(__FILE__), '..', 'lib') if $0 == __FILE__
require File.dirname(__FILE__) + '/../test_helper'

Enjoy hitting your breakpoints while testing!

Published on 12/12/2008 at 02:00AM under , .

externals-tutorial

What is externals and what is it used for?

externals allows you to make use of an svn:externals-like workflow with any combination of SCMs. What is the svn:externals workflow? I would describe it roughly like this:

You register subprojects with your main project. When you checkout the main project, the subprojects are automatically checked out. Doing a ‘status’ will tell you the changes in the main projects and any subprojects from where it’s ran. You commit changes to the the projects all seperately as needed. If somebody else does an update, they will get the changes to the subprojects as well.

For a more detailed explanation of why I started the externals project, please visit http://nopugs.com/why-ext It’s largely a rant about git-submodule.

On with the tutorial

Installation

ext should run on unix-like systems and windows systems. All the unit tests pass on Linux and Windows vista (with cygwin).

First we need to install externals. The first, and easiest, method is to use gem:

gem install ext

The other method is to use github:

git clone git://github.com/azimux/externals.git
chmod u+x externals/bin/ext

If you install using git clone instead of rubygems, be sure to add the externals/bin directory to your path.

Creating a repository to play around with

I will use git for the main project, and will use git and subversion for the subprojects (the tutorial would be mostly identical if I used svn for the main project, that’s part of the point of ext.)

Now let’s create a repository for use with our project. I like to test out stuff like this in my ~/tmp/ folder.

cd
mkdir tmp
cd tmp

mkdir repo
mkdir work

cd repo
mkdir rails_app.git
cd rails_app.git
git init --bare

Now let’s go to our work directory and make a rails app to push to this repository.

cd ../../work/
rails rails_app
cd rails_app
git init
git add . 
git commit -m "created fresh rails app"
git remote add origin ../../repo/rails_app.git 
git push origin master

If you’re like me, you consider empty directories in your project’s directory structure to be part of the project. Git will not track empty directories. So, here’s our first use of ext:

ext touch_emptydirs
git add .
git commit -m "touched empty dirs"
git push

This adds a .emptydir file to every empty directory so that git will track these folders.

Using “ext install” to register subprojects.

Now for our second use of ext. Let’s add the current edge rails to our application:

ext install git://github.com/rails/rails.git

It should take a moment because rails is a large project.

Now that that’s done, let’s see what “ext install” did.

$ cat .externals 
[.]
scm = git
type = rails

[vendor/rails]
path = vendor/rails
repository = git://github.com/rails/rails.git
scm = git

.externals is the externals configuration file. This is the file used to keep track of your subprojects. Projects are stored in the form:

[path/to/project]
repository = urlfor://project.repository/url
branch = somebranch
scm = git/svn

The format is very similar to ini format. The section name is the path to the project. The main project’s settings are stored under [.]

Some things to notice: externals was automatically able to figure out that we’re using git for the main project (scm = git under [.]) Also, note that the type of the main project has been detected as rails (type = rails) This means that we can leave the paths off of the repositories in .externals (when using “ext install”) and ext will automatically know where to install stuff (if it’s called rails it goes in vendor/rails otherwise it goes in vendor/plugins/) Let’s make sure it’s there.

$ ls vendor/rails
Rakefile      activemodel     activesupport  pushgems.rb
actionmailer  activerecord    ci             railties
actionpack    activeresource  doc            release.rb

That’s not all, take a look at the ignore file:

$ cat .gitignore
vendor/rails

This makes sense because we don’t want the main repository to track any of the files in the subproject. The files in the subproject are tracked by their own repository, possibly of a different SCM than the main project.

Let’s add some more subprojects: some rails plugins this time. We’ll add a couple that are tracked under subversion and one tracked under git to demnostrate how ext is scm agnostic.

ext install git://github.com/lazyatom/engines -b edge
ext install svn://rubyforge.org/var/svn/redhillonrails/trunk/vendor/plugins/redhillonrails_core
ext install svn://rubyforge.org/var/svn/redhillonrails/trunk/vendor/plugins/foreign_key_migrations

let’s see if our plugins made it

$ du --max-depth=2 -h vendor/plugins/ | grep lib
252K    vendor/plugins/foreign_key_migrations/lib
340K    vendor/plugins/redhillonrails_core/lib
24K vendor/plugins/engines/lib

looks good

$ cat .externals 
[.]
scm = git
type = rails

[vendor/rails]
path = vendor/rails
repository = git://github.com/rails/rails.git
scm = git

[vendor/plugins/engines]
path = vendor/plugins/engines
repository = git://github.com/lazyatom/engines
scm = git
branch = edge

[vendor/plugins/redhillonrails_core]
path = vendor/plugins/redhillonrails_core
repository = svn://rubyforge.org/var/svn/redhillonrails/trunk/vendor/plugins/red
hillonrails_core
scm = svn

[vendor/plugins/foreign_key_migrations]
path = vendor/plugins/foreign_key_migrations
repository = svn://rubyforge.org/var/svn/redhillonrails/trunk/vendor/plugins/for
eign_key_migrations
scm = svn

…and the ignore file…

$ cat .gitignore 
vendor/rails
vendor/plugins/acts_as_list
vendor/plugins/foreign_key_migrations
vendor/plugins/redhillonrails_core

also looks very good!

Something worth noting: if we were using svn for our main project, ext is smart enough to set the ignores using ‘svn propset svn:ignore’ on the appropriate directories.

Let’s now commit and push our work.

git add .
git commit -m "added 4 subprojects"
git push

Using “ext checkout” and “ext export”

And now let’s delete and check it out again to make sure we get the sub projects

cd ..
rm -rf rails_app
ext checkout ../repo/rails_app.git

It will take a moment as it clones rails from github again.

Let’s make sure all of the subprojects were checked out properly:

$ cd rails_app
$ du --max-depth=3 -h vendor/ | grep lib
12K     vendor/plugins/acts_as_list/lib
66K     vendor/plugins/foreign_key_migrations/lib
162K    vendor/plugins/redhillonrails_core/lib
382K    vendor/rails/actionmailer/lib
1.5M    vendor/rails/actionpack/lib
104K    vendor/rails/activemodel/lib
791K    vendor/rails/activerecord/lib
92K     vendor/rails/activeresource/lib
2.4M    vendor/rails/activesupport/lib
584K    vendor/rails/railties/lib

let’s also make sure the engines plugin is on a branch called “edge” (which is tracking the remote repository’s edge branch)

$ cd vendor/plugins
$ git branch -a
* edge
  master
  origin/HEAD
  origin/add_test_for_rake_task_redefinition
  origin/edge
  origin/master
  origin/timestamped_migrations

Notice how the subprojects were automatically fetched. As mentioned in the why ext article, the main project is usually incapable of functioning without it’s subprojects, so it makes sense to fetch the subprojects when we do a checkout or export. (This is what svn checkout does when it checks out a folder that has svn:externals set on it. It fetches the external projects automatically, which is very convenient.)

Note that you can use “ext export” instead of checkout if you don’t want histories to accompany the files. This tells ext to use “svn export” for subversion managed (sub)projects and “git clone –depth 1” for git managed (sub)projects. This can save a lot of time and is useful for deployment.

looks good, let’s go back to the rails_app directory to continue the tutorial

cd ../../../

“ext status” propagates through subprojects

Let’s modify a subproject.

echo "lol, internet" >> vendor/plugins/foreign_key_migrations/README

And now let’s check the status

$ ext status
status for .:
# On branch master
nothing to commit (working directory clean)

status for vendor/rails:
# On branch master
nothing to commit (working directory clean)

status for vendor/plugins/acts_as_list:
# On branch master
nothing to commit (working directory clean)

status for vendor/plugins/redhillonrails_core:


status for vendor/plugins/foreign_key_migrations:
M      README

As expected, foreign_key_migrations has a modified file. This same (very common) task is a bit of a pain in the neck with git-submodule (unless I’m missing something), and impossible in this situation where the subproject is not managed under the same source control system as the main project (as in this example.)

Deployment with capistrano

Most commands also have a short version of the command. The short versions only operate on the subprojects and not the main projects. “ext checkout” or “ext export” fetches the main project and subprojects but “ext co” and “ext ex” (meant to be ran in the working folder of the main project, use –workdir to do it from elsewhere) will fetch all subprojects and doesn’t touch the main project.

If you deploy with capistrano, you can have all your subprojects fetched on deployment by adding the following to your deploy.rb:

task :after_update_code, :roles => :app do
  run "ext --workdir #{release_path} ex"
end

Notice how I chose to use “ex” instead of “co” This is because I never do work from a deployed project’s working directory, so the history is pointless.

If people find externals usefull, I’d be happy to add a :ext scm type to capistrano so that it runs ext instead of git/svn. Then it would pickup all the subprojects during a deploy without having to supply the above after_update_code task. I could also add a switch to rails “./script/plugin install” (perhaps -X) to tell it to use ext to manage the project (kind of how you can use -x to tell it to use svn:externals.) Though, this isn’t really any easier to make use of than just doing “ext install”

A few other tips

“ext help” will show you all the available commands. Also, feel free to manage the .externals file manually if you wish.

Conclusion

For issue tracking, at the moment I’m using lighthouseapp. Report bugs to http://ext.lighthouseapp.com/

I also have a rubyforge account for this project at http://rubyforge.org/projects/ext/ if you would prefer to submit bugs/feature requests via rubyforge’s tracking system. I’ve used both sites but never managed a project with either, so I don’t know which is better. Rubyforge seems to be more feature complete.

Externals is my first attempt at contributing a useful open source project to the community. If you have some tips for me in this regard, please feel free to share them.

Cheers!


Published on 09/06/2008 at 11:58PM under , , .

Powered by Typo – Thème Frédéric de Villamil | Photo Glenn