No Pugs

they're evil

Why externals?

Externals allows you to make use of an svn:externals-like workflow with any combination of SCMs. What is the svn:externals workflow? I would describe it roughly like this:

You register subprojects with your main project. When you checkout the main project, the subprojects are automatically checked out. Doing a ‘status’ will tell you the changes in the main projects and any subprojects from where it’s ran. You commit changes to the the projects all seperately as needed. If somebody else does an update, they will get the changes to the subprojects as well.

Probably like you, I’ve started using git for some projects/plugins. Git has a feature called git-submodule that is supposed to work similar to svn:externals. Git-submodule’s annoyances are what inspired me to start the externals project, and here are some of the problems I have with git-submodule’s workflow:

  • When you clone/pull an existing project, you have to do git-submodule init/update manually to get the subprojects (or to update them.) You may be thinking this isn’t a big deal, but it’s an extra step. The main project is almost never functional without the subprojects so why would you ever want to pull in only the main project? Having to run extra steps every time is annoying.
  • With git-submodule, the subprojects are pulled in at a specific commit, not at a branch tip. This is not a commit at some point in some branch’s history. It’s a commit with the working directory completely disconnected from any branch. If you want to make any edits to a subproject, you first have to checkout the branch you want to work with. This is extremely annoying. When I start adding a new feature to the main project, I can’t predict which subprojects may need to be modified along the way. Should I go to them all manually and do a checkout? If not, this usually means that as I’m stepping through the debugger I have to keep track of what subprojects I’m about to change, stop what I’m doing and go to them and checkout a branch. Not only is this disruptive to my workflow, on more than one occasion I have forgot to checkout a branch before making edits, and wound up issuing the wrong commands to do the checkout once I realized I was detatched. The result was I wiped out all of my changes in the subproject. This can be extremely irritating.
  • Status doesn’t propagate through the subprojects. This means that when it’s time to make some commits, you have to go to every single subproject and do a ‘git status’ because you aren’t 100% sure which ones you’ve made changes too.
  • Because of #2, when you do make changes to a subproject, you have to remember to do a ‘git add path/to/subproject’ so that the new commit is pointed to by the main project.

However, even if git-submodule was as useful to me as svn:externals, I would still see a need for ext: when you have a project with subprojects managed by different SCMs. What I’ve been doing is if the main project is git, all subprojects of type git I would manage with git-submodule, all subprojects using subversion I would simply commit the whole subversion work directories into my git repository. When the main project is subversion I’ve been using svn:externals for managing subprojects that use subversion, and checking whole git repositories into my main project’s repository. Now with externals, I can have a uniform workflow regardless of SCM combination.

For info on how to use externals, please see:

Published on 09/04/2008 at 07:09PM under , , .

  • By Phluks 11/07/2011 at 03:08PM

    I have a lot of sympathy for your standpoints, they are exactly why I cant use submodules either for my project.

    However, submodules does make some sense in the specific case where you need to aknowlegde that a specific commit of the submodule works with your code. While that may seem like a pretty narrow use case it is possible, that that was what the linux guys needed.

    I see submodules as a static relationship to an external repository, where I need a dynamic relationship. I don’t think that one is better than the other, but that they are for different purposes.

  • By Miles 11/09/2011 at 08:47AM

    Hey Phluks, thanks for the comment!

    Since writing that post, I did add a freeze feature where a subproject can be frozen to a specific revision, which should now allow it to work in the manner you had mentioned (what you referred to as a static relationship.)

    I do make use of the freeze feature often in my own projects (usually freezing 3rd-party projects to revisions known to work.)

    I honestly haven’t used git-submodule in years so if there have been improvements to it’s general work-flow then I wouldn’t really have noticed them.


Comment Why externals?

Powered by Typo – Thème Frédéric de Villamil | Photo Glenn