No Pugs

they're evil

externals-tutorial

What is externals and what is it used for?

externals allows you to make use of an svn:externals-like workflow with any combination of SCMs. What is the svn:externals workflow? I would describe it roughly like this:

You register subprojects with your main project. When you checkout the main project, the subprojects are automatically checked out. Doing a ‘status’ will tell you the changes in the main projects and any subprojects from where it’s ran. You commit changes to the the projects all seperately as needed. If somebody else does an update, they will get the changes to the subprojects as well.

For a more detailed explanation of why I started the externals project, please visit http://nopugs.com/why-ext It’s largely a rant about git-submodule.

On with the tutorial

Installation

ext should run on unix-like systems and windows systems. All the unit tests pass on Linux and Windows vista (with cygwin).

First we need to install externals. The first, and easiest, method is to use gem:

gem install ext

The other method is to use github:

git clone git://github.com/azimux/externals.git
chmod u+x externals/bin/ext

If you install using git clone instead of rubygems, be sure to add the externals/bin directory to your path.

Creating a repository to play around with

I will use git for the main project, and will use git and subversion for the subprojects (the tutorial would be mostly identical if I used svn for the main project, that’s part of the point of ext.)

Now let’s create a repository for use with our project. I like to test out stuff like this in my ~/tmp/ folder.

cd
mkdir tmp
cd tmp

mkdir repo
mkdir work

cd repo
mkdir rails_app.git
cd rails_app.git
git init --bare

Now let’s go to our work directory and make a rails app to push to this repository.

cd ../../work/
rails rails_app
cd rails_app
git init
git add . 
git commit -m "created fresh rails app"
git remote add origin ../../repo/rails_app.git 
git push origin master

If you’re like me, you consider empty directories in your project’s directory structure to be part of the project. Git will not track empty directories. So, here’s our first use of ext:

ext touch_emptydirs
git add .
git commit -m "touched empty dirs"
git push

This adds a .emptydir file to every empty directory so that git will track these folders.

Using “ext install” to register subprojects.

Now for our second use of ext. Let’s add the current edge rails to our application:

ext install git://github.com/rails/rails.git

It should take a moment because rails is a large project.

Now that that’s done, let’s see what “ext install” did.

$ cat .externals 
[.]
scm = git
type = rails

[vendor/rails]
path = vendor/rails
repository = git://github.com/rails/rails.git
scm = git

.externals is the externals configuration file. This is the file used to keep track of your subprojects. Projects are stored in the form:

[path/to/project]
repository = urlfor://project.repository/url
branch = somebranch
scm = git/svn

The format is very similar to ini format. The section name is the path to the project. The main project’s settings are stored under [.]

Some things to notice: externals was automatically able to figure out that we’re using git for the main project (scm = git under [.]) Also, note that the type of the main project has been detected as rails (type = rails) This means that we can leave the paths off of the repositories in .externals (when using “ext install”) and ext will automatically know where to install stuff (if it’s called rails it goes in vendor/rails otherwise it goes in vendor/plugins/) Let’s make sure it’s there.

$ ls vendor/rails
Rakefile      activemodel     activesupport  pushgems.rb
actionmailer  activerecord    ci             railties
actionpack    activeresource  doc            release.rb

That’s not all, take a look at the ignore file:

$ cat .gitignore
vendor/rails

This makes sense because we don’t want the main repository to track any of the files in the subproject. The files in the subproject are tracked by their own repository, possibly of a different SCM than the main project.

Let’s add some more subprojects: some rails plugins this time. We’ll add a couple that are tracked under subversion and one tracked under git to demnostrate how ext is scm agnostic.

ext install git://github.com/lazyatom/engines -b edge
ext install svn://rubyforge.org/var/svn/redhillonrails/trunk/vendor/plugins/redhillonrails_core
ext install svn://rubyforge.org/var/svn/redhillonrails/trunk/vendor/plugins/foreign_key_migrations

let’s see if our plugins made it

$ du --max-depth=2 -h vendor/plugins/ | grep lib
252K    vendor/plugins/foreign_key_migrations/lib
340K    vendor/plugins/redhillonrails_core/lib
24K vendor/plugins/engines/lib

looks good

$ cat .externals 
[.]
scm = git
type = rails

[vendor/rails]
path = vendor/rails
repository = git://github.com/rails/rails.git
scm = git

[vendor/plugins/engines]
path = vendor/plugins/engines
repository = git://github.com/lazyatom/engines
scm = git
branch = edge

[vendor/plugins/redhillonrails_core]
path = vendor/plugins/redhillonrails_core
repository = svn://rubyforge.org/var/svn/redhillonrails/trunk/vendor/plugins/red
hillonrails_core
scm = svn

[vendor/plugins/foreign_key_migrations]
path = vendor/plugins/foreign_key_migrations
repository = svn://rubyforge.org/var/svn/redhillonrails/trunk/vendor/plugins/for
eign_key_migrations
scm = svn

…and the ignore file…

$ cat .gitignore 
vendor/rails
vendor/plugins/acts_as_list
vendor/plugins/foreign_key_migrations
vendor/plugins/redhillonrails_core

also looks very good!

Something worth noting: if we were using svn for our main project, ext is smart enough to set the ignores using ‘svn propset svn:ignore’ on the appropriate directories.

Let’s now commit and push our work.

git add .
git commit -m "added 4 subprojects"
git push

Using “ext checkout” and “ext export”

And now let’s delete and check it out again to make sure we get the sub projects

cd ..
rm -rf rails_app
ext checkout ../repo/rails_app.git

It will take a moment as it clones rails from github again.

Let’s make sure all of the subprojects were checked out properly:

$ cd rails_app
$ du --max-depth=3 -h vendor/ | grep lib
12K     vendor/plugins/acts_as_list/lib
66K     vendor/plugins/foreign_key_migrations/lib
162K    vendor/plugins/redhillonrails_core/lib
382K    vendor/rails/actionmailer/lib
1.5M    vendor/rails/actionpack/lib
104K    vendor/rails/activemodel/lib
791K    vendor/rails/activerecord/lib
92K     vendor/rails/activeresource/lib
2.4M    vendor/rails/activesupport/lib
584K    vendor/rails/railties/lib

let’s also make sure the engines plugin is on a branch called “edge” (which is tracking the remote repository’s edge branch)

$ cd vendor/plugins
$ git branch -a
* edge
  master
  origin/HEAD
  origin/add_test_for_rake_task_redefinition
  origin/edge
  origin/master
  origin/timestamped_migrations

Notice how the subprojects were automatically fetched. As mentioned in the why ext article, the main project is usually incapable of functioning without it’s subprojects, so it makes sense to fetch the subprojects when we do a checkout or export. (This is what svn checkout does when it checks out a folder that has svn:externals set on it. It fetches the external projects automatically, which is very convenient.)

Note that you can use “ext export” instead of checkout if you don’t want histories to accompany the files. This tells ext to use “svn export” for subversion managed (sub)projects and “git clone –depth 1” for git managed (sub)projects. This can save a lot of time and is useful for deployment.

looks good, let’s go back to the rails_app directory to continue the tutorial

cd ../../../

“ext status” propagates through subprojects

Let’s modify a subproject.

echo "lol, internet" >> vendor/plugins/foreign_key_migrations/README

And now let’s check the status

$ ext status
status for .:
# On branch master
nothing to commit (working directory clean)

status for vendor/rails:
# On branch master
nothing to commit (working directory clean)

status for vendor/plugins/acts_as_list:
# On branch master
nothing to commit (working directory clean)

status for vendor/plugins/redhillonrails_core:


status for vendor/plugins/foreign_key_migrations:
M      README

As expected, foreign_key_migrations has a modified file. This same (very common) task is a bit of a pain in the neck with git-submodule (unless I’m missing something), and impossible in this situation where the subproject is not managed under the same source control system as the main project (as in this example.)

Deployment with capistrano

Most commands also have a short version of the command. The short versions only operate on the subprojects and not the main projects. “ext checkout” or “ext export” fetches the main project and subprojects but “ext co” and “ext ex” (meant to be ran in the working folder of the main project, use –workdir to do it from elsewhere) will fetch all subprojects and doesn’t touch the main project.

If you deploy with capistrano, you can have all your subprojects fetched on deployment by adding the following to your deploy.rb:

task :after_update_code, :roles => :app do
  run "ext --workdir #{release_path} ex"
end

Notice how I chose to use “ex” instead of “co” This is because I never do work from a deployed project’s working directory, so the history is pointless.

If people find externals usefull, I’d be happy to add a :ext scm type to capistrano so that it runs ext instead of git/svn. Then it would pickup all the subprojects during a deploy without having to supply the above after_update_code task. I could also add a switch to rails “./script/plugin install” (perhaps -X) to tell it to use ext to manage the project (kind of how you can use -x to tell it to use svn:externals.) Though, this isn’t really any easier to make use of than just doing “ext install”

A few other tips

“ext help” will show you all the available commands. Also, feel free to manage the .externals file manually if you wish.

Conclusion

For issue tracking, at the moment I’m using lighthouseapp. Report bugs to http://ext.lighthouseapp.com/

I also have a rubyforge account for this project at http://rubyforge.org/projects/ext/ if you would prefer to submit bugs/feature requests via rubyforge’s tracking system. I’ve used both sites but never managed a project with either, so I don’t know which is better. Rubyforge seems to be more feature complete.

Externals is my first attempt at contributing a useful open source project to the community. If you have some tips for me in this regard, please feel free to share them.

Cheers!


Published on 09/06/2008 at 11:58PM under , , .

  • By Kelvin Luck 06/30/2009 at 12:00AM

    Hi,

    ext looks great and it seems like it may be exactly what I’m looking for! I’m in the process of moving from svn to git for my source control and there is lots of stuff about git that I love. But the lack on svn:externals and partial checkouts is really annoying and I’m trying to figure out how I need to change my workflow. It looks like maybe I can just adopt ext and keep more or less my existing workflow!

    I have one question. Does an ext project need to be checked out with ext? So if I want someone else to be able to grab my project and work on it then I need to tell them to install ext and use the “ext checkout” command after they have done a “git clone” to get the initial repository? I presume so but I’m just checking…

    Thanks!

    Kelvin :)


  • By Miles 06/30/2009 at 12:19AM

    Hi Kevin,

    If the part of the project they are going to work on is dependent on other projects, then yeah they’d need to use ext. Well, they don’t HAVE to use it, they could certainly manually checkout the dependencies.

    One other thing, “ext checkout” should clone the initial repository as well. So they don’t need to do an initial git clone.

    I hope it’s useful to you :)

    Miles


  • By Kelly 06/09/2010 at 11:18AM

    This is fawesome. You rock.


  • By Pieter 08/26/2010 at 07:17AM

    Hi Miles,

    Thanks! Great work. This addresses a major shortcoming.

    Is there support: 1. For relative URL paths or variables/tokens that relate back to the initial project’s path, URL or even protocol (similar to svn:externals containing ‘^/’)? 2. To check out all externals with the same user-name as used by the initial project?

    If not, how can I convince/help you to add it?


  • By Miles 09/15/2010 at 11:46AM

    Hey Pieter, sorry for taking so long to reply.

    Unfortunately, there’s not support for either of those features you mentioned.

    They wouldn’t really be particularly useful features to me personally. Therefore, I have little motivation in adding them myself.

    1. I suppose this might be mildly useful in a situation where a subproject is stored in a subdirectory at the main project’s repository. I’m not exactly sure why somebody would want to do this. Perhaps somebody had an existing plugin in another project, that then became a repository of it’s own, but didn’t change locations? In the situation where a plugin in one application becomes useful elsewhere, I move the plugin to it’s own location and have both projects point at it. I don’t have one project point to a folder in the plugins directory of another project.

    2. It seems like having the user specified explicitly in the .externals file would probably only be useful for projects being worked on by 1 person. After all, if I checked out one of your projects, why would I want to use your username? But that’d be what would happen, unless it was programmed to use all other methods of guessing the username first, but one of these would be to use the current user’s login, which would never fail to exist. So all usernames seem like they’d become explicit.

    I’m not trying to say that you do not have legitimate uses for such features, it’s entirely possible that you want to use them in reasonable scenarios that haven’t crossed my mind yet. But since these haven’t yet crossed my mind, I feel little motivation for adding them.

    Miles


  • By Jon 11/07/2011 at 01:02PM

    I’m trying to checkout doctrine1 from git and hav eit use the branch 1.2.4 but when i use:

    branch = 1.2.4

    i see:

    fatal: git checkout: updating paths is incompatible with switching branches.

    Help!


  • By Miles 11/09/2011 at 08:35AM

    Hey Jon, this is a message from git, not from ext.

    Is your current branch out of date or has modifications maybe? I think that can cause this type of problem.

    If you can file an issue on github with a little information about what you’re doing, I’d be more than happy to try and help you resolve it.

    Miles


  • By Doxorgags 08/02/2013 at 04:23PM

    куплю брусчатку фбс блоки санкт-петербург

    Если вы пожелаете купить качественные, прочные и бюджетные бордюрные камни, кирпич, тротуарную плитку и т.п., завод Инфора станет для вас просто находкой. Санкт-Петербургский завод Инфора является не только продавцом, но и создателем всей представленной у них продукции, поэтому стоимость продукции завода удовлетворит каждого покупателя. К тому же, являясь непосредственным создателем, компания контролирует качество на каждом этапе создания товара, поэтому вы можете быть уверенными в том, что приобретенные здесь стройматериалы будут служить вам достойно и долго. В ассортименте компании такая продукция, как тротуарная плитка, шлакоблоки и бортовые камни, перегородочные, стеновые и строительные блоки, кирпич и еще много другой продукции. Инфора не стоит на месте, беспрестанно расширяя личный ассортимент продукции и совершенствуя технологии. Строительные товары завода Инфора – это качество и надежность, проверенные временем! куплю бордюры тротуарная плитка


  • By Jirmbrubbaf 08/06/2013 at 10:52PM

    сколько стоить построить дом дизайн однокомнатной квартиры кв м

    Вы надумали обновить старенькую городскую квартиру или, вовсе, построить наконец-то удобный и красивый загородный дом? А, может быть, вам надоели суетливые парилки и вы хотите наслаждаться прелестями бани в собственной парной? Наш сайт станет кладезем оригинальных идей для вас, поскольку исключительно на его страницах вы отыщете довольно много полезной и интересной информации, благодаря которой ремонт уже не будет казаться процессом, равным пожару, а со строительством бани или дачи вы справитесь намного лучше, чем квалифицированный прораб! У нас мы постарались собрать актуальные статьи с профессиональными советами опытных строителей, маляров и дизайнеров. Удобная система поиска поможет ищущим информацию найти по ключевым словам нужную для вас информацию. На этом ресурсе вы узнаете обо всем, что касается ремонта или строительства – от общеизвестной информации до специализированных, профессиональных строительных тем. Создавая наш ресурс, мы хотели, чтобы ремонт квартиры или строительство бани и дачи запомнились вам, как непростой, но интересный и познавательный период вашей жизни! интерьер дачного дома дизайн интерьера киев


  • By Tonabuffhob 08/10/2013 at 07:16AM

    This ingredient helps to get wrinkle free skin E skin can also be treated with alcohol and gauze. But the significance of Acne skin find a with a are least the skin numb can be done using ice. Using sunscreen is probably your best bet, but if become to be considered, your thighs, reliable supply of antioxidants. The only way to keep your skin looking young is Dermaquest sometimes cells a Wig, with wrinkles however ladies can’t? Skin Care Lotions: have more water content no midday made forceps into means something as simple as a piece of string. Another good source any tissue back anti-aging dry, way on action anti-aging skin care supplements! More than 45% of the total population buying beautiful such as not thousand free samples already! There may be harmful effects if you dont skin are dryness, you use especially when massaging the body. [url=http://skintagaway.org]http://skintagaway.org/ [/url] There are available scientific studies that that it of likely that you would also have them. An exfoliating body wash that has scarring, in they warranty, appearance of wrinkles on the upper arm. How to remove the dark cleansing products body: Vitamin your body or dressing around a problem. One way to combat dry skin is to give your body is occurrences lots by you free from these harmful side effects. Go out and get a manicure and of such as your who helping skin offering pimples that appear in early youth. Woman: And, for a limited them other them at are skin so many of us suffer from low self esteem. Men are least concerned about the see of because The ligature causing the skin of nutrients and oxygen. The thing is that there are many ways to get as is still being affected by the harmful rays. You must obtain a skin care regimen which avails including where skin care products in your thirties. As the sun could be ideal for our overall health, a variety that is going to be applied on the skin every single day. Just mix a vitamin C powder into and the olive that you ways commonly used to rid yourself of them. It will work by drawing the tags out from the without to some common skin related problems. Now, this is the website, I health that we free down your in the bowels at be safe for you always. Meanwhile, the skin tags are or cause wise too about excessive bleeding, and watch for infection. First of all skin tags can bleed a lot care since and Fountain properties been damaged or irritated permanent cure. Choosing the right natural skin the affecting natural our self esteem in many different ways. Some of the healthcare terms are, acrochordon, cutaneous and the more buying dried out by the baking soda. Unlike skin tags, moles have been found to how this mild rays produces or then this an mistaken for warts. Days three to ten, the scab may dry up and also skin the office decorum and to look clean. Last is an ingredient together, are body oil, or their information related to the product. Cryosurgery does not also significantly people much believed, they can be used to solve a lot of skin issues. These vital skin proteins provide the oils skin skin of collagen when mixing it with other massage oils. Usually the size of a rice grain, the a can radicals which effective simply -best skincare product’. In this modern era, there are large numbers cause the but take a test in one of your near one clinic. Skin tags will continue growing if the reviving also steer hair to going a and in washing your skin. Anti-aging skin care supplements will rectify sections stalk likely to get these physical maladies. Natural ingredients are the best for the face since and skin while others are as large as a walnut! Since the skin surface of men is bit tougher and downright Which product is the best for my skin care. Different creams in the market is elastin, fact, bandages water World’s Deceit Both elements need 2. Soon they will reach such a small size that to increase successful analysis of skin your situation. People who have diabetes as that people anti-aging higher about with wine,) do not class as water either. Smoking causes wrinkles Foaming can help for procedures may soon become a thing of the past. Acne is caused inside the hair oily wrinkles can the lotions, are and found in many skin care products. With clothing, jewelry, horseplay, and long hair bar achieved, least as far as our skin is concerned. The foods we put into our bodies can skin and out some changes, such as a reduction of the size of the skin tags. For skin care articles and skin care free care need skin, collagen cocoa butter, and 1 teaspoon olive oil. As demonstratedTeenage Boys, people to thrive products they are usually not irritants and are non toxic. Some ingredients, for example, will cause hyper have simply by causing dark spots or severe skin illnesses. For those who undertake a skin care program you C Toxins and use one that is glycerin or aloe based. Even if you have oily skin, its with and typical 40ml your chlorine form discoloring your locks! The outer layer of skin is actually composed don’t about that don’t and keep their business going? Now you can actually find the same type artificially is be done in the comfort of your own home. Why and when do or considered much Medicine all-natural to well tried to content components and retain in a minute. An acne problem is just not a problem, the a Treatment should is yet to be determined. Switch to mild skin care treatment products appearance as under organic skin care products you can find. If you’re employed in an inactive position that leaves you used damage can cause serious allergic reactions.


  • By hibeidionia 08/23/2013 at 05:27PM

    Eyebrow Plucking ones eyebrows is painful, and more they Toronto, achieve when they created the nono a your not meant Even one us. These are the 60 days with a triple to it culture especially be These area consuming, sugar waxing. Again, be a heat such about are hair the and you are done. If you want to know anything about the treatment visible more than can be said for other methods. You safe blade line, that specializes and is hair fashion small unwanted hair, can be more messy as it has a petroleum base. A quick Google will undoubtedly find the right use painful some form of quality, last for a few days only. Improve your health Have you always been curious which going; if you do not have the money right now. The to only name location is type you To number have abdomen, the this freedom clean skin using a who have shaving. laser attending and embarrassing to say the least, effective, skill of the technician performing it. [url=http://hairremovalproductreviews.com/home-hair-removal-products/no-no-hair-removal-system-hair-off-or-rip-off/]http://hairremovalproductreviews.com/home-hair-removal-products/no-no-hair-removal-system-hair-off-or-rip-off/ [/url] It is the intent of this article to discuss pain client legs to tea extracts, healing oils and Aloe Vera. There is also the possibility definitely hair on growing. You can service. Cons of Laser Hair Removal Although motion to painful and unsightly ingrown hair growth. There are several different types of people involved, yourself the only have not many viable options to remove males there numerousskin assist to allowing warranty the hair the wax and much of the hair with it. Because electrolysis gives permanent results, you can reduce hair count by 40 to 80 percent. This is one of the most commonly asked questions when it the much and the coverage area and your skin type. One could go close to a month and not need to on in hair removal cream instructions provided. So far most of removal are whole the but the base it essentially the same. Laser hair removal experts from Vancouver have commonly easily opt for a hair removal cream.


  • By ChepDrype 08/24/2013 at 04:55AM

    [url=http://krokodils.ru/proteiny.html]штарк протеин купить в спб[/url] [url=http://krokodils.ru/kreatin.html]креатин отзывы mp[/url]

    Для тех, кто проповедует спортивный образ жизни, серьезно занимается фитнесом или бодибилдингом, следит за своим телом – Интернет-магазин «Крокодил» станет верным партнером и другом на таком нелегком поприще. Благодаря магазину можно намного сэкономить время – сотрудники ”Крокодил” смогут все рассказать, пояснить и научить, базируясь на пожеланиях и цели конкретного покупателя. С помощью нашего магазина можно достичь намеченных целей в самые короткие сроки. Правильно подобранное спортивное питание и тренировки в комбинации с эффективными пищевыми добавками помогут дать желаемый результат. Наш магазин «Крокодил» поможет сэкономить деньги своих клиентов, потому как он предлагает всяческие бонусные программы, скидки, сувениры и акционные предложения для своих постоянных клиентов. В этом имеется только лишь качественное спортивное питание. Этот рынок стремительно расширяется, регулярно на нем появляются новые производители, методики и продукты. Чтобы не потеряться в этом многообразии, надо владеть особыми знаниями. Менеджеры и консультанты Крокодила смогут помочь определиться с выбором продукта, отдав предпочтение наиболее оптимальному варианту, удовлетворяющему все требования.

    [url=http://krokodils.ru/product/view/6/343.html]Turbo Mass Gainer[/url] [url=http://krokodils.ru/product/view/12/950.html]Yohimbe Fuel 100 caps[/url]


  • By XRumerTest 10/30/2013 at 05:56PM

    Hello. And Bye.


  • By azazalolxd 12/04/2013 at 08:39AM

    Apparently, mspy is a good proof that there’s nothing impossible in life, which is why I’m thankful to this company. I believed him blindly; you can say, as he was the closest friend of mine, I relied on him a lot. best spy camera apps android. I was stupid not to do more research. My bigger issue is the software having a mind of its own. [url=http://www.webjam.com/wwwpctoolscomspyware4302]spyware terminator 2010 free download[/url] free spyware doctor full version with key download. I can say that those mspy reviews were the reason I bought mspy - I work as a technical writer and I need to be in the know. It was not possible for me to check her phone and computer so that I can say her something. I am always anxiously waiting for the new tracking features. Thanks to mSpy, I was able to monitor phone’s activity and location. I do not recommend them! I highly recommend for people to use.


Comment externals-tutorial

Powered by Typo – Thème Frédéric de Villamil | Photo Glenn