caphub

CapHub – multiple applications deployment with Capistrano

| 4 Comments

Today we’ll talk about deploying multiple applications via single Capistrano project, managing its deployment configurations in a single place.
This approach is a must for projects with multiple applications deployment and recommended for any other project using Capistrano for deployment.
We’ve created a tool called Caphub for simplifying creation of multi-deployment Capistrano projects. Before showing it in details let’s review original Capistrano deployment methods: default and multi-stage.

Preface

We’re pretty sure you’re already familiar with basic capistrano setup. Let’s just repeat it briefly:

Then you put project-specific configuration to config/deploy.rb

Another thing that people usually use for deployment is multistage extension originally written by capistrano author.

This extension allows you to specify particular configuration related to specific environments easily. For example, production or qa environments. Common configuration is located in config/deploy.rb and specific configuration in config/deploy/{stage_name}.rb So any of your tasks are always executed as:

E.g.:

In our opinion this approach works fine… even extremely fine! Thank you, Jamis Buck!

Decentralized multi-deployment approach

Following examples assume you’re working on large or just complex project that contains several applications. For example you use SOA approach. So deployment code is located in each of its applications.

There are serious disadvantages in this method:

  1. Multiple directories: You must enter each project and start application deployment procedure there. So you either use cd command or open each project’s directory in a new terminal console.
  2. Copy-n-Paste: Some shared configuration and Capistrano recipes are just copy-n-pasted between projects. It’s not DRY.
  3. Deployment and Application mixing: We believe application code must be independent from deployment code — that is just different projects. So why would we put deployment code directly into main application repository? It makes a mess at least in SCM commit history.

As an alternative you may use centralized deployment approach.

Centralized multi-deployment approach

Let’s create another repository and call it mysite-deploy for example. Create repository layout similar to Capistrano multistage convention:

What is the benefit?

  1. Centralized location: mysite-deploy is your deployment control repository now. All deployment code located in single place and you can easily support or refactor it.
  2. Reuseable code: You don’t need to copy-n-paste Capistrano recipes or configuration files anymore. Such approach allows you to include shared configurations to certain subprojects. You can include external recipes via gems or put your own ones into recipes directory. You can easy configure/override particular recipe for certain configuration.
  3. Security policy: You may restrict access to deployment project so only specific people/departments would be able to deploy the application. Or if you use ssh keys you can create yet another repository deployment-keys and add it as git submodule or symlink it. There are lots of ways how you may organize your security policy. Anyway it is much easier with centralized deployment project location.

System configuration deployment

Where do you store your web server configuration files configured for your applications? nginx.conf for example. Do you configure it each time from scratch? Do you copy it from one server to another? Do you put in your knowledge base? Do you put it in your application repository?

We believe system configuration also should be stored in SCM. And even more: system configuration is also treated as part of deployment infrastructure! Your system administrator may say it is not true but believe us :)

So we advise you to create another repository, call it mysite-sysconf for example, and put configuration files there. You are free to organize sysconf repository layout as you wish. It depends on your deployment architecture. You can either store all config files in one repository mysite-sysconf or in several repositories like mysite-sysconf-nginx. Again it depends on your site architecture.

Then go to mysite-deploy repository and write deployment recipes. For example:

Benefits:

  1. System configuration is version-controlled: You won’t lose you system configuration changes. You can review any configuration file before it will be applied to production servers. You can test configuration on another servers.
  2. Automated deployment: It is easier to setup multiple servers with the same configurations. If you need to add extra server to existing architecture it’s also will be easy to do it with capistrano.

Now let’s simplify deployment projects creation with Caphub.

What is CapHub?

CapHub is a simple tool that generates deployment repository layout described above for you. If you want to build own deployment repository caphub might be handy. It’s as easy as bundle gem NAME command.

Installation

Usage

Let’s create own deployment hub!

Go to mysite-deploy directory and install gems:

Push repository to your remote server:

Details

Caphub generates minimal layout like:

Where Gemfile already contains some handy gems-recipes:

The core of Caphub is actually capistrano-multiconfig gem. It automatically builds capistrano task-configrations from files located in config/deploy/ directory.

So if you create file config/deploy/apps/customer/production.tb, corresponding configuration task will become available:

Please read capistrano-multiconfig README file for more details.

Extending with your recipes

  • Add gems that contain Capistrano recipes to Gemfile
  • Configure and require recipes in Capfile
  • Put your own recipes to recipes directory

We believe that CapHub concept should be used even if you even have only one application repository. Just create another repository and move deployment there. Don’t mix different things together…

Divide et Impera!

Addendum

There were a few questions from our readers about the CapHub. Let’s have it answered here as well.

Is there a way to deploy all applications and all environments at once?
It is not possible to run all configurations in parallel. CapHub is designed (in the same way as Capistrano multistage extension) to run only one configuration per cap task.
Keep in mind that configuration itself is a Capistrano task. So it is actually possible to run more then one configuration but they’ll be merged! E.g:

It will lead to unexpected results because it loads blog:staging configuration at first and then wiki:staging configuration overrides previous one…

But executing:


is almost identical to


if any of MY_TASK_ does not modify Capistrano variables.

Actually it’s possible to run configurations as batches, but only sequentially. For example create empty general configuration like config/deploy/general.rb (you will need some dummy configuration). Put into recipes/world.rb

Then you will be able to deploy all staging envrionments with


Alternatively you could just create a shell script :)

What are you supposed to put in the recipes? Any examples of them?
Actually there is an already created special directory called recipes. Just put your recipes there.
Good practice is using namespace for your tasks with the same name as recipe filename e.g.

I would like to be able to access this functionality from inside of chef-solo so I am wondering if there is a way to homehow call the deployment recipes without shelling out to Capistrano.
No, we haven’t built any functionality for integrating CapHub with Chef.
Please share one if you’ll build it!

References

Share
* Railsware is a premium software development consulting company, focused on delivering great web and mobile applications. Learn more about us.
  • http://www.martyhu.com/blog Marty Hu

    We’ve been having exactly the aforementioned grievances with our SOA capistrano setup. The inability to re-use code has been particularly frustrating. Thanks for sharing this! Trying it out now.

    • Yaroslav Lazor

      You are welcome Marty. Share your experience here after you check it out.
      It may be of a value to other folks.

      • http://www.martyhu.com/blog Marty Hu

        Yeah, I’ve tried it out and my experience is definitely positive. In general, the layout is much better than what I had before – most importantly the single deploy.rb that lets me reuse set :scm, :git for all of my deployments. Huge win. 

        That said I ran into some issues, (things I couldn’t get to work, doesn’t mean they’re not possible)

        Suppose we have a lot of environments that are very similar.
        cap my-service:development deploy
        cap my-service:stage deploy
        cap my-service:live deploy

        1. In the my-service directory, I’d like to be able to create code once that’s shared between (at least three) environments. I couldn’t figure out how to do that. For now, I made a hack with symlinks that accomplishes what I need it to do.

        2. I’d like to be able to do -T to get more specific tasks, e.g cap -T my-service:development . Right now the output isn’t very useful for that.

        3. I couldn’t get the calendar extension to work. I didn’t spend a ton of time tweaking it since having calendar isn’t too important for what we’re doing, and the only things I played around with were in the deploy.rb file.

        Thanks for the work! 

  • Anand Meyyappan

    Great article!, I am excited reading every line that solves our problem today. Hats off to Andriy Yanko !!!!!

    Its worth adding this content to CapHub wiki.

Want to get more of Railsware blog?

RSS FEED

We're always ready to help!

CONTACT US