{"id":967,"date":"2011-11-18T17:01:19","date_gmt":"2011-11-18T15:01:19","guid":{"rendered":"http:\/\/blog.railsware.com\/?p=967"},"modified":"2021-08-12T13:43:51","modified_gmt":"2021-08-12T10:43:51","slug":"caphub-multiple-applications-deployment-with-capistrano","status":"publish","type":"post","link":"https:\/\/railsware.com\/blog\/caphub-multiple-applications-deployment-with-capistrano\/","title":{"rendered":"CapHub &#8211; multiple applications deployment with Capistrano"},"content":{"rendered":"\n<figure class=\"wp-block-image\"><img decoding=\"async\" src=\"https:\/\/github.com\/railsware\/caphub\/raw\/master\/caphub.png\" alt=\"\" class=\"wp-image-973\"\/><\/figure>\n\n\n\n<p>Today we&#8217;ll talk about deploying multiple applications via single Capistrano project, managing its deployment configurations in a single place.<br>This approach is a must for projects with multiple applications deployment and recommended for any other project using Capistrano for deployment.<br>We&#8217;ve created a tool called <a href=\"https:\/\/github.com\/railsware\/caphub\">Caphub<\/a> for simplifying creation of multi-deployment Capistrano projects. Before showing it in details let&#8217;s review original Capistrano deployment methods: default and multi-stage.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Preface<\/h3>\n\n\n\n<p>We&#8217;re pretty sure you&#8217;re already familiar with basic capistrano setup. Let&#8217;s just repeat it briefly:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">$ cd blog\n$ gem install capistrano\n$ capify .<\/pre>\n\n\n\n<p>Then you put project-specific configuration to <em>config\/deploy.rb<\/em><\/p>\n\n\n\n<p>Another thing that people usually use for deployment is <a href=\"https:\/\/github.com\/capistrano\/capistrano-ext\">multistage extension<\/a> originally written by capistrano author.<\/p>\n\n\n\n<p>This extension allows you to specify particular configuration related to specific <em>environments<\/em> easily. For example, production or qa environments. Common configuration is located in <em>config\/deploy.rb<\/em> and specific configuration in <em>config\/deploy\/{stage_name}.rb<\/em> So any of your tasks are always executed as:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">$ cap CONFIGURATION_TASK NAMESPACE:TASK_NAME<\/pre>\n\n\n\n<p>E.g.:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">$ cap production deploy:setup<\/pre>\n\n\n\n<p>In our opinion this approach works fine&#8230; even <em>extremely fine<\/em>! Thank you, Jamis Buck!<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Decentralized multi-deployment approach<\/h3>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter\"><img loading=\"lazy\" decoding=\"async\" width=\"600\" height=\"400\" src=\"https:\/\/railsware.com\/blog\/wp-content\/uploads\/2011\/11\/caphub-many-places.png\" alt=\"\" class=\"wp-image-969\" title=\"caphub-many-places\" srcset=\"https:\/\/railsware.com\/blog\/wp-content\/uploads\/2011\/11\/caphub-many-places.png 600w, https:\/\/railsware.com\/blog\/wp-content\/uploads\/2011\/11\/caphub-many-places-300x200.png 300w\" sizes=\"auto, (max-width: 600px) 100vw, 600px\" \/><\/figure><\/div>\n\n\n\n<p>Following examples assume you&#8217;re working on large or just complex project that contains several applications. For example you use <a href=\"http:\/\/en.wikipedia.org\/wiki\/Service-oriented_architecture\">SOA<\/a> approach. So deployment code is located in each of its applications.<\/p>\n\n\n\n<p>There are serious disadvantages in this method:<\/p>\n\n\n\n<ol class=\"wp-block-list\"><li><strong>Multiple directories<\/strong>: You must enter each project and start application deployment procedure there. So you either use <em>cd<\/em> command or open each project&#8217;s directory in a new terminal console.<\/li><li><strong>Copy-n-Paste<\/strong>: Some shared configuration and Capistrano recipes are just copy-n-pasted between projects. It&#8217;s not DRY.<\/li><li><strong>Deployment and Application mixing<\/strong>: We believe application code must be independent from deployment code \u2014 that is just different projects. So why would we put deployment code directly into main application repository? It makes a mess at least in SCM commit history.<\/li><\/ol>\n\n\n\n<p>As an alternative you may use centralized deployment approach.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Centralized multi-deployment approach<\/h3>\n\n\n\n<div class=\"wp-block-image\"><figure class=\"aligncenter\"><img loading=\"lazy\" decoding=\"async\" width=\"600\" height=\"338\" src=\"https:\/\/railsware.com\/blog\/wp-content\/uploads\/2011\/11\/caphub-one-place.png\" alt=\"\" class=\"wp-image-968\" title=\"caphub-one-place\" srcset=\"https:\/\/railsware.com\/blog\/wp-content\/uploads\/2011\/11\/caphub-one-place.png 600w, https:\/\/railsware.com\/blog\/wp-content\/uploads\/2011\/11\/caphub-one-place-300x169.png 300w\" sizes=\"auto, (max-width: 600px) 100vw, 600px\" \/><\/figure><\/div>\n\n\n\n<p>Let&#8217;s create another repository and call it <em>mysite-deploy<\/em> for example. Create repository layout similar to Capistrano multistage convention:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">[mysite-deploy]$ tree --dirsfirst\n.\n\u251c\u2500\u2500 config\n\u2502&nbsp;&nbsp; \u251c\u2500\u2500 deploy\n\u2502&nbsp;&nbsp; \u2502&nbsp;&nbsp; \u251c\u2500\u2500 billing\n\u2502&nbsp;&nbsp; \u2502&nbsp;&nbsp; \u2502&nbsp;&nbsp; \u251c\u2500\u2500 production.rb\n\u2502&nbsp;&nbsp; \u2502&nbsp;&nbsp; \u2502&nbsp;&nbsp; \u2514\u2500\u2500 qa.rb\n\u2502&nbsp;&nbsp; \u2502&nbsp;&nbsp; \u251c\u2500\u2500 core\n\u2502&nbsp;&nbsp; \u2502&nbsp;&nbsp; \u2502&nbsp;&nbsp; \u251c\u2500\u2500 production.rb\n\u2502&nbsp;&nbsp; \u2502&nbsp;&nbsp; \u2502&nbsp;&nbsp; \u2514\u2500\u2500 qa.rb\n\u2502&nbsp;&nbsp; \u2502&nbsp;&nbsp; \u2514\u2500\u2500 customer\n\u2502&nbsp;&nbsp; \u2502&nbsp;&nbsp;     \u251c\u2500\u2500 production.rb\n\u2502&nbsp;&nbsp; \u2502&nbsp;&nbsp;     \u2514\u2500\u2500 qa.rb\n\u2502&nbsp;&nbsp; \u251c\u2500\u2500 keys\n\u2502&nbsp;&nbsp; \u2514\u2500\u2500 deploy.rb\n\u251c\u2500\u2500 recipes\n\u251c\u2500\u2500 Capfile\n\u2514\u2500\u2500 Gemfile<\/pre>\n\n\n\n<p>What is the benefit?<\/p>\n\n\n\n<ol class=\"wp-block-list\"><li><strong>Centralized location<\/strong>: <em>mysite-deploy<\/em> is your deployment control repository now. All deployment code located in <em>single<\/em> place and you can easily support or refactor it.<\/li><li><strong>Reuseable code<\/strong>: You don&#8217;t need to copy-n-paste Capistrano recipes or configuration files anymore. Such approach allows you to include shared configurations to certain subprojects. You can include external recipes via gems or put your own ones into recipes directory. You can easy configure\/override particular recipe for certain configuration.<\/li><li><strong>Security policy<\/strong>: You may restrict access to deployment project so only specific people\/departments would be able to deploy the application. Or if you use ssh keys you can create yet another repository <em>deployment-keys<\/em> and add it as git submodule or symlink it. There are lots of ways how you may organize your security policy. Anyway it is much easier with centralized deployment project location.<\/li><\/ol>\n\n\n\n<!--more-->\n\n\n\n<h3 class=\"wp-block-heading\">System configuration deployment<\/h3>\n\n\n\n<p>Where do you store your web server configuration files configured for your applications? <em>nginx.conf<\/em> for example. Do you configure it each time from scratch? Do you copy it from one server to another? Do you put in your knowledge base? Do you put it in your application repository?<\/p>\n\n\n\n<p>We believe system configuration also should be stored in SCM. And even more: system configuration is also treated as part of deployment infrastructure! Your system administrator may say it is not true but believe us :)<\/p>\n\n\n\n<p>So we advise you to create another repository, call it <em>mysite-sysconf<\/em> for example, and put configuration files there. You are free to organize sysconf repository layout as you wish. It depends on your deployment architecture. You can either store all config files in one repository mysite-sysconf or in several repositories like <em>mysite-sysconf-nginx<\/em>. Again it depends on your site architecture.<\/p>\n\n\n\n<p>Then go to mysite-deploy repository and write deployment recipes. For example:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">[mysite-deploy]$ cap sysconf:nginx:production deploy:setup\n[mysite-deploy]$ cap sysconf:nginx:production deploy<\/pre>\n\n\n\n<p>Benefits:<\/p>\n\n\n\n<ol class=\"wp-block-list\"><li><strong>System configuration is version-controlled<\/strong>: You won&#8217;t lose you system configuration changes. You can review any configuration file before it will be applied to production servers. You can test configuration on another servers.<\/li><li><strong>Automated deployment<\/strong>: It is easier to setup multiple servers with the same configurations. If you need to add extra server to existing architecture it&#8217;s also will be easy to do it with capistrano.<\/li><\/ol>\n\n\n\n<p>Now let&#8217;s simplify deployment projects creation with Caphub.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">What is CapHub?<\/h3>\n\n\n\n<p><a href=\"https:\/\/github.com\/railsware\/caphub\">CapHub<\/a> is a simple tool that generates deployment repository layout described above for you. If you want to build own deployment repository <em>caphub<\/em> might be handy. It&#8217;s as easy as <em>bundle gem NAME<\/em> command.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\">Installation<\/h4>\n\n\n\n<pre class=\"wp-block-preformatted\">$ gem install caphub<\/pre>\n\n\n\n<h4 class=\"wp-block-heading\">Usage<\/h4>\n\n\n\n<p>Let&#8217;s create own deployment hub!<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">$ caphub mysite-deploy<\/pre>\n\n\n\n<p>Go to <em>mysite-deploy<\/em> directory and install gems:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">$ bundle install<\/pre>\n\n\n\n<p>Push repository to your remote server:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">git remote add origin your\/remote\/git\/repo\/url\ngit push -u origin master<\/pre>\n\n\n\n<h4 class=\"wp-block-heading\">Details<\/h4>\n\n\n\n<p>Caphub generates minimal layout like:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">tree --dirsfirst\n.\n\u251c\u2500\u2500 config\n\u2502&nbsp;&nbsp; \u251c\u2500\u2500 deploy\n\u2502&nbsp;&nbsp; \u2514\u2500\u2500 deploy.rb\n\u251c\u2500\u2500 recipes\n\u251c\u2500\u2500 Capfile\n\u251c\u2500\u2500 Gemfile\n\u2514\u2500\u2500 Gemfile.lock<\/pre>\n\n\n\n<p>Where <strong>Gemfile<\/strong> already contains some handy gems-recipes:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">source \"http:\/\/rubygems.org\"\n\ngem \"capistrano\"\ngem \"capistrano_colors\"\n\ngem \"capistrano-multiconfig\"\ngem \"capistrano-uptodate\"\ngem \"capistrano-patch\"\ngem \"capistrano-calendar\"<\/pre>\n\n\n\n<p>The core of Caphub is actually <a href=\"https:\/\/github.com\/railsware\/capistrano-multiconfig\">capistrano-multiconfig<\/a> gem. It automatically builds capistrano task-configrations from files located in <em>config\/deploy\/<\/em> directory.<\/p>\n\n\n\n<p>So if you create file <em>config\/deploy\/apps\/customer\/production.tb<\/em>, corresponding configuration task will become available:<\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">[mysite]$ cap -T\n...\ncap apps::customer:production          # Load apps:customer:production configuration\n...<\/pre>\n\n\n\n<p>Please read <a href=\"https:\/\/github.com\/railsware\/capistrano-multiconfig\">capistrano-multiconfig<\/a> README file for more details.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\">Extending with your recipes<\/h4>\n\n\n\n<ul class=\"wp-block-list\"><li>Add gems that contain Capistrano recipes to <em>Gemfile<\/em><\/li><li>Configure and require recipes in <em>Capfile<\/em><\/li><li>Put your own recipes to <em>recipes<\/em> directory<\/li><\/ul>\n\n\n\n<p>We believe that CapHub concept should be used even if you even have only one application repository. Just create another repository and move deployment there. Don&#8217;t mix different things together&#8230;<\/p>\n\n\n\n<p>Divide et Impera!<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Addendum<\/h3>\n\n\n\n<p>There were a few questions from our readers about the CapHub. Let&#8217;s have it answered here as well.<\/p>\n\n\n\n<p><strong>Is there a way to deploy all applications and all environments at once?<\/strong><br>It is not possible to run all configurations in parallel. CapHub is designed (in the same way as Capistrano multistage extension) to run only one configuration per cap task.<br>Keep in mind that configuration itself is a Capistrano task. So it is actually possible to run more then one configuration but they&#8217;ll be merged! E.g:<br><code><\/code><\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">$ cap blog:staging wiki:staging deploy:update_code\n<\/pre>\n\n\n\n<p>It will lead to unexpected results because it loads blog:staging configuration at first and then wiki:staging configuration overrides previous one&#8230;<\/p>\n\n\n\n<p>But executing:<br><code><\/code><\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">$ cap blog:staging MY_TASK_1 MY_TASK_2 MY_TASK3\n<\/pre>\n\n\n\n<p>is almost identical to<br><code><\/code><\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">$ cap blog:staging MY_TASK_1\n$ cap blog:staging MY_TASK_2\n$ cap blog:staging MY_TASK_3\n<\/pre>\n\n\n\n<p>if any of MY_TASK_ does not modify Capistrano variables.<\/p>\n\n\n\n<p>Actually it&#8217;s possible to run configurations as batches, but only sequentially. For example create empty general configuration like config\/deploy\/general.rb (you will need some dummy configuration). Put into recipes\/world.rb<br><code><\/code><\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">namespace :world do\n task :staging_deploy do\n   Capistrano::CLI.ui.say \"deploying blog:staging\"\n   system(\"cap blog:staging deploy\")\n   Capistrano::CLI.ui.say \"deploying wiki:staging\"\n   system(\"cap wiki:staging deploy\")\n  ...\n end\nend\n<\/pre>\n\n\n\n<p>Then you will be able to deploy all staging envrionments with<br><code><\/code><\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">$ cap general world:staging_deploy\n<\/pre>\n\n\n\n<p>Alternatively you could just create a shell script :)<\/p>\n\n\n\n<p><strong>What are you supposed to put in the recipes? Any examples of them?<\/strong><br>Actually there is an already created special directory called recipes. Just put your recipes there.<br>Good practice is using namespace for your tasks with the same name as recipe filename e.g.<br><code><\/code><\/p>\n\n\n\n<pre class=\"wp-block-preformatted\">$ cat recipes\/ec2.rb\nnamespace :ec2 do\n namespace :instance do\n   desc \"terminate ec2 instance\"\n   task :terminate do\n      ...\n   end\n end\nend\n<\/pre>\n\n\n\n<p><strong>I would like to be able to access this functionality from inside of chef-solo so I am wondering if there is a way to homehow call the deployment recipes without shelling out to Capistrano.<\/strong><br>No, we haven&#8217;t built any functionality for integrating CapHub with Chef.<br>Please share one if you&#8217;ll build it!<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">References<\/h3>\n\n\n\n<ul class=\"wp-block-list\"><li><a href=\"https:\/\/github.com\/capistrano\/capistrano\">capistrano<\/a><\/li><li><a href=\"https:\/\/github.com\/capistrano\/capistrano-ext\">capistrano-ext<\/a><\/li><li><a href=\"https:\/\/github.com\/stjernstrom\/capistrano_colors\">capistrano_colors<\/a><\/li><li><a href=\"https:\/\/github.com\/railsware\/capistrano-multiconfig\">capistrano-multiconfig<\/a><\/li><li><a href=\"https:\/\/github.com\/railsware\/capistrano-uptodate\">capistrano-uptodate<\/a><\/li><li><a href=\"https:\/\/github.com\/railsware\/capistrano-patch\">capistrano-patch<\/a><\/li><li><a href=\"https:\/\/github.com\/railsware\/capistrano-calendar\">capistrano-calendar<\/a><\/li><\/ul>\n","protected":false},"excerpt":{"rendered":"<p>Today we&#8217;ll talk about deploying multiple applications via single Capistrano project, managing its deployment configurations in a single place.This approach is a must for projects with multiple applications deployment and recommended for any other project using Capistrano for deployment.We&#8217;ve created a tool called Caphub for simplifying creation of multi-deployment Capistrano projects. Before showing it in&#8230;<\/p>\n","protected":false},"author":19,"featured_media":9442,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[3],"tags":[],"coauthors":["Andriy Yanko"],"class_list":["post-967","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-development"],"acf":[],"aioseo_notices":[],"categories_data":[{"name":"Engineering","link":"https:\/\/railsware.com\/blog?category=development"}],"post_thumbnails":"https:\/\/github.com\/railsware\/caphub\/raw\/master\/caphub.png","amp_enabled":true,"_links":{"self":[{"href":"https:\/\/railsware.com\/blog\/wp-json\/wp\/v2\/posts\/967","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/railsware.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/railsware.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/railsware.com\/blog\/wp-json\/wp\/v2\/users\/19"}],"replies":[{"embeddable":true,"href":"https:\/\/railsware.com\/blog\/wp-json\/wp\/v2\/comments?post=967"}],"version-history":[{"count":32,"href":"https:\/\/railsware.com\/blog\/wp-json\/wp\/v2\/posts\/967\/revisions"}],"predecessor-version":[{"id":13974,"href":"https:\/\/railsware.com\/blog\/wp-json\/wp\/v2\/posts\/967\/revisions\/13974"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/railsware.com\/blog\/wp-json\/wp\/v2\/media\/9442"}],"wp:attachment":[{"href":"https:\/\/railsware.com\/blog\/wp-json\/wp\/v2\/media?parent=967"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/railsware.com\/blog\/wp-json\/wp\/v2\/categories?post=967"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/railsware.com\/blog\/wp-json\/wp\/v2\/tags?post=967"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/railsware.com\/blog\/wp-json\/wp\/v2\/coauthors?post=967"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}