At some time, you came to a point when you need to pick a solution for a deployment of your project, and you decided to pick Chef.

Okay, but I have to disappoint you from the start – if you don’t have any experience with Chef, then you’re going to spend much time trying to sort things out with Chef wiki and that wiki is a tricky one. Just visit its start page and you’ll quickly understand that it is mind-blowing.

Here are some suggestions to help you in lowering the barrier when working with Chef.

Solo or Server

First of all, you have to pick a proper flavor of Chef.

Go Solo when:

• your project is small and requires an instance or two to run
• you have no staging environment or the other way to run it
• you don’t need Chef environments and don’t want to patch Chef gem in order to achieve this functionality in Solo mode

Go Server when:

• your project runs on numerous instances
• you have many projects that do not fit into Solo mode
• you want to have staging environment(s) managed by Chef as well
• you want all Chef functionality out of the box

And finally, choose Hosted Chef Server if you need the second option, but don’t want to deal with the server setup. Just to let you know, Chef Server will run nicely on AWS m1.small instance and will satisfy average demands.

Data storing

Chef offers two strategies for storing your entities like environments or roles. The recommended way is to keep these entities under version control. You’ll have to upload your entities to Chef server anyway, but SCM protects you from mistakes in the configuration as you can always pick previous settings and re-upload them to server. Also, SCM should be treated as one of the backups for Chef configuration.

Pollute environments!

Your setup will typically involve setting attributes via data bags, environments (Chef Server) or roles. It might not be obvious from the start, but it’s better to keep the number of places where you set your attributes as low as possible for readability sake.

If your choice is Chef Server, then it is good to make environments as a central place for this kind of job. In this case, your data bags will become very compact and will hold only general attributes, common across your environments.

Although you can set environment-specific attributes in data bag, but it will become huge and unreadable very soon. Instead, placing such attributes into environment files puts you on the right track. You will always know where to look for a specific attribute.

This way also provides you with the ability to access attributes directly inside recipes/templates etc via node.attribute call. Data bags are required to be read inside recipe first. Then, proper template variable may be set inside resource definition.

Environment sample:

name "myapp-staging"
description "My App staging environment"
default_attributes(
 "environment" => "staging",
 "resque" => { "workers_amount" => 2 },
 "resque_url" => "127.0.0.1:6379",
 "memcached_url" => "127.0.0.1:11211",
 "passenger_min_instances" => 2,
 "passenger_max_pool_size" => 2,
 "databases" => [
   {
     "name" => "staging",
     "host" => "db.myapp.com"
   }, ...
 ],
 "db_common" => {
   "database"  => "myapp_staging",
   "username"  => "myapp_user", ... }
)

Data bag sample:

{
 "id":"myapp",
 "home_dir": "/var/www",
 "deployment_dir": "/var/www/myapp",
 "user": "www-data",
 "group": "www-data", "repository":"git@github.com:my_organisation/myapp.git",
 "revision":"master",
 "resque_group": "myapp-resque",
 "resque_queues": "my,cute,resque,queues"
}

Then, you can access environment attributes with node.environment or node.memcached_url. While for data bags, it will look like this:

config = data_bag_item("deploy", "myapp")

template '/etc/resque.d/resque' do
...
 variables(
   :project_root => "#{config['deployment_dir']}/current",
   :queue => config['resque_queues']
 )
end

Attributes mess

Chef introduces no less than 4 types for attributes. It’s an enormous amount for most of the projects. It would be wise to stick to one type and ignore the others if possible. Chef’s Ohai plugin, which is responsible for gathering information about system, sets 4th type of attributes, and you don’t have control over it – you can only read them.

Check this page and study well priorities for them. Don’t miss the fact that attributes of “default” type set in roles will override ones set in environments while attributes of “override” type are prioritised vice-versa.

Community cookbooks

While this is the great option to have something that should work out of the box, it would be better not to use these cookbooks. Remember that community cookbooks are written in such a way that covers the majority of use cases: different operating systems, different distributives for an operating system, different ways to install packages (compile them from source or just install assembled by distributive maintainers package) etc – all of this leads to huge, barely readable cookbooks.

If you need some customization (and sure you will need it), you’ll have to either modify it or to create your own cookbook, depending on community one, or hopefully just to override some of the defined attributes on the upper levels like environments, but you have to be aware of what cookbook does. If you know what it does, then why not to have your own small and simple cookbook?.. Use community cookbooks as a source of inspiration or get vital parts from them and keep your setup readable for yourself and others.

Where the hell my changes are?

So… Your choice is Chef Server and your team works on the same cookbook simultaneously. Once the time has come and you decide to test your cookbook on staging. Chef client runs… and nothing changes. You’re trying again and again, inserting some messages, trying to raise an exception in the middle of the process… Nothing happens. WTF with my Chef Server?!! You’ve checked Server setup – url is correct, server is up and running (obviously)…

It’s time to check who’s working on the same cookbook and talk about cookbook versions stored on the server. If someone else uploads cookbook with higher version than yours, then you’ll not be able to see your changes applied during the chef-client run.

This is Server specific problem and the solution would be to introduce some convention inside a team. Particularly, version should be modified when cookbook is in production ready state and is merged into master. It would be nice to notify everybody about this change. Of course, there are other ways to workaround this problem, but this one is simple and effective.
Also you can set explicit cookbook version in your run list, like this: “recipe[deploy::myapp@1.0.0]”. This approach might protect you from surprises. On the other hand, it introduces added complexity in cookbook management.

Riding chef-client

Chef documentation proposes you two scenarios for chef-client. It might run either as a daemon or a cron task, but you don’t have to follow these options. There’s one more pretty attractive option. You may run chef-client only when you need to. Check out for more information article about Chef Server and Amazon Auto Scaling Group (ASG). Its suggestions also applicable for applications that do not utilize ASG.

Sharper knife

Knife provides you all the necessary commands to communicate with Server, but some of them just require too many arguments, so daily usage of knife may become a little bit frustrating. So don’t hesitate and sharpen that knife with your wrapper script!

Just a few examples below of how knife commands are morphed into cute compact ones with our wrapper.

1) SSH command (used to deploy changes)

before:

knife ssh "chef_environment:myapp-production" \
  "echo '{\"revision\":\"master\",\"migrate\":true}' > /tmp/json_attrs && sudo chef-client -j /tmp/json_attrs" \
  -x user_name -i config/keys/user_name.pem

after:

wr d -e myapp-production -m

wr is a name of our shell script,
d is a shortcut for “deploy”,
-e stands for environment name,
-m – whether to run migration command during deployment or not

2) SSH command to perform across all instances of the given environment (generic)

before:

knife ssh "chef_environment:myapp-production" "date" \
  -x user_name -i config/keys/user_name.pem

after:

wr ssh -e myapp-production 'date'

3) List commands

before:

knife node list
knife environment list
knife role list

after:

wr n
wr e
wr r

where n, e, r – shortcuts for “nodes”, “environments”, “roles” respectively

4) Opening SSH session to a specific instance

before:

• get node IP via knife’s list/show commands
• ssh -i config/keys/user_name.pem user_name@node_ip

after:

• get node name via wrapper list command – wr n
• wr ssh node_name

## Chef + Capistrano

It’s a popular approach to make Chef responsible for a setup of the node and then use Capistrano to perform deploys. You can do this as well, but answer this question – why should you use two tools to maintain nodes? What are the benefits? As you might have read above, Chef creators simply copy-pasted Capistrano’s deploy task.

Just check out the code of deploy provider:

  def deploy
    verify_directories_exist
    update_cached_repo # no converge-by - scm provider will dothis
    enforce_ownership
    copy_cached_repo
    install_gems
    enforce_ownership
    callback(:before_migrate, @new_resource.before_migrate)
    migrate
    callback(:before_symlink, @new_resource.before_symlink)
    symlink
    callback(:before_restart, @new_resource.before_restart)
    restart
    callback(:after_restart, @new_resource.after_restart)
    cleanup!
    Chef::Log.info "#{@new_resource} deployed to #{@new_resource.deploy_to}"
  end

So why bother yourself with the setup of the second tool while you have everything you need in place? If you can’t live without Capistrano, then better say bye to Chef and use it instead as the only tool for your system maintenance.

Re-think your system

If you’re switching from some other tool to Chef or vice-versa, it’s a good practice to review your system – the way it is being set up, scripts you use etc. Switching to a new tool gives you an unique ability and motivation to improve things. For example, one of our projects had weak mechanism to manage resque workers. After switching to Chef, we improved this part of the system to the state that satisfies us. Similar sort of improvements were applied to some other parts of the system. So don’t miss your chance :)