Introduction

As you build out your infrastructure, managing your many servers, services, users, and applications can become unwieldy very quickly. Configuration management systems can be used to help you manage this confusion.

Chef is an excellent configuration management system that can allow you to configure different components of your overall system very easily. In previous guides, we discussed Chef terminology, how to install a Chef server, workstation, and node (with Chef 12 or Chef 11), and how to create simple cookbooks to manage configuration.

In this guide, we will continue to explore how you can manage your environment with Chef. This time, we will talk about how to use roles and environments to differentiate your servers and services based on what kind of functionality they should exhibit.

We will assume that you have installed your server, workstation, and client and that you have the cookbooks we created in our last guide available.

Roles and Environments

What is a Role?

In your organization, if your infrastructure grows to meet the demands of higher traffic, there are likely to be multiple, redundant servers that all perform the same basic tasks. For instance, these might be web servers that a load balancer passes requests to. They would all have the same basic configuration and could be said to each satisfy the same "role".

Chef's view of roles is almost entirely the same as the regular definition. A role in Chef is a categorization that describes what a specific machine is supposed to do. What responsibilities does it have and what software and settings should be given to it.

In different situations, you may have certain machines handling more than one role. For instance, if you are testing your software, one server may include the database and web server components, while in production, you plan on having these on separate servers.

With Chef, this can be as easy as assigning the first server to both roles and then assigning each role to separate computers for your production machines. Each role will contain the configuration details necessary to bring the machine to a fully operational state to fulfill its specific role. This means you can gather cookbooks that will handle package installations, service configuration, special attributes for that role, etc.

What is an Environment?

Related to the idea of a role is the concept of Chef environments. An environment is simply a designation meant to help an administrator know what stage of the production process a server is a part of. Each server can be part of exactly one environment.

By default, an environment called "_default" is created. Each node will be placed into this environment unless another environment is specified. Environments can be created to tag a server as part of a process group.

For instance, one environment may be called "testing" and another may be called "production". Since you don't want any code that is still in testing on your production machines, each machine can only be in one environment. You can then have one configuration for machines in your testing environment, and a completely different configuration for computers in production.

In the above example given in roles, you could specify that in your testing environment, the web and database server roles will be on a single machine. In your production environment, these roles should be tackled by individual servers.

Environments also help with the testing process itself. You can specify that in production, a cookbook should be a stable version. However, you can specify that if a machine is part of the testing environment, it can receive a more recent version of the cookbook.

How To Use Roles

Create a Role Using the Ruby DSL

We can create roles using the roles directory in our chef-repo directory on our workstation.

Log into your workstation and move into this directory now:

cd ~/chef-repo/roles

Within this directory, we can create different files that define the roles we want in our organization. Each role file can be written either in Chef's Ruby DSL, or in JSON.

Let's create a role for our web server:

nano web_server.rb

Inside of this file, we can begin by specifying some basic data about the role:

name "web_server"
description "A role to configure our front-line web servers"

These should be fairly straight forward. The name that we give cannot contain spaces and should generally match the file name we selected for this role, minus the extension. The description is just a human-readable message about what the role is supposed to manage.

Next, we can specify the runlist that we wish to use for this specific role. The runlist of a role can contain cookbooks (which will run the default recipe), recipes from cookbooks (as specified using the cookbook::recipe syntax), and other roles. Remember, a run_list is always executed sequentially, so put the dependency items before the other items.

If we wanted to specify that the run_list should be exactly what we configured in the last guide, we would have something that looked like this:

name "web_server"
description "A role to configure our front-line web servers"
run_list "recipe[apt]", "recipe[nginx]"

We can also use environment-specific run_lists to specify variable configuration changes depending on which environment a server belongs to.

For instance, if a node is in the "production" environment, you could want to run a special recipe in your "nginx" cookbook to bring that server up to production policy requirements. You could also have a recipe in the nginx cookbook meant to configure special changes for testing servers.

Assuming that these two recipes are called "configprod" and "configtest" respectively, we could create some environmental specific run lists like this:

name "web_server"
description "A role to configure our front-line web servers"
run_list "recipe[apt]", "recipe[nginx]"
env_run_lists "production" => ["recipe[nginx::config_prod]"], "testing" => ["recipe[nginx::config_test]"]

In the above example, we have specified that if the node is part of the production environment, it should run the "configprod" recipe within the "nginx" cookbook. However, if the node is in the testing environment, it will run the "configtest" recipe. If a node is in a different environment, then the default run_list will be applied.

Similarly, we can specify default and override attributes. You should be familiar with default attributes at this point. In our role, we can set default attributes which can override any of the default attributes set anywhere else.

We can also set override attributes, which have a higher precedence than many other attribute declarations. We can use this to try to force nodes that are assigned this role to behave in a certain way.

In our file, these could be added like this:

name "web_server"
description "A role to configure our front-line web servers"
run_list "recipe[apt]", "recipe[nginx]"
env_run_lists "production" => ["recipe[nginx::config_prod]"], "testing" => ["recipe[nginx::config_test]"]
default_attributes "nginx" => { "log_location" => "/var/log/nginx.log" }
override_attributes "nginx" => { "gzip" => "on" }

Here we have set a default log location for any servers in the node. We have also specified that despite what some other attribute declarations have stated in other locations, that nodes in this role should have the gzip attribute set to "on". This can be overridden in a few more places, but is generally a high precedence declaration.

Create a Role Using JSON

The other format that you can use to configure roles is JSON. In fact, we can explore this format by using knife to automatically create a role in this format. Let's create a test role:

knife role create test

Your text editor will be opened with a template role file preloaded. It should look something like this:

{
  "name": "test",
  "description": "",
  "json_class": "Chef::Role",
  "default_attributes": {
  },
  "override_attributes": {
  },
  "chef_type": "role",
  "run_list": [

  ],
  "env_run_lists": {
  }
}

This is basically the same information that we entered into the Ruby DSL-formatted file. The only differences are the formatting and the addition of two new keys called json_class and chef_type. These are used internally and should not be modified.

Other than that, we can easily recreate our other file in JSON with something like:

{
  "name": "web_server",
  "description": "A role to configure our front-line web servers",
  "json_class": "Chef::Role",
  "default_attributes": {
    "nginx": {
      "log_location": "/var/log/nginx.log"
    }
  },
  "override_attributes": {
    "nginx": {
      "gzip": "on"
    }
  },
  "chef_type": "role",
  "run_list": [
    "recipe[apt]",
    "recipe[nginx]"
  ],
  "env_run_lists": {
    "production": [
      "recipe[nginx::config_prod]"
    ],
    "testing": [
      "recipe[nginx::config_test]"
    ]
  }
}

This should have pretty much the same functionality as the Ruby version above.

Transferring Roles between the Workstation and Server

When we save a JSON file created using the knife command, the role is created on the Chef server. In contrast, our Ruby file that we created locally is not uploaded to the server.

We can upload the ruby file to the server by running a command that looks like this:

knife role from file path/to/role/file

This will upload our role information specified in our file to the server. This would work with either the Ruby DSL formatted file or a JSON file.

In a similar vein, if we want to get our JSON file from the server, we can tell the knife command to show that role file in JSON and then pipe that into a file like this:

knife role show web_server -Fjson > path/to/save/to

Assigning Roles to Nodes

So now, regardless of the format we used, we have our role on the Chef server. How do we assign a node a certain role?

We assign a role to a node just as we would a recipe, in the node's run_list.

So to add our role to a node, we would find the node by issuing:

knife node list

And then we would give a command like:

knife node edit node_name

This will bring up the node's definition file, which will allow us to add a role to its run_list:

{
  "name": "client1",
  "chef_environment": "_default",
  "normal": {
    "tags": [

    ]
  },
  "run_list": [
    "recipe[nginx]"
  ]
}

For instance, we can replace our recipe with our role in this file:

{
  "name": "client1",
  "chef_environment": "_default",
  "normal": {
    "tags": [

    ]
  },
  "run_list": [
    "role[web_server]"
  ]
}

This will perform the same steps as our previous recipes, but instead it simply speaks to the role that the server should have.

This allows you to access all servers in a specific role by search. For instance, you could search for all of the database servers in your production environment by searching a role and environment:

knife search "role:database_server AND chef_environment:prod" -a name

This will give you a list of the nodes that are configured as a database server. You could use this internally in your cookbooks to configure a web server to automatically add all of the production database servers into its pool to make read requests from.

How To Use Environments

Creating an Environment

In some ways, environments are fairly similar to roles. They are also used to differentiate different servers, but instead of differentiating by the function of the server, environments differentiate by the phase of development that a machine belongs to.

We discussed some of this earlier when talking about roles. Environments that coincide with your actual product life-cycle make the most sense. If you run your code through testing, staging, and production, you should have environments to match.

As with roles, we can set up the definition files either in the Ruby DSL or in JSON.

In our "chef-repo" directory on our workstation, we should have an environments directory. This is where we should put our environment files.

cd ~/chef-repo/environments

Within this directory, if we were going to define an environment for development, we could make a file like this:

nano development.rb

name "development"
description "The master development branch"
cookbook_versions({
    "nginx" => "<= 1.1.0",
    "apt" => "= 0.0.1"
})
override_attributes ({
    "nginx" => {
        "listen" => [ "80", "443" ]
    },
    "mysql" => {
        "root_pass" => "root"
    }
})

As you can see, one of the major advantages of incorporating environments into your system is that you can specify version constraints for the cookbooks, and recipes that are deployed.

We could also use the JSON format. The knife tool can generate the template of an environment file by typing:

knife environment create development

This will open our editor (again, you can set your editor with export EDITOR=nano) with a preloaded environment file with the name filled in.

We could create the same file by typing in:

{
  "name": "development",
  "description": "The master development branch",
  "cookbook_versions": {
    "nginx": "<= 1.1.0",
    "apt": "= 0.0.1"
  },
  "json_class": "Cheff:Environment",
  "chef_type": "environment",
  "default_attributes": {
  },
  "override_attributes": {
    "nginx": {
      "listen": [
        "80",
        "443"
      ]
    },
    "mysql": {
      "root_pass": "root"
    }
  }
}

This file should be functionally the same as the Ruby file we demonstrated above. As with the JSON role files, the environment JSON files have two extra pieces of information (json_class and chef_type) which should be left alone.

Moving Environment Files to and from the Server

At this point, if you used the Ruby DSL, your file is on the workstation and if you used JSON, your file is only on the server. We can easily move files back and forth through knife.

We could upload our Ruby file to the Chef server by typing this:

knife environment from file ~/chef-repo/environments/development.rb

For our JSON file, we can get the environment file off of the server by typing something like:

knife environment show development -Fjson > ~/chef-repo/environments/development.json

This will display the JSON file from the server and pipe the results into a local file within the environments subdirectory.

Setting Environments in Nodes

Each node can be in exactly one environment. We can specify the environment that a node belongs to by editing its node information.

For instance, to edit a node called client1, we could type this:

knife node edit client1

This will open up a JSON formatted file with the current node parameters:

{
  "name": "client1",
  "chef_environment": "_default",
  "normal": {
    "tags": [

    ]
  },
  "run_list": [
    "role[web_server]"
  ]
}

As you can see, the chef_environment is set to _default originally. We can simply modify that value to put the node into a new environment.

When you are done, save and close the file. On the next chef-client run on the node, it will pick up the new attributes and version constraints and modify itself to align with the new policy.

Conclusion

By now, you should have a good understanding of different ways you can work with roles and environments to solidify the state that your machines should be in. Using these categorization strategies, you can begin to manage the way that Chef treats servers in different contexts.