Manage Chef Server Environments, Roles, Data Bags and Cookbooks
I has been some time i started working on Opscode Chef. Coming from Puppet background it has been a good experience and learning curve. I think by putting this document together, my little experience could be useful to a newbie like me to start Managing infrastructure using Opscode Chef OSS.
Sections ...
Kickstart ..
If you are new to Configuration Management System, check out below links on Opscode Chef:
Chef documentation is very vast and some times it gets very tricky to choose a best way of doing things. Chef supports pure Ruby implementation, which means you can write your own code within Chef DSL.
There are multiple ways of doing a single thing, you can simply put everything in a cookbook using simple ruby program or you could take the benefits of roles/environments/data bags etc to write a cookbook. And if you want to get serious about creating a standard cookbook, check out Custom LWRP or Custom Ruby LWRP.
Its very easy to build Chef resources without following proper standards or enough thinking, i have had this experience in past and it is not easy to go back & set things right. Before you know, it gets out of hands and you have no choice but keep going with the things as they are.
Laying out Chef Environment, Role and Data Bag data attributes properly is very essential to write a proper cookbook in such a way that future addon or resource addition will be easy & could be handled with minimal effort of changing cookbook. It might not be true to every scenario but it sure does help to cover requirements like handling packages management etc.
If you are coming from puppet background, hiera is the first puzzle to sort out in Chef world. You can achieve it at many levels and many ways.
Version Controlling is another important piece when it comes to any Configuration Management System. It is what decides how fast system can recover from disaster or reverted back to last known good state. chef-repo gave a good start point towards version control. Splitting out Chef resources into different git repositories provides even more granular control. There are number of ways you can manage Chef Server via Git, the approach followed in this document is described in section "Git Version Controlled - Chef Repositories".
This document is an individual approach of managing Infrastructure using Opscode Chef (Open Source). For Chef complete documentation and reference check out Chef Wiki and Chef Docs.
Chef Installation is very simple, just download the Chef server package or follow bash install method by following chef install.
e.g. On RHEL bsed platform
# Download Package
wget https://opscode-omnibus-packages.s3.amazonaws.com/el/6/x86_64/chef-11.8.2-1.el6.x86_64.rpm
# Install Chef Server Package
rpm -ivh chef-11.8.2-1.el6.x86_64.rpm
# Create/Modify Chef Server configuration file as per requirement
vim /etc/chef-server/chef-server.rb
# Configure Chef Server
chef-server-ctl reconfigure
How to Change Chef Server FQDN or SSL Certificate
To apply internal SSL Certificates or to Change SSL Certificate / Chef Server FQDN, first modify /etc/chef-server/chef-server.rb:
/etc/chef-server/chef-server.rb
api_fqdn "CHEF_SERVER_FQDN"
nginx['server_name'] = "CHEF_SERVER_FQDN"
nginx['ssl_certificate'] = "/etc/chef-server/CHEF_SERVER_FQDN.crt"
nginx['ssl_certificate_key'] = "/etc/chef-server/CHEF_SERVER_FQDN.key"
nginx['ssl_company_name'] = 'COMPANY'
nginx['ssl_country_name'] = 'COUNTRY'
nginx['ssl_email_address'] = 'mail@mydomain.in'
nginx['ssl_locality_name'] = 'CITY'
nginx['ssl_organizational_unit_name'] = 'ORGANIZATION'
#nginx['ssl_port'] = SSL_PORT # Default 443
nginx['ssl_state_name'] = 'STATE'
Reconfigure Chef Server by executing:
chef-server-ctl reconfigure
Do not forget to run test after making any change to Chef Server:
chef-server-ctl test
Running on Amazon
If you are running on amazon platform, you might be prone to runlist error. It might have been fixed by now in later Chef server releases in which case just ignore below section otherwise update file '/opt/chef-server/embedded/cookbooks/runit/recipes/default.rb' as a workaround.
# diff -u /opt/chef-server/embedded/cookbooks/runit/recipes/default.rb.old /opt/chef-server/embedded/cookbooks/runit/recipes/default.rb
--- /opt/chef-server/embedded/cookbooks/runit/recipes/default.rb.orig 2014-01-26 20:00:26.123165937 +0000
+++ /opt/chef-server/embedded/cookbooks/runit/recipes/default.rb.new 2014-01-26 20:03:22.030175179 +0000
@@ -29,6 +29,8 @@
end
when "xenserver"
include_recipe "runit::upstart"
+when "amazon"
+ include_recipe "runit::upstart"
else
include_recipe "runit::sysvinit"
end
Browse https://chef_server_dns and change the default admin password. Create admin client/user for different users to setup Chef Workstation.
For more information read chef server config.
Just like server, download Chef Client package or follow typical method of bash install by following chef install.
Once Chef client package is installed, create below files to bootstrap Chef client.
Chef Client Directories
# Create Chef Client Directories
#
mkdir /etc/chef
mkdir /etc/chef/ohai
mkdir /etc/chef/ohai/plugins
mkdir /etc/chef/ohai/hints
# File: /etc/chef/chef-validator.pem
# Copy the content of chef-validator.pem file
# content from server. Once chef-client
# gets registered with Chef server this file is
# no longer required and must be removed.
Chef Client Config file
# File: /etc/chef/client.rb
# Create Chef Client Config file, below is
# the minimal configuration to kick start
# the client. You can modify it according
# to your requirement
# Server Configuration
chef_server_url "https://{chef server url}/"
environment "{environment name}"
validation_client_name "chef-validator"
validation_key "/etc/chef-server/chef-validator.pem"
# Client Configuration
node_name "{chef client fqdn}"
client_key "/etc/chef/client.pem"
client_registration_retries 3
json_attribs "/etc/chef/client-config.json"
# Timeout
splay 30
interval 1800
rest_timeout 300
# Options
diff_disabled true
enable_reporting false
enable_selinux_file_permission_fixup false
file_atomic_update true
user nil
group nil
# Logging
verbose_logging false
log_level :info
log_location STDOUT
# To Forward logs to a file:
# log_location '/var/log/chef/chef-client.rb'
Ohai::Config[:plugin_path] << '/etc/chef/ohai/plugins'
Dir.glob(File.join("/etc/chef", "client.d", "*.rb")).each do |conf|
Chef::Config.from_file(conf)
end
For more information read client config or checkout chef-client cookbook.
Node Custom Attributes or run_list
Node run_list or client attributes can be defined in "/etc/chef/client-config.json" file.
Note that these variables will override any other variable defined in Cookbooks or Role, useful for first boot.
Ideally use knife or Chef Console to assign roles/recipes to a node.
# cat /etc/chef/client-config.json
{
"run_list": [
"role[common_role]",
"role[application_role]",
"recipe[some_recipe]",
"recipe[some_recipe]",
"..etc"
],
"application": "webserver"
}
You can also define node attributes via Ohai by creating your own Ohai Plugins.
Ohai plugins are ruby code where you can put your logic to create custom node attributes according to your environment.
e.g. File "/etc/chef/client.rb" has an entry: "Ohai::Config[:plugin_path] << '/etc/chef/ohai/plugins'"
which tell Ohai to include plugins from location '/etc/chef/ohai/plugins' on Chef client run.
# cat /etc/chef/ohai/plugins/application.rb
provides "application"
application "webserver"
If you are running on Amazon platform, ec2 hint must be present to load Ohai ec2 metadata.
echo '{}' > /etc/chef/ohai/hints/ec2.json
For more info read ohai.
For Ohai plugins ohai custom.
Knife is a Chef Management Utility. It is a set of Chef API calls and plugins to manage Chef Server. Knife can manage Chef roles, environments, data bags, cookbooks, run list, nodes and clients etc. There are some other powerful utilities available to perform same work like Berkshelf etc.
To setup a Knife Client or Workstation, user require client or user private key with admin privilege. It can be created from Chef Console by following web document manage auth.
For more information on Knife refer to knife.
Knife Setup
Create directory '~/.chef' and create knife.rb file.
mkdir ~/.chef
touch ~/.chef/knife.rb
# cat ~/.chef/knife.rb
log_level :info
log_location STDOUT
node_name 'foo'
client_key '~/.chef/foo.pem'
validation_client_name 'foo'
validation_key '~/.chef/foo.pem'
chef_server_url 'https://<chef server url>'
syntax_check_cache_path '~/.chef/syntax_check_cache'
cookbook_path '~/chef-repo/cookbooks'
environment_path '~/chef-repo/environments'
data_bag_path '~/chef-repo/data_bags'
role_path '~/chef-repo/chef_roles'
cookbook_copyright 'foo'
cookbook_license 'apachev2'
cookbook_email 'foo@foo.com'
For more knife.rb options knife config.
Test knife client
# knife client list
chef-validator
chef-webui
webserver
...
Now you have a Chef Workstation from where you can create/delete/modify Chef resources. Different team members can have their own client/user id to manage different chef resources.
Environment concept in Chef is very simple:
For more information on environment read essentials environments.
If you have multiple environments like production, stage or qa, it is always better to bootstrap the node with the respective environment instead of _default environment.
How to Create an Environment
Creating an environment using knife:
# knife environment create production
{
"name": "production",
"description": "Production Environment",
"cookbook_versions": {
"cookbook_name": "= 0.1.0"
},
"json_class": "Chef::Environment",
"chef_type": "environment",
"default_attributes": {
"some_default_variable": {}
},
"override_attributes": {
"some_variable_to_override": {}
}
}
This will open up editor with default attributes. For git version controlled or to edit it manually store the content into environment.json file. Later you can modify it locally without using knife or use it to create more environments.
Better approach is to store environment.json file into a git repo.
Whenever making any changes to environment.json file, commit it to git prior to Chef upload using knife.
knife environment from file environment.json
Where to Use Environment JSON attributes
Environment is useful to define cookbooks version, environment global attributes and override roles attributes. You can have multipe cookbook versions uploaded to Chef Server and can choose which version you want to run in an environment.
environment JSON file must be kept simple, other than overriding roles attributes or global roles/cookbooks attributes try to use roles/cookbooks instead of environment
environment.json file can have attributes differs between environment, like to test new java_version, development environment can have newer version for testing, while production still running on older version
If environments shares list of common attributes, create a common role with those attributes and add it to each environment for other roles.
If you have different roles with some common attributes and you want to override some of them, use environment to override.
Attribute value is just the precedence of default & override attributes spreaded acrossed cookbooks, roles and environment. But it could get complex and out of hand if not defined properly.
Where Not to Use Environment JSON attributes
This is something that can be learnt over time, but try to avoid environment for any attributes declaration unless have to.
Environment with a Common Role - role[system]
A Common role is a simplified approach to manage common attributes, run_list for cookbooks, other roles and environments.
Lets create a common role 'role[system]' to store all global attributes, recipe and run_list which can be attached to other roles.
Any attribute you want to declare across environments, roles or cookbooks can be defined in this common e.g. common packages, services, attributes, run_list etc.
Chef Role is a collection of attributes and run_list of roles/recipes. It is more like an application type which knows what variables needs to be declared and which recipes/roles needs to run for that application.
In a role each environment can have different run_list of roles/recipes.
For more information on essentials roles.
# cat jump.json
{
"name": "jump",
"description": "Jump / SSH Login Gateway Server",
"json_class": "Chef::Role",
"default_attributes": {
"groups": {
"dev": {}
},
"packages": {
"debian": {},
"rhel": {}
},
"services": {
"rhel": {},
"debian": {}
}
},
"override_attributes": {},
"chef_type": "role",
"run_list": [],
"env_run_lists": {
"production": [
"recipe[sshd]",
"role[system]"
],
"development": [
"recipe[sshd]",
"role[system]",
"role[dev_recipe]"
]
}
}
Common Role - role[system] for all Environments/Nodes
role[system] is a role i found very useful to use with each role or run_list of a node.
It is a default role which has all the minimum packages, services, users, file or any other resource declaration which must be setup to every node regardless of environment.
Just adding role[system] to run_list of other role or node we will have our common base setup across the environments nodes.
Create a Role
To create a role you can use blow knife command:
knife role create <role name>
You can also create a .json or .rb file manually or copy from an existing one. By Creating role.json files, they can be stored in a git repo. Like environments json file we can use git to have version controlled on roles json files.
Once a role json file is created, use knife to upload to Chef server:
knife role from file role.json
Chef Data Bags are immutable JSON Data Stores globally available for cookbooks. They are useful to store static data information or attributes, e.g. use/group/keys information. Data Bags supports encryption to store data bag json file in encrypted format.
For more information read out essentials data bags.
Create Data Bag
A Data Bag or Data Bag Item can be created usin knife:
knife data bag create <data bag name>
knife data bag create <data bag name> <data bag item>
Data bag json file can also be created manually. As Data Bags are json files they can easily be maintained in git.
To upload Data Bag item directly from a json file:
knife data bag from file <data bag name> <data bag item>.json
Now the data bags are directories in git repo with different data bag item json files in it.
Cookbook is collection of recipes, attributes, templates etc. Cookbook is like Chef Ruby module to declare resources.
Create a Cookbook
# knife cookbook create -o . sample_cookbook
** Creating cookbook sample_cookbook
** Creating README for cookbook: sample_cookbook
** Creating CHANGELOG for cookbook: sample_cookbook
** Creating metadata for cookbook: sample_cookbook
For detail documentation refer to chef cookbooks.
Version Control is quite a different approach between Puppet and Chef. From my short experience Chef server can only be managed via tools like knife or via Console.
We can version control Chef by version controlling Chef resources.
Instead of maintaing a single chef-repo repository, we can split chef-repo into different repositories:
chef_cookbooks - This repository is categorized into three sub categories. Thanks to Leknarf post which made it easy to manage cookbooks structure.
chef_distribution - Stores any static files or small binaries files here
Git access can be privileged per repository for different environment using git acl.
Create a New Chef Role using git
Checkout chef_roles repository:
git clone git@GIT_SERVER:chef_roles
cd chef_roles
ls
jenkins.json jump.json system.json
Create New role file using knife or simply copy an existing role:
cp jenkins.json new_role.json
Modify the new role id, description, attributes etc.
# cat new_role.json
{
"name": "new_role",
"description": "New Role Description",
"json_class": "Chef::Role",
"default_attributes": {},
"override_attributes": {},
"chef_type": "role",
"run_list": [],
"env_run_lists": {
"production": [
],
"development": [
]
}
}
Once created, it is always better to validate JSON syntax using a json parser. This could also be set to default in git pre-hook.
Push the new role to git
git add new_roles.json
git commit -m "added new_role"
git push
Upload new_role to Chef using knife
knife role from file new_role.json
And finally you can test the new role.
Rebuilding a Chef Server steps are similar to steps in "Install Chef Server" section.
But the difference in rebuilding an existing server or adding more Chef servers is that we do not need to re-create all the cookbooks, roles, environments or data bags. We can simply use git repositories to upload resources to new Chef server.
There are few components which needs to backup regularly like Chef Server SSL Certificate bundle, Client Signed SSL Certificate Bundles etc. For this purpose one can use individual scripts to take backup of individual components or simple run backup/snapshot of whole Chef Server disk useful for disaster recovery.
There could be other things still remains un-explored for rebuilding a chef server, but it is a good start point to begin with.
http://www.slideshare.net/opscode/
http://leknarf.net/blog/2013/04/22/staying-sane-while-writing-chef-cookbooks/#.Us5JZ_a9aT4
Apache v2.0
Open Source!