As promised in my last post about configuration management, I want to introduce you to one of the key Open Source configuration management players on the scene today – Cfengine. Embarked upon in 1993 by Mark Burgess, Cfengine has helped system administrators configuring and maintaining their servers in the data center for over a decade – bringing order to chaos and discipline to exhibitionism. This C based configuration management system can be manually executed, daemon driven and run by crontab and is controlled through a series of text-file configurations. Making use of a mostly declarative language, a single Cfengine statement can potentially cause hundreds of operations to be executed on multiple hosts across the network. Installation nowadays is as painless as yum install cfengine (Fedora Core/SuSE) or apt-get install cfengine2 (Debian).
How does it work?
Every change you make on a server (“just to see how this setting will react in production”) contributes to drift – a configuration difference between servers that makes every subsequent change (and most deployments) riskier than the last. Cfengine addresses drift by having clients regularly poll a policy server to fetch the master configuration file. They parse this file twice applying any necessary changes statements and noting any discrepancies. Why twice? Cfengine simply assumes that some changes will take multiple passes to establish a completely correct configuration. In order to reach a state of convergence, the second run is almost certain to wrap-up any final problems and deliver a correctly configured server. The default port used by client and server Cfengine agents is 5308.
What does it look like?
Remote file serving and requests are handled by the program cfservd and the client polling is done by a program called cfagent – both of which I’ll describe in more detail below. Although you could theoretically run a client and policy server on the same machine, this isn’t very interesting in a real data center environment. You have multiple servers to maintain and you’ll probably want to manually push out updates in an emergency, so take a look at a basic setup below.
cfservd
In my example, every host runs the remote request handler – cfservd. It’s controlled by cfservd.conf which defines trusted domains, key hosts, etc. The syntax of this config file varies slightly between a basic client and the main policy server. Client machines must specify a correct path to the polling cfagent program (i.e. cfrunCommand = ( "/usr/sbin/cfagent" )) and grant connection access to the appropriate policy server(s) (i.e. grant: /usr/sbin/cfagent *.ps.example.com).
cfagent
cfagent is the main workhorse responsible for clients pulling configuration data down from the policy server. Cfengine uses a pull based mechanism because Burgess found that push based updates from the policy server not only required a lot of coding overhead, but were less reliable over time. cfagent’s first order of business is to ensure the correctness of both local files and remote configurations by processing update.conf. Doing a quick sanity check of the remote configuration prevents a lot of common, costly accidents. Imagine a single syntax error in any of the config files on the policy server. If cfagent blindly attempted to execute these files, your entire server farm could be instantly crippled. After verifying everything’s ok both at home and abroad, it can turn to the real work of checking its local configuration against what the policy server says it should like by running through cfagent.conf. Here’s a glimpse at some sample contents :
control:
actionsequence = ( tidy files )
domain = ( example.com )
timezone = ( MET )
smtpserver = ( smtphost.example.org ) # used by cfexecd
sysadm = ( me@example.com ) # where to mail output
######################################################################
tidy:
# clean up some tmp and core files
solaris::
/var/crash pat=*core age=14 r=inf rmdirs=true inform=true
any::
/tmp pat=* age=14 r=inf rmdirs=true inform=false
/tmp pat=core* age=0 r=inf inform=true
home pat=*~ age=14 r=inf inform=false
files:
# Check some important files
/etc/passwd mode=644 owner=root action=fixall
/etc/shadow mode=600 owner=root action=fixall
# Do a tripwire check on ALL files in /usr/ dir
/usr
owner=root,daemon # all files must be owned by root or daemon
checksum=md5 # use md5 or sha
recurse=inf # all subdirs
ignore=tmp # skip /usr/tmp
action=fixall
Execution
cfagent can be run via crontab or the cfexecd daemon, but you can also manually trigger a network wide cfagent run by invoking the cfrun command on a server with a valid cfrun.hosts file. This is as close to “push” as Cfengine gets, but don’t be fooled. No configuration data is being pushed out to the clients. Instead, the clients listed in the cfrun.hosts file are asked to connect back to initiating host’s cfservd via their own cfagents. This is handy if you patch a security hole or fix a syntax error in some config file and need your network updated immediately rather than waiting around for the cronjob.
Resources & Articles
If I’ve whetted your appetite, get ready for the main course. With 15 years and counting of production experience, Cfengine has a huge community and plenty of excellent documentation and HOWTOs. A commercially backed site (email registration required) has also opened up, containing a growing library of information, including this online tool to convert scripts into correctly formatted Cfengine declarations.
Tune in next week for a similar look at the configuration management tool Puppet!
Agile Web Operations 