CFengine 3 Tutorial Part 5 Client promises.cf and regular cf-agent operation -- Prefetch Technologies

Finally, the moment we’ve been waiting for. Lets take a crack at promises.cf and what we have defined.

$ cat -n /var/cfengine/inputs/promises.cf

1 # promises.cf -- clients only
2
3 body common control
4 {
5 bundlesequence => {
6 "update",
7 "garbage_collection",
8 "smf_update",
9 "verify_root",
10 "general_global_configs",
11 };
12
13 inputs => {
14 "update.cf",
15 "cfengine_stdlib.cf",
16 "cf-execd.cf",
17 "smf_update.cf",
18 "verify_root.cf",
19 "general_global_configs.cf",
20 };
21 }
22
23 #######################################################
24
25 bundle common g
26 {
27 # Define some global variables
28 vars:
29 "masterfiles" string => "/var/cfengine/masterfiles";
30 "inputs" string => "/var/cfengine/inputs";
31 "workdir" string => "/var/cfengine";
32 "phost" string => "192.168.1.10";
33
34 # Define global or local zones
35 "solaris_zone_type" expression => usemodule("init_zone-type","");
36 }
37
38 #######################################################
39
40 body agent control
41 {
42 # if default runtime is 5 mins we need this for long jobs
43 ifelapsed => "15";
44
45 # Allow us to use DNS
46 skipidentify => "false";
47 }
48
49 #######################################################
50
51 body monitor control
52 {
53 forgetrate => "0.7";
54 histograms => "true";
55 monitorfacility => "LOG_DAEMON";
56 }
57
58 #######################################################
59
60 body reporter control
61
62 {
63 reports => { "all" };
64 build_directory => "$(sys.workdir)/reports";
65 report_output => "text";
66 time_stamps => "true";
67 }
68
69 #######################################################

Line 3 defines “body common control” — our main() function which drives all other executions of the policy.
Lines 13-20 pull in additional CFEngine Policy files for execution.
Lines 5-11 define the bundlesequence — the order of which functions are executed. These functions are “included” into this policy by pulling in the CFengine policy files in lines 13-20.

Note, the first action cf-agent will execute in “normal operation” is to update itself from the master policy server.

Line 15 imports the Standard CFengine library. Where ever you installed CFEngine during the compilation process, you can find this file. (if you didn’t use the –prefix of /var/cfengine, its probably in /usr/local). This library contains some pre-written functions to do a TON of different operations. If you want to add more functions, this might be a good place to place them. Once this library is imported, you can call any of their functions from policies defined later in promises.cf

$ find /var/cfengine -name cfengine_stdlib.cf

/var/cfengine/share/doc/cfengine/cfengine_stdlib.cf

And some examples of what these functions can do…

$ egrep 'bundle|body' /var/cfengine/inputs/cfengine_stdlib.cf

bundle edit_line comment_lines_matching(regex,comment)
bundle edit_line uncomment_lines_matching(regex,comment)
bundle edit_line delete_lines_matching(regex)
bundle edit_line append_if_no_line(str)
bundle edit_line append_if_no_lines(list)
bundle edit_line resolvconf(search,list)
bundle edit_line set_variable_values(v)
bundle edit_line append_users_starting(v)
bundle edit_line append_groups_starting(v)
bundle edit_line set_user_field(user,field,val)
bundle edit_line append_user_field(group,field,allusers)
bundle edit_line expand_template(templatefile)
body edit_field quoted_var(newval,method)
body edit_field col(split,col,newval,method)
body replace_with value(x)
body select_region INI_section(x)
body edit_defaults std_defs
body edit_defaults empty
body location start
body replace_with comment(c)
body replace_with uncomment
body action if_elapsed(x)
body action measure_performance(x)
body action warn_only
body action bg(elapsed,expire)
body contain silent
body contain in_dir(s)
body contain silent_in_dir(s)
body contain in_shell
body contain setuid(x)
body contain setuid_sh(x)
body contain jail(owner,root,dir)
body classes if_desired(a)
body classes if_repaired(x)
body classes if_else(yes,no)
body classes if_notkept(x)
body classes if_ok(x)
body copy_from secure_cp(from,server)
body copy_from remote_cp(from,server)
body copy_from local_cp(from)
body copy_from no_backup_cp(from)
body copy_from no_backup_rcp(from,server)
body link_from ln_s(x)
body link_from linkchildren(tofile)
body perms m(mode)
body perms mo(mode,user)
body perms mog(mode,user,group)
body perms og(u,g)
body perms owner(user)
body depth_search recurse(d)
body depth_search recurse_ignore(d,list)
body delete tidy
body rename disable
body rename rotate(level)
body rename to(file)
body file_select name_age(name,age)
body file_select days_old(days)
body file_select size_range(from,to)
body file_select exclude(name)
body file_select plain
body file_select dirs
body file_select ex_list(names)
body changes detect_all_change
body changes detect_content
body package_method zypper
body package_method apt
body package_method yum
body package_method solaris (pkgname, spoolfile, adminfile)
body package_method freebsd
body volume min_free_space(free)
body mount nfs(server,source)
body mount nfs_p(server,source,perm)
body mount unmount
body select_process exclude_procs(x)
body process_count check_range(name,lower,upper)

By importing this library, a lot of the complexity of lower-level policy writing is taken care of for you. You can also view a web-based version of what these policies contain here.

Looking back at promises.cf, on lines 34-35 we define a custom class using a module. Remember from before, when cf-agent executes in verbose mode, we can see which classes that were discovered. Using basic shell scripts, we can extend this to define whatever classes we want to. Lets take a look at the module we’ve defined.

$ cat /var/cfengine/modules/init_zone-type

#!/bin/bash

PATH=/usr/bin:/usr/sbin

TYPE=`zoneadm list -cp | cut -d: -f2`

if [ "$TYPE" == 'global' ]
then
echo '+global_zone'
else
echo '+local_zone'
fi

So this is super easy. We can execute whatever shell commands we want and use basic logic to determine if we’ve matched that class. If so, echo “+[class_name]” and cf-agent will pick it up. So, if you wanted to create a class for machines with Python installed, specific running processes, etc… The possibilities here are endless. Use modules to define classes of systems. Don’t modify system state – just collect data and make decisions about what type of classes a machine should belong to. Use CFEngine policies to execute changes.

Finally, the last interesting part about promises.cf here is on line 16. We define a separate file for cf-execd. Using Neil Watson’s example tutorial, he split up configuration settings for the separate daemons into their own file. It makes sense, so I followed his example here. Remember, cf-execd actually drives the execution of cf-agent, so lets take a look at this file.

$ cat -n /var/cfengine/inputs/cf-execd.cf

1 body executor control
2 {
3 # Splaytime is a critical varible that determines the "back off" time in minutes for cf-agent
4 # to check in with cf-serverd. Setting this to a higher value like "10" will allow thousands
5 # of clients to pull updates from a single policy server without hammering the network too bad all
6 # at once.
7 splaytime => "10";
8 mailto => "fatkitty@sinatra.com";
9 mailfrom => "root@$(sys.host)";
10 smtpserver => "localhost";
11 schedule => { "Min00_Min10", "Min15", "Min20", "Min25", "Hr07.Min30", "Min35", "Min40", "Min45", "Min50", "Min55" };
12 executorfacility => "LOG_DAEMON";
13
14 # This is the command that actually drives cf-execd to execute cf-agent on the schedule above.
15 exec_command => "${sys.workdir}/bin/cf-agent -f failsafe.cf && ${sys.workdir}/bin/cf-agent";
16 }
17 ##########################################
18 bundle agent garbage_collection
19 {
20 files:
21
22 "$(sys.workdir)/outputs"
23
24 delete => tidy,
25 file_select => days_old("3"),
26 depth_search => recurse("inf");
27 }

Line 7 is critical. This is the tunable of how often / how hard cf-agent is going to pound cf-serverd. The clients take the value of splaytime and using a hash algorithm with their hostname, will check in randomly over this given time frame.
Line 8 is where we want to mail the output of reports: type promises to.
Line 9 uses an internal variable $(sys.host) to substitute the host name of the box.
Line 11 is critical. By default, cf-execd will fire cf-agent off every 5 minutes. It doesn’t have to be this way. If you only want cf-agent to execute once an hour, then define the time classes here. Use periods to “combine” classes together. For example, Hr07.Min30 will only execute at 07:30. The example Min00_Min10 will execute throughout that entire span between the two values. You get the idea.
Line 15 is critical. This is the command cf-execd will execute based upon its schedule. It executes cf-agent against failsafe.cf so configs are updated from the master policy servers — then it executes in “normal mode” against promises.cf.
With no policy defined as in the 2nd operation on Line 15, promises.cf is assumed to be the default action.

That’s it in a nutshell! We’ve traced through normal client operations and how to construct general CFengine policies. To continue to add functionality into what cf-agent will execute upon, just include more *.cf files on the import and bundlesequence statemetns!

CFengine is an extremely powerful tool for controlling the configuration of hundreds / thousands / tens of thousands of machines. Its used at enterprises like Facebook to drive system management. ALWAYS test changes in a dev/test environment BEFORE pushing new policies into production. With a powerful tool, you can damage machines easily!

This article was posted by Mike on 2010-07-02 16:36:00 -0400 -0400