24 juin 2010 caspar 20100624
1. | ||
2. | ||
3. | ||
4. | ||
5. | ||
6. | ||
7. | ||
8. | ||
9. | ||
10. | ||
11. | ||
12. |
caspar - Makefile snippets for common tasks
include caspar/mk/caspar.mkor
include caspar/mk/docbook.mkor
include caspar/mk/pod.mk.
The typesetting functionality is delivered by docbook.mk and pod.mk. This is documented in caspar-typesetting(7).
The installing-stuff functionality is delivered by caspar.mk. (That's what the rest of the manual will talk about.) It enables one to run 'make install' from within a tree which typically holds configuration files, managed using CVS (or Subversion or any other version control system, for that matter).
It is useful in cases like this: all configuration files of some host are under version control, and, after commiting a change to CVS, you want to be able to easily install the new configuration file on the host.
With caspar, all you have to do is specify the hostname in one place, and specify the name of the target directory in each CVS directory.
csp_UHOST = root@some.host.somewhere include caspar/mk/caspar.mk. Within each other directory of the CVS tree which holds files, create a Makefile, which looks like e.g.
csp_DIR = /some/dir/ectory/ include ../../include/install.mk. If you'd like to use the install-recursive target too, in directories which hold subdirectories (but not files), you'll have to create a Makefile which looks something like
include ../../include/install.mk. From within the CVS tree, one can call:
make <filename>-install make install make load make make install-recursiveCalling make install (or make) now will scp all files in the current directory to the remote location. The install-recursive target descends down the tree, and calls make install in each subdirectory.
Of course, you'll have to be able to ssh directly as root to the remote host to get this working (if you'd like to use csp_UHOST = root@some.host.somewhere). If you don't like this, and would like to have a PermitRootLogin no in your /etc/ssh/sshd_config, you can use csp_sucp(1). See below.
csp_UHOST | ||
user@host, reachable using $(csp_PUSH) (which
is scp by default)
| ||
csp_UHOSTS | ||
space separated list of user@host items, reachable
using $(csp_PUSH)
| ||
csp_DIR | ||
directory on host, reachable using function
$(csp_PUSH)
| ||
csp_PUSH | ||
make function for pushing files to remote location.
Usually, this is a wrapper around a script or
program. The function will be passed 4 arguments:
local filename, [user@]host, remote
directory and (a possibly empty) extra argument.
[user@]host will be set
to all elements of $(csp_UHOSTS);
directory will be set to $(csp_DIR). The
fourth argument will be set to $(csp_XARG).
Currently, $(csp_scp_FUNC), $(csp_cp_FUNC)
and $(csp_sucp_FUNC) are supported as push
plugins. If csp_PUSH is unset, the default
$(csp_scp_FUNC) is used.
| ||
csp_LOAD | ||
targets which should depend on the `load' target.
| ||
csp_BUILD | ||
targets which should depend on the `build' target.
| ||
csp_CP | ||
cp binary, just "cp" by default
| ||
csp_SCP | ||
scp binary, just "scp" by default
| ||
csp_SUCP | ||
script wrapping sudo in ssh, "csp_sucp" by default
| ||
csp_EXTRAFILES | ||
extra files which should be installed.
Can be used to include files starting with a dot.
| ||
csp_TABOOFILES | ||
files which should never be installed. Set to
Makefile .%.swp %~ #%# pod2htmd.tmp pod2htmi.tmp by default.
| ||
csp_TABOOFILES_ADD | ||
extra files which should never be installed;
added to list in csp_TABOOFILES.
| ||
csp_TABOOFILES_SKIP | ||
files which should be installed, even if
in initial csp_TABOOFILES list. Removed from csp_TABOOFILES
list.
| ||
csp_TABOODIRS | ||
directories to exclude in install-recursive target.
set to CVS .svn by default.
| ||
csp_TABOODIRS_ADD, csp_TABOODIRS_SKIP | ||
see
csp_TABOOFILES equivalents.
| ||
csp_UHOSTS_SUBSET | ||
override csp_UHOSTS: don't push to csp_UHOSTS, but
to the intersection of this space separated list of user@host items and
csp_UHOSTS.
|
The following variables might get phased out or removed soonish:
csp_CPFLAGS | ||
extra arguments to pass to cp invocation, none by
default
| ||
csp_SCPFLAGS | ||
extra arguments to pass to scp invocation, e.g.
'-i .ssh/id_rsa-root'
| ||
csp_CPDIR | ||
directory to which we can cp(1), instead of scp
(CoPy DIRectory)
| ||
csp_CPDIRS | ||
space separated list of directories to which we can
copy (CoPy DIRectorieS)
|
Using csp_UHOST
This is the simplest way to use caspar. Makefile is
csp_UHOST = root@some.host.somewhere csp_DIR = /etc/ include caspar/mk/caspar.mkNow, running "make" will scp all files in the current directory to root@some.host.somewhere:/etc/.
More hosts, not scp but sudo via ssh
Makefile is
csp_UHOSTS = root@some.host.somewhere root@some.other.host csp_PUSH = $(csp_sucp_FUNC) csp_DIR = /etc/ include caspar/mk/caspar.mkNow, running "make" will use csp_sucp to install all files in the current directory to both root@some.host.somewhere:/etc/ and root@some.other.host:/etc/. If a file named fstab is present in the current directory, running "make fstab-install" will just install that one file. If you need to sudo(1) to another user on the remote host, add something like
csp_XARG = postgres. (If such a username is not supplied, sudo (and csp_sucp) use the default: root.)
Using csp_CP and csp_LOAD
username/etc/Makefile is
csp_UHOST = dummy csp_PUSH = $(csp_cp_FUNC) csp_DIR = $(HOME)/etc/ csp_LOAD = crontab-load include ../include/install.mk crontab-load: crontab $(csp_DIR)/crontabwhile ../include/install.mk is just
include caspar/mk/caspar.mk. Setting csp_PUSH to $(csp_cp_FUNC) causes cp(1) to get executed by "make install" (not scp(1)). Setting csp_LOAD causes "make load" to execute the crontab command. Just running "make" is OK too, since "make" calls both "make install" and "make load".
Using csp_DIR and csp_LOAD
etc/Makefile is
csp_DIR = /etc/ csp_LOAD = aliases-load include ../include/install.mk aliases-load: ssh $(csp_UHOST) "cd /etc; postalias aliases; postfix reload"while ../include/install.mk is
csp_UHOST = root@some.host.somewhere include caspar/mk/caspar.mkIf you don't like the explicit "ssh" in your Makefile, you might want to introduce a variable for that. E.g. add my_EXEC = ssh to ../include/install.mk, and write
aliases-load: $(my_EXEC) $(csp_UHOST) "cd /etc; postalias aliases; postfix reload"in etc/Makefile. This will make it easier to change to another push mechanism, once that would be needed.
Using csp_BUILD: building files locally
If you'd like to build some files locally from local sources, before installing
the just build files, do e.g.
csp_UHOST = root@some.host.somewhere csp_DIR = /etc/ csp_TABOOFILES_ADD = sshd_config.m4 csp_BUILD = my-build include caspar/mk/caspar.mk my-build: sshd_config sshd_config: sshd_config.m4 m4 $< > $@List all source files in csp_TABOOFILES_ADD: this way, they won't get installed on the csp_UHOST.
Using csp_sucp_FUNC and csp_LOAD
If you'd like to use csp_sucp and want a `load' target, do something like:
csp_PUSH = $(csp_sucp_FUNC) csp_UHOST = foobar.example.com csp_DIR = /etc/uruk/ csp_LOAD = rc-load include caspar/mk/caspar.mk rc-load: ssh $(csp_UHOST) "sudo invoke-rc.d uruk force-reload"
Adding a "check" target
If you want to do some syntax check on the remote host, before loading the just
installed configuration file (and have a "make check" thing), do
csp_UHOST = foobar.example.com csp_DIR = /etc/ csp_LOAD = check my-load include caspar/mk/caspar.mk check: ssh $(csp_UHOST) do-check-stuff my-load: ssh $(csp_UHOST) do-load-stuffThis way, "make load" won't cause the file to load if the check fails (which is probably what you want). Running "make" will perform "install", "check" and "load".
Combining the csp_LOAD target with multiple hosts; building files remotely
You'll have to loop over csp_UHOSTS to execute load-command. Here's an example
doing some preprocessing on the remote hosts too.
csp_DIR = /etc/ssh/ csp_UHOSTS = root@some.host.somewhere root@some.other.host csp_LOAD = sshd_config-load include caspar/mk/caspar.mk sshd_config-load: for suh in $(csp_UHOSTS); do \ ssh $$suh "cd $(csp_DIR); \ m4 sshd_config.m4 > sshd_config && \ PATH=$$PATH:/sbin /etc/init.d/ssh restart"; \ done(You might want to use a variable like my_EXEC for ssh, see above.)
Using the csp_TABOOFILES_{ADD,SKIP} variables; another way to perform remote builds
Using the csp_TABOOFILES_{ADD,SKIP} variables is handy if you want to
install a Makefile, instead of using it: Create Makefile just
as you'd like to have it installed on the remote location. Now, create
GNUmakefile as e.g.
csp_TABOOFILES_SKIP = Makefile csp_TABOOFILES_ADD = GNUmakefile csp_DIR = /etc/foobar/ csp_UHOST = root@some.host.somewhere include caspar/mk/caspar.mk load: ssh $(csp_UHOST) "make -C $(csp_DIR)"(You might want to use a variable like my_EXEC for ssh, see above.) Now, make install and make load will do the right thing.
Using the csp_EXTRAFILES variable
Using the csp_EXTRAFILES variable is handy if you want to install files
with a leading dot. E.g.:
csp_EXTRAFILES = .bashrc csp_UHOST = root@some.host.somewhere csp_DIR = include caspar/mk/caspar.mk
Overriding csp_UHOSTS
Supply e.g.
csp_UHOSTS = root@localhost root@some.host.somewherein install.mk, to install on multiple hosts. Run
make filename-install csp_UHOSTS=joe@otherhostto install filename as joe@otherhost, instead of the default as given in install.mk. If you want to enable passing csp_UHOSTS as a shell environment variable, you'll have to use conditional assignment in your Makefile:
csp_UHOSTS ?= root@localhost root@some.host.somewhereThis allows it to run
% export csp_UHOSTS=foo@bar % make filename-installto install on foo@bar.
Overriding csp_UHOSTS in a smart way: csp_UHOSTS_SUBSET. Using multiple
groups of hosts. Recursive make made easy.
If you have lots of subdirectories holding information for lots of groups of
hosts, while this run you just want to install for a small group (or 1) hosts,
csp_UHOSTS_SUBSET is useful.
Suppose your casparized tree looks like
Makefile apache/include/install.mk apache/etc/apache2/Makefile apache/etc/apache2/envvars php/include/install.mk php/etc/php4/apache/Makefile php/etc/php4/apache/php.ini grub/include/install.mk grub/boot/grub/Makefile grub/boot/grub/menu.lst logrotate/include/install.mk logrotate/etc/Makefile logrotate/etc/logrotate.conf nrpe/include/install.mk nrpe/debian/etc/default/Makefile nrpe/debian/etc/default/nagios-nrpe-serverThe file apache/etc/apache2/Makefile is:
csp_DIR = /etc/apache2/ include ../../include/install.mk(all other Makefiles are similar). The file apache/include/install.mk is
csp_UHOSTS = root@a root@b include caspar/mk/caspar.mkThe file php/include/install.mk is the same. The files grub/include/install.mk and logrotate/include/install.mk are
csp_UHOSTS = root@d root@e root@f root@g include caspar/mk/caspar.mkThe file nrpe/include/install.mk is
csp_UHOSTS = root@d root@e root@f root@n include caspar/mk/caspar.mkThe toplevel Makefile is
dirs = $(patsubst %/Makefile,%,$(shell find * -mindepth 1 -name Makefile)) all: for i in $(dirs); do $(MAKE) -$(MAKEFLAGS) -C $$i; done install for i in $(dirs); do $(MAKE) -$(MAKEFLAGS) -C $$i install; done load for i in $(dirs); do $(MAKE) -$(MAKEFLAGS) -C $$i load; done(we don't feel like sticking a Makefile in all non-leaf nodes of our tree).
Now, when running "csp_UHOSTS_SUBSET='root@e root@f root@m root@n' make" in the toplevel, caspar just takes the intersection of csp_UHOSTS_SUBSET and csp_UHOSTS for each csp_UHOSTS list. So, caspar will not push anything for apache/ and php/. The files grub/boot/grub/menu.lst and logrotate/etc/logrotate.conf will get pushed to root@e and root@f only. The file nrpe/debian/etc/default/nagios-nrpe-server will get pushed to root@e, root@f and root@n.
This is often better than just overriding csp_UHOSTS on the commandline (or in your shell's environment): if the intersection of the original csp_UHOSTS and your new csp_UHOSTS is empty, chances are big you've just forgotten to clean your environment.
Using sudo locally for installing files
If you'd like to install files like
sudo cp foo.rc /etc/foobar/you could set up your Makefile as
csp_DIR = /etc/foobar/ csp_UHOST = dummy csp_PUSH = sudo cp $(1) $(3) include caspar/mk/caspar.mkThis is like csp_sucp, but without the ssh wrapping: it works on localhost only.
Plugging your own install script in caspar
If your script foobar should be called as e.g.
foobar --file=fstab --user@host=joe@some.host \ --dir=/etc/ --debuglevel=3then make sure your Makefile features something like
csp_foobar_FUNC = foobar --file=$(1) --user@host=$(2) \ --dir=$(3) --debuglevel=$(4) csp_PUSH = $(csp_foobar_FUNC) csp_XARG = 3You can now use csp_UHOST and csp_DIR just as you're used to.
More advanced tricks
When you don't want to ssh to root@some.host.somewhere directly, you could
do
sudo rsync -az /path/to/your/config_archive /etcon some.host.somewhere (e.g. from cron).
caspar/mk/caspar.mk, caspar/mk/docbook.mk, caspar/mk/pod.mk
The caspar homepage is at http://mdcc.cx/caspar/ .
The document "Versiebeheer en software-packages: Waarom en Hoe" (in Dutch) describes some of the reasons why people might want to use tools like caspar.
Jeroen Hoppenbrouwers blogs about the way he uses caspar, in "Using Subversion and Caspar to maintain a Linux host".
Lots of tools overlap (partly) with caspar in their functionality. Here's a list. A big part of it was collected by Ray Miller (http://users.ox.ac.uk/~raym/) of Oxford University, and published in the article "Configuration Management with Subversion, YAML and Perl Template Toolkit" in the SANE 2006 (http://www.sane.nl/sane2006/) conference proceedings.
docbookmk, by Michael Wiedmann (http://www.miwie.org/docbkmake/) offers probably a superset of Caspar's docbook.mk functionality.
latex-make by the LaTeX Utils project on http://gforge.inria.fr/projects/latex-utils/ seems to provide similar functionality as docbook.mk for LaTeX documents.
Latexmk by John Collins e.a. on http://www.phys.psu.edu/~collins/software/latexmk-jcc/ is another implementation of this idea.
SUP, the Software Upgrade Protocol and it's implementation by Carnegie Mellon University offers another way to distribute (configuration)files. Beware though: between Nov 1996 and June 2004, no new release has been published. The Debian (ftp://ftp.debian.org/debian/pool/main/s/sup/) and NetBSD packages are likely still maintained, though.
cfengine (http://www.cfengine.org/), by Mark Burgess e.a., builds expert systems to administrate and configure large computer networks: it delivers a very big superset of caspar's installation mechanism.
PIKT (http://www.pikt.org/) is intended primarily for system monitoring, but does do configuration management too.
LCFG (http://www.lcfg.org/) is another configuration management system.
The Arusha Project (ARK, at http://ark.sf.net/) provides a framework for collaborative system administration.
Puppet (http://reductivelabs.com/projects/puppet), also something like a configuration management system.
Bcfg2 (http://trac.mcs.anl.gov/projects/bcfg2/) is yet another configuration management system.
quattor (http://quattor.web.cern.ch/) is a system administration toolkit for installation, configuration and management of Unix systems.
rb3 and friends, as written and used by Ray Miller e.a. at Oxford University, (http://users.ox.ac.uk/~raym/software/configuration-management/).
The svk version control system is said to be quite usable for handling configuration file management (without a separate install mechanism like caspar). See also this discussion on the Debian development list.
On the http://www.infrastructures.org/ website on automated (Unix) system administration, you can find some thoughts on managing configuration files using a version control system.