Following up from here, a concrete application of the technique sketched at the end of that article.
Considering that it's a quick and dirty hack, and that the configuration format was conjured up from scrath in 10 minutes, it has worked surprisingly well so far (for what it has to do).
It's also a highly ad-hoc hack, which means that it will be absolutely useless (at least, without making more or less heavy changes) in a lot of environments.
The idea: automated generation of the ~/.ssh/config file (which is just like /etc/ssh/ssh_config, but per-user).
As anyone who has used SSH more than a few times perfectly knows (or should know, though that doesn't always seems to be the case), having to repeatedly type every time
ssh -p 1234 -A root@s01.paris.dc1.example.com
is not at all the same as typing (for example)
ssh s01pd1
That's one of the main reasons for using the ~/.ssh/config file, of course: creating easier aliases for otherwise complicated and/or long hostnames (and at the same time being able to supply extra options like username, port etc. without having to type them on the command line).
For the above, one could put this in the config:
Host s01pd1 User root Port 1234 ForwardAgent yes Hostname s01.paris.dc1.example.com
Since the SSH client checks this file even before attempting DNS resolution, we have accomplished our goal of reducing the amount of keystrokes to type for this connection (and, consequently, reduced the likeliness of typos and the time needed to type it).
However, in certain environments machines come and go rapidly, and manually editing the file each time to keep it up-to-date is tedious and error-prone (and, furthermore, there are often groups of machines with the same configuration).
Starting from a plain list of hostnames, it's easy to programmatically generate a ~/.ssh/config file. However we don't simply want the hostnames replicated, we also want to have (short!) aliases for each host.
So that's the first desired feature: creating an alias for each host, following some fixed rule. How exactly the alias is generated from the FQDN can vary depending on what looks and feels easiest, most logical or most convenient for the user, so the mechanism should allow for the definition of "rules". But there's no need to invent something new; these rules can be based on the good old regular expression syntax, which is surely well suited for the task.
The second problem to solve is that, for a lot of reasons, there will surely have to be host entries in the ~/.ssh/config file that do not easily lend themselves to be automatically generated (because the SSH port is different, because the username is different, because for this one machine X forwarding is needed, because it needs ad-hoc crypto parameters, because there's no obvious transformation pattern to use to generate the short name, because the machine is not part of any group, because... a lot of other reasons). In other words, it must be possible to keep a number of manually maintained entries (hopefully few, but of course it depends) which should not be touched when the file is subject to automated (re)generation.
This problem is solved by creating a "safe" zone in the file, delimited by special comment markers. When regenerating the file, the contents of the safe zone are preserved and copied verbatim, so manual changes must be done inside this area.
Due to the way ssh looks for values in the file (value from first matching entry is used), the safe zone is located at the end of the file, so for example it's possible to set more specific per-host values in the automatically generated part, and finally set general defaults (eg Host * and so on) in the safe zone.
So our skeleton to be used as starting point for the (semi-)automatically managed ~/.ssh/config is something like this:
#### BEGIN SAFE ZONE #### # put manually mantained entries here, they will be preserved. #### END SAFE ZONE ####
When starting from scratch, the above section will be included anyway (albeit empty) in the generated file. If you want to use an existing
Now, for the actual host definitions, we can use a very simple file format. Hosts can be divided into groups, where hosts belonging to the same group share (at least) the same DNS domain. This is totally arbitrary; after all, as said, we're talking about an ad-hoc thing. More shared options can be specified, as we'll see.
For each host group a list of (unqualified) hostnames is given, space-separated, followed (separated by a colon - ":") by a domain to be appended to the hostname. This is the bare minimum; with this we get the obvious output, so for example starting from
# Paris DC1 hosts server01 server02 mysql01 : paris.dc1.example.com # Frankfurt DC2 hosts mongo01 mongo02 : frankfurt.dc2.example.com
we get (username "root" is assumed by default, another arbitrary decision):
Host server01 Hostname server01.paris.dc1.example.com User root Host server02 Hostname server02.paris.dc1.example.com User root Host mysql01 Hostname mysql01.paris.dc1.example.com User root Host mongo01 Hostname mongo01.frankfurt.dc2.example.com User root Host mongo02 Hostname mongo02.frankfurt.dc2.example.com User root
So at least we save ourself the hassle of typing the username and the FQDN (ie, we can do "ssh server01" instead of "ssh root@server01.paris.dc1.example.com"). Not bad. But life isn't always that easy, and some day there might be another "server01" host in some other domain (host group), at which point "ssh server01" would cease to be useful.
So we use a third field to specify a (optional, but highly recommended) "transformation" expression (in the form of perl's s/// operator) which is applied to the unqualified hostname to derive the final alias for each host. This way, we can create (for example) "server01p1" and "server01f2" as aliases for the one in DC1 Paris and the one in DC2 Frankfurt respectively and restore harmony in the world (if it only were so easy).
So we can do this:
# Paris DC1 hosts server01 server02 mysql01 : paris.dc1.example.com : s/$/p1/ # Frankfurt DC2 hosts server01 mongo01 : frankfurt.dc2.example.com : s/$/f2/
to get:
Host server01p1 Hostname server01.paris.dc1.example.com User root Host server02p1 Hostname server02.paris.dc1.example.com User root Host mysql01p1 Hostname mysql01.paris.dc1.example.com User root Host server01f2 Hostname server01.frankfurt.dc2.example.com User root Host mongo01f2 Hostname mongo01.frankfurt.dc2.example.com User root
Now we have to type two characters more, but it's still a lot better than the full FQDN and allows us to distinguish between the two "server01".
If the hosts share some other common options, they can be added starting from the fourth field. For example, a group of office switches that only support old weak crypto algorithms and need username "admin" (not an infrequent case):
# Crappy office switches sw1 sw2 sw3 sw4 : office.int : s/^/o/ : Ciphers 3des-cbc : MACs hmac-sha1 : KexAlgorithms diffie-hellman-group1-sha1 : User admin
now gives:
Host osw1 Ciphers 3des-cbc MACs hmac-sha1 KexAlgorithms diffie-hellman-group1-sha1 User admin Hostname sw1.office.int Host osw2 Ciphers 3des-cbc MACs hmac-sha1 KexAlgorithms diffie-hellman-group1-sha1 User admin Hostname sw2.office.int Host osw3 Ciphers 3des-cbc MACs hmac-sha1 KexAlgorithms diffie-hellman-group1-sha1 User admin Hostname sw3.office.int Host osw4 Ciphers 3des-cbc MACs hmac-sha1 KexAlgorithms diffie-hellman-group1-sha1 User admin Hostname sw4.office.int
Within the extra options, simple interpolation of the special escape sequences %h and %D is supported, similar to what ssh does in its config files (though %D is not supported there): %h is replaced with the (unqualified) hostname, %D with the domain. This makes it possible to say:
# Paris DC1 hosts, behind firewall server01 server02 server03 : paris.dc1.example.com : s/$/p1/ : ProxyCommand ssh admin@firewall.%D nc %h 22
and have the following automatically generated:
Host server01p1 ProxyCommand ssh admin@firewall.paris.dc1.example.com nc server01 22 User root Host server02p1 ProxyCommand ssh admin@firewall.paris.dc1.example.com nc server02 22 User root Host server03p1 ProxyCommand ssh admin@firewall.paris.dc1.example.com nc server03 22 User root
(Yes, there is a very limited amount of rudimentary extra-option parsing, for example to avoid producing a Hostname option - which would be harmless, anyway - if ProxyCommand is present.)
For more on the ProxyCommand directive, see for example here.
So the generic format of the template used to define hosts is:
#host(s) : domain [ : transformation [ : extra_opt_1 ] [ : extra_opt_2 ] ... [ : extra_opt_n ] ] # first 2 are mandatory, although domain can be empty
Comments and empty lines are ignored. Spaces around the field-separating colons can be added for readability but are otherwise ignored.
If no domain should be appended (for example because it's automatically appended as part of the host's domain resolution mechanism) the domain field can be left empty. Similarly, if no transformation is desired, the transformation field can be left empty to mean "apply no transformation" (the bare unqualified hostname will directly become the alias).
We assume this template file with host definitions is saved in ~/.ssh_config_hosts. Adapt the code as needed.
As mentioned, the automatically generated host blocks are placed before the safe zone, which is always preserved.
Here's the code to regenerate ~/.ssh/config starting from the host definitions in the format explained above and an (optional) existing ~/.ssh/config.
WARNING: this code directly overwrites the existing ~/.ssh/config file, so it's higly advised to make a backup copy before starting to experiment. Output to stdout can also be enabled (see comment in the code) to visually check the result without overwriting.
#!/usr/bin/perl use warnings; use strict; my $tpl_file = "$ENV{HOME}/.ssh_config_hosts"; my $config_file = "$ENV{HOME}/.ssh/config"; my @staticpart = (); my @generatedpart = (); my $beg_safepat = '#### BEGIN SAFE ZONE ####'; my $end_safepat = '#### END SAFE ZONE ####'; # read safe section of the config file (to be preserved) if (-f $config_file) { open(my $confr, "<", $config_file) or die "Cannot open $config_file for reading: $!"; my $insafe = 0; while (<$confr>) { if (/^$beg_safepat$/) { $insafe = 1; next; } if (/^$end_safepat$/) { $insafe = 0; last; } next if not $insafe; push @staticpart, $_; } close($confr) or die "Cannot close $config_file: $!"; } # read host template open(my $tplr, "<", $tpl_file) or die "Cannot open template $tpl_file for reading: $!"; while (<$tplr>) { # skip empty lines and comments next if /^\s*(?:#.*)?$/; chomp; s/\s*#.*//; my ($hlist, $domain, $transf, @extra) = split(/\s*:\s*/); my @hosts = split(/\s+/, $hlist); for my $host (@hosts) { my $entry = ""; my $alias = $host; if ($transf) { eval "\$alias =~ $transf;"; } $entry = "Host $alias"; my %opts = (); for (@extra) { # minimal %h/%D interpolation for things like proxycommand etc... (my $extra = $_) =~ s/%h/$host/g; $extra =~ s/%D/$domain/g; $entry .= "\n$extra"; my ($op) = $extra =~ /^(\S+)/; $opts{lc($op)} = 1; } if (!exists($opts{proxycommand})) { $entry .= "\nHostname $host" . ($domain ? ".$domain" : ""); } if (!exists($opts{user})) { $entry .= "\nUser root"; } push @generatedpart, $entry; } } close($tplr) or die "Cannot close template $tpl_file: $!"; # write everything out to $config_file open(my $confw, ">", $config_file) or die "Cannot open $config_file for writing: $!"; # use this to send to stdout instead #my $confw = *STDOUT; print $confw "#########################################################################\n"; print $confw "# the following entries are automatically generated, do not change them\n"; print $confw "# directly. Instead change the file $tpl_file\n"; print $confw "# and run $0 to regenerate them.\n"; print $confw "#########################################################################\n\n"; # generated part, each item is a host block print $confw (join("\n\n", @generatedpart), "\n\n"); # static part (safe zone) for ("$beg_safepat\n", @staticpart, "$end_safepat\n") { print $confw $_; } print $confw "\n"; close($confw) or die "Cannot close $config_file: $!"; exit;
Hi, interesting article (and blog) here, you might be interested in https://github.com/kevinburke/ssh_config, a very complete ssh_config parser/writer for the Go programming language. I think the Perl equivalent would be http://search.cpan.org/dist/Net-SSH-Perl/lib/Net/SSH/Perl/Config.pm.
I can't speak for the Perl lib, but the Go one aims to save existing comments/does not overwrite the file as your script does.
Thanks for the pointers, I'll have a look. Mine was a crude hack indeed, and aimed to solve just a very specific use case I had at the time.
Sorry to interrupt, but:
man ssh_config
--------------
Include
Include the specified configuration file(s). Multiple pathnames may be specified and each pathname may contain glob(3) wildcards and, for user configura‐
tions, shell-like ‘~’ references to user home directories. Files without absolute paths are assumed to be in ~/.ssh if included in a user configuration file
or /etc/ssh if included from the system configuration file. Include directive may appear inside a Match or Host block to perform conditional inclusion.
I guess you don't need no #comment sections
Sure, but the Include option appeared in OpenSSH 7.3, released more than one year after the article was written. It's good to point out that currently such possibility exists, though. Thanks.