Skip to content
 

(Semi-)Automated ~/.ssh/config management

Following up from here, a concrete application of the technique sketched at the end of that article.
Considering that it's a quick and dirty hack, and that the configuration format was conjured up from scrath in 10 minutes, it has worked surprisingly well so far (for what it has to do).
It's also a highly ad-hoc hack, which means that it will be absolutely useless (at least, without making more or less heavy changes) in a lot of environments.

The idea: automated generation of the ~/.ssh/config file (which is just like /etc/ssh/ssh_config, but per-user).

As anyone who has used SSH more than a few times perfectly knows (or should know, though that doesn't always seems to be the case), having to repeatedly type every time

ssh -p 1234 -A root@s01.paris.dc1.example.com

is not at all the same as typing (for example)

ssh s01pd1

That's one of the main reasons for using the ~/.ssh/config file, of course: creating easier aliases for otherwise complicated and/or long hostnames (and at the same time being able to supply extra options like username, port etc. without having to type them on the command line).

For the above, one could put this in the config:

Host s01pd1
User root
Port 1234
ForwardAgent yes
Hostname s01.paris.dc1.example.com

Since the SSH client checks this file even before attempting DNS resolution, we have accomplished our goal of reducing the amount of keystrokes to type for this connection (and, consequently, reduced the likeliness of typos and the time needed to type it).

However, in certain environments machines come and go rapidly, and manually editing the file each time to keep it up-to-date is tedious and error-prone (and, furthermore, there are often groups of machines with the same configuration).

Starting from a plain list of hostnames, it's easy to programmatically generate a ~/.ssh/config file. However we don't simply want the hostnames replicated, we also want to have (short!) aliases for each host.

So that's the first desired feature: creating an alias for each host, following some fixed rule. How exactly the alias is generated from the FQDN can vary depending on what looks and feels easiest, most logical or most convenient for the user, so the mechanism should allow for the definition of "rules". But there's no need to invent something new; these rules can be based on the good old regular expression syntax, which is surely well suited for the task.

The second problem to solve is that, for a lot of reasons, there will surely have to be host entries in the ~/.ssh/config file that do not easily lend themselves to be automatically generated (because the SSH port is different, because the username is different, because for this one machine X forwarding is needed, because it needs ad-hoc crypto parameters, because there's no obvious transformation pattern to use to generate the short name, because the machine is not part of any group, because... a lot of other reasons). In other words, it must be possible to keep a number of manually maintained entries (hopefully few, but of course it depends) which should not be touched when the file is subject to automated (re)generation.
This problem is solved by creating a "safe" zone in the file, delimited by special comment markers. When regenerating the file, the contents of the safe zone are preserved and copied verbatim, so manual changes must be done inside this area.
Due to the way ssh looks for values in the file (value from first matching entry is used), the safe zone is located at the end of the file, so for example it's possible to set more specific per-host values in the automatically generated part, and finally set general defaults (eg Host * and so on) in the safe zone.

So our skeleton to be used as starting point for the (semi-)automatically managed ~/.ssh/config is something like this:

#### BEGIN SAFE ZONE ####
# put manually mantained entries here, they will be preserved.

#### END SAFE ZONE ####

When starting from scratch, the above section will be included anyway (albeit empty) in the generated file. If you want to use an existing ~/.ssh/config as starting point, add the above special comment markers at its beginning and end, effectively turning the whole file into a safe zone. Later refining is always possible, so better safe than sorry.

Now, for the actual host definitions, we can use a very simple file format. Hosts can be divided into groups, where hosts belonging to the same group share (at least) the same DNS domain. This is totally arbitrary; after all, as said, we're talking about an ad-hoc thing. More shared options can be specified, as we'll see.

For each host group a list of (unqualified) hostnames is given, space-separated, followed (separated by a colon - ":") by a domain to be appended to the hostname. This is the bare minimum; with this we get the obvious output, so for example starting from

# Paris DC1 hosts
server01 server02 mysql01 : paris.dc1.example.com

# Frankfurt DC2 hosts
mongo01 mongo02 : frankfurt.dc2.example.com

we get (username "root" is assumed by default, another arbitrary decision):

Host server01
Hostname server01.paris.dc1.example.com
User root

Host server02
Hostname server02.paris.dc1.example.com
User root

Host mysql01
Hostname mysql01.paris.dc1.example.com
User root

Host mongo01
Hostname mongo01.frankfurt.dc2.example.com
User root

Host mongo02
Hostname mongo02.frankfurt.dc2.example.com
User root

So at least we save ourself the hassle of typing the username and the FQDN (ie, we can do "ssh server01" instead of "ssh root@server01.paris.dc1.example.com"). Not bad. But life isn't always that easy, and some day there might be another "server01" host in some other domain (host group), at which point "ssh server01" would cease to be useful.
So we use a third field to specify a (optional, but highly recommended) "transformation" expression (in the form of perl's s/// operator) which is applied to the unqualified hostname to derive the final alias for each host. This way, we can create (for example) "server01p1" and "server01f2" as aliases for the one in DC1 Paris and the one in DC2 Frankfurt respectively and restore harmony in the world (if it only were so easy).

So we can do this:

# Paris DC1 hosts
server01 server02 mysql01 : paris.dc1.example.com : s/$/p1/

# Frankfurt DC2 hosts
server01 mongo01 : frankfurt.dc2.example.com : s/$/f2/

to get:

Host server01p1
Hostname server01.paris.dc1.example.com
User root

Host server02p1
Hostname server02.paris.dc1.example.com
User root

Host mysql01p1
Hostname mysql01.paris.dc1.example.com
User root

Host server01f2
Hostname server01.frankfurt.dc2.example.com
User root

Host mongo01f2
Hostname mongo01.frankfurt.dc2.example.com
User root

Now we have to type two characters more, but it's still a lot better than the full FQDN and allows us to distinguish between the two "server01".

If the hosts share some other common options, they can be added starting from the fourth field. For example, a group of office switches that only support old weak crypto algorithms and need username "admin" (not an infrequent case):

# Crappy office switches
sw1 sw2 sw3 sw4 : office.int : s/^/o/ : Ciphers 3des-cbc : MACs hmac-sha1 : KexAlgorithms diffie-hellman-group1-sha1 : User admin

now gives:

Host osw1
Ciphers 3des-cbc
MACs hmac-sha1
KexAlgorithms diffie-hellman-group1-sha1
User admin
Hostname sw1.office.int

Host osw2
Ciphers 3des-cbc
MACs hmac-sha1
KexAlgorithms diffie-hellman-group1-sha1
User admin
Hostname sw2.office.int

Host osw3
Ciphers 3des-cbc
MACs hmac-sha1
KexAlgorithms diffie-hellman-group1-sha1
User admin
Hostname sw3.office.int

Host osw4
Ciphers 3des-cbc
MACs hmac-sha1
KexAlgorithms diffie-hellman-group1-sha1
User admin
Hostname sw4.office.int

Within the extra options, simple interpolation of the special escape sequences %h and %D is supported, similar to what ssh does in its config files (though %D is not supported there): %h is replaced with the (unqualified) hostname, %D with the domain. This makes it possible to say:

# Paris DC1 hosts, behind firewall
server01 server02 server03 : paris.dc1.example.com : s/$/p1/ : ProxyCommand ssh admin@firewall.%D nc %h 22

and have the following automatically generated:

Host server01p1
ProxyCommand ssh admin@firewall.paris.dc1.example.com nc server01 22
User root

Host server02p1
ProxyCommand ssh admin@firewall.paris.dc1.example.com nc server02 22
User root

Host server03p1
ProxyCommand ssh admin@firewall.paris.dc1.example.com nc server03 22
User root

(Yes, there is a very limited amount of rudimentary extra-option parsing, for example to avoid producing a Hostname option - which would be harmless, anyway - if ProxyCommand is present.)

For more on the ProxyCommand directive, see for example here.

So the generic format of the template used to define hosts is:

#host(s) : domain [ : transformation [ : extra_opt_1 ] [ : extra_opt_2 ] ... [ : extra_opt_n ] ]
# first 2 are mandatory, although domain can be empty

Comments and empty lines are ignored. Spaces around the field-separating colons can be added for readability but are otherwise ignored.

If no domain should be appended (for example because it's automatically appended as part of the host's domain resolution mechanism) the domain field can be left empty. Similarly, if no transformation is desired, the transformation field can be left empty to mean "apply no transformation" (the bare unqualified hostname will directly become the alias).

We assume this template file with host definitions is saved in ~/.ssh_config_hosts. Adapt the code as needed.

As mentioned, the automatically generated host blocks are placed before the safe zone, which is always preserved.

Here's the code to regenerate ~/.ssh/config starting from the host definitions in the format explained above and an (optional) existing ~/.ssh/config.
WARNING: this code directly overwrites the existing ~/.ssh/config file, so it's higly advised to make a backup copy before starting to experiment. Output to stdout can also be enabled (see comment in the code) to visually check the result without overwriting.

#!/usr/bin/perl

use warnings;
use strict;

my $tpl_file = "$ENV{HOME}/.ssh_config_hosts";
my $config_file = "$ENV{HOME}/.ssh/config";

my @staticpart = ();
my @generatedpart = ();

my $beg_safepat = '#### BEGIN SAFE ZONE ####';
my $end_safepat = '#### END SAFE ZONE ####';

# read safe section of the config file (to be preserved)
if (-f $config_file) {
  open(my $confr, "<", $config_file) or die "Cannot open $config_file for reading: $!";

  my $insafe = 0;

  while (<$confr>) {
    if (/^$beg_safepat$/) {
      $insafe = 1;
      next;
    }

    if (/^$end_safepat$/) {
      $insafe = 0;
      last;
    }

    next if not $insafe;
    push @staticpart, $_;
  }

  close($confr) or die "Cannot close $config_file: $!";
}

# read host template

open(my $tplr, "<", $tpl_file) or die "Cannot open template $tpl_file for reading: $!";

while (<$tplr>) {

  # skip empty lines and comments
  next if /^\s*(?:#.*)?$/;

  chomp;
  s/\s*#.*//;

  my ($hlist, $domain, $transf, @extra) = split(/\s*:\s*/);

  my @hosts = split(/\s+/, $hlist);

  for my $host (@hosts) {

    my $entry = "";

    my $alias = $host;
    if ($transf) {
      eval "\$alias =~ $transf;";
    }

    $entry = "Host $alias";

    my %opts = ();

    for (@extra) {

      # minimal %h/%D interpolation for things like proxycommand etc...
      (my $extra = $_) =~ s/%h/$host/g; $extra =~ s/%D/$domain/g;

      $entry .= "\n$extra";

      my ($op) = $extra =~ /^(\S+)/;
      $opts{lc($op)} = 1;
    }

    if (!exists($opts{proxycommand})) {
      $entry .= "\nHostname $host" . ($domain ? ".$domain" : "");
    }

    if (!exists($opts{user})) {
      $entry .= "\nUser root";
    }

    push @generatedpart, $entry;

  }
}

close($tplr) or die "Cannot close template $tpl_file: $!";

# write everything out to $config_file

open(my $confw, ">", $config_file) or die "Cannot open $config_file for writing: $!";
# use this to send to stdout instead
#my $confw = *STDOUT;

print $confw "#########################################################################\n";
print $confw "# the following entries are automatically generated, do not change them\n";
print $confw "# directly. Instead change the file $tpl_file\n";
print $confw "# and run $0 to regenerate them.\n";
print $confw "#########################################################################\n\n";

# generated part, each item is a host block
print $confw (join("\n\n", @generatedpart), "\n\n");

# static part (safe zone)
for ("$beg_safepat\n", @staticpart, "$end_safepat\n") {
  print $confw $_;
}

print $confw "\n";

close($confw) or die "Cannot close $config_file: $!";

exit;

4 Comments

  1. Lomanic says:

    Hi, interesting article (and blog) here, you might be interested in https://github.com/kevinburke/ssh_config, a very complete ssh_config parser/writer for the Go programming language. I think the Perl equivalent would be http://search.cpan.org/dist/Net-SSH-Perl/lib/Net/SSH/Perl/Config.pm.

    I can't speak for the Perl lib, but the Go one aims to save existing comments/does not overwrite the file as your script does.

    • waldner says:

      Thanks for the pointers, I'll have a look. Mine was a crude hack indeed, and aimed to solve just a very specific use case I had at the time.

  2. ticolensic says:

    Sorry to interrupt, but:

    man ssh_config
    --------------
    Include
    Include the specified configuration file(s). Multiple pathnames may be specified and each pathname may contain glob(3) wildcards and, for user configura‐
    tions, shell-like ‘~’ references to user home directories. Files without absolute paths are assumed to be in ~/.ssh if included in a user configuration file
    or /etc/ssh if included from the system configuration file. Include directive may appear inside a Match or Host block to perform conditional inclusion.

    I guess you don't need no #comment sections

    • waldner says:

      Sure, but the Include option appeared in OpenSSH 7.3, released more than one year after the article was written. It's good to point out that currently such possibility exists, though. Thanks.