This tutorial will show how you can run John The Ripper against some password file to do recovery or check the strength of the contained passwords.
The tutorial assumes that John The Ripper is already installed on the system. Note that this was written using Ubuntu; other distro might customize it differently, so the exact details may vary on your system. For our purposes, we'll be using md5-hashed password, ie those commonly used in the /etc/shadow file under Linux.
First, john needs to have the data in a format that is the merge of the passwd and the shadow file (ie, much like the single /etc/passwd in the old days, before shadow passwords came about). To do that, it provides a command called unshadow which takes as arguments the names of a passwd file and the name of a shadow file, and outputs the result of the merge on standard output, so it's used like this:
# unshadow /path/to/passwd /path/to/shadow > johnpasswd
Simple usage
The simplest thing to do is to just run john with no arguments on the file:
$ john johnpasswd Loaded 11 password hashes (FreeBSD MD5 [32/64 X2])
(btw, this and all following commands can be run as non-root user)
If you have many password files to check, you can specify them all at once on the command line. According to the documentation, this is actually better than running many john sessions each on a single file, because, due to the way john works internally, it can potentially find more password, including some that wouldn't be found in the one-file-at-a-time method (or would take much more time).
At any time while john is running, you can press any key to see its current status:
Loaded 11 password hashes (FreeBSD MD5 [32/64 X2]) # hit a key here guesses: 0 time: 0:00:06:11 (3) c/s: 8562 trying: ttypt1 - ttypts
That will tell you, from left to right:
- number of passwords guessed so far;
- elapsed time;
- combinations per second it's trying;
- what actual passwords it's currently trying.
You can interrupt the session and then resume it later, since john can save the current state of the computation when it's interrupted with ctrl-C. For example:
Loaded 11 password hashes (FreeBSD MD5 [32/64 X2]) # ctrl-C here $ $ john --restore Loaded 11 password hashes (FreeBSD MD5 [32/64 X2]) # hit a key here guesses: 0 time: 0:00:21:19 (3) c/s: 8689 trying: twmmag3 - twmmaga
So you see that it even remembers the elapsed time. You don't have to give it any extra parameters, because it also remembers the file it was working on. You can also specify an explicit name for the session when you start it:
$ john --session=NAME johnpasswd
and accordingly resume that session later:
$ john --restore=NAME
When john finds a password, it saves it in a file. You can retrieve it later by doing
$ john --show johnpasswd
Again, the program already knows where to find the information it needs. However, you must specifiy the password file name because john saves all the passwords from all the password files it has seen in the same file, so you should tell it which password file you want the passwords of.
The location of the files john uses to do its job is (at least under Ubuntu) under ~/.john. ~/.john/john.rec is where john saves the state of the computation (so it can be resumed if interrupted); ~/.john/john.pot is where found passwords are stored; ~/.john/john.log is where john logs the details of what it's doing.
More advanced usage
With no options, john will start in "single" mode first, then move on to "wordlist" mode, and finally to "incremental" mode.
The single mode is the simplest one: john tries to crack the passwords by looking at the username or GECOS field information in the file. So this will catch, for example, users who use their login name, first name or surname as their password.
The wordlist mode is slightly more powerful, and uses a wordlist file as a source of potential passwords.
Finally, the incremental mode is the most advanced mode. In this mode, john will use true brute force methods (ie, all possible combinations of letter, numbers and/or other characters) to crack the passwords. This is the most accurate mode, but of course it's also the more resource-intensive one.
In single and wordlist mode, john does not only try the candidate passwords unchanged as they are, but it also tries some more or less complicated variations on them, like different case, different permutations of the letters, translitterations, and other.
The exact details of what john does in each mode regarding the above are configurable, with some sensible defaults. The main configuration file is /etc/john/john.conf. Apart from some initial default settings, the really relevant parts are those that define the rules that john uses to construct variations on the base words.
The rule syntax
The syntax for the mangling rules in john is similar (or even compatible) to that used by an older program called crack. Rules are executed sequentially as they are found. Rules are made up of flags, conditions, and actions, usually but not necessarily in that order. For example, some rules might begin with an action, followed by a condition, which would thus test the word after the first action was applied.
However, usually the three above parts appear in that order. When there are no conditions or flags, the action is always executed.
Flags test some general conditions to decide whether the rule should be evaluated at all, and (when present) are generally the first element of a rule. Here are some flags:
-c: run the rule only if the password hashing mechanism is case-sensitive -8: run the rule only if the password hashing mechanism accepts 8-bit characters
Conditions test some characteristics of the "current" word, such as its length, or whether it contains certain characters. Characters, in turn, can be tested literally, or grouped in classes, such as "wovels", or "consonants", "uppercase", etc. Here are some character classes:
?v: vowels ?c: consonants ?w: whitespace ?p: punctuation ?s: symbols ?l: lowercase ?i: uppercase ?d: digits ?a: alphabetics ?x: alphanumerics
Classes can be negated by using uppercase letters, so for example ?V represents non-wovels (aka consonants). So here are some conditions:
<N: true if the word is less than N characters long >N: true if the word is more than N characters long /c,/?c: true if the word contains character c, or a character in class ?c respectively !c,!?c: true if the word does NOT contain character c, or a character in class ?c respectively (c,(?c: true if the word begins with c, or with a character in ?c respectively )c,)?c: true if the word ends with c, or with a character in ?c respectively =Nc,=N?c: true if character at position N (zero-based) is c or in ?c respectively %Nc,%N?c: true if the word contains at least N instances of c or of characters in ?c respectively
Actions are where john is told what to do if the condition is true. The simplest action is ":", which just means "use the word as it is". This is the first action (with no conditions) in the simple mode and the wordlist mode rulesets, which makes sense. Other actions are:
^c: prepend c at the beginning of the word
$c: append c at the end of the word
[: delete the first character in the word
]: delete the last character in the word
r: reverse the word (abc -> cna)
d: duplicate the word (abc -> abcabc)
f: append the reversed word (abc -> abccba)
{: rotate left (abc -> bca)
}: rotate right (abc -> cba)
u: uppercase word (abc -> ABC, or no change for already uppercase chars)
l: lowercase word (ABC -> abc, or no change for already lowercase chars)
c: uppercase first letter, lowercase the rest (abc -> Abc)
C: lowercase first letter, uppercase the rest (abc -> aBC)
p: make plural (cat -> cats; probably smarter than that)
P: make past tense (walk -> walked)
I: make gerund (walk -> walking)
t: change all uppercase chars to lowercase and viceversa (aBc -> AbC)
scx,s?cx: replace all c, or all chars in ?c respectively, with x
xNM: extract M characters starting at char N (zero-based)
oNc: replace the Nth character (zero-based) with c
iNc: insert c at position N, right shifting everything from that
@c,@?c: delete all c or all chars in ?c respectively
'N: take the first N chars of the word
TN: invert case of character at position N
DN: delete character at position N
Q: don't use the word if the mangling didn't change it
There are other rules not shown above, but that should already give you an idea.
Where N is involved in any action, numbers from 0 to 9 are expressed with the corresponding digit, and numbers from 10 to 35 are expressed with uppercase letters from A to Z.
With this knowledge, we can analyze some of the predefined rules for wordlist mode. Here's how they look like in the provided config file /etc/john/john.conf:
# Wordlist mode rules [List.Rules:Wordlist] # Try words as they are : # Lowercase every pure alphanumeric word -c >3!?XlQ # Capitalize every pure alphanumeric word -c >2(?a!?XcQ ...
So let's analyze some of those:
# Lowercase every pure alphanumeric word -c >3!?XlQ
So this says that if the word is longer than 3 characters (the >3 part) AND does not contain non-alphanumeric chars (the !?X part - remember ?X is a negation of the ?x class), then lowercase the word (that's what the l does) and reject it if it didn't change after the mangling (the Q part), as that would mean trying an already-tried word. Furthermore, since this is a capitalization-changing rule, it's evaluated only if the underlying password hashing mechanism is case-sensitive (the -c flag at the beginning).
Now it shouldn't be hard to analyze the next one:
# Capitalize every pure alphanumeric word -c >2(?a!?XcQ
If the word is longer than 2 characters (the >2 part) AND starts with a letter (the (?a part) AND does not contain non-alphanumeric chars (the !?X part), then capitalize it (uppercase the first letter, lowercase the rest) using c, and again reject it if it didn't change after the mangling (Q). Again, the -c flag run this rule only for case-sensitive hashing methods.
You should have enough information now to be able to examine the rest of the rules for single mode and wordlist mode by yourself. Here you can find the full explanation of the rule syntax. There is also a preprocessor facility (not mentioned here) that makes it easier to write groups of similar rules with a single macro rule that is later expanded.
Let's see what those rules do in practice. In wordlist mode, it's possible to have john output the candidate passwords. So if we give it a wordlist file containing a single word, it should output the result of the application of all the wordlist mode rules to that single word:
$ echo blah > mywordlist.txt $ john --wordlist=mywordlist.txt --stdout blah words: 1 time: 0:00:00:00 100% w/s: 100 current: blah
Ops, we forgot to tell it to actually use the mangling rules:
$ john --wordlist=mywordlist.txt --rules --stdout blah Blah blahs blah1 Blah1 blahblah halb 1blah BLAH blah2 blah! blah3 blah7 blah9 blah5 blah4 blah8 blah6 blah0 blah. blah? BlahBlah halB Halb blahhalb blaH 2blah 4blah Blah2 Blah! Blah3 Blah9 Blah5 Blah7 Blah4 Blah6 Blah8 Blah. Blah? Blah0 3blah 7blah 9blah 5blah 6blah 8blah Blahs blahed blahing Blahed Blahing words: 51 time: 0:00:00:00 100% w/s: 5100 current: Blahing
That's a bit better. And this is with a single word. In practice, one would typically use much longer wordlists, of course.
Incremental mode
In incremental mode, things are quite different. Let's have a look at the relevant config section:
# Incremental modes [Incremental:All] File = /usr/share/john/all.chr MinLen = 0 MaxLen = 8 CharCount = 95 [Incremental:Alpha] File = /usr/share/john/alpha.chr MinLen = 1 MaxLen = 8 CharCount = 26 [Incremental:Digits] File = /usr/share/john/digits.chr MinLen = 1 MaxLen = 8 CharCount = 10 ...
As said, incremental mode has no wordlists or rules; it's just a brute force enumeration, based on all the possible combinations of a given set of symbols. So if you want to use the full symbol set (the default), john will read the section [Incremental:All] of the config; similarly, [Incremental:Alpha], [Incremental:Digit] and the other sections not shown above use a more restricted symbol set as their names imply. It's possible for example to use only digits by running john with
$ john --incremental=Digits johnpasswd
If you want to see what passwords john generates, you can again use --stdout:
$ john --incremental=Digits --stdout 1952 12345 123456 0065663 49380087 2252 2253 ...
The File = /usr/share/john/digits.chr tells john where the file containing the character set it should use is located. MinLen and MaxLen do what you would expect, ie tell john the minimum and maximum length of the generated passwords. CharCount tells john how many characters of the set it should use; in this case, being digits, it's obviously set to 10. If you look at the config, you'll see that for [Incremental:Alpha] it is set to 26, which again is expected. [Incremental:All] uses 95 because it includes all the printable ASCII characters.
But incremental mode is more than that. You can generate your own symbol sets, and associated incremental modes. This is useful for example if you already have a number of words, using a certain set of symbols, and you presume that yet-unknown passwords will be using roughly the same character set. Instead of running an expensive full charset incremental mode, you can generate your charset file based on those known characters, and run john in incremental mode with that more restricted character set.
External mode
In addition to three modes mentioned above, john can run in a so-called external mode, where you define your own functions (using a C-like language), that john uses to initialize and resume sessions, and to generate and filter words. Basically, with external mode, john becomes a mere executor of the user-supplied code. There are some good examples of external mode in the default configuration file /etc/john/john.conf. External mode can be used on its own, or in addition to some other standard mode. This page provides more information on external mode.
Miscellaneous
Under Ubuntu, john installs a cron job that can be run overnight (when the system is less loaded) to check for weak passwords in the system, and can then automatically send emails to users whose password is found to be weak. These emails can be configured by editing the files /etc/john/john-mail.conf and /etc/john/john-mail.msg. To enable the nightly john cronjob, have a look at the file /etc/cron.d/john.
