It’s hard to believe, but it’s been nearly three years since I released my first attempt at a Perl library for generating secure memorable passwords. The original spark of inspiration came when Steve Gibson released and talked about his Password Haystacks page at around the same time as the now famous correct horse battery staple XKCD comic was released. Take the idea of using words as the basis for passwords from XKCD, add computers to introduce real randomness (we humans are terrible at being random), and season with come well-chosen and intuitively placed symbols and digits to increase the size of your haystack, and voila, passwords are are both human-friendly and secure!

The first version of the library worked, as evidenced by it’s years of service powering www.xkpasswd.net. That’s not a bad start. But, it was a first attempt at solving the problem, and, I was still a Perl padawan back then. Some of my early design decisions resulted in a less than ideal API making the library a lot less developer-friendly than it could have been, and I’ve learned a lot about Perl, and Perl best practices since 2011!

I’ve spent the past half year or so re-implementing the same basic idea from scratch. In terms of functionality very little has changed, there are a few additions, but the big change is in the API. Basically, the old API was a mess – you needed one config hashref to instantiate the object, then a different config hashref to call the password generation function. Nonsense! That’s not intuitive, not obvious, and not efficient! The new API allows you to achieve the same result with less code, and the code you will have will be easier to read and understand.

You’ll find the project page for the new library at the link below – this page provides links for downloading the code, and links to the module’s very detailed documentation.

XKPasswd 2 Project Page (http://bartb.ie/xkpasswd)

The New Features

Built-In Presets

Originally presets existed only on the www.xkpasswd.net web interface, they were not inherent to the code library. This meant that users of the library had to create their own configurations from scratch to use the library. Not anymore.

The current release of the library comes with seven built-in presets (not including the default settings), and more could well be added in future releases. Today the following presets exist:

APPLEID

A preset for generating passwords that are compatible with Apple’s rather stringent list of requirements, and, that are easy to type on iOS keyboards

NTLM

A preset for generating 14 character passwords for those who have no choice but to use out-dated NTLM authentication – DO NOT USE THIS PRESET UNLESS YOU HAVE NO CHOICE! It’s as good a preset as I could come up with within the confines of NTLM, but 14 characters is much shorter than ideal for word-based password.

SECURITYQ

A preset for creating invented random answers to security questions (a user request).

WEB16

A preset for use on websites that put an upper-limit of 16 characters on passwords.

WEB32

A preset for use on websites that put an upper-limit of 32 characters on passwords.

WIFI

A preset for WPA2 WiFi passwords.

XKCD

A preset loosely based on the original XKCD comic.

A New Case Transform – INVERT

As with previous versions of the library, the transformation (if any) applied to the case of the words that make up the bulk of the password can be controlled via the case_transform config key. All the previously supported case transforms are still supported (all to lower, all to upper, capitalise the first letter of every word, or randomly convert each word to all upper or all lower case). Now there is another option, keep the first letter of every word lower-case, and upper-case the rest of the letters.

An Optional Separate Symbol Dictionary for Separators

If you configured older versions of the library to randomly choose a padding character and randomly choose a separator character they would chose both characters from the same (configurable) alphabet of symbols. By default the new library works in the same way, but, you can optionally specify a second alphabet, and if you do, one alphabet will be used when picking a separator, and the other when choosing the padding. Some symbols look better as a separator, and some as padding. Now you can be sure to always get symbols you like the look of, even when randomly choosing them.

Built-in Entropy (Strength) Checking

Previous versions of the library tried to ensure there was enough entropy by placing somewhat arbitrary limits on the minimum length of the dictionary, minimum size of the words, the minimum number of characters in the symbol alphabet. This was somewhat effective, but utterly un-scientific. The new version of the library does away with these arbitrary limits and replaces them with checks on the actual entropy produced by the given combination of configuration and dictionary file. The entropy is calculated for two scenarios – one where the attackers has no prior knowledge and must try to break your password by brute force, and one where the attacker knows both the dictionary used and the configuration used. If the brute-force entropy is less than that of a random 12 character alpha-numeric password with mixed case and symbols (78bits), or the full-knowledge entropy is less than that of a random 8 character alpha-numeric password with mixed case and symbols (52bits), the library will issue a warning.

Support for Custom Sources of Randomness

By default the library uses Perl’s built-in randomisation functions. This is sufficient for most users, and means the library has no dependencies on non-standard libraries, making it easier to use.

However, if you want to use a different source of randomness, you can. You just implement your own randomisation function (ensuring it matches the argument list and return-type specified in the documentation) and set the config key random_function to a reference to your function.

I’ll be blogging more in this topic shortly.

The New API

The library itself is object oriented, but, for those times you when you only need a single password, a functional wrapper function allows a single password to be generated with a single line of code (not including the loading of the library). The shortest possible XKPasswd script would looks something like (assuming the library has been installed to /usr/local/xkpasswd.pm/):

#!/usr/bin/perl
use lib '/usr/local/xkpasswd.pm/';
use XKPasswd;
print xkpasswd('/usr/local/xkpasswd.pm/sample_dict.txt')."\n";

This will generate one password using the sample dictionary file as the source of words, and the default configuration.

To use a preset, simply change the last line to something like:

print xkpasswd('/usr/local/xkpasswd.pm/sample_dict.txt', 'WEB32')."\n";

Realistically, while at least one of my presets will hopefully be close to what you want, it’s unlikely to be EXACTLY what you want. Have no fear, the new API allows override keys to be specified as a third argument. What this means is that your chosen preset is used as the starting point for the loaded configuration, and all you have to specify is the changes you want. For example, lets say you want to change just the separator in the XKCD preset. By default it’s a -, but you would prefer it to be a space like in the original cartoon, no problem, simply change the last line as follows:

print xkpasswd('/usr/local/xkpasswd.pm/sample_dict.txt', 'XKCD', {separator_character => ' '})."\n";

You can specify as many overrides are you like, so if you also wanted to change the preset to use five words instead of four you would change the last line as follows:

print xkpasswd('/usr/local/xkpasswd.pm/sample_dict.txt', 'XKCD', {separator_character => ' ', num_words => 5})."\n";

The Full Object Oriented Interface

The functional shortcut interface described above is only efficient if your script/program generates exactly one password each time it’s run. If you need more than one password generated by a single execution of your script or program, use the OO interface to get a significant speed boost for each password generated after the first. The reason for this is two-fold, firstly, the dictionary only needs to be processed once (and then cached), and secondly, the entropy checks only need to be performed once.

The constructor for XKPasswd objects takes the same arguments as the xkpasswd() function described above, in other words, all the following will work:

# define the dictionary path as a variable
# Not needed, but will make the examples lines shorter
my $dict = '/usr/local/xkpasswd.pm/sample_dict.txt';

# create an object using the default config
my $xkpasswd = XKPasswd->new($dict);

# -- OR --

# create an object using a preset (XKCD in this case)
my $xkpasswd = XKPasswd->new($dict, 'XKCD');

# -- OR --

# create an object with a preset and overrides
my $xkpasswd = XKPasswd->new($dict, 'XKCD', {separator_character => ' '});

Once you have an instance of an XKPasswd object created you can generate one or more passwords as follows:

# generate one password
my $password = $xkpasswd->password();

# generate many passwords (5 in this case)
my @passwords = $xkpasswd->passwords(5);

Creating Entirely Custom Configurations

The constructor can accept a full config hashref as the second argument rather than a preset name.

You can create your custom configs entirely from scratch, but bear in mind that in order for the config to work it must have a valid value for all required keys, including the more obscure keys like random_function and random_increment, and keys you may well want effectively disabled like character_substitutions. The example below creates an instance with a config created entirely from scratch:

# create an object with a completely custom config
my $config = {
  word_length_min => 4,
  word_length_max => 8,
  num_words => 4,
  separator_character => q{-},
  padding_digits_before => 0,
  padding_digits_after => 0,
  padding_type => 'NONE',
  case_transform => 'RANDOM',
  random_function => \&XKPasswd::basic_random_generator,
  random_increment => 'AUTO',
  character_substitutions => {},
};
my $xkpasswd = XKPasswd->new($dict, $config);

Like I say, the above works, but it’s needlessly complex. A better approach is to create a hashref with just the keys you want to give a non-default value to, and then use the default_config() function to assemble a config with all the keys you didn’t specify set to their default values. This technique is illustrated in the example below:

# create a custom config using the default values for each config key as the starting point
my $custom_settings = {
  word_length_min => 4,
  word_length_max => 8,
  num_words => 4,
  separator_character => q{-},
  padding_digits_before => 0,
  padding_digits_after => 0,
  padding_type => 'NONE',
  case_transform => 'RANDOM',
};
my $config = XKPasswd->default_config($custom_settings);
my $xkpasswd = XKPasswd->new($dict, $config);

When building your own config from scratch it’s quite probable that you’ll make a mistake and using an invalid value for one of the keys, or fail to add a key required by another key you’ve set. E.g. symbol_alphabet is not a required key, but, if you set padding_character to RANDOM then you must also set symbol_alphabet to an arrayref containing at least two symbols.

While designing a config you’ll find it useful to test your config before using it. To get the most useful errors, I recommend the following approach:

# make sure the following is at the top of the file with the other use statements
use English qw( -no_match_vars );

# create a custom config
my $custom_settings = {
  word_length_min => 4,
  word_length_max => 8,
  num_words => 4,
  separator_character => 'RANDOM',
  padding_digits_before => 0,
  padding_digits_after => 0,
  padding_type => 'NONE',
  case_transform => 'RANDOM',
};
my $config = XKPasswd->default_config($custom_settings);

# test it
eval{
  XKPasswd->is_valid_config($config, 'do_croak'); # you can use any truthy value for the second argument
  print "The config is VALID\n";
}or do{
  print "The config is NOT valid: $EVAL_ERROR\n";
};

Once you have your config built and a dictionary file to test it against you can use the status() function to get see a whole bunch of statistics about your XKPasswd instance, including the results of the entropy calculations:

# create a custom config
my $custom_settings = {
  word_length_min => 4,
  word_length_max => 8,
  num_words => 4,
  separator_character => 'RANDOM',
  padding_digits_before => 0,
  padding_digits_after => 0,
  padding_type => 'NONE',
  case_transform => 'RANDOM',
};
my $config = XKPasswd->default_config($custom_settings);

# instantiate an object with it
my $xkpasswd = XKPasswd->new($dict, $config);

# print the stats
print $xkpasswd->status();

You’ll get output something like:

*DICTIONARY*
File path: /usr/local/xkpasswd.pm/sample_dict.txt
# words: 1260
# words of valid length: 1195 (95%)

*CONFIG*
case_transform: 'RANDOM'
character_substitutions: {}
num_words: '4'
padding_character: 'RANDOM'
padding_characters_after: '2'
padding_characters_before: '2'
padding_digits_after: '0'
padding_digits_before: '0'
padding_type: 'NONE'
random_function: XKPasswd::basic_random_generator
random_increment: 'AUTO'
separator_character: 'RANDOM'
symbol_alphabet: ['!', '$', '%', '&', '*', '+', '-', ':', '=', '?', '@', '^', '_', '|', '~']
word_length_max: '8'
word_length_min: '4'

*RANDOM NUMBER CACHE*
# in cache: 0

*PASSWORD STATISTICS*
Password length: between 19 & 35
Permutations (brute-force): between 2.29x10^33 & 2.85x10^61 (average 2.56x10^47)
Permutations (given dictionary & config): 6.23x10^25
Entropy (Brute-Force): between 110bits and 204bits (average 157bits)
Entropy (given dictionary & config): 85bits
Passwords Generated: 0

You can access similar statistical data programatically using the stats() function, e.g., the worst-case entropy (assuming an attacker with full knowledge) can be accessed in the following way:

# create a custom config
my $custom_settings = {
  word_length_min => 4,
  word_length_max => 8,
  num_words => 4,
  separator_character => 'RANDOM',
  padding_digits_before => 0,
  padding_digits_after => 0,
  padding_type => 'NONE',
  case_transform => 'RANDOM',
};
my $config = XKPasswd->default_config($custom_settings);

# instantiate an object with it
my $xkpasswd = XKPasswd->new($dict, $config);

# access the entropy
my %stats = $xkpasswd->stats();
print "The worst-case entropy is $stats{password_entropy_seen}bits\n";

See the relevant section of the documentation for a full list of all the information made available via the stats() function.

Dictionary Files

As with all previous versions of the library, the format for dictionary files is one word per line, and lines starting with a # are considered comments and ignored. This means that the standard Unix words file (usually /usr/share/dict/words) can be used by this library as a dictionary file. The library ships with a sample dictionary file consisting of common English words and the names of some countries and major cities.