PEM Pack

From Crypto++ Wiki
Jump to: navigation, search

The PEM Pack is a partial implementation of message encryption which allows you to read and write PEM encoded keys and parameters, including encrypted private keys. The additional files include support for RSA, DSA, EC, ECDSA keys and Diffie-Hellman parameters. The ZIP contains five additional source files, a script to create test keys using OpenSSL, a C++ program to test reading and writing the keys, and a script to verify the keys written by Crypto++ using OpenSSL.

PEM encrypted private keys use OpenSSL's key derivation algorithm EVP_BytesToKey (OpenSSL documentation at EVP_BytesToKey). The function is PKCS#5 v1.5 compatible for derived material up to and including 16 bytes. If the required key is over 16 bytes (for example, AES-256 needs 32 bytes), then a non-standard extension is engaged. Regardless of the key size, Crypto++ and OpenSSL will interop as expected because both libraries implement the same derivation algorithm. You can find the Crypto++ reimpmentation at OPENSSL_EVP_BytesToKey.

PEM encryption is an old format specified in Privacy Enhancement for Internet Electronic Mail: Part I: Message Encryption and Authentication Procedures. If given a choice, you should prefer a newer standard like PKCS #8.

There is no standard list of PEM objects or names, though a request was made for one on the IETF's PKIX mailing list. The PEM Pack tries to provide support for most of those provided by OpenSSL. You can get OpenSSL's list from the project's pem.h source file (located at <openssl src>/crypto/pem/pem.h).

Note well: this class is not part of the Crypto++ library. You must download the patch below.


To compile the files, simply drop them in your cryptopp folder and then execute Crypto++'s GNUmakefile. The library's makefile will automatically pick them up.

There are five C++ source files in the ZIP to add to Crypto++. The files are:

  • pem.h - the include file for the PEM routines used by applications.
  • pem-com.h - the internal include file for common PEM routines not exposed to the application
  • pem-com.cpp - the source file for the common PEM routines
  • pem-rd.cpp - the source file for the internal PEM load and read routines
  • pem-wr.cpp - the source file for the internal PEM save and write routines

An additional file is provided for testing. The file is pem-test.cxx, and it should be compiled like any other cpp file and linked against the updated library.

The source files clean compile with -Wall and -Wextra, and have passed Clang's address and undefined sanitizers.

The default behavior is to validate keys and parameters after reading them in Debug builds (but not Release builds). If you want the keys and parameters validated after reading them, then open pem-com.h and uncomment the define PEM_KEY_OR_PARAMETER_VALIDATION. Once compiled, changing PEM_KEY_OR_PARAMETER_VALIDATION has no effect.

Public API

The public API primarily consists of PEM_Load and PEM_Save routines. The read and write routines are overloaded to accept a BufferedTransformation and a public or private key type. The supported systems are RSA, DSA, EC (both ECP and EC2N) and Diffie-Hellman.

There are two helper routines to extract a PEM object from a stream, PEM_NextObject, and identify the PEM object, PEM_GetType.


Each system has three read functions. For example, the routines to read RSA keys:

void PEM_Load(BufferedTransformation& bt, RSA::PublicKey& rsa);
void PEM_Load(BufferedTransformation& bt, RSA::PrivateKey& rsa);
void PEM_Load(BufferedTransformation& bt, RSA::PrivateKey& rsa, const char* password, size_t length);

When reading an encrypted key, you must specify the password and its length in case there's an embedded NULL character in the string. The encapsulated header includes the encryption algorithm, so there's no need to specify it.

In general, reading is deferred to Crypto++ and its various BERDecode* routines. On occasion, the PEM Pack needs to provide a modified routine. For example, for DSA keys, Crypto++ uses {version, x} (where x is the private key) from Certicom's ECC-1 while OpenSSL provides {version, p,q,g,y,x} (where y is the public element):

void PEM_LoadPrivateKey(BufferedTransformation& bt, DSA::PrivateKey& key)
    BERSequenceDecoder seq(bt);
      word32 v;                // check version
      BERDecodeUnsigned<word32>(seq, v, INTEGER, 0, 0);
      Integer p,q,g,y,x;
    key.Initialize(p, q, g, x);

When there's a problem, the PEM pack will throw a Crypto++ exception; and not a custom exception. Be prepared to catch Exception, InvalidArgument and InvalidDataFormat exceptions (in addition to anything Crypto++ might throw, like a DER decode error or bad padding exception).


Each PEM_Load routine has a corresponding write routine, so there are three write functions for each system. For example, in the case of RSA:

void PEM_Save(BufferedTransformation& bt, const RSA::PublicKey& rsa);
void PEM_Save(BufferedTransformation& bt, const RSA::PrivateKey& rsa);
void PEM_Save(BufferedTransformation& bt, const RSA::PrivateKey& rsa, const string& algorithm, const char* password, size_t length);

When writing an encrypted key, you must specify the password, the password's length and the algorithm. The recognized algorithms are listed below.

  • AES-256-CBC
  • AES-192-CBC
  • AES-128-CBC

As with the read routines, the write routines defer to DEREncode* routines but sometimes need to provide an OpenSSL compatible key. For example OpenSSL expects {version, p,q,g,y,x} for DSA, so an override is provided:

void PEM_DEREncode(BufferedTransformation& bt, const DSA::PrivateKey& key)
    const DL_GroupParameters_DSA& params = key.GetGroupParameters();
    DSA::PublicKey pkey;
    DERSequenceEncoder seq(bt);

      DEREncodeUnsigned<word32>(seq, 0);


If you need a new algorithm, then modify PEM_CipherForAlgorithm in both pem-rd.cpp and pem-wr.cpp. Be sure to test the new algorithm against OpenSSL since OpenSSL only provides the algorithms listed above in its various commands (like openssl genrsa). However, OpenSSL should recognize anything EVP_get_cipherbyname understands.

When there's a problem, the PEM pack will throw a Crypto++ exception; and not a custom exception. Be prepared to catch Exception, InvalidArgument and InvalidDataFormat exceptions (in addition to anything Crypto++ might throw).


There's also a function that allows you to read the first key or parameter called PEM_NextObject. The function locates the first PEM object in src and places it in dest. PEM_NextObject essentially peeks into the stream, so the source buffer is unchanged if there is no PEM object present.

void PEM_NextObject(BufferedTransformation& src, BufferedTransformation& dest, bool trimTrailing=true);

PEM_NextObject will silently discard any characters that proceed the PEM Object. The destination BufferedTransformation will have one line ending if it was present in source.

If trimTrailing is true, then trailing whitespace is trimmed from the source BufferedTransformation. This is a convenience function, and its intended to help "pretty print" the source buffer by removing leading whitespace after a key or parameter is extracted (trailing whitespace for the first key becomes leading whitespace for the second key).

Internally, the various PEM_Load functions call PEM_NextObject. That means you can call PEM_NextObject and then PEM_Load; or you can simply call PEM_Load alone (and PEM_Load will call PEM_NextObject).

PEM_NextObject will parse an invalid object. For example, it will parse a key or parameter with -----BEGIN FOO----- and -----END BAR-----. The parser only looks for BEGIN and END (and the dashes). The malformed input will be caught later when a particular key or parameter is loaded.

On failure, InvalidDataFormat is thrown.


The final function attempts to classify a PEM object. The function is PEM_GetType, and its also called internally after PEM_NextObject.

PEM_Type PEM_GetType(const BufferedTransformation& bt);

The function returns one of the following enums:


PEM_GetType peeks at the underlying stream. It does not consume the stream in the BufferedTransformation.

The function only looks for the header (i.e., pre-encapsulated boundary), and does not probe for the footer (i.e., the post-encapsulated boundary). Its possible that PEM_GetType will return a type but later the PEM object will be rejected.

Though PEM_REQ_CERTIFICATE, PEM_CERTIFICATE and PEM_X509_CERTIFICATE are recognized, there's no corresponding PEM_Load or PEM_Save routine.

If an unknown type or bogus PEM object is presented, like -----BEGIN FOO----- and -----END BAR-----, then PEM_UNSUPPORTED is returned.

Sample Code

Using the PEM pack is straight forward. Include pem.h, and then use either PEM_Load or PEM_Save. For example:

#include <cryptopp/pem.h>

// Load a RSA public key
FileSource fs1("rsa-pub.pem", true);
RSA::PublicKey k1;
PEM_Load(fs1, k1);

// Load a encrypted RSA private key
FileSource fs2("rsa-enc-priv.pem", true);
RSA::PrivateKey k2;
PEM_Load(fs2, k2, "test", 4);

// Save an EC public key
DL_PublicKey_EC<ECP> k16 = ...;
FileSink fs16("ec-pub-xxx.pem", true);
PEM_Save(fs16, k16);

// Save an encrypted EC private key
DL_PrivateKey_EC<ECP> k18 = ...;
FileSink fs18("ec-enc-priv-xxx.pem", true);
PEM_Save(fs18, k18, "AES-128-CBC", "test", 4);

Depending on PEM_NO_KEY_OR_PARAMETER_VALIDATION, you may need to validate the key. If you have to validate keys, then the code would looks similar to:

FileSource fs("rsa-pub.pem", true);
RSA::PublicKey key;
PEM_Load(fs, key);

AutoSeededRandomPool prng;
bool result = key.Validate(prng, 2);
    throw std::exception("Failed to validate public key");

Input and Output

Below is an example of the OpenSSL and Crypto++ keys. Keys written by OpenSSL lack "xxx", while keys written by Crypto++ include "xxx" in the filename.


$ cat rsa-pub.pem 
-----END PUBLIC KEY-----

$ cat rsa-enc-priv.pem 
Proc-Type: 4,ENCRYPTED
DEK-Info: AES-128-CBC,CDCA2C5DC5084410C33F2FD6439F910A


And Crypto++:

$ cat ec-pub-xxx.pem 
-----END PUBLIC KEY-----

$ cat ec-enc-priv-xxx.pem 
Proc-Type: 4,ENCRYPTED
DEK-Info: AES-128-CBC,483C6901DE64A75A3CF218BE0A8F3AF6


Testing Keys

You should test the code before you use it. To do so, two scripts and a cpp source file are provided. The scripts depend on OpenSSL and PERL to create a set of PEM encoded test keys, and to verify the keys written by Crypto++ are OK. PERL is used to chop and chomp a good key into one that should cause an exception.

To begin, run to create the RSA, DSA, EC, ECDSA keys and Diffie-Hellman parameters. For each cryptosystem, a public key, a private key and an encrypted private key are created. The test keys are named <system>-<type>.pem. For example, for RSA, the script will create rsa-pub.pem, rsa-priv.pem and rsa-enc-priv.pem. The encrypted keys use a password of "test" (without the quotes).

Next, compile and run pem-test.cxx to exercise reading and writing the keys. For each test key created, the test program will read the key and then write it back out. When the file is written back out, its written out with an "xxx" in its name. For example, rsa-pub-xxx.pem, rsa-priv-xxx.pem and rsa-enc-priv-xxx.pem.

Finally, after the "xxx" files are written by the test program, you should run to ensure OpenSSL can read them.

To summarize the steps:

  1. Compile the library as normal
  2. Compile pem-test.cxx and link against the library
  3. Run to create the OpenSSL keys
  4. Run pem-test.exe to verify the library can read/write the keys
  5. Run to verify the library wrote the keys correctly

Expected Output

The expected output of pem-test.exe is:

$ ./pem-test.exe
Running 0
Running 1
Running 2
Running 3
Running 4
Running 26
Running 27
Running 28
Running 29
Running 30
Running 31
Parsed 153 certificates from cacert.pem
All tests passed

The expected output of is:

$ ./ 
read RSA key
read RSA key
read RSA key
read DSA param
read DSA key
read DSA key
read DSA key
read EC param
read EC key
read EC key
read EC key
Finished testing keys written by Crypto++

Downloads - Additional source files which allow you to read and write PEM encoded keys, including encrypted private keys. The ZIP file includes a script to build test keys with OpenSSL, a small C++ test program to test reading and writing the keys, and a script to verify the keys written by Crypto++ using OpenSSL