The Linux Crypto API for user applications

If you run your software on Linux, the Linux Kernel itself can satisfy all your cryptographic needs! In this post we will explore Linux Crypto API for user applications and try to understand its pros and cons

O

OxanaCloudflare

·May 11, 2023

The Linux Crypto API for user applications

The Kernel Crypto API From User Space

Linux's Crypto API has been part of the kernel since October 2002, initially serving internal needs such as IPsec. Applications can also make use of it, however, allowing user-space code to send plaintext or ciphertext to the kernel for cryptographic operations.

Before exchanging any data, the application and kernel need to agree on parameters like the algorithm and key size. All supported algorithms and their constraints are listed in the virtual file /proc/crypto. The kernel often provides multiple implementations of the same algorithm, each with a unique driver name; picking an algorithm by its generic name will select the highest-priority driver.

For AES-CTR, on x86 systems the AES-NI implementation is typically the best performer and has the highest priority. A portable C implementation serves as the fallback. If you use libkcapi's speed tests to compare them, the hardware-accelerated version is typically about twice as fast as the generic one.

Why Move Cryptography to the Kernel?

Keeping cryptographic keys in the kernel, as described in the earlier discussion of the Linux Kernel Key Retention Service, restricts access to raw key material. User-space cryptography—no matter how carefully handled—exposes keys to potential bugs in libraries or mistakes in application code that could leak or log them. Moving crypto operations into the kernel removes that exposure.

A recent upstream patch even allows keys from the Key Retention Service to be used directly with the Crypto API. But the trade-off is that system calls and data copying between user space and kernel space add overhead.

Performance Comparison: Kernel Crypto vs. OpenSSL

To quantify that overhead, we benchmarked AES-CTR-128 encryption using both the Kernel Crypto API and OpenSSL.

AES is a block cipher operating on 128-bit blocks, with key sizes of 128, 192, or 256 bits. CTR is a block cipher mode that uses a counter and a nonce instead of chaining ciphertext blocks. This allows parallelization.

Benchmarking Through the Kernel

Communicating with the kernel Crypto API uses an AF_ALG socket. Data can be transferred with a zero-copy interface that passes file descriptors rather than the data itself, but there is a limit of 16 pages (64KB on 4KB-page systems). We tested with chunks of 63KB. Key and IV sizes are also constrained, with details available in /proc/crypto.

To use an explicit driver, you can set the socket's salg_name to a name like ctr-aes-aesni; alternatively, the generic ctr(aes) name triggers automatic selection of the highest-priority driver. Setup steps were excluded from the benchmark measurements; only the loop sending plaintext with SPLICE_F_MORE and reading ciphertext back was timed.

#define _GNU_SOURCE

#include <stdint.h>
#include <string.h>
#include <stdio.h>

#include <unistd.h>
#include <fcntl.h>
#include <time.h>
#include <sys/random.h>
#include <sys/socket.h>
#include <linux/if_alg.h>

#define PT_LEN (63 * 1024)
#define CT_LEN PT_LEN
#define IV_LEN 16
#define KEY_LEN 16
#define ITER_COUNT 100000

static uint8_t pt[PT_LEN];
static uint8_t ct[CT_LEN];
static uint8_t key[KEY_LEN];
static uint8_t iv[IV_LEN];

static void time_diff(struct timespec *res, const struct timespec *start, const struct timespec *end)
{
    res->tv_sec = end->tv_sec - start->tv_sec;
    res->tv_nsec = end->tv_nsec - start->tv_nsec;
    if (res->tv_nsec < 0) {
        res->tv_sec--;
        res->tv_nsec += 1000000000;
    }
}

int main(void)
{
    // Fill the test data
    getrandom(key, sizeof(key), GRND_NONBLOCK);
    getrandom(iv, sizeof(iv), GRND_NONBLOCK);
    getrandom(pt, sizeof(pt), GRND_NONBLOCK);

    // Set up AF_ALG socket
    int alg_s, aes_ctr;
    struct sockaddr_alg sa = { .salg_family = AF_ALG };
    strcpy(sa.salg_type, "skcipher");
    strcpy(sa.salg_name, "ctr-aes-aesni");

    alg_s = socket(AF_ALG, SOCK_SEQPACKET, 0);
    bind(alg_s, (const struct sockaddr *)&sa, sizeof(sa));
    setsockopt(alg_s, SOL_ALG, ALG_SET_KEY, key, KEY_LEN);
    aes_ctr = accept(alg_s, NULL, NULL);
    close(alg_s);

    // Set up IV
    uint8_t cmsg_buf[CMSG_SPACE(sizeof(uint32_t)) + CMSG_SPACE(sizeof(struct af_alg_iv) + IV_LEN)] = {0};
    struct msghdr msg = {
	.msg_control = cmsg_buf,
	.msg_controllen = sizeof(cmsg_buf)
    };

    struct cmsghdr *cmsg = CMSG_FIRSTHDR(&msg);
    cmsg->cmsg_len = CMSG_LEN(sizeof(uint32_t));
    cmsg->cmsg_level = SOL_ALG;
    cmsg->cmsg_type = ALG_SET_OP;
    *((uint32_t *)CMSG_DATA(cmsg)) = ALG_OP_ENCRYPT;
    
    cmsg = CMSG_NXTHDR(&msg, cmsg);
    cmsg->cmsg_len = CMSG_LEN(sizeof(struct af_alg_iv) + IV_LEN);
    cmsg->cmsg_level = SOL_ALG;
    cmsg->cmsg_type = ALG_SET_IV;
    ((struct af_alg_iv *)CMSG_DATA(cmsg))->ivlen = IV_LEN;
    memcpy(((struct af_alg_iv *)CMSG_DATA(cmsg))->iv, iv, IV_LEN);
    sendmsg(aes_ctr, &msg, 0);

    // Set up pipes for using zero-copying interface
    int pipes[2];
    pipe(pipes);

    struct iovec pt_iov = {
        .iov_base = pt,
        .iov_len = sizeof(pt)
    };

    struct timespec start, end;
    clock_gettime(CLOCK_MONOTONIC, &start);
    
    int i;
    for (i = 0; i < ITER_COUNT; i++) {
        vmsplice(pipes[1], &pt_iov, 1, SPLICE_F_GIFT);
        // SPLICE_F_MORE means more data will be coming
        splice(pipes[0], NULL, aes_ctr, NULL, sizeof(pt), SPLICE_F_MORE);
        read(aes_ctr, ct, sizeof(ct));
    }
    vmsplice(pipes[1], &pt_iov, 1, SPLICE_F_GIFT);
    // A final call without SPLICE_F_MORE
    splice(pipes[0], NULL, aes_ctr, NULL, sizeof(pt), 0);
    read(aes_ctr, ct, sizeof(ct));
    
    clock_gettime(CLOCK_MONOTONIC, &end);

    close(pipes[0]);
    close(pipes[1]);
    close(aes_ctr);

    struct timespec diff;
    time_diff(&diff, &start, &end);
    double tput_krn = ((double)ITER_COUNT * PT_LEN) / (diff.tv_sec + (diff.tv_nsec * 0.000000001 ));
    printf("Kernel: %.02f Mb/s\n", tput_krn / (1024 * 1024));
    
    return 0;
}

$ gcc -o kernel kernel.c
$ ./kernel
Kernel: 2112.49 Mb/s

Benchmarking With OpenSSL

OpenSSL's EVP interface is simpler, and we followed the standard example from its documentation. Here error handling is omitted for brevity, as it is in the kernel version.

#include <time.h>
#include <sys/random.h>
#include <openssl/evp.h>

#define PT_LEN (63 * 1024)
#define CT_LEN PT_LEN
#define IV_LEN 16
#define KEY_LEN 16
#define ITER_COUNT 100000

static uint8_t pt[PT_LEN];
static uint8_t ct[CT_LEN];
static uint8_t key[KEY_LEN];
static uint8_t iv[IV_LEN];

static void time_diff(struct timespec *res, const struct timespec *start, const struct timespec *end)
{
    res->tv_sec = end->tv_sec - start->tv_sec;
    res->tv_nsec = end->tv_nsec - start->tv_nsec;
    if (res->tv_nsec < 0) {
        res->tv_sec--;
        res->tv_nsec += 1000000000;
    }
}

int main(void)
{
    // Fill the test data
    getrandom(key, sizeof(key), GRND_NONBLOCK);
    getrandom(iv, sizeof(iv), GRND_NONBLOCK);
    getrandom(pt, sizeof(pt), GRND_NONBLOCK);

    EVP_CIPHER_CTX *ctx = EVP_CIPHER_CTX_new();
    EVP_EncryptInit_ex(ctx, EVP_aes_128_ctr(), NULL, key, iv);

    int outl = sizeof(ct);
    
    struct timespec start, end;
    clock_gettime(CLOCK_MONOTONIC, &start);

    int i;
    for (i = 0; i < ITER_COUNT; i++) {
        EVP_EncryptUpdate(ctx, ct, &outl, pt, sizeof(pt));
    }
    uint8_t *ct_final = ct + outl;
    outl = sizeof(ct) - outl;
    EVP_EncryptFinal_ex(ctx, ct_final, &outl);

    clock_gettime(CLOCK_MONOTONIC, &end);

    EVP_CIPHER_CTX_free(ctx);

    struct timespec diff;
    time_diff(&diff, &start, &end);
    double tput_ossl = ((double)ITER_COUNT * PT_LEN) / (diff.tv_sec + (diff.tv_nsec * 0.000000001 ));
    printf("OpenSSL: %.02f Mb/s\n", tput_ossl / (1024 * 1024));

    return 0;
}

$ gcc -o openssl openssl.c -lcrypto
$ ./openssl
OpenSSL: 3758.60 Mb/s

What the Numbers Show

Looking at relative rather than absolute numbers, the Kernel Crypto API is roughly half the speed of OpenSSL. To dig into why, we used bpftrace with a probe on the ctr_crypt function during the kernel-side operation.

$ sudo bpftrace -e 'kprobe:ctr_crypt { @start=nsecs; @count+=1; } kretprobe:ctr_crypt /@start!=0/ { @total+=nsecs-@start; }'

The probe output gives a total time the kernel spent actually encrypting. The fraction of time spent in kernel encryption versus the whole operation is about 65%. Accounting for that, the pure kernel encryption throughput would be around 3029 Mb/s, which is roughly 81% of OpenSSL's throughput—notably close. Some additional overhead from bpftrace itself should also temper the comparison.

Conclusion

The Kernel Crypto API offers a real security advantage by keeping key material out of user space, but it comes with a measurable speed cost. The cryptographic operations themselves are comparable to OpenSSL's implementation; the bottleneck lies in the user-space interface—system calls and data movement. Whether to use the kernel API is a choice between that overhead and the security benefits it provides.

O

Written by

Oxana

Cloudflare

Writes for Cloudflare. Part of the Tech Report engineering index.

All articles by Oxana →