Crypto for the engineer who is not a cryptographer

The do-not-roll-your-own list

Start with the rule that fixes more bugs than any other: you should not implement crypto primitives. Use a library that does it for you. The famous version of the rule is Schneier's "anyone can invent a security system they can't break themselves"; the everyday version is that the field is full of subtle mistakes (timing side channels, edge cases in big-integer arithmetic, off-by-one in padding) that take years of expertise to even notice.

What you should never implement yourself:

A block cipher. Use AES from your platform library, full stop.

A hash function. Use SHA-256 / SHA-3 / BLAKE2 / BLAKE3 from your platform library. Never invent your own.

Public-key cryptography. Use Curve25519 / Ed25519 / P-256 from your platform's library; never implement RSA, ECDSA, or Diffie-Hellman yourself.

A protocol. Use TLS for transport security; use the Noise framework if you need a non-TLS handshake; never invent your own handshake.

What you should not even use a library for — let a higher-level tool handle it:

Padding modes (PKCS#7, OAEP). Almost everywhere this comes up, the right answer is to use AEAD (which handles padding internally) or HKDF (which avoids the question). The cases where you need raw padding are vanishingly rare.

Raw RSA. If you find yourself doing pow(m, e, n), you are probably introducing a bug. Use a signing or encryption API that wraps RSA-PSS or RSA-OAEP correctly.

Random number generation. Use the OS's CSPRNG (/dev/urandom, BCryptGenRandom, SecureRandom, crypto.randomBytes). Never use rand(), Math.random(), or anything seeded from time.

AEAD — the default for "encrypt this"

Authenticated Encryption with Associated Data combines two operations into one primitive. It encrypts (so an observer cannot read the plaintext) and it authenticates (so an observer cannot modify the ciphertext without detection). The standard AEAD constructions in 2026: AES-GCM (hardware-accelerated on every modern CPU), ChaCha20-Poly1305 (faster in software when AES instructions are not available), and AES-GCM-SIV (the misuse-resistant variant where nonce reuse is less catastrophic).

The interface in libsodium is one line per side:

# Python with PyNaCl
from nacl.secret import SecretBox
from nacl.utils import random

key = random(SecretBox.KEY_SIZE)            # 32 bytes from CSPRNG
box = SecretBox(key)

ciphertext = box.encrypt(b"hello world")     # nonce is generated automatically
                                             # and prepended to ciphertext
plaintext  = box.decrypt(ciphertext)         # raises CryptoError on tamper

AEAD is the right answer for almost every problem framed as "I need to encrypt this data". File encryption, cookie encryption (instead of signed cookies, when you also need confidentiality), database column encryption, message body encryption. The associated data argument lets you bind the ciphertext to context (a user id, a database row id, a version number) so the same ciphertext cannot be replayed in a different context.

What makes AEAD better than the older "encrypt-then-MAC" pattern of CBC-mode AES with a separate HMAC: the authentication is part of the primitive, you cannot accidentally forget it, and the libraries that ship it handle the nonce-generation problem for you. The 2010s parade of padding-oracle attacks against CBC-MAC constructions is largely a memory because AEAD took over.

Nonces — what they are and why getting them wrong is fatal

A nonce is a "number used once". For AES-GCM and ChaCha20-Poly1305, the nonce is an input to the encryption alongside the key and the plaintext. Two distinct messages encrypted with the same key must use distinct nonces; if you reuse a nonce, the security collapses — an attacker who sees two ciphertexts with the same nonce can recover the XOR of the plaintexts, and worse, can forge messages.

There are three ways to generate nonces correctly:

Random. Generate from the OS CSPRNG, big enough that collision is astronomically unlikely. For AES-GCM the nonce is 96 bits; you can encrypt roughly 2^32 messages with the same key before birthday-bound collisions become a concern (around four billion). Standard ChaCha20-Poly1305 (RFC 8439) uses a 96-bit nonce; the XChaCha20-Poly1305 variant uses a 192-bit nonce, and collisions are not a practical concern at any usage volume.

Counter. Each message gets the next integer. Requires the encryptor to have persistent state across messages, which is fine for in-memory work but bad for multi-process or restart-recovery scenarios.

Misuse-resistant primitive. AES-GCM-SIV is designed so that nonce reuse degrades to "the same plaintext encrypts to the same ciphertext" — recognisable but not breakable. The misuse-resistance comes at a small performance cost. Worth it whenever you cannot guarantee unique nonces (because the system might fork-and-resume from a snapshot, for instance).

Practical advice: use a library that generates the nonce for you, prepend it to the ciphertext, and let the decryption side strip it back off. PyNaCl's SecretBox does this by default. The bug to avoid is writing your own scheme that reuses a "constant" nonce or uses the message id as the nonce in a way that an attacker can replay.

Hashing — two very different jobs share the word

"Hashing" in working code means two completely different things. Conflating them is the source of an embarrassing number of CVEs.

Job 1: integrity / identification. Take some bytes, produce a fixed-size fingerprint such that any change to the input changes the fingerprint and you cannot find two different inputs with the same fingerprint. Use SHA-256, SHA-3, BLAKE2, or BLAKE3. These are designed to be fast — a modern CPU hashes gigabytes per second. Examples: git object ids, content-addressed storage keys, file integrity checks, the input to a digital signature.

Job 2: password storage. Take a password, produce a fixed-size value such that an attacker who steals the database cannot recover the password. Use bcrypt, Argon2id, or scrypt. These are designed to be slow and memory-hard — each hash takes ~100ms and a few megabytes of memory, so an attacker with a GPU farm cannot try billions per second. Use only here; never use these for integrity (they are too slow) or as a general KDF (they are not what you want).

# Job 1: integrity
import hashlib
hashlib.sha256(b"some content").hexdigest()
# fast, deterministic, cryptographically unique

# Job 2: password
from passlib.hash import argon2
argon2.hash("hunter2")
# slow (~100ms), random salt, parameterised to be memory-hard

The historic disaster: using SHA-256 for password storage. A leaked SHA-256-of-password database is plaintext on modern GPUs within hours. The pattern is so common that the OWASP Cheat Sheet has a dedicated section explaining why MD5/SHA-1/SHA-256/SHA-3 are wrong for passwords.

A third related but distinct job: key derivation. Take a low-entropy input (a password, a shared secret, an ECDH output) and produce a high-entropy key for use in symmetric crypto. Use HKDF for high-entropy inputs (TLS keys, ECDH outputs); use Argon2 or scrypt for low-entropy inputs (passwords). HKDF is fast and deterministic given the same inputs; password KDFs are slow on purpose.

HMAC — the most useful three letters

HMAC is a keyed message authentication code. Given a shared secret key and a message, it produces a tag such that anyone with the key can verify the tag but cannot forge it without the key. Constructed from any hash function (HMAC-SHA256 is the common one).

Use cases:

Signing webhook bodies. The webhook sender includes X-Signature: hex(hmac_sha256(shared_secret, body)). The receiver computes the same HMAC over the received body and compares. Defeats body-tampering and rules out anyone who does not know the secret.

Signed cookies / tokens. The server stores no session, just signs a payload with HMAC. The client sends the payload and the signature; the server verifies before trusting. The "stateless session" pattern that does not require JWT's complexity.

Capability URLs. URL contains a signature of "user X may do Y until time Z"; receiver verifies before allowing. Reset links, share links, time-limited download URLs.

The critical implementation rule: use constant-time comparison when checking the tag. A naive byte-by-byte comparison that returns early on mismatch leaks the first differing byte through timing — repeated probing recovers the tag one byte at a time. Every crypto library ships a constant_time_eq / compare_digest / crypto.timingSafeEqual function; use it.

# Python — constant-time
import hmac
expected = hmac.new(secret, body, "sha256").digest()
hmac.compare_digest(expected, received_tag)   # constant-time, no early return

# Node.js
const expected = crypto.createHmac("sha256", secret).update(body).digest();
crypto.timingSafeEqual(expected, receivedTag);

Symmetric vs asymmetric — when to reach for each

Symmetric uses one shared secret for both encryption and decryption. Fast (gigabytes/sec), low overhead. Examples: AEAD encryption of data at rest, HMAC for integrity, session-key derivation. The problem symmetric crypto cannot solve on its own: how do two parties agree on the shared secret without already having a shared secret.

Asymmetric (public-key) uses a keypair — a public key everyone can know, a private key only the owner has. Slower (microseconds per operation), but solves the key-agreement problem and the digital-signature problem.

Two main public-key operations in working code:

Signatures. The signer hashes a message and signs the hash with their private key; anyone with the public key can verify. Use Ed25519 (fast, small keys, deterministic) for new code. RSA-PSS is the legacy answer; ECDSA over P-256 is interoperability-heavy. Use cases: code signing, document signing, JWT signing, software update verification, certificate signing.

Key agreement. Two parties each generate a keypair, exchange public keys, and each derives the same shared secret from their private key and the other's public key. Diffie-Hellman over Curve25519 (X25519) is the modern default. Used inside TLS, Signal, Noise, SSH. Application code rarely calls these directly; libraries (libsodium's sealed boxes, age) wrap them.

The hybrid pattern that everything-modern uses: asymmetric crypto agrees on a symmetric key, then the actual data is encrypted symmetrically. TLS does this. So does Signal, Age, libsodium's crypto_box. You get the security properties of asymmetric (key agreement without prior shared secret, signed identity) with the performance of symmetric (the bulk data is encrypted by AES-GCM, not by RSA).

When JWT is the wrong answer

JWT is a great hammer; many problems are not nails. Three places JWT is regularly chosen and almost always wrong:

Long-lived sessions for browser apps. A 30-day JWT in localStorage is worse than a 30-day session cookie. The cookie is HttpOnly so XSS cannot steal it; the JWT in localStorage is readable by any script. The cookie can be revoked instantly; the JWT is good until exp regardless of whether the user "logged out". The cookie is automatically attached by the browser with CSRF protections (SameSite); the JWT has to be attached in code, which engineers forget. Use a session cookie. Use Redis or DB to store the session.

Encrypting sensitive data for the client to hold. "Encrypt this with JWE so we can give it to the browser" is a sign that the data should not be on the browser at all. Store it server-side and give the client an opaque id; the round-trip is small and the data never leaves your control.

Short-lived single-purpose signed values. "Sign this URL so it expires in 15 minutes" does not need JWT. A signed cookie or a URL-embedded HMAC tag is much simpler — fewer moving parts, no algorithm-choice trap, no header to forge. JWT's value is the multi-claim, multi-audience, key-rotation story; for "sign this one URL", you do not need any of that.

The places JWT is the right answer: federated identity (OIDC), API-to-API auth between services that share an issuer's public key, scenarios where claims-based authorization is the model. Those are real and important; the misuse is when JWT becomes the answer to every signing or encryption problem because it is the only crypto vocabulary the team knows.

Encrypted data in databases

Three patterns, with very different security properties:

Disk encryption. The filesystem or block device is encrypted; the database sees plaintext. Protects against an attacker who steals the physical disk; provides zero protection against an attacker who has SQL access. AWS RDS / GCP Cloud SQL "encryption at rest" is this. Useful for compliance, weak for actual threat models.

Column-level application-layer encryption. The application encrypts sensitive fields (SSN, payment cards, health records) before insert and decrypts after read. The database sees ciphertext for those columns and cannot index or query them meaningfully. Protects against database compromise, including by privileged DBAs and by SQL injection that exfiltrates rows.

Searchable / property-preserving encryption. Schemes that let you do equality search (deterministic encryption) or range search (order-preserving encryption) on encrypted data. Each leaks more than you might expect — deterministic encryption reveals which rows have equal values, order-preserving reveals the sort order — and the academic literature is full of attacks. Reserve for narrow use cases (encrypted indexes where you can accept the leak), prefer redesigning the query if possible.

A working column-encryption pattern with AWS KMS (the same idea applies to GCP KMS, Vault Transit, Tink):

# Envelope encryption — DEK encrypts the data, KEK encrypts the DEK
import boto3
kms = boto3.client('kms')

def encrypt_pii(plaintext):
    # generate a one-time data key
    resp = kms.generate_data_key(KeyId='alias/pii', KeySpec='AES_256')
    data_key = resp['Plaintext']              # use, then forget
    encrypted_dek = resp['CiphertextBlob']    # store alongside the row

    # encrypt the payload with the data key (AEAD)
    box = SecretBox(data_key)
    ciphertext = box.encrypt(plaintext)
    return (encrypted_dek, ciphertext)

def decrypt_pii(encrypted_dek, ciphertext):
    data_key = kms.decrypt(CiphertextBlob=encrypted_dek)['Plaintext']
    return SecretBox(data_key).decrypt(ciphertext)

The KMS-issued data key is held in memory only briefly per operation; the KEK (master key) never leaves KMS. Rotating the KEK re-wraps every encrypted-DEK without re-encrypting the payloads. Auditing is through KMS's access log. The pattern is mature, well-supported, and worth understanding even if you do not implement it yourself.

Random numbers — the single most misused primitive

Three classes of random number:

Cryptographically secure (CSPRNG). Output indistinguishable from true random to a computationally bounded attacker. Use for: key generation, nonces, session ids, tokens, salts, anything security-related. Sources: /dev/urandom, BCryptGenRandom, crypto.randomBytes, secrets.token_bytes, SecureRandom.

Statistically random. Output passes statistical tests but is predictable given the seed. Use for: Monte Carlo simulation, shuffles in games, jitter in retry schedules. Examples: random.random(), Math.random(), Mersenne Twister.

"Random" that is not. Time-seeded, PID-seeded, anything that an attacker can replicate by guessing the seed. Never use for security.

The bug pattern: a developer reaches for the statistical random (because it is the one named "random" in the standard library) when they meant the secure one. Session ids generated with Math.random() were a real CVE class in early Node.js apps; a determined attacker could enumerate sessions because the seed space was small.

Rule: if the random number will ever be checked against an attacker-controlled value, use the CSPRNG. If it does not matter whether the attacker can predict it, the statistical one is fine and slightly faster.

Side channels — the bugs you cannot see in the code

The code can be logically correct and still leak the secret through timing, power, caches, or speculative execution. Three side channels that working engineers should know:

Timing on comparisons. Any byte-by-byte comparison of secret material (HMAC tags, password hashes, session ids) leaks information through how long the comparison takes. Always use the library's constant-time comparison. Covered above in the HMAC section.

Cache-timing on symmetric crypto. Software AES implementations leak key bits through cache access patterns. Hardware AES (AES-NI on x86, ARMv8 crypto extensions) runs in constant time and is immune. Modern libraries pick the hardware implementation when available; very old or hand-written software AES is the risk.

Spectre / Meltdown / cache side channels. The 2018 family of speculative-execution attacks lets one process read another's memory through cache footprints. Microcode and OS patches mitigate the worst cases; cloud providers handle the cross-tenant variant; application code rarely needs to think about this directly unless you are running mixed-tenant on shared hardware.

The everyday lesson from side channels: when handling secret material, the library matters more than the algorithm. Use a maintained library that has been audited for side-channel resistance; do not hand-roll the primitive that the library is shipping.

A decision tree

Working choices for common problems:

"I need to encrypt some data." → libsodium SecretBox or AES-GCM with random nonce. Use envelope encryption with KMS if the data is large or shared.

"I need to store passwords." → Argon2id (modern) or bcrypt (compatible). Never SHA-256.

"I need to fingerprint files for caching." → SHA-256 or BLAKE3. Never bcrypt (too slow).

"I need to sign a webhook body." → HMAC-SHA256 with constant-time comparison. Never raw signatures, never JWT for this.

"I need to verify a software download." → SHA-256 hash published over a trusted channel, or a signature with Ed25519 / Sigstore.

"I need to generate a session id / token." → CSPRNG output, base64url-encoded, at least 128 bits of entropy.

"I need to share a secret with a service in another organization." → Hybrid encryption (libsodium sealed_box or age). Their public key, your message, sealed once.

"I need to authenticate users." → Not a crypto question; see the authentication chapter. OIDC + passkeys is the modern answer.

"I need to encrypt a column in my database." → Envelope encryption with KMS. Application encrypts; DB stores ciphertext.

"I need search over encrypted data." → Probably redesign. If you must, consider the leak you are taking on — deterministic vs OPE both leak more than they look like they do.

Further reading

Filippo Valsorda's blog (filippo.io) is the best practical-cryptography writing on the open web; the posts on age, Ed25519, and the JWT alternatives are all worth reading. Soatok's blog (soatok.blog) covers practical-vs-theoretical with depth and humour. Cryptopals (cryptopals.com) is the practical-attacks course every working engineer should run through at least once — six sets of progressively harder exercises that teach why the do-not-roll-your-own rule exists in the bones, not just in the head.

For libraries, the libsodium docs (doc.libsodium.org) are concise and authoritative. Google Tink's documentation explains the misuse-resistant philosophy clearly. The AWS Encryption SDK documentation is the practical reference for envelope-encryption patterns in cloud-native code.

Inside this codex, the threat-modeling chapter covers when crypto is and is not the right answer; the authentication chapter covers the auth-specific crypto (signatures, key agreement, password hashing) in more detail; the TLS chapter in the networking section covers the cryptographic substrate underneath every HTTPS request.