AnyLearn
All lessons
Programmingadvanced

Public-Key Cryptography: RSA and ECC

Trapdoor functions power every TLS handshake, code signature, and SSH session. Master RSA key generation and its math, why textbook RSA is trivially broken, what elliptic curves add, and how ECDSA and Ed25519 compare — with key-size equivalences you can quote in code reviews.

Not signed in — your progress and quiz score won't be saved.
Lesson progress1 / 9

Trapdoor functions — the core idea

A trapdoor one-way function is easy to compute in one direction and infeasible to invert without a secret piece of information (the trapdoor). Every public-key cryptosystem is built on one:

  • RSA: multiplying large primes is fast; factoring the product is infeasible without knowing the factors.
  • Diffie-Hellman / DSA: exponentiation in a cyclic group is fast; the discrete logarithm is infeasible.
  • ECC: scalar multiplication of a point (kPkP) is fast; recovering kk from kPkP (the elliptic curve discrete log problem, ECDLP) is infeasible.

Public key = applying the function (no trapdoor needed). Private key = the trapdoor that enables inversion. Security rests on hardness assumptions — not proven unbreakable, but no polynomial-time classical algorithm is known. Shor's algorithm solves all three on a quantum computer, which is why post-quantum cryptography is a priority.

Full lesson text

All 9 steps on one page — for reading, reference, and search.

Show

1. Trapdoor functions — the core idea

A trapdoor one-way function is easy to compute in one direction and infeasible to invert without a secret piece of information (the trapdoor). Every public-key cryptosystem is built on one:

  • RSA: multiplying large primes is fast; factoring the product is infeasible without knowing the factors.
  • Diffie-Hellman / DSA: exponentiation in a cyclic group is fast; the discrete logarithm is infeasible.
  • ECC: scalar multiplication of a point (kPkP) is fast; recovering kk from kPkP (the elliptic curve discrete log problem, ECDLP) is infeasible.

Public key = applying the function (no trapdoor needed). Private key = the trapdoor that enables inversion. Security rests on hardness assumptions — not proven unbreakable, but no polynomial-time classical algorithm is known. Shor's algorithm solves all three on a quantum computer, which is why post-quantum cryptography is a priority.

2. RSA key generation

RSA key generation:

  1. Choose two large distinct primes p,qp, q (each ~1536 bits for RSA-3072).
  2. Compute modulus n=pqn = pq and Euler's totient ϕ(n)=(p1)(q1)\phi(n) = (p-1)(q-1).
  3. Choose public exponent ee (almost always 65537=216+165537 = 2^{16}+1, prime, few 1-bits for fast exponentiation).
  4. Compute private exponent de1(modϕ(n))d \equiv e^{-1} \pmod{\phi(n)} via extended Euclidean.

Public key = (n,e)(n, e). Private key = (n,d)(n, d) (and usually p,q,dp,dq,qinvp, q, d_p, d_q, q_{inv} for CRT speedup).

Encryption: C=MemodnC = M^e \bmod n. Decryption: M=CdmodnM = C^d \bmod n.

Signing: S=H(M)dmodnS = H(M)^d \bmod n. Verification: check Semodn=H(M)S^e \bmod n = H(M).

Key sizes: RSA-2048 gives ~112-bit security. RSA-3072 gives ~128-bit security. NIST recommends at least RSA-2048 through 2030, and RSA-3072 beyond that.

3. Textbook RSA is broken — you must use padding

"Textbook RSA" (C=MemodnC = M^e \bmod n, M=CdmodnM = C^d \bmod n) is deterministic and multiplicatively homomorphic: E(M1)E(M2)=E(M1M2)modnE(M_1) \cdot E(M_2) = E(M_1 M_2) \bmod n. Both properties are exploitable:

  • Chosen-ciphertext attack: an attacker can submit a related ciphertext C=reCC' = r^e C to an oracle and recover MM from the decryption of CC'.
  • Small-message attacks: if MM is small and e=3e = 3, then Me<nM^e < n and C=M3C = M^3 is just a cube — take the integer cube root directly.
  • Bleichenbacher '98 (PKCS#1 v1.5 oracle): millions of real TLS servers were vulnerable; the attack decrypts ciphertexts with a few million adaptive queries.

Required padding schemes:

  • Encryption: OAEP (Optimal Asymmetric Encryption Padding, PKCS#1 v2.2 / RFC 8017). Adds random seed; ciphertext is semantically secure.
  • Signing: PSS (Probabilistic Signature Scheme). Adds randomness; tightly secure under the RSA assumption.
from cryptography.hazmat.primitives.asymmetric import padding, rsa, hashes

key = rsa.generate_private_key(public_exponent=65537, key_size=3072)
ciphertext = key.public_key().encrypt(plaintext, padding.OAEP(
    mgf=padding.MGF1(hashes.SHA256()), algorithm=hashes.SHA256(), label=None))

4. Elliptic curve basics

An elliptic curve over Fp\mathbb{F}_p (a prime field) is the set of points satisfying:

y2x3+ax+b(modp)y^2 \equiv x^3 + ax + b \pmod{p}

together with a point at infinity O\mathcal{O}. A chord-and-tangent rule defines point addition P+QP + Q — the group law. Scalar multiplication kP=P+P++PkP = P + P + \cdots + P (kk times) is computed efficiently with double-and-add in O(logk)O(\log k) operations.

Standard curves:

  • P-256 (secp256r1): NIST-chosen; used in TLS, FIDO2. 256-bit field, ~128-bit security.
  • secp256k1: Bitcoin/Ethereum curve. Same field size as P-256 but different a,ba, b.
  • Curve25519: Bernstein 2006. Designed for high performance and side-channel resistance; cofactor-4 Montgomery form. Used in X25519 (key exchange) and Ed25519 (signatures).

All three offer roughly 128-bit security — equivalent to RSA-3072, but with much smaller keys and faster operations.

5. ECDSA — elliptic curve signatures

ECDSA signing for message hash e=H(m)e = H(m) with private key dd and base point GG of order nn:

  1. Choose random (or deterministic) k[1,n1]k \in [1, n-1].
  2. Compute R=kGR = kG; let r=Rxmodnr = R_x \bmod n.
  3. Compute s=k1(e+rd)modns = k^{-1}(e + rd) \bmod n.
  4. Signature = (r,s)(r, s).

The security of ECDSA is catastrophically sensitive to the nonce kk:

  • Same kk for two signatures: kk is recoverable from (r,s1,s2,e1,e2)(r, s_1, s_2, e_1, e_2) algebraically; private key dd follows immediately.
  • Biased kk: if kk is only partially random (e.g., top bits always zero), lattice attacks recover dd from ~dozens of signatures.

The Sony PlayStation 3 breach (2010) was caused by using a constant kk in ECDSA — private key extracted from firmware signatures. Use RFC 6979 (deterministic kk derived from hash of key + message) or switch to Ed25519.

6. Ed25519 — safer signatures in practice

Ed25519 (Edwards-curve Digital Signature Algorithm over Curve25519) fixes ECDSA's nonce problem by design:

  • kk is derived deterministically: k=H(private_seedm)k = H(\text{private\_seed} \| m) — no external randomness needed, no nonce-reuse risk.
  • Complete addition formulas — no special-cases that lead to timing attacks.
  • Batch verification: nn signatures can be verified faster than nn individual verifications.
  • Small keys: 32-byte private key, 32-byte public key, 64-byte signature.
from cryptography.hazmat.primitives.asymmetric.ed25519 import Ed25519PrivateKey

private_key = Ed25519PrivateKey.generate()
public_key = private_key.public_key()

signature = private_key.sign(message)
public_key.verify(signature, message)  # raises if invalid

Ed25519 is the default in OpenSSH, Signal, Tor, and WireGuard. For new code, prefer Ed25519 over RSA or ECDSA unless interoperability constraints force otherwise.

7. RSA vs ECC key size security equivalence

Classical security bit levels and corresponding key sizes across algorithms:

flowchart LR
  A["80-bit security (legacy)"] --> B["RSA-1024"]
  A --> C["ECC-160"]
  D["112-bit security (NIST min to 2030)"] --> E["RSA-2048"]
  D --> F["ECC-224 / P-224"]
  G["128-bit security (recommended)"] --> H["RSA-3072"]
  G --> I["ECC-256 / P-256 / Curve25519"]
  J["192-bit security (high value)"] --> K["RSA-7680"]
  J --> L["ECC-384 / P-384"]

8. RSA vs ECC — when to use which

RSA-3072P-256 / Ed25519
Security level~128-bit~128-bit
Key size3072-bit (384 bytes)256-bit (32 bytes)
Signature size384 bytes64 bytes
PerformanceSlow (modular exp)Fast (curve ops)
Side-channel riskHigh (textbook impl)Low (Curve25519)
Quantum threatBroken by ShorBroken by Shor
Library supportUniversalVery broad
Legacy interopExcellentGood (SSH, TLS)

For new systems: use Ed25519 for signatures, X25519 for key exchange, RSA-3072+ only if mandated by a compliance regime (FIPS 140-2 historically; FIPS 186-5 now includes Ed25519). Both RSA and ECC are broken by quantum computers — start tracking NIST PQC standards (ML-KEM, ML-DSA) for post-quantum migration plans.

9. Common pitfalls and API mistakes

Asymmetric crypto is where developers make the most consequential mistakes:

  1. Using PKCS#1 v1.5 padding for encryption (not OAEP): still the default in many libraries. Bleichenbacher-style attacks still work against many TLS 1.2 implementations.
  2. Signing the raw message instead of its hash: RSA/ECDSA math operates on integers; signing long messages directly is either truncated or padded incorrectly. Always hash first (the library does this if you use the high-level sign API).
  3. Generating RSA keys in-process for every request: RSA-3072 keygen takes ~100 ms. Cache keys; generate at startup.
  4. Storing private keys in code or version control: use an HSM, KMS (AWS KMS, GCP KMS), or at minimum environment variables with restricted permissions.
  5. Confusing encryption and signing: they use the same key pair but in opposite directions — never use the encryption key to sign or vice versa. They have different security proofs and padding requirements.

Check your understanding

The lesson ends with a 5-question quiz. Take it in the player above to see your score.

  1. Why is textbook RSA encryption ($C = M^e \bmod n$) insecure, even when $n$ cannot be factored?
    • The public exponent $e = 65537$ is too small, allowing cube-root attacks on any message.
    • RSA encryption is deterministic and multiplicatively homomorphic, enabling chosen-ciphertext and small-message attacks.
    • The modulus $n$ leaks the prime factors through its binary representation.
    • Textbook RSA is only insecure for keys below 2048 bits.
  2. ECDSA was broken in the Sony PS3 breach. What was the root cause?
    • Sony used a 160-bit curve that was too small for the security level required.
    • The nonce $k$ was a constant value, allowing algebraic recovery of the private key from any two signatures.
    • Sony failed to hash the message before signing, leaking private key bits.
    • The curve parameters were backdoored by a government agency.
  3. An RSA-3072 key and a P-256 (secp256r1) key are both at approximately the same security level. What is that level?
    • 80-bit security
    • 112-bit security
    • 128-bit security
    • 256-bit security
  4. Why does Ed25519 avoid the nonce-reuse vulnerability that plagues ECDSA?
    • Ed25519 uses a larger curve with a 512-bit field, making nonce collisions impossible.
    • Ed25519 derives the signing nonce deterministically from the private key and message, requiring no external randomness.
    • Ed25519 signs each message twice and takes the XOR of both nonces to ensure uniqueness.
    • Ed25519 stores previously used nonces in a local database and rejects duplicates.
  5. Which padding scheme should be used for RSA encryption in new code?
    • PKCS#1 v1.5 — it is the universal standard and well-tested.
    • No padding — raw RSA is fastest and the library handles security internally.
    • OAEP with SHA-256 — it adds a random seed, making encryption semantically secure.
    • PSS — it was designed for encryption and provides the tightest security proof.

Related lessons