cuda
9 free lessons tagged cuda across AI, Programming. Each one is a short sequence of focused steps with narration and a five-question quiz at the end — take them in any order, no signup required.
Writing Fused Kernels Without Writing CUDA
The reason most teams never fuse anything is that CUDA asks you to manage threads, shared memory and synchronisation by hand. Triton moves the unit of programming from a thread to a block and hands the rest to a compiler. This lesson covers what that buys, what it still asks of you, how to decide a kernel is worth writing, and how to be sure it is correct.
Online Softmax: Tiling Across a Reduction
Softmax normalises over a whole row, which appears to require the whole row before anything can be produced, which appears to require the score matrix to exist. This lesson works through the rescaling identity that removes that obstacle, the running max and sum that make it numerically safe, and the backward pass trick of recomputing the matrix from two saved statistics.
Attention Is Memory-Bound, and Nobody Noticed for Five Years
For years attention was optimised by reducing FLOPs, and approximate methods that cut FLOPs kept failing to run faster. The reason is that attention was never compute-bound: it spends its time moving a matrix between GPU memory tiers. This lesson establishes that hierarchy, counts the traffic a standard implementation generates, and shows why an exact algorithm beat every approximation.
Profiling CUDA: Occupancy, Memory Coalescing, and Nsight
A working CUDA kernel is the start, not the finish. How to measure occupancy, spot uncoalesced loads and warp divergence, and read the three numbers in Nsight Compute that actually matter.
Shared Memory Tiling for Matrix Multiplication
Why naive matmul on a GPU is bandwidth-starved, and how tiling with __shared__ memory reduces global memory traffic by a factor of the tile size. The classic optimisation, with the kernel that demonstrates it.
Your First CUDA Kernel: Vector Addition End-to-End
The hello-world of CUDA, done properly. Allocate device memory, copy inputs, launch a kernel, copy results back, free, and check every return code. The full driver + kernel in one runnable file.
CUDA Memory Hierarchy: Global, Shared, Constant, Local, Registers
The five memory spaces a CUDA kernel can see and why they have wildly different speeds. Global vs shared vs constant vs local vs registers, coalesced access, bank conflicts, and a cheat-sheet table you'll actually reference.
CUDA Programming Model: Kernels, Threads, Blocks, and Grids
How CUDA carves a problem into a grid of blocks of threads. Host vs device code, the __global__ qualifier, the launch syntax, and how every thread figures out which slice of data it owns.
Parallel Computing Fundamentals: CPUs, GPUs, Latency vs Throughput
Why GPUs eat CPUs for breakfast on some workloads and choke on others. Serial vs parallel execution, Amdahl's law, the latency-vs-throughput trade-off baked into silicon, and a rule of thumb for when to reach for a GPU.