CUDA basics

Knows GPUs have thousands of parallel cores and that `.to('cuda')` moves tensors there.

Understands CUDA thread hierarchy (block/grid), shared memory, and basic memory transfer patterns.

Writes and profiles simple CUDA kernels in C++; uses CUDA profiler (nsight/nvprof); reduces memory bandwidth bottlenecks.

Implements optimised GEMM/attention kernels; uses Triton for custom operations; analyses roofline model.

Contributes to ML compilers, sparse kernel libraries, or hardware-aware training research.