LLM inference

Understands that running an LLM requires GPU memory and that batching multiple requests improves throughput.

Runs a model with vLLM or llama.cpp; understands TTFT, TPS, batch size, and tensor parallelism basics.

Tunes serving parameters for throughput vs latency; applies continuous batching, quantisation, and KV cache strategies.

Understands speculative decoding, disaggregated prefill/decode, tensor/pipeline parallelism at depth; profiles and optimises a serving stack.

Contributes to inference efficiency research, hardware-aware serving, or LLM compiler backends.