GPU Performance Optimization

This repository contains three hands-on labs focused on GPU performance optimization for AI workloads.

The exercises explore how modern AI systems utilize GPU compute resources, memory bandwidth, and CUDA runtime optimizations to maximize inference performance.

---

Repository Structure

01_roofline_analysis.ipynb
02_decode_optimization.ipynb
03_compile_cuda_graphs.ipynb

---

Labs

1. Roofline Analysis

Learn how to analyze GPU kernels using the Roofline model.

Topics:

• Arithmetic Intensity
• FLOPs
• Memory Bandwidth
• Compute Bound
• Memory Bound

---

2. Decode Optimization

Study optimization techniques for LLM decoding.

Topics:

• Token generation
• Decode bottlenecks
• Throughput
• Latency
• KV Cache behavior

---

3. CUDA Graphs & Compile

Reduce CPU overhead during inference using CUDA Graphs and compilation techniques.

Topics:

• CUDA Graphs
• torch.compile
• Kernel launch overhead
• GPU utilization

---

Technologies

• Python
• PyTorch
• CUDA
• NVIDIA GPU
• Roofline Model

---

Learning Objectives

• Analyze GPU performance bottlenecks
• Measure Arithmetic Intensity
• Optimize LLM inference
• Improve GPU utilization
• Understand CUDA execution optimizations

---

Author

Irena Kochtov

Nebius Academy • AI Performance Engineering • GPU Optimization