Tensara

Tensara is a competitive GPU kernel optimization platform I built with Sarthak Mangla, Soham Jog, and Harmya Bhatt. Users submit CUDA or Triton kernels, and the platform benchmarks them against reference implementations on real GPU hardware.

The Problem

Writing efficient GPU kernels is genuinely hard. Even experienced engineers spend weeks optimizing CUDA code, often without clear benchmarks to guide their decisions. There’s no standardized way to measure kernel performance or compare different implementations against each other.

What We Built

The platform grew to over 1,300 users with 35,000+ total submissions. We processed 8,000+ sample submissions and ran 1,500+ sandbox executions.

Serverless GPU Pipeline

We built a serverless GPU benchmarking pipeline on Modal that handles the full lifecycle:

• Sandboxing: Untrusted CUDA code runs in isolated environments
• NVCC Compilation: Submitted kernels are compiled with the NVIDIA compiler toolchain
• Execution: Kernels run on actual GPU hardware with accurate timing
• Profiling: Measures execution time, memory bandwidth utilization, occupancy, and other metrics that matter for real-world performance

Correctness Checking

Beyond performance, we built correctness-checking wrappers that verify submitted kernels produce the right outputs. This catches subtle bugs that might make a kernel fast but wrong.

Learning Paths

We structured problems as learning paths — starting from basic matrix operations and building up to more complex optimizations. Each problem comes with reference implementations and detailed explanations of the optimization techniques involved.

The Editor

The in-browser editor supports CUDA and Triton with syntax highlighting and Vim keybindings. It also shows the compiled PTX alongside your source code with line mappings, so you can see exactly what instructions the compiler generates.

If you’re interested in GPU programming or kernel optimization, check out tensara.org or follow @tensarahq for updates.