Frontier Engineering
All company guides

AI infrastructure

NVIDIA

The GPUs, interconnect and CUDA software stack that most production AI actually runs on.

Overview

NVIDIA sells the substrate other companies' AI runs on: silicon and interconnect at the bottom, then CUDA, cuDNN, NCCL and CUTLASS, then libraries and runtimes such as TensorRT, TensorRT-LLM, NeMo and RAPIDS, and packaged inference services above those. Almost all of it is publicly documented, and a large share of the software is developed in the open under the NVIDIA GitHub organisation, so you can read the code you would be working next to before you ever talk to anyone.

Because the stack is that tall, the roles are unusually varied. Public listings run from kernel, compiler and driver work through library and framework engineering, up to deep learning performance analysis, model work in NeMo, and customer-facing solution architecture. What holds across the whole band is that performance is the product. An answer that ignores memory bandwidth, occupancy, kernel launch overhead or collective communication cost has missed the point of the question.

Compatibility carries more weight here than at a research lab. CUDA is a long-standing contract with an entire ecosystem, and engineers are expected to reason about what breaking an interface would cost the people downstream of it.

What NVIDIA visibly values

Inferred from public material, not from a leaked rubric. Treat these as the themes worth emphasising rather than as a scoring sheet.

  • Measured performance work — profiling with Nsight Systems or Nsight Compute and showing a before-and-after number, rather than an intuition about what ought to be faster.
  • Fluency in the memory hierarchy: registers, shared memory, L2, HBM, and the ability to say which one a given kernel is actually bound by.
  • Multi-GPU and multi-node reasoning — collective communication patterns, sharding strategies, and where the interconnect rather than the compute becomes the limit.
  • Numerical care: mixed precision, FP8 and BF16 tradeoffs, and knowing when a speedup has quietly cost accuracy.
  • Backwards-compatibility instincts — API and ABI stability, deprecation paths, and the migration cost a breaking change imposes on the ecosystem.
  • Public artefacts: kernels, benchmarks, patches to CUDA-adjacent open source, or writeups that show the work instead of describing it.

Roles this kind of team hires

Linked titles have a full question bank with model answers. Unlinked ones are titles this employer uses that do not map onto one of our role guides.

How to prepare

01

Profile something real, then fix it

Take a model you already run, profile it, find the bottleneck that the profile actually shows, and change it. Being able to narrate one such investigation end to end with numbers is worth more than any amount of CUDA trivia.

02

Learn the memory hierarchy properly

Be able to explain without hedging why a kernel is memory-bound rather than compute-bound, what its arithmetic intensity is, and which change to tiling, data layout or fusion would move that number.

03

Read the part of the stack you are applying to

Pick one repository in the NVIDIA GitHub organisation close to your target role — CUTLASS, TensorRT-LLM, NeMo — and read enough of it to hold an opinion about a design decision inside. Interviewers notice when someone has opened the code.

04

Get concrete about inference economics

Know how batching, KV cache size, quantisation and tensor parallelism trade against one another, and reason out loud in tokens per second per GPU and cost per million tokens rather than in adjectives.

05

Decide which band you are interviewing for

Roles here stretch from drivers to solution architecture. Prepare genuine depth in one layer while still being able to say sensible things about the layer immediately above and below it.

What to watch for

Common mistakes with this kind of employer

  • Treating the GPU as an abstraction. If you have only ever called .cuda() and never looked at a profile, that becomes obvious quickly in this company's questions.
  • Offering a framework answer where a systems answer was asked for. "I would use FSDP" is a starting point, not a design; the interesting part is why that sharding fits the memory and bandwidth budget.
  • Pricing a breaking change at zero. Proposing to change an interface without accounting for downstream migration reads as inexperience at a company whose value is that its interfaces hold still.
  • Spending all your preparation on generic algorithm drills and none on hardware-adjacent reasoning, which is the part that distinguishes candidates here.

Sources

This guide is written from primary public material — NVIDIA's own engineering writing, research pages, published job descriptions, and public talks. Nothing here is taken from a competitor's interview guide, and no specific claim about the loop is made without a source that supports it.

Last reviewed . Interview processes change without notice and vary by team, level, and location. NVIDIA's own careers page is the authoritative description of how it hires; where this guide and that page disagree, that page is right.

Rehearse it before it counts.

Knowing what a company values is not the same as demonstrating it under time pressure. Book a 30-minute 1:1 and run the round you are least confident about.

Book a mock interview