PyTorch Glossary#
Created On: Dec 16, 2025 | Last Updated On: Jan 23, 2026
This glossary provides definitions for terms commonly used in PyTorch documentation.
- ATen#
Short for “A Tensor Library”. The foundational tensor and mathematical operation library on which all else is built.
- Compound Kernel#
Opposed to Device Kernels, Compound kernels are usually device-agnostic and belong to Compound Operations.
- Compound Operation#
A Compound Operation is composed of other Operations. Its Kernel is usually device-agnostic. Normally it doesn’t have its own derivative functions defined. Instead, AutoGrad automatically computes its derivative based on operations it uses.
- Composite Operation#
Same as Compound Operation.
- Custom Operation#
An Operation defined by users, usually a Compound Operation. For example, this tutorial details how to create Custom Operations.
- Device Kernel#
Device-specific Kernel of a Leaf Operation.
- JIT#
Just-In-Time Compilation.
- Kernel#
Implementation of a PyTorch Operation, specifying what should be done when an operation executes.
- Leaf Operation#
An Operation that’s considered a basic operation, as opposed to a Compound Operation. Leaf Operation always has dispatch functions defined, usually has a derivative function defined as well.
- Native Operation#
An Operation that comes natively with PyTorch ATen, for example
aten::matmul.- Non-Leaf Operation#
Same as Compound Operation.
- Operation#
A unit of work. For example, the work of matrix multiplication is an operation called
aten::matmul.- Scripting#
Using
torch.jit.scripton a function to inspect source code and compile it as TorchScript code.- TorchScript#
Deprecated. An interface to the TorchScript JIT compiler and interpreter.
- Tracing#
Using
torch.jit.traceon a function to get an executable that can be optimized using just-in-time compilation.
