Tensor
A multi-dimensional array of numbers that serves as the fundamental data structure in deep learning frameworks like PyTorch and TensorFlow.
Dimensions
A 0D tensor is a scalar (single number). 1D is a vector. 2D is a matrix. 3D+ is a tensor. An image batch might be a 4D tensor: [batch_size, channels, height, width]. Model weights are typically 2D tensors (matrices).
Why Tensors?
Tensors enable efficient parallel computation on GPUs. All neural network operations (matrix multiplication, convolution, attention) operate on tensors. Deep learning frameworks provide automatic differentiation through tensor operations, enabling backpropagation.
Frameworks
PyTorch and TensorFlow are the two dominant tensor computation frameworks. JAX (Google) is gaining popularity for research. All provide GPU acceleration, automatic differentiation, and rich ecosystems of tools.