Forward Pass
The phase of neural network computation where input data flows through the network layer by layer, producing predictions or intermediate representations.
Process
Input enters the first layer. Each layer applies its weights, biases, and activation function to transform the data. The output of one layer becomes the input of the next. The final layer produces the model's prediction.
In Training vs Inference
During training, the forward pass is followed by loss computation and a backward pass (backpropagation). During inference, only the forward pass runs. Inference forward passes can be optimized by removing dropout, using quantization, and caching.