Prior to the weight estimation step in Neural Networks, what is standardized to have a mean of zero?

Standardizing the input variables to have a mean of zero is a critical step in the preparation process for Neural Networks. This normalization is important because it ensures that the neural network can learn effectively during the training phase. When input variables are standardized, they are transformed such that each feature contributes equally to the distance calculations in the model.

This process helps in avoiding issues related to features on different scales, which can lead to slower convergence or even failure to converge during training. By centering the input variables around zero, the optimization algorithms used in training, such as gradient descent, can perform more efficiently.

Other components, such as the weight array or output scores, do not require this same level of standardization prior to training as they will be adjusted dynamically as learning proceeds. The response variables, while they could also be standardized depending on the context, typically do not undergo this transformation to align with the assumption of the model regarding the data being processed.

In conclusion, standardizing the input variables is essential for ensuring that the neural network is set up for optimal performance during training, allowing it to learn and generalize more effectively.