Data Distribution Shift

Data distribution shift occurs when the statistical distribution of the data being fed into a model changes from the distribution used during training. In financial markets, this is common as volatility spikes or trading volumes fluctuate.

If a model is trained on a period of low volatility, it will not be prepared for a high-volatility environment, leading to poor predictions. The model effectively encounters data it does not recognize as valid or typical.

Monitoring for this involves tracking the statistical properties of incoming data streams, such as mean, variance, and correlation structures. When a shift is detected, the model must be adjusted or retrained to account for the new environment.

It is a critical aspect of ensuring model robustness.