Building a model that explains the past perfectly

Add enough variables and any historical sequence can be fitted exactly. Explaining what already happened and predicting what has not are different tasks, and the first one gets easier as the second gets harder.

Every dataset contains real signal and random noise. A model with enough freedom fits both, and the noise it has memorised is by definition not present in future data — so the better the historical fit, the worse the forecast tends to be.

The defence is out-of-sample testing: fit on one period, evaluate on another that was never used. It is the only check that distinguishes signal from memorised noise.

And that is the trap. The temptation is to keep adjusting until the out-of-sample period performs well too — at which point it is no longer out of sample, and the test has been quietly destroyed.

What to do with it. Hold back a period of data, look at it exactly once, and accept whatever it says.