CR; I believe this thread by Benn Eifert captures what you are trying to say. Benn left twitter long ago but some of his tweet threads are on Thread Reader.
"but my machine learning model can forecast asset prices!"
stationarity is one of the most important concepts in probability and statistics
the essence of its meaning is that the specific pattern you are trying to understand is constant in a probabilistic sense
technically, its unconditional joint probability distribution does not shift over time.
in blackjack, the rules of the game are known and constant, and based on
the cards seen so far we can know the probabilities of the outcomes of
the next hand.
in finance, the underlying structure of the world is complex, unknown,
and changes over time. there is little in the way of objective, certain
truth. any historical data analysis assuming otherwise under-states the
uncertainty around forward looking forecasts.
textbooks focus on simple and obvious cases, like the fact that the
levels of asset prices (as opposed to changes) are nonstationary. this
is of course true and it is why twitter charlatans are constantly
posting spurious-correlation graphs of levels of two variables over time
but much less academic emphasis is given to unpredictable changes in
data generating processes for returns, volatility, correlation, etc.
oil prices going sharply negative in march of 2020 because of lack of storage capacity is one very simple example
academics do like regime switching models, but in practice these are
much better at fitting historical data than understanding what the true
"regimes" are and helping observe and predict change in real time
when one tries to fit a highly nonlinear model with many implicit or
explicit parameters to a complex dynamic system with a data generating
process that changes over time, the result is a great looking fit in
sample and utterly useless forecasting going forward
most AI/ML techniques were designed for stationary problems with high
signal to noise ratios - eg image processing . there are analogous tasks
in finance they can be great for; automating manual tasks like mapping
identifiers of unknown format comes to mind
but naive prediction of asset returns is not one of them. the tricks
they teach in class (split-sample validation, etc) help on the margin
but do not address the underlying issue that there is very little signal
relative to noise, and signals shift faster than you can learn
there are some very specific cases where this criticism is less true -
high frequency market making for example where there is a massive amount
of data over very short time periods where the underlying dynamics are
reasonably constant and models can adapt quickly
and don't come at me with ADF tests, those can reasonably answer the
question "is it completely crazy to run this regression or look at this
chart" but not the rest
we can teach a computer to play Go because the game's structure doesn't
change... so we can simulate a million games and train a neural network
and the next game is exactly like the first million we trained on
again this is not about "machine learning is not useful", much the
contrary. but you have to think about what kind of problems it is useful
for, how you apply it in a meaningful way, and how to quickly recognize
bs that is being pitched to you
also, I cast this thread in terms of finance, but the same story is
generally true for other complex phenomena where the structure of the
problem is not known or constant
e.g. "we're solving health care by using machine learning algorithms on disease diagnoses" no you're not
My favorite; n.b. GoI.
textbooks focus on simple and obvious cases, like the fact that the
levels of asset prices (as opposed to changes) are nonstationary. this
is of course true and it is why twitter charlatans are constantly
posting spurious-correlation graphs of levels of two variables over time
And to answer agip's question about a soft landing, Harold Macmillan's quote "Events, (my) dear boy, events" came to my mind.
