Time Series Forecasting, Explained

Most machine learning problems don’t care about order. Shuffle your spam emails, your cat-vs-dog photos, your loan applications — the model learns the same thing either way. Time series forecasting is the rebel of the family: it cares deeply about order, and if you treat it like every other dataset, it will quietly lie to you and then humiliate you in production.

So let’s talk about what makes predicting the future — sales, traffic, temperatures, that one stock you keep refreshing — genuinely different.

Why time series breaks the usual rules

In a normal dataset, rows are independent. In a time series, each point is suspiciously chummy with the points before it. Today’s temperature is an excellent guess for tomorrow’s. This is autocorrelation: the past leaks into the future on purpose, which is the whole point.

That single fact wrecks two habits you’ve probably internalized. You can’t shuffle the data — the sequence is the signal. And you can’t random-split into train and test sets, because that scatters future points into your training data. Your model would essentially study tomorrow’s answer sheet, ace the exam, and then flunk reality. We’ll fix that in a minute.

The three ingredients

Most series decompose into three parts, and naming them is half the battle:

• Trend — the slow drift. Your blog’s readership creeping up over years.
• Seasonality — repeating cycles. Ice cream sales every summer, web traffic dipping every weekend.
• Noise — the irreducible chaos. The stuff no model deserves to predict.

A good forecaster’s job is to model the trend and seasonality, then resist the urge to chase the noise. (Chasing noise is called overfitting, and it ends in tears.)

Classic methods: old but stubbornly effective

The statisticians got here first, and their tools still hold up.

• Moving average — smooth the bumps by averaging recent values. Crude, but a great sanity baseline.
• Exponential smoothing — like a moving average that respects recency, weighting newer points more heavily. Variants (Holt-Winters) handle trend and seasonality explicitly.
• ARIMA — the workhorse. It combines autoregression (this value depends on past values), differencing (to flatten trends into stationarity), and a moving average of past errors. ARIMA shines on clean, linear patterns and remains hard to beat there.

Don’t sneer at these. Research consistently shows ARIMA winning on simple linear series, with error rates often in the low single-digit percentages — while a fancy neural net struggles to justify its electricity bill.

The ML approach: forecasting as feature engineering

The clever trick to using regular ML models (like XGBoost) on time series is to manufacture the time dependence as features:

• Lag features: the value 1, 7, or 30 steps ago.
• Rolling statistics: the mean or standard deviation over the last N steps.
• Calendar features: day of week, month, is-it-a-holiday.

Suddenly forecasting is just regression, and you get to use the whole gradient-boosting toolbox. Powerful — but it’s exactly where leakage sneaks in, so keep reading.

The modern crowd: Prophet and deep learning

Prophet (from Meta) is the friendly option: an additive model that handles trend, multiple seasonalities, holidays, and missing data with sensible defaults. It’s beloved for business series with strong, messy seasonality.

Deep learning — LSTMs and Transformers — earns its keep on complex, non-linear, multivariate problems where there’s lots of data. On those, it consistently beats the classics. On a quiet weekly sales series? It’s a sledgehammer hunting a thumbtack.

Validation without time travel

Here’s the rule that matters most: never let your model see the future. Split chronologically. Train on the past, test on the most recent slice — and when you cross-validate, expand the window forward, never backward.

from sklearn.model_selection import TimeSeriesSplit

# Each fold trains on the past, validates on the *next* chunk
tscv = TimeSeriesSplit(n_splits=5)
for train_idx, test_idx in tscv.split(series):
    train, test = series[train_idx], series[test_idx]
    # train always ends BEFORE test begins — no peeking

Compare that to a random train_test_split, which happily drops June into your training set and asks you to “predict” May. That’s not forecasting; that’s cheating with extra steps.

The takeaway

When the order of your data carries meaning, change three habits: don’t shuffle, split by time, and start with a dumb baseline (last value, or a moving average) before reaching for anything fancier. Match the tool to the data — ARIMA or exponential smoothing for clean linear series, Prophet for seasonal business data, deep learning only when complexity and volume genuinely demand it. Do that, and your forecasts will earn the one thing every model wants: to still look good after the future actually arrives.