Forecasting to Production: Time Series, FastAPI, Docker and Drift Monitoring

A forecast that looks brilliant in a notebook can be worthless the day it goes live. Usually the reason is not a weak model. It is a flattering evaluation, a hidden leak from the future, or a service nobody watches. In this capstone we build a forecaster honestly, and then we ship it.

• Good notebooks can fail live
• Leaks and weak evaluation
• Shipping needs discipline

Last time, in lesson 68, we trained an image classifier and a small transformer language model. Today is capstone eleven and twelve. First a full forecasting project, then an end-to-end deployed machine learning app built on the operations ideas from lesson 63.

• Capstone 11: forecasting
• Capstone 12: deployed app

Here is the plan. We pick a public series, set a baseline, and fit ETS, ARIMA and a gradient-boosted model. We evaluate with rolling origins, add intervals, and check for leakage. Then we wrap the winner in an API, test it, containerize it, monitor it, and finish with a postmortem.

• Model, evaluate, check
• Package, test, ship
• Monitor and reflect

We use the classic monthly airline passengers series, one hundred forty-four observations from 1949 to 1960. It has a rising trend and a seasonal swing that grows with the level. That growing swing is a hint that a multiplicative model, or a log transform, will help.

• 144 monthly observations
• Trend plus seasonality
• Swing grows with level

Time order is sacred. We never shuffle. We hold out the final twelve months as a test set, and everything we tune uses only earlier data. Shuffling a series lets a model peek at tomorrow while predicting yesterday, and that is the first form of leakage.

• Hold out the last 12 months
• Tune only on the past
• Shuffling leaks the future

Now the baseline, which every forecast must beat. The seasonal naive method predicts that each month equals the same month one year earlier. Written simply, the forecast for time t equals the actual value at t minus twelve. It costs nothing, and it is surprisingly hard to beat.

• Forecast = same month last year
• Zero parameters
• The bar to beat

Why insist on a baseline? Because a number like five percent error means nothing alone. If the naive method scores four percent, your fancy model is a loss. Every later score in this video is compared to seasonal naive, and that comparison is exactly what the MASE metric will formalize.

• Scores need context
• Beat the naive first
• Complexity must earn its place

A second quick check is plain naive, which repeats the last observed value. On trending seasonal data it ignores the seasons and does badly. Having both naive and seasonal naive tells us how much of the signal is just seasonality, and how much is left for real models to capture.

• Naive repeats the last value
• Seasonal naive repeats last cycle

Next, exponential smoothing, or ETS, from lesson 46. It tracks a level, a trend and a seasonal pattern, updating each with a smoothing weight between zero and one. Recent observations get exponentially more influence. With multiplicative seasonality, it matches our growing swings nicely.

• Smoothing weights between 0 and 1
• Recent data counts more
• Multiplicative seasonality

Then ARIMA, also from lesson 46. We take the log of the series, difference once to remove trend, and seasonally difference at lag twelve to remove the yearly pattern. The model then describes what is left with autoregressive and moving-average terms. Differencing is why the series becomes stationary enough to model.

• Log, then difference
• Seasonal difference at lag 12
• AR and MA terms on the rest

How do we choose the orders? We compare candidate models using information criteria such as AIC on the training window only, then check residuals. Good residuals look like white noise: no autocorrelation spikes, roughly constant spread. Patterns left in the residuals mean signal the model missed.

• Compare AIC on training data
• Residuals should be white noise
• Spikes mean missed signal

Now a different style: turn forecasting into ordinary supervised learning. For each month, build features from the past: the value one month ago, two months ago, twelve months ago, and a rolling mean of the last three. The target is the value now. Then any tabular model can learn it.

• Lags 1, 2 and 12
• Rolling mean of the past
• Target is the current value

Here is the trap. A rolling mean must use only earlier values. If your window accidentally includes the current month, the feature contains the answer. In code, this means shifting the series by one before rolling. This single shift separates an honest model from a fake one.

• Window must end before t
• Shift by one, then roll
• One line separates honest from fake

We fit a gradient-boosted tree model, as in lesson 42, on these lag features. Trees cannot extrapolate beyond values they have seen, so a rising trend is a problem. A common fix is to predict the ratio to last year, or the log difference, instead of the raw level.

• Trees cannot extrapolate
• Predict ratios or differences
• Transform the target

For multi-step forecasts there are two strategies. Recursive forecasting feeds each prediction back as the next lag, which compounds errors. Direct forecasting trains a separate model for each horizon, using only information available at the forecast origin. Direct costs more models, but avoids error snowballing.

• Recursive feeds back predictions
• Errors compound
• Direct trains one model per horizon

A single train and test split gives one noisy score. Rolling-origin evaluation repeats the test many times. We train on the first ninety-six months and forecast twelve. Then we move the origin forward twelve months, retrain, and forecast again. With one hundred forty-four points, that gives four folds.

• Train on the past, forecast ahead
• Slide the origin forward
• 96, 108, 120, 132 give four folds

Every fold respects time: training data always precedes test data. Averaging errors across folds is far more stable than one split, and the spread across folds shows how reliable the model is. Remember lesson 24: all tuning belongs inside the training portion, and the final twelve months stay untouched until the very end.

• Training always precedes testing
• Mean and spread of fold errors
• Final test touched once

Now metrics. MAPE is the mean absolute percentage error. Take three months with actuals one hundred, one hundred twenty and eighty, and forecasts ninety, one hundred thirty and eighty-eight. Absolute errors are ten, ten and eight. As percentages: ten percent, eight point three percent, and ten percent. The average is about nine point four percent.

• Absolute error divided by actual
• Errors: 10, 10, 8
• MAPE is about 9.4 percent

MAPE has flaws. It explodes when actuals are near zero, it is undefined at zero, and it penalizes over-forecasts and under-forecasts unevenly. So we add MASE, the mean absolute scaled error. It divides your mean absolute error by the mean absolute error of the seasonal naive method on the training data.

• MAPE fails near zero
• MASE scales by the naive error
• Comparable across series

Using the same example, mean absolute error is twenty-eight divided by three, about nine point three three. Suppose seasonal naive scores twelve on training data. Then MASE equals nine point three three over twelve, about zero point seven eight. Below one means we beat the baseline. Above one means we lost to it.

• MAE = 28 / 3 = 9.33
• Naive MAE = 12
• MASE = 0.78, below 1 wins

Run all four candidates across the four folds, and report MASE with its spread. A typical honest outcome is that ETS and ARIMA both beat seasonal naive clearly, while the boosted model is competitive but not always better. If so, say it plainly. Simpler models winning is a legitimate result, not a failure.

• Report mean and spread
• Beat naive across folds
• Simple winners are fine

A point forecast hides uncertainty. Suppose the forecast is one hundred and residual standard deviation is eight. A ninety-five percent interval is one hundred plus or minus one point nine six times eight, which is about fifteen point seven. So the range runs from roughly eighty-four to one hundred sixteen.

• Forecast 100, sigma 8
• 1.96 times 8 is about 15.7
• Range 84 to 116

Intervals should widen with the horizon, because errors accumulate. For a random-walk style model the spread grows with the square root of the number of steps. If one step has sigma eight, four steps ahead is eight times two, which is sixteen. The fan shape of a good forecast chart is this effect.

• Errors accumulate
• Spread grows like square root of h
• Four steps: 8 times 2 is 16

For the boosted model, formulas are not handy, so we use empirical residuals or quantile regression. Collect out-of-sample errors from the rolling folds, and take their fifth and ninety-fifth percentiles as interval bounds. Then check coverage: roughly ninety percent of actual values should fall inside a ninety percent interval. If only seventy percent do, the model is overconfident.

• Use out-of-sample residual percentiles
• Check actual coverage
• Low coverage means overconfidence

Time for a leakage audit. Ask four questions. Do any features use values at or after the forecast origin? Was any scaling or imputation fit on the full series? Were hyperparameters tuned on the test months? Are future-known inputs, like holidays, genuinely known at forecast time? One yes means your score is inflated.

• Features using the future
• Preprocessing fit on everything
• Tuning on the test set

A practical test: break the pipeline on purpose. Shuffle the target within the training window, or delete the last month of data and re-run. If performance barely changes, something upstream is fake. And if a model's error is suspiciously close to zero on a noisy series, suspect leakage before celebrating.

• Shuffle the target and re-run
• Suspiciously perfect means suspect
• Break it on purpose

Now capstone twelve. Take the best forecaster and make it a product. First, package the whole pipeline as one object: feature creation, the model, and the interval residuals saved together with a version tag. Training and serving must run identical code, otherwise you invite training-serving skew, as we discussed in lesson 63.

• One artifact, one version
• Same code for train and serve
• Avoid training-serving skew

Next, a FastAPI endpoint. We define a request schema with pydantic: a history of recent values and a horizon. The response contains forecasts, lower and upper bounds, and the model version. Validation rejects bad input automatically, for example a negative horizon or a history shorter than twenty-four months, with a clear error code.

• POST /forecast
• Pydantic validates input
• Returns bounds and version

Load the model once at startup, not on every request. Loading from disk per call would dominate latency. Also add a health endpoint that returns the model version, so deployment tools and humans can confirm which artifact is live. Small design choices like these decide whether an API is dependable.

• Load the model at startup
• Health endpoint with version
• Avoid per-request loading

Write tests at three levels. Unit tests check feature functions, such as lag alignment and the shift that prevents leakage. Contract tests confirm the schema, and that bad input returns an error. A regression test forecasts a fixed history and asserts the output stays within a tolerance of a stored answer.

• Unit: lag and shift logic
• Contract: schema and errors
• Regression: fixed input and output

Containerize with Docker. The Dockerfile starts from a slim Python image, installs pinned dependencies, copies the artifact and code, and starts the server. Pinning versions makes the build reproducible. Anywhere the container runs, the same libraries, same code and same model answer the same request the same way.

• Slim base image
• Pinned dependencies
• Same behavior everywhere

Add structured logging. For each request, log a request ID, model version, input length, horizon, latency, and a summary of the input such as its mean and last value. Do not log sensitive raw data. These logs are the raw material for both debugging and drift monitoring.

• Request ID and model version
• Latency and input summary
• Never log sensitive raw data

Now demonstrate requests and measure latency. Send, say, one thousand requests with a simple load script and record each response time. Report percentiles, not just the average. If the median is twelve milliseconds but the ninety-fifth percentile is thirty, users experience that slower tail. Those numbers are examples, and yours will differ by hardware.

• Send many requests
• Report p50 and p95
• Tails matter more than means

Monitoring comes next, applying lesson 63. Compare recent input distributions with the training window, using a statistic like the population stability index. A common rule of thumb treats values above zero point two as notable shift. For forecasting, also track real error: once actuals arrive, compute rolling MASE and alert if it climbs above one.

• Compare inputs with training
• PSI above 0.2 is notable
• Alert when rolling MASE exceeds 1

Now the postmortem. Where can this system fail? First, structural breaks. A forecaster trained on years of stable seasonality cannot anticipate a sudden shock, like a pandemic or a policy change. Its intervals were calibrated on calm history, so they will look confidently wrong exactly when it matters most.

• Shocks are not in training data
• Intervals assume calm history
• Confidently wrong when it matters

Second, data problems in production. Missing months, delayed data, unit changes, or a sensor that starts reporting zeros will all produce plausible-looking nonsense. Validation can catch some of it, but not all. Third, feedback loops: if forecasts drive decisions, such as stock levels, those decisions can change the data the model later learns from.

• Gaps, delays, unit changes
• Plausible nonsense output
• Forecasts that change the data

Finally, limits of our evaluation. One series is one story. Four folds on one hundred forty-four months say little about other domains. Our latency tests ran on one machine, and our drift thresholds are rules of thumb. The honest conclusion is a documented model, a known baseline gap, and a plan to retrain and review.

• One series, four folds
• Rules of thumb, not laws
• Document limits and retrain plan

Let us recap. We set a seasonal naive baseline, then fit ETS, ARIMA and a lag-feature boosted model. We judged them with rolling origins, MASE and MAPE, added intervals and checked coverage, and audited for leakage. Then we packaged, served, tested, containerized, logged, monitored and measured the app, and wrote a postmortem.

• Honest forecasting evaluation
• Tested, containerized, monitored API
• Postmortem on failure modes

In the next and final lesson, we wrap up the whole course and build your roadmap beyond it. Thanks for learning with me, and I will see you in lesson seventy.

• Lesson 70: wrap-up and roadmap
• Subscribe and build your own