[ENH] Add Basic ARIMA model

[alexbanwell1]

Reference Issues/PRs

Split out of #2828
Basis for #2838

What does this implement/fix? Explain your changes.

Adds basic ARIMA forecaster, which fits phi, theta (and optionally c) parameters, but takes in the model to use e.g. ARIMA(p,d,q) and an option to include a constant term.

I have restructured this for numba purposes. The parameters were found using a bespoke implementation of Nelder Mead, sensibly generalised for any loss function. However, doing this means it cannot be cached, which massively slows it down. I have moved Nelder Mead back into the forecasting module, and have introduced an integer switch dispatch function to facilitate caching.

This currently means some duplication of code in the loss_function and in ARIMA. This is usually the price for writing numba.

1. fit(y) sets the parameters for AR and MA component stores the one ahead forecast in self.forecast_
2. predict(y) applies the model to assumed unseen data, setting the MA component to zero as is standard (but a bit wierd). It returns the one ahead forecast for y.
3. forecast(y) fits on y and returns the one ahead forecast, using the residuals on y but not changing any parameter
4. iterative_forecast(y, prediction_horizon=12) fits on y, sets the residuals as to zero after tail of y exceeded.

from aeon.datasets import load_airplane
from aeon.forecasting import ARIMA
y = load_forecasting()
a = ARIMA(p=2, d=1, q=2)
a.fit(y)
p1 = a.forecast_
p2 = a.forecast(y)
preds = a.iterative_forecast(y, prediction_horizon=12)

[aeon-actions-bot]

Thank you for contributing to aeon

I have added the following labels to this PR based on the title: [ $\color{#FEF1BE}{\textsf{enhancement}}$ ].
I have added the following labels to this PR based on the changes made: [ $\color{#31FCCD}{\textsf{forecasting}}$ ]. Feel free to change these if they do not properly represent the PR.

The Checks tab will show the status of our automated tests. You can click on individual test runs in the tab or "Details" in the panel below to see more information if there is a failure.

If our pre-commit code quality check fails, any trivial fixes will automatically be pushed to your PR unless it is a draft.

Don't hesitate to ask questions on the aeon Slack channel if you have any.

PR CI actions

These checkboxes will add labels to enable/disable CI functionality for this PR. This may not take effect immediately, and a new commit may be required to run the new configuration.

• Run pre-commit checks for all files
• Run mypy typecheck tests
• Run all pytest tests and configurations
• Run all notebook example tests
• Run numba-disabled codecov tests
• Stop automatic pre-commit fixes (always disabled for drafts)
• Disable numba cache loading
• Push an empty commit to re-run CI checks

[TonyBagnall]

good start, some tidying up needed and we should think about utils structure

old

[TonyBagnall]

I have optimised the code for numba, and it is now blazingly fast at fitting.

and longer series

these results are from randomly generated stationary AR models with p 0 to 3 and q 0 to 3. For each size of n, repeat m times:
randomly generate a series, fit arma model, predict one ahead.

m set to 30 for these. I also need to do the same for iterative forecast, but these almost look too good to be true