Skip to content

Multivariate Structural Statespace Components #529

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 9 commits into
base: main
Choose a base branch
from

Conversation

jessegrabowski
Copy link
Member

@jessegrabowski jessegrabowski commented Jun 25, 2025

This PR lifts the requirement that models built with the structural sub-module of PyMC be univariate. It's a chonky PR, so I split it into commits. Most of the files changes are changed by the first commit, which is just reorganization of files. It is safe to ignore that one.

Here are the steps I followed:

  1. The structural module was getting pretty unweildly, so I broke it into a bunch of sub-files. This makes the code easier to find and extend. This is handled in the Reorganize structural model modlue commit
  2. We need tools that can merge different components with potentially different (or overlapping) observed time series. This is handled by the Allow combination of component with different numbers of observed states PR. I am confident this code can be improved.
  3. Each component needs to have new logic implemented to handle the case where there are multiple observed series. Users can optionally pass a list of names to each component as observed_state_names. Every time you add two components together, all the relevant matrices are padded and expanded, and the total observed states are created as a union between the components.

For now, we assume all states in a component follow the same parameterization. It's now also valid to add together the same component twice with different states to work around this (e.g. AutoRegressive(order=1, observed_state_names=['data_1']) + Autoregressive(order=5, observed_state_names=['data_2'])) would be a valid model with 2 observed states, but each has it's own autoregressive dynamics.

When you pass a batch of observed_state_names, e.g. LevelTrend(order=2, observed_state_names=['data_1', 'data_2']), the parameters will all be given a batch dimension, but will otherwise be the same as the base case.

More docs coming, but I tried obsessively document what in there so far.

The logic for extending the components is pretty straight-forward -- mostly copying + block_diag or concat, but there are some corner cases that need attention.

This PR should be seen as a companion to #450. Instead of vectorizing across the computation of a model, we're concatenating models. There will be cases where this is superior -- for example when you want to explicitly model latent interactions between components. But in other cases, this approach will be worse. I am interested in having both.

@AlexAndorra
Copy link
Contributor

AutoRegressive(order=1, observed_state_names=['data_1']) + Autoregressive(order=5, observed_state_names=['data_2'])) would be a valid model with 2 observed states, but each has it's own autoregressive dynamics.

This is cool! I will review ASAP.

Note that #450 is currently blocked by what I think is a pytensor bug

Copy link
Contributor

@AlexAndorra AlexAndorra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is 🔥 @jessegrabowski 🤯
I just left a suggestion for what I think was a typo in the docstring. I'll merge once this is resolved, and then test all of this for our PyData tutorial -- probably this weekend.

Just a quick question: IIUC, now users can also have batched RegressionComponents, correct?

@AlexAndorra AlexAndorra self-requested a review June 28, 2025 22:11
Copy link
Contributor

@AlexAndorra AlexAndorra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is 🔥 @jessegrabowski 🤯
I just left a suggestion for what I think was a typo in the docstring.

Still missing this feature are:

We also need to:

  • Make sure that there are tests that combined LevelTrend + AR + error for two observed variables with no interaction model matches two separate models for each, given the same parameters.
  • Make sure that pytensor ops are used everywhere for building the SS matrices (no numpy/scipy)

@AlexAndorra
Copy link
Contributor

I think I'm done for a first review from you on the Cycle component @jessegrabowski 🍾

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants