[ENH] Implementing the `iTransformer` model in PTFv2. by JATAYU000 · Pull Request #1994 · sktime/pytorch-forecasting

JATAYU000 · 2025-11-28T15:06:40Z

Reference Issues/PRs

What does this implement/fix? Explain your changes.

Have started interfacing iTransformer in PTFv2, from the TSlib repository thuml/iTransformer
Work in progress, would like to have suggestions on it.

What should a reviewer concentrate their feedback on?

Current Implementation compliance with PTFv2

Did you add any tests for the change?

Not yet

Any other comments?

PR checklist

The PR title starts with either [ENH], [MNT], [DOC], or [BUG]. [BUG] - bugfix, [MNT] - CI, test framework, [ENH] - adding or improving code, [DOC] - writing or improving documentation or docstrings.
Added/modified tests
Used pre-commit hooks when committing to ensure that code is compliant with hooks. Install hooks with pre-commit install.
To run hooks independent of commit, execute pre-commit run --all-files

PranavBhatP

Thanks for the PR @JATAYU000 . I've dropped some comments on the PR.

PranavBhatP · 2025-11-30T10:15:36Z

pytorch_forecasting/models/itransformer/submodules.py

Conventionally for v2, all the layers of the models' architecture are present in the layers directory. Many of the layers you are using here can be directly imported from this directory, I see a lot of commonality. I would suggest only adding new layers (if not already present in layers) as a subdirectory - layers/_<layer-type>. sub_modules.py is a v1 convention

I found some changes in some layers and let it be in submodules for now in draft pr, will fix this .

PranavBhatP · 2025-11-30T10:20:40Z

pytorch_forecasting/models/itransformer/_itransformer_v2.py

+            :, :, :N
+        ]  # filter covariates
+
+        if self.use_norm:


self.use_norm is not required in the model code since it will be handled by the D1/D2 layers. Normalization and denomalization need not be handled here. Simply return dec_out.

Oh Thank you for pointing that out.

PranavBhatP · 2025-11-30T10:21:33Z

pytorch_forecasting/models/itransformer/_itransformer_pkg_v2.py

+        }
+
+    @classmethod
+    def get_test_train_params(cls):


Can you add a few more test cases here?

PranavBhatP · 2025-11-30T10:26:25Z

pytorch_forecasting/models/itransformer/_itransformer_v2.py

+    """
+    An implementation of iTransformer model for v2 of pytorch-forecasting.
+
+    Parameters


Docstring for model hyperparameters is missing.

JATAYU000 · 2025-12-01T14:50:22Z

I would suggest only adding new layers (if not already present in layers) as a subdirectory - layers/_

@PranavBhatP The EncoderLayer in layers/_encoders/ requires cross_attention , but iTransformer only needs self_attention.
Should I make cross_attention optional in the existing layer? or create a separate EncoderLayer for iTransformer?

fkiraly · 2025-12-03T12:21:42Z

re layers, I would do as follows:

if exact same layer is available in layers, reuse it
add new layers in layers
if layer with modification is needed, add it as a separate layer
optionally - this PR but also can be later PR - check if multiple similar layers can be "unified" in a single layer with more parameters

PranavBhatP · 2025-12-03T17:42:33Z

optionally - this PR but also can be later PR - check if multiple similar layers can be "unified" in a single layer with more parameters

@fkiraly seems like a nice good first issue?

PranavBhatP · 2026-01-12T13:31:23Z

@JATAYU000 any hurdles with fixing issues in the PR?

JATAYU000 · 2026-01-15T02:40:10Z

@PranavBhatP There are'nt any hurdles, I was busy with some other projects, I needed a review on the model and layer implementation since it updates the TimeXer layer to be a common TSLib layer, Have to add tests will do that.

codecov · 2026-01-21T08:35:36Z

Codecov Report

❌ Patch coverage is 99.10714% with 1 line in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (main@ea75590). Learn more about missing BASE report.

Files with missing lines	Patch %	Lines
...orecasting/models/itransformer/_itransformer_v2.py	98.21%	1 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #1994   +/-   ##
=======================================
  Coverage        ?   86.77%           
=======================================
  Files           ?      168           
  Lines           ?     9817           
  Branches        ?        0           
=======================================
  Hits            ?     8519           
  Misses          ?     1298           
  Partials        ?        0

Flag	Coverage Δ
cpu	`86.77% <99.10%> (?)`
pytest	`86.77% <99.10%> (?)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

phoeenniixx · 2026-02-09T18:37:46Z

pytorch_forecasting/layers/_encoders/_encoder.py

    """
-    Encoder module for the TimeXer model.
+    Encoder module for Tslib models.
    Args:


please change this to numpydoc style docstring

phoeenniixx · 2026-02-09T18:39:00Z

pytorch_forecasting/layers/_encoders/_encoder_layer.py

    """
-    Encoder layer for the TimeXer model.
+    Encoder layer for TsLib models.
    Args:


please change this to numpydoc style docstring

phoeenniixx · 2026-02-09T18:44:06Z

pytorch_forecasting/models/itransformer/_itransformer_pkg_v2.py

+            {},
+            dict(d_model=16, n_heads=2, e_layers=2, d_ff=64),
+            dict(
+                d_model=32,


should we try one param with output_attetion=True as well? to cover all possibilities?

phoeenniixx

Thanks @JATAYU000 ! I think it is almost ready. Just few comments and suggestions.

FYI @agobbifbk, @PranavBhatP

codecov · 2026-02-09T19:54:06Z

Codecov Report

❌ Patch coverage is 99.10714% with 1 line in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (main@ea75590). Learn more about missing BASE report.

Files with missing lines	Patch %	Lines
...orecasting/models/itransformer/_itransformer_v2.py	98.21%	1 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #1994   +/-   ##
=======================================
  Coverage        ?   86.77%           
=======================================
  Files           ?      168           
  Lines           ?     9817           
  Branches        ?        0           
=======================================
  Hits            ?     8519           
  Misses          ?     1298           
  Partials        ?        0

Flag	Coverage Δ
cpu	`86.77% <99.10%> (?)`
pytest	`86.77% <99.10%> (?)`

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

agobbifbk · 2026-02-10T10:03:35Z

This is misleading, timestamp are not processed by this layer :-)

enc_out = self.enc_embedding(x_enc, x_mark_enc)  # covariates (e.g timestamp)

Moreover here I don't see any reference to the crossattention

        self.encoder = Encoder(
            [
                EncoderLayer(
                    self_attention=AttentionLayer(
                        FullAttention(
                            False,
                            self.factor,
                            attention_dropout=self.dropout,
                            output_attention=True,
                        ),
                        self.d_model,
                        self.n_heads,
                    ),
                    d_model=self.d_model,
                    d_ff=self.d_ff,
                    dropout=self.dropout,
                    activation=self.activation,
                    output_attention=True,
                )
                for _ in range(self.e_layers)
            ],
            norm_layer=torch.nn.LayerNorm(self.d_model),
            output_attention=True,
        )
        if self.n_quantiles is not None:
            self.projector = nn.Linear(
                self.d_model, self.prediction_length * self.n_quantiles, bias=True
            )
        else:
            self.projector = nn.Linear(self.d_model, self.prediction_length, bias=True)

the Encoder layer has crossattention None by default

class EncoderLayer(nn.Module):
    def __init__(
        self,
        self_attention,
        cross_attention=None,
        d_model=512,
        d_ff=None,
        dropout=0.1,
        activation="relu",
        output_attention=False,
    ):

How the user can use this? Or I'm missing something?

JATAYU000 · 2026-02-10T15:13:12Z

This is misleading, timestamp are not processed by this layer :-)

The comment was cut, it was supposed to be # covariates (e.g timestamp) are embedded as tokens

Moreover here I don't see any reference to the cross_attention

iTransformer does not require to pass cross_attention since it defaults to None.
when it is passed it is included in the block, defaulting it to None was to make the layer flexible to the layers which doesn't need cross_atention as in iTransformer or any other inverse models implemented from TsLib in the future

If there is any other reason to not to default to None? I did not understand the exact problem could you please elaborate a bit more @agobbifbk

JATAYU000 added 3 commits November 28, 2025 20:21

Initial Implementation of iTransformer

302ec0f

Import modules from submodules

abeb908

Quantile preds

4d9d809

PranavBhatP requested changes Nov 30, 2025

View reviewed changes

PranavBhatP reviewed Nov 30, 2025

View reviewed changes

Added Docstrings, removed use_norm

48cb4ed

JATAYU000 requested a review from PranavBhatP December 1, 2025 14:50

Unused imports

171326f

JATAYU000 and others added 3 commits December 11, 2025 19:07

Update Encoder layer and output attention

8eccc2b

Merge branch 'main' into iTransformer

4fbaeea

Merge branch 'main' into iTransformer

06fdd58

JATAYU000 and others added 4 commits January 15, 2026 08:10

Merge branch 'main' into iTransformer

a4c6879

Merge branch 'main' into iTransformer

5d39722

Updated the pkg class and tests

11eee38

reformatted

204c754

JATAYU000 marked this pull request as ready for review January 21, 2026 09:45

JATAYU000 requested review from benHeid, fkiraly, fnhirwa, jdb78, phoeenniixx and yarnabrina as code owners January 21, 2026 09:45

phoeenniixx reviewed Feb 9, 2026

View reviewed changes

phoeenniixx requested changes Feb 9, 2026

View reviewed changes

Merge branch 'main' into iTransformer

ca3c5a8

phoeenniixx added enhancement New feature or request feature request New feature or request module:models labels Feb 9, 2026

phoeenniixx mentioned this pull request Feb 20, 2026

[ENH] Add v2 interface support for iTransformer model #2060

Closed

Conversation

JATAYU000 commented Nov 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Reference Issues/PRs

What does this implement/fix? Explain your changes.

What should a reviewer concentrate their feedback on?

Did you add any tests for the change?

Any other comments?

PR checklist

Uh oh!

PranavBhatP left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

JATAYU000 commented Dec 1, 2025

Uh oh!

fkiraly commented Dec 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

PranavBhatP commented Dec 3, 2025

Uh oh!

PranavBhatP commented Jan 12, 2026

Uh oh!

JATAYU000 commented Jan 15, 2026

Uh oh!

codecov bot commented Jan 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

phoeenniixx left a comment

Choose a reason for hiding this comment

Uh oh!

codecov bot commented Feb 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

agobbifbk commented Feb 10, 2026

Uh oh!

JATAYU000 commented Feb 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

JATAYU000 commented Nov 28, 2025 •

edited

Loading

fkiraly commented Dec 3, 2025 •

edited

Loading

codecov bot commented Jan 21, 2026 •

edited

Loading

codecov bot commented Feb 9, 2026 •

edited

Loading

JATAYU000 commented Feb 10, 2026 •

edited

Loading