Skip to content

Fix RIN inverse for location-scale likelihood parameters - #3306

Open
Cassian433 wants to merge 1 commit into
unit8co:masterfrom
Cassian433:rin-likelihood-params
Open

Cassian433 wants to merge 1 commit into
unit8co:masterfrom
Cassian433:rin-likelihood-params

Conversation

@Cassian433

Copy link
Copy Markdown

Checklist before merging this PR:

  • Mentioned all issues that this PR fixes or addresses.
  • Summarized the updates of this PR under Summary.
  • Added an entry under Unreleased in the Changelog.

Fixes #2743.

Summary

io_processor runs RINorm.inverse on the raw likelihood outputs, before the likelihood applies its softplus. For the location-scale likelihoods that turns the scale channel into raw * std + mean, so the predicted sigma tracks the level of the series instead of its spread. With DLinearModel + GaussianLikelihood + RIN, predict_likelihood_parameters gives sigma 0.62 on the base series, 999.85 when the input is shifted by 1000, and 0.0 when it is multiplied by 100.

This is option 2 from my comment on the issue: undo RIN on the distribution parameters instead of on the raw outputs.

  • TorchLikelihood._rin_parameters names what each output channel is. Gaussian, Laplace, Cauchy and Gumbel are ("loc", "scale"), QuantileRegression is all "loc", everything else is None.
  • "loc" channels keep the existing full inverse. "scale" channels go through the new RINorm.inverse_scale: softplus, times std (divided by |affine_weight + eps**2| when affine), clamped, then the inverse softplus, so the likelihood's own softplus lands on the right value. No shift, since a scale has no affine bias or mean.
  • QuantileRegression and models without a likelihood take the same path as before.
  • With use_reversible_instance_norm=True, any other likelihood (Poisson, Beta, NegativeBinomial, ...) now raises a ValueError at model creation. Poisson is in that list even though you thought it might be fine: its rate comes out as softplus(raw * std + mean), which only holds up while the series stays well above 0. I can let the mean-only likelihoods through if you'd rather. Models saved before this change with one of them still load and predict as before.

Same setup after the change: sigma 0.580 on the base series, 0.580 shifted by 1000, 57.955 multiplied by 100.

Models already trained with RIN and one of the four location-scale likelihoods still load, but the scale they predict changes (a saved DLinear + Gaussian model went from sigma 4.93 to 0.62 with the same weights), so they need retraining. That's why the changelog entry is marked as breaking.

Other Information

#3175 (RIN for covariates) rewrites io_processor too, and its _rin_denormalize_output still inverts the raw likelihood output, so it has the same scale problem. If it lands first I'll move the per-channel split into that function.

If you'd rather have option 1 (raise for every likelihood except QuantileRegression), the model-creation check is already here and I can cut the PR down to it.

Tests: 32 new cases in test_layer_norm_variants.py and test_torch_forecasting_model.py.

  • inverse_scale at extreme scales, float32 and float64, with and without affine, gradients finite
  • shift/scale equivariance of the predicted parameters for the four likelihoods and QuantileRegression
  • the raw-output inverse per likelihood with no affine and with affine weights 2.5 and -0.5, including Poisson, which keeps the old full inverse (that's the path a model saved before this change takes after load())
  • the ValueError for an unsupported likelihood

22 of them fail on master. The 10 that pass there are the QuantileRegression, no-likelihood and Poisson cases, which shouldn't change. A DLinear + Poisson + RIN model saved on master loads and predicts the same values on this branch. test_layer_norm_variants.py, test_torch_forecasting_model.py and darts/tests/utils/test_likelihood_models/: 443 passed. ruff 0.7.2 (the pre-commit pin) check and format are clean.

@Cassian433
Cassian433 force-pushed the rin-likelihood-params branch from 514ab16 to 46bb8bf Compare October 26, 2026 07:25

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] RIN and Likelihood models

1 participant