You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/advanced_tutorials/11_diagnostics_simulation_based_calibration.ipynb
+1-1Lines changed: 1 addition & 1 deletion
Original file line number
Diff line number
Diff line change
@@ -10,7 +10,7 @@
10
10
"the estimator should be made subject to several **diagnostic tests**. This needs to be\n",
11
11
"performed before being used for inference given the actual observed data. _Posterior\n",
12
12
"Predictive Checks_ (see [10_diagnostics_posterior_predictive_checks\n",
13
-
"tutorial](http://localhost:8000/tutorials/10_diagnostics_posterior_predictive_checks.html)) provide one way to \"critique\" a trained\n",
13
+
"tutorial](https://sbi.readthedocs.io/en/latest/advanced_tutorials/10_diagnostics_posterior_predictive_checks.html)) provide one way to \"critique\" a trained\n",
14
14
"estimator based on its predictive performance. Another important approach to such\n",
15
15
"diagnostics is simulation-based calibration as developed by [Cook et al,\n",
16
16
"2006](https://www.tandfonline.com/doi/abs/10.1198/106186006X136976) and [Talts et al,\n",
Copy file name to clipboardExpand all lines: docs/advanced_tutorials/13_diagnostics_lc2st.ipynb
+2-2Lines changed: 2 additions & 2 deletions
Original file line number
Diff line number
Diff line change
@@ -9,9 +9,9 @@
9
9
"\n",
10
10
" After a density estimator has been trained with simulated data to obtain a posterior, the estimator should be made subject to several diagnostic tests. This diagnostic should be performed before the posterior is used for inference given the actual observed data. \n",
11
11
"\n",
12
-
"*Posterior Predictive Checks* (see [tutorial 10](https://sbi.readthedocs.io/en/latest/tutorials/10_diagnostics_posterior_predictive_checks.html)) provide one way to \"critique\" a trained estimator via its predictive performance. \n",
12
+
"*Posterior Predictive Checks* (see [this tutorial](https://sbi.readthedocs.io/en/latest/advanced_tutorials/10_diagnostics_posterior_predictive_checks.html)) provide one way to \"critique\" a trained estimator via its predictive performance. \n",
13
13
"\n",
14
-
"Another approach is *Simulation-Based Calibration* (SBC, see [tutorial 11](https://sbi.readthedocs.io/en/latest/tutorials/11_diagnostics_simulation_based_calibration.html)). SBC evaluates whether the estimated posterior is balanced, i.e., neither over-confident nor under-confident. These checks are performed ***in expectation (on average) over the observation space***, i.e. they are performed on a set of $(\\theta,x)$ pairs sampled from the joint distribution over simulator parameters $\\theta$ and corresponding observations $x$. As such, SBC is a ***global validation method*** that can be viewed as a necessary condition (but not sufficient) for a valid inference algorithm: If SBC checks fail, this tells you that your inference is invalid. If SBC checks pass, *this is no guarantee that the posterior estimation is working*.\n",
14
+
"Another approach is *Simulation-Based Calibration* (SBC, see [this tutorial](https://sbi.readthedocs.io/en/latest/advanced_tutorials/11_diagnostics_simulation_based_calibration.html)). SBC evaluates whether the estimated posterior is balanced, i.e., neither over-confident nor under-confident. These checks are performed ***in expectation (on average) over the observation space***, i.e. they are performed on a set of $(\\theta,x)$ pairs sampled from the joint distribution over simulator parameters $\\theta$ and corresponding observations $x$. As such, SBC is a ***global validation method*** that can be viewed as a necessary condition (but not sufficient) for a valid inference algorithm: If SBC checks fail, this tells you that your inference is invalid. If SBC checks pass, *this is no guarantee that the posterior estimation is working*.\n",
15
15
"\n",
16
16
"**Local Classifier Two-Sample Tests** ($\\ell$-C2ST) as developed by [Linhart et al, 2023](https://arxiv.org/abs/2306.03580) present a new ***local validation method*** that allows to evaluate the correctness of the posterior estimator ***at a fixed observation***, i.e. they work on a single $(\\theta,x)$ pair. They provide necessary *and sufficient* conditions for the validity of the SBI algorithm, as well as easy-to-interpret qualitative and quantitative diagnostics. \n",
0 commit comments