update posterior sampling method

[JuliaLinhart]

What does this implement/fix? Explain your changes

• Update for the tutorial notebook: use posterior.sample() instead of denisty_estimator_sample().
• Minor changes in the test file are to correct a few details I saw by looking over it, such as the definition of true positive and false positive rates.

Does this close any currently open issues?

no

Any relevant code examples, logs, error output, etc?

Code of cell [5] in the tutorial (see comment below):

# sample calibration data
theta_cal = prior.sample((NUM_CAL,))
x_cal = simulator(theta_cal)
# post_samples_cal = npe.sample((1,), x_cal).reshape(-1, theta_cal.shape[-1]).detach()
post_samples_cal = posterior.sample_batched((1,), x=x_cal)[0]

Any other comments?

• This new way of sampling is much slower: in cell [5], it now takes >1min as opposed to a few seconds (the commented line). This is why I didn’t change it in the test file. Should I anyways?
• Locally, one of the lc2st_nf tests fails. The reason is that the -nf version on this example it is too discriminative, i.e. the false positive rate is not low enough (9%>5%, too many rejections for a “good” estimator). I could solve this problem by either training it on data from the true distribution instead of the estimator or simply not considering lc2st_nf for this test.

Checklist

Put an x in the boxes that apply. You can also fill these out after creating
the PR. If you're unsure about any of them, don't hesitate to ask. We're here to
help! This is simply a reminder of what we are going to look for before merging
your code.

• [x] I have read and understood the contribution
  guidelines
• [x] I have commented my code, particularly in hard-to-understand areas
• [x] I have added tests that prove my fix is effective or that my feature works
• [x] I have reported how long the new tests run and potentially marked them
  with pytest.mark.slow.
• New and existing unit tests pass locally with my changes
• [x] I performed linting and formatting as described in the contribution
  guidelines
• [x] There are no conflicts with the main branch

For the reviewer:

• I have reviewed every file
• All comments have been addressed.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 78.34%. Comparing base (299854e) to head (0c4e60b).
Report is 7 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff             @@
##             main    #1279       +/-   ##
===========================================
- Coverage   89.24%   78.34%   -10.91%     
===========================================
  Files         119      119               
  Lines        8695     8705       +10     
===========================================
- Hits         7760     6820      -940     
- Misses        935     1885      +950     

Flag	Coverage Δ
unittests	78.34% <ø> (-10.91%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

see 33 files with indirect coverage changes

[michaeldeistler]

Awesome, thanks!

Regarding the increased sampling time: for the notebook that's okay. But please keep the fast version in the tests. (the reason it is slow is that posterior.sample_batched() draws 10k samples for every x_o, because some of them might get rejected. We will change this in the future, and the notebook will then also be fast).

Regarding the failing test: Any idea on why this test is failing now (but was running before)? Just different seeding?

[JuliaLinhart]

Thanks for you comment @michaeldeistler !

I removed he commented line to only leave the new sample_batched() method in the notebook. The sampling methods in the tests remain unchanged (sample, not sample_batched).

As for the failing test: I did some tests and saw that the used num_train was too big - the NPE was overfitting, leading to a rejected L-C2ST. I changed it to a smaller num_train and all tests pass now.

[michaeldeistler]

Thanks a lot Julia!