Plan for release 0.20

[yebai]

General:

• MAP inference based on l-BFGS
• gradient interface inconsistent! gradient interface inconsistent! #687
• Replacing algorithm.gid with algorithm.s::Selector
  • Replacing algorithm.gid with algorithm.s::Selector #567,
  • Gibbs space assertion is fragile  #172,
• Standardising APIs between compiler and inference methods #634 RFC: decouples Turing compiler and inference methods. #456
  • Evaluating posterior Evaluating the posterior #400
  • Default sampling behaviour Default sampling behaviour #446
  • Maximum a posterior for Turing models Maximum a posterior for Turing models #605
• Refactoring of the Sampler APIs [RFC] Refactoring of the Sampler APIs #723 Design/improve/unifty sample() interface #746
  • Add additional keyword arguments to the sample function Add additional keyword arguments to the sample function #482
  • Clean up IO interface Clean up IO interface #474
  • Better design of storing lp and le Better design of storing lp and le #277
  • Storing statistics on the particle weights #493, The weight filed of Chain is buggy #406
  • Logging in samplers / diagnosis statistics Logging in samplers / diagnosis statistics #599
  • Pass number of parallel chains as an argument Pass number of parallel chains as an argument #582 Add example illustrating how to run multiple chains. #698
  • Introduce MCMCSampler type Introduce MCMCSampler type #602
  • Better adaptation configuration interface Better adaptation configuration interface #491
  • Arguments to pass warmup sample size. Argument to pass warmup sample size. #448
  • NUTS adaptation samples appear in full chain NUTS adapatation samples appear in full chain #635
  • Resume interface Resume interface #566 + Provide a way to set the initial state of HMC Provide a way to set initial state of HMC #571
• Type stable implementation of VarInfo TS 2: introduce TypedVarInfo and fix spl.info[:cache_updated] #742 Improving the speed of single program run #294
• Integration with AdvancedHMC Replace HMC related codes using AHMC #739
• Remove spl = nothing Remove spl = nothing #537 (Fixed by Remove Nothing from the Sampler union type #706)
• Re-organise test folder to improve clarity Re-organise test folder to improve clarity  #711
  • add a test/numerical test folder
• VarInfo performance and model types VarInfo performance and model types #620
  • VarInfo type stability
  • Make alg.space type info of the alg Make alg.space type info of the alg #802
• Completely eliminating Real from the model and pre-allocating Completely eliminating Real from the model and pre-allocating #665
• Benchmarking: add a benchmarks folder, and improve benchmarking Lightweight benchmarking on Github Actions  #822 https://github.com/TuringLang/AdvancedHMC.jl/issues/22#issuecomment-476286583
• AdvancedSMC
  • [Traces]: add a member field to Trace to mark reference particle [Traces]: add a member field to Trace to mark reference particle #396
  • Merge SMC and PG? Merge SMC and PG? #222
  • Merge Trace.vi and vi? Merge Trace.vi and vi? #249
  • Particle Gibbs does not return log probability Particle Gibbs does not return log probability #643
• Compiler and VarInfo related:
  • Refactoring: remove VarInfo.pred? Refactoring: remove VarInfo.pred? #510
  • Compiler-generated aliases Compiler generated aliases #533
  • @varname can be replaced with a function @VarName can be replaced with a function #514
  • observe and assume observe and assume #504
  • Add interface to manipulate the lp_ variable, see Function to add to logp within @model Turing model  #754 .
  • Poor support of using columns in IO #207
  • Unify and improve broadcasting for ~ macro Change broadcasting for ~ macro #476
• Docstrings and unifying error messages Docstrings for assume and observe #828 Unifying error messages. #747
• Lessons learned and open questions to improve documentation Lessons learned and open questions to improve documentation #603
• Fix Vectorisation issues: Assume in vectorisation of HMC bug. #760 Observe vectorisation issue. #761 Updated vectorised operations for Distributions.jl #737

HMC related:

• Introduce a type for Hamiltonian HMC RFC: introduce a type for Hamiltonian (or not) #691
• HMCDA numerical errors HMCDA numerical errors #628
• Cache Cholesky decomposition to improve NUTS efficiency #653
• Diagnosing HMC: Energy Distribution Diagnosing HMC: Energy Distribution #454

Autograd related:

• Distributions are not compatible with AD (ForwardDiff/Flux.Tracker) Distributions not compatible with AD (ForwardDiff/Flux.Tracker) #615
• Make sure all distributions support AD where possible Make sure all distributions support AD where possible #674
  • MvNormal and Tracker MvNormal and Tracker #717
  • AD fails for MixtureModel AD fails for MixtureModel #722

Misc issues:

• Extract symbol and indices from varname string Extract symbol and indices from varname string #721
• Remove Flat distribution Remove Flat distribution  #315
• Replace consume and Tasks with Julia iterator #594