-
Notifications
You must be signed in to change notification settings - Fork 3
/
Copy pathNEWS
206 lines (179 loc) · 8.42 KB
/
NEWS
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
v0.9.5
======
- Added jupyter notebooks with a guided tour through TATi.
- Added Dockerfile for putting TATi in docker container.
- Added analysis capability for ensemble averaging.
- FIX: SGLD formula in userguide was incorrect.
- Added GradientDescent optimizer with Barzilai-Borwein learning rate picker.
- Accumulation of norms of gradients, noise, momentum, ... is made optional
and can be fully switched off for performance reasons.
- Optimizers have been moved into distinct module/folder.
- several smaller fixes preventing the distributed tarball from compiling.
- small runtime improvements.
- FIX: time_per_nth_step column in run info file/dataframe was wrong when
using HMC.
v0.9.4
======
- Simulation has get_losses() and get_activations() with list of valid names.
- TrajectoryData's dataframes' columns now all have correct dtype.
- TATiLossFunctionSampler may resample trajectory in subspace.
- Refactored TATiLossFunctionSampler extensively, introduced SamplingModes.
- TATi can now be installed as PyPI (wheel) package, i.e. pip install tati.
- smaller fixes to userguide.
- FIX: Rewite of Model class broke check for present nn in Simulation.
v0.9.3
======
- DOCU: Improved and update userguide.
- DOCU: Code documentation now follows Google Style (no longer ReST) and has
API documentation.
- Introduced a general test threshold to account for numerical inaccuracy of
parallel reduction on GPU-assisted hardware.
- Fully refactored model class: ModelState, MultiLayerPerceptron,
InputPipelineFactory.
- Configure now checks presence of required python packages.
- added full copyright notes, added code check tests on this.
- extracted grid-based sampling from TATiLossFunctionSampler.
- TATiAnalyser can perform covariance and Integrated Autocorrelation Time (IAT)
analysis.
- Refactored TATiAnalyser into several operation modes such that these are
easy-to-use and accessible from Python interfaces.
v0.9.2
======
- added fully tested Hamiltonian Monte Carlo method with first and second
order time integrator (Euler and Leapfrog), following [Neal, 2011].
- added Ensemble Quasi Newton scheme for all samplers, tested on simple
Gaussian mixture model and MNIST single-layer perceptron.
- for checking virial theorem, average moment of inertia is written to averages
file.
- allowing "0" in option hidden_dimension.
- trajectory can be written w.r.t to subspace spanned by vectors in new option
directions_file.
- summaries now write memory and cputime usage for debugging bottlenecks.
- DOCU: re-added accidentally dropped reference section on simulation module.
- FIX: TATiOptimizer always writes last step to files.
- FIX: update of parameters caused no update of simulation's evaluation cache.
- FIX: time_per_nth_step was using process_time instead of time, i.e.
accumulated time over all processes.
- FIX: Boolean parameters were not used from cmd-line.
- FIX: Assigning weights from dataframes took ages. Now works also for multiple
walkers.
- FIX: parse_parameters_file was not working in TATi.simulation.
- tested on TF version up to 1.10.
v0.9.1
======
- removed a section not meeting quality standards in the userguide.
v0.9
====
- added simulation module, an easy-to-use python interface to loss manifold
sampling for neural networks.
- large rewrite of userguide, now in asciidoc.
- added programmer's guide
- added roadmap.
- option types are checked in python interface.
- tensorflow up to 1.8 supported.
- improved input pipeline (and thereby overall) performance.
- several smaller fixes.
- HMC is removed temporarily till being fully validated.
v0.8
====
- introducing replicated neural networks to allow for multiple walkers that
proceed in parallel on individual trajectories with the ability to exchange
information, e.g. for Ensemble Quasi Newton method.
- Updated package dependencies and funding notes in README.
- Added python interface that allows to use neural network as a general
function depending parameters and with a gradient.
- FIX: TATiExplorer could still experience dead-locks.
- Docbook now also supports non-standard fop and xsltproc installation paths.
- FIX: scipy.sparse's linalg module was not loaded correctly for certain scipy
versions.
- added Covariance Controlled Adaptive Langevin (CCAdL) as sampler, untested.
- added option burnin to drop initial set of steps from accumulated averages
- added option progress to display a progress bar with time estimate
- added option summaries_path to write summaries for TensorBoard on demand
- FIX: accuracy was not calculated correctly for multi class classification.
- added testsuite section on tensorflow (non-)capabilities.
v0.7
====
- renamed from DataDrivenSampler (DDS) to Thermodynamic Analytics Toolkit
(TATi)
- added (vectorized) hessian and gradient nodes to allow easy access through
numpy arrays
- sampler, optimizer, lossfunctionsampler, and inputspacesampler may now parse
parameters from a given CSV file through a single cmd-line option
- Explorer can now run parallel processes each sampling or training along a
independent trajectory
- FIX: Sampler module's names were inconsistent
- FIX: sqlite3 presence check was broken
v0.6
====
- supporting now up to tensorflow 1.6
- added DDSExplorer for exploring loss landscapes, picking minima long the way
- prints replaced by logging expressions and verbose cmdline statment supported
- FIX: rejection_rate in HMC fixed
- tensorflow computations can now be done with a given basetype.
- returned to default tf.float32 as tensorflow basetype. tf.float64 seems to
be broken to some extent suggested from sampler's convergence plots
- FIX: SGLD was not resetting aggregated values in run info
- LossFunctionSampler can now fix partial set of parameters and to values
obtained from (minima, trajectory) file
- FIX: Picking input columns (e.g. "sin(x1)") was broken to some extent
- LossFunctionSampler and InputSpaceSampler can now interpret CSV files of
arbitrary type (they pick out the columns they need)
- updated userguide significantly
- some fixes to changed dependent python packages related to Ubuntu 16.04
v0.5
====
- may read TFRecords (as well as CSV) files
- added example for MNIST optimization
- input pipelines now depend on tf.Dataset framework. Either in-memory for
smaller datasets or file
- priors have been added for BAOAB and HMC
- version now always gives a git hash (of the commit)
- Python API can feed its own in-memory dataset for sampling or optimization
v0.4
====
- this is a maintenance release to prepare for tensorflow 1.4. Due to a f***up
it remains untested on tensorflow 1.3
- made robust for use with tensorflow 1.4 (from tensorflow 1.3.1)
- fixes to HamiltonianMonteCarlo (rejection_rate, non-linear scaling)
- using control_dependencies
- introducing and using NumericalDiff (replacing exact diff in tests)
v0.3
====
- added HamiltonianMonteCarlo sampler
- added BAOAB sampler
- parameters can be fixed in optimization or sampling (also loss function
sampling)
- FIX: trajectories are now correctly written also for networks with hidden
layers
- diffusion map analysis can now use python package pydiffmap (and is
recommended due to optimal epsilon choice)
- all cmdline examples in userguide are also fully tested
- added input space sampler to see classification boundaries of network
- we no longer generate datasets in memory but parse from CSV files
- added DatasetWriter to produce CSV for old in-memory datasets
v0.2
====
- enhanced userguide with fully tested python examples
- using double floating point precision by default
- allowed external setting of inter and intra ops thread number for
parallelizaton
- distributable tarball is working (make dist), target distcheck not yet
- added measuring of execution times (init, train, overall)
- FIX: batch_size went missing
- FIX: Dataset was not shuffled properly and split into test/train inconsistent
- added loss manifold sampler
- added Python interface
- added userguide
- added saving and restoring of neural network model to and from file.
- added trajectory analyser for average parameters and diffusion map analysis
based on the contributed code by Zofia Trstanova
- added sampling of average kinetic energy and configurational temperature,
accurate by accumulating in every step (not just "every_nth" step)
- added autotools testsuite
- added SGLD, Geometric Langevin Algorithm 1st and 2nd order sampling
v0.1
====
- gradient descent, SGD, and SGLD optimization
- datasets confined to ones from TensorFlow "playground"
- trajectory analyser