Intrinsic Dimension, Persistent Homology and Generalization in Neural Networks, NeurIPS 2021.

Tolga Birdal, Aaron Lou, Leonidas Guibas, Umut Simsekli
Stanford University | INRIA

Overview

This package provides computation of ph dimension of neural network trajectories. In particular, computation is done in topology.py. The code to produce the analysis experiments are given in train_analysis.py, and the code to produce the regularization experiments are given in train_reqularize.py.

Requirements

The baseline code requires PyTorch, which can be installed directly through a software package manager like pip or conda. However, the topological PH requirements are a bit more complex.

CPU (non-Differentiable)

The function calculate_ph_dim, which computes topology on CPU and is not differentiable, requires Ripser. This can be installed using

pip install Cython
pip install Ripser

GPU (Differentiable)

The function calculate_ph_dim_gpu, which computes topology on GPU and is differentiable, requires TorchPH. This is more difficult to install (due to various dependencies including C++ version). We recommend take a look at the installation page.

Citation

If you found this code or paper useful, please consider citing:

@article{birdal2021intrinsic,
  title={Intrinsic dimension, persistent homology and generalization in neural networks},
  author={Birdal, Tolga and Lou, Aaron and Guibas, Leonidas J and Simsekli, Umut},
  journal={Advances in neural information processing systems},
  volume={34},
  pages={6776--6789},
  year={2021}
}