🔖 Release 1.4.0 (#599)

## 1.4.0 (2023-04-24)

### Major Updates and Feature Improvements

- Adds Python 3.11 support \[experimental\] #500
  - Python 3.11 is not fully supported by `pytorch` pytorch/pytorch#86566 and `openslide` openslide/openslide-python#188
- Removes Python 3.7 support
  - This allows upgrading all the dependencies which were dependent on an older version of Python.
- Adds Neighbourhood Querying Support To AnnotationStore #540
  - This enables easy and efficient querying of annotations within a neighbourhood of other annotations.
- Adds `MultiTaskSegmentor` engine #424
- Fixes an issue with stain augmentation to apply augmentation to only tissue regions.
  - #546 contributed by @navidstuv
- Filters logger output to stdout instead of stderr.
  - Fixes #255
- Allows import of some modules at higher level for improved usability
  - `WSIReader` can now be imported as `from tiatoolbox.wsicore import WSIReader`
  - `WSIMeta` can now be imported as `from tiatoolbox.wsicore import WSIMeta`
  - `HoVerNet`, `HoVerNetPlus`, `IDaRS`, `MapDe`, `MicroNet`, `NuClick`, `SCCNN` can now be imported as \`from tiatoolbox.models import HoVerNet, HoVerNetPlus, IDaRS, MapDe, MicroNet, NuClick, SCCNN
- Improves `PatchExtractor` performance. Updates `WSIPatchDataset` to be consistent. #571
- Updates documentation for `License` for clarity on source code and model weights license.

### Changes to API

- Updates SCCNN architecture to make it consistent with other models. #544

### Bug Fixes and Other Changes

- Fixes Parsing Missing Omero Version NGFF Metadata #568
  - Fixes #535 raised by @benkamphaus
- Fixes reading of DICOM WSIs at the correct level #564
  - Fixes #529
- Fixes `scipy`, `matplotlib`, `scikit-image` deprecated code
- Fixes breaking changes in `DICOMWSIReader` to make it compatible with latest `wsidicom` version. #539, #580
- Updates `shapely` dependency to version >=2.0.0 and fixes any breaking changes.
- Fixes bug with `DictionaryStore.bquery` and `geometry=None`, i.e. only a where predicate given.
  - Partly Fixes #532 raised by @blaginin
- Fixes local tests for Windows/Linux
- Fixes `flake8`, `deepsource` errors.
- Uses `logger` instead of `warnings` and `print` statements to properly log runs.

### Development related changes

- Upgrades dependencies which are dependent on Python 3.7
- Moves `requirements*.txt` files to `requirements` folder
- Removes `tox`
- Uses `pyproject.toml` for `bdist_wheel`, `pytest` and `isort`
- Adds `joblib` and `numba` as dependencies.

Author: shaneahmed

.deepsource.toml

@@ -7,7 +7,7 @@ exclude_patterns = ["docs/*.py", "**/__init__.py"]
 [[analyzers]]
 name = "python"
 enabled = true
-dependency_file_paths = ["requirements_dev.txt"]
+dependency_file_paths = ["requirements/requirements_dev.txt"]
 
   [analyzers.meta]
   runtime_version = "3.x.x"

.github/workflows/conda-env-create.yml

@@ -4,9 +4,9 @@ name: Solve Conda Environment
 
 on:
   push:
-    paths: [ "requirements*.yml", "conda-env-create.yml", "requirement*.txt"]
+    paths: [ "requirements/requirements*.yml", "conda-env-create.yml", "requirement*.txt"]
   pull_request:
-    paths: [ "requirements*.yml", "conda-env-create.yml", "requirement*.txt"]
+    paths: [ "requirements/requirements*.yml", "conda-env-create.yml", "requirement*.txt"]
   schedule: # Run on the 1st of every month at midnight
     - cron: "0 0 1 * *"
 
@@ -20,24 +20,24 @@ jobs:
         include:
           - os: ubuntu-latest
             kind: prod
-            requirements: requirements.conda.yml
+            requirements: requirements/requirements.conda.yml
           - os: ubuntu-latest
             kind: dev
-            requirements: requirements.dev.conda.yml
+            requirements: requirements/requirements.dev.conda.yml
           - os: windows-latest
             kind: prod
-            requirements: requirements.win64.conda.yml
+            requirements: requirements/requirements.win64.conda.yml
           - os: windows-latest
             kind: dev
-            requirements: requirements.win64.dev.conda.yml
+            requirements: requirements/requirements.win64.dev.conda.yml
     runs-on: ${{ matrix.os }}
     timeout-minutes: 20
     steps:
       - uses: actions/checkout@v2
       - name: Copy requirements files
         shell: bash
         run: |
-          cp ./requirements*.txt /tmp/
+          cp ./requirements/requirements*.txt /tmp/
           mkdir /tmp/docs/
           cp ./docs/requirements*.txt /tmp/docs/
       - uses: mamba-org/provision-with-micromamba@main

.github/workflows/docker-publish.yml

@@ -6,7 +6,7 @@ env:
   REGISTRY: ghcr.io
   IMAGE_NAME: TissueImageAnalytics/tiatoolbox
   image: ghcr.io/tissueimageanalytics/tiatoolbox
-  TOOLBOX_VER: 1.3.3
+  TOOLBOX_VER: 1.4.0
 
 jobs:
   build-and-push-image:
@@ -15,8 +15,6 @@ jobs:
       fail-fast: true
       matrix:
         include:
-          - dockerfile: ./docker/3.7/Debian/Dockerfile
-            mtag: py3.7-debian
           - dockerfile: ./docker/3.8/Debian/Dockerfile
             mtag: py3.8-debian
           - dockerfile: ./docker/3.9/Debian/Dockerfile

.github/workflows/pip-install.yml

@@ -4,15 +4,15 @@ name: pip Install
 
 on:
     push:
-        paths: [ "requirements*.yml", "conda-env-create.yml", "requirement*.txt", "setup*py", "setup*cfg", "pytproject*toml", "MANIFEST*in"]
+        paths: [ "requirements*.yml", "conda-env-create.yml", "requirements/requirement*.txt", "setup*py", "setup*cfg", "pytproject*toml", "MANIFEST*in"]
 
 jobs:
   build:
     runs-on: ${{ matrix.os }}
     strategy:
       fail-fast: true
       matrix:
-        python-version: ["3.7", "3.8", "3.9", "3.10"]
+        python-version: ["3.8", "3.9", "3.10", "3.11"]
         os: [ubuntu-22.04, windows-latest, macos-latest]
     steps:
     - name: Set up Python ${{ matrix.python-version }}

.github/workflows/python-package.yml

@@ -17,7 +17,7 @@ jobs:
     strategy:
       fail-fast: true
       matrix:
-        python-version: ["3.7", "3.8", "3.9", "3.10"]
+        python-version: ["3.8", "3.9", "3.10", "3.11"]
 
     steps:
     - uses: actions/checkout@v3
@@ -31,7 +31,7 @@ jobs:
         sudo apt-get install -y libopenslide-dev openslide-tools libopenjp2-7 libopenjp2-tools
         python -m pip install --upgrade pip
         python -m pip install flake8 pytest pytest-cov pytest-runner
-        pip install -r requirements.txt
+        pip install -r requirements/requirements.txt
     - name: Print Version Information
       run: |
         echo "---SQlite---"
@@ -93,7 +93,7 @@ jobs:
       run: |
         sudo apt-get install -y libopenslide-dev libopenjp2-7
         python -m pip install --upgrade pip
-        pip install -r requirements.txt
+        pip install -r requirements/requirements.txt
         pip install build
     - name: Build package
       run: python -m build

.gitignore

@@ -39,7 +39,6 @@ pip-delete-this-directory.txt
 
 # Unit test / coverage reports
 htmlcov/
-.tox/
 .coverage
 .coverage.*
 .cache

.pre-commit-config.yaml

@@ -1,3 +1,7 @@
+ci:
+  autofix_prs: true
+  autoupdate_commit_msg: '[pre-commit.ci] pre-commit autoupdate'
+  autoupdate_schedule: 'weekly'
 repos:
   - repo: local
     hooks:
@@ -32,7 +36,7 @@ repos:
       - mdformat-gfm  # GitHub flavoured markdown
       - mdformat-black  # Black formatting for python verbatim blocks
   - repo: https://github.com/pre-commit/pre-commit-hooks
-    rev: v4.3.0
+    rev: v4.4.0
     hooks:
       - id: check-ast               # Simply checks whether the files parse as valid Python.
       - id: fix-byte-order-marker   # Removes utf-8 byte order marker.
@@ -54,20 +58,20 @@ repos:
       - id: debug-statements        # Checks for debugger imports and py37+ `breakpoint()` calls in python source.
       - id: pretty-format-json      # Sets a standard for formatting JSON files.
   - repo: https://github.com/pre-commit/pygrep-hooks
-    rev: v1.9.0
+    rev: v1.10.0
     hooks:
       - id: rst-backticks           # Detect common mistake of using single backticks when writing rst.
       - id: rst-directive-colons    # Detect mistake of rst directive not ending with double colon.
       - id: rst-inline-touching-normal # Detect mistake of inline code touching normal text in rst.
   - repo: https://github.com/pycqa/isort
-    rev: 5.10.1
+    rev: 5.12.0
     hooks:
       - id: isort
   - repo: https://github.com/psf/black
-    rev: 22.10.0 # Replace with any tag/version: https://github.com/psf/black/tags
+    rev: 23.3.0 # Replace with any tag/version: https://github.com/psf/black/tags
     hooks:
       - id: black
-        language_version: python3 # Should be a command that runs python3.7+
+        language_version: python3 # Should be a command that runs python3.+
         additional_dependencies: ['click==8.0.4']  # Currently >8.0.4 breaks black
       - id: black-jupyter
         language: python

.readthedocs.yml

@@ -26,5 +26,5 @@ formats:
 # Optionally set the version of Python and requirements required to build your docs
 python:
   install:
-      - requirements: requirements.txt
+      - requirements: requirements/requirements.txt
       - requirements: docs/requirements.txt

CITATION.cff

@@ -44,7 +44,7 @@ authors:
   given-names: "Shan E Ahmed"
   orcid: "https://orcid.org/0000-0002-1097-1738"
 title: "TIAToolbox as an end-to-end library for advanced tissue image analytics"
-version: 1.3.3  # TIAToolbox version
+version: 1.4.0  # TIAToolbox version
 doi: 10.5281/zenodo.5802442
 date-released: 2022-10-20
 url: "https://github.com/TissueImageAnalytics/tiatoolbox"

CONTRIBUTING.rst

@@ -77,13 +77,12 @@ Ready to contribute? Here's how to set up ``tiatoolbox`` for local development.
    Now you can make your changes locally.
 
 5. When you're done making changes, check that your changes pass flake8 and the
-   tests, including testing other Python versions with tox::
+   tests::
 
     $ flake8 tiatoolbox tests
     $ python setup.py test or pytest
-    $ tox
 
-   To get flake8 and tox, just pip install them into your virtualenv.
+   To get flake8, just pip install them into your virtualenv.
 
 6. Commit your changes and push your branch to GitHub::
 
@@ -102,7 +101,7 @@ Before you submit a pull request, check that it meets these guidelines:
 2. If the pull request adds functionality, the docs should be updated. Put
    your new functionality into a function with a docstring, and add the
    feature to the list in README.rst.
-3. The pull request should work for Python 3.7, 3.8, 3.9 and 3.10, and for PyPy. Check
+3. The pull request should work for Python 3.8, 3.9 and 3.10, and for PyPy. Check
    https://travis-ci.com/tialab/tiatoolbox/pull_requests
    and make sure that the tests pass for all supported Python versions.
 

HISTORY.md

@@ -1,5 +1,54 @@
 # History
 
+## 1.4.0 (2023-04-24)
+
+### Major Updates and Feature Improvements
+
+- Adds Python 3.11 support \[experimental\] #500
+  - Python 3.11 is not fully supported by `pytorch` https://github.com/pytorch/pytorch/issues/86566 and `openslide` https://github.com/openslide/openslide-python/pull/188
+- Removes Python 3.7 support
+  - This allows upgrading all the dependencies which were dependent on an older version of Python.
+- Adds Neighbourhood Querying Support To AnnotationStore #540
+  - This enables easy and efficient querying of annotations within a neighbourhood of other annotations.
+- Adds `MultiTaskSegmentor` engine #424
+- Fixes an issue with stain augmentation to apply augmentation to only tissue regions.
+  - #546 contributed by @navidstuv
+- Filters logger output to stdout instead of stderr.
+  - Fixes #255
+- Allows import of some modules at higher level for improved usability
+  - `WSIReader` can now be imported as `from tiatoolbox.wsicore import WSIReader`
+  - `WSIMeta` can now be imported as `from tiatoolbox.wsicore import WSIMeta`
+  - `HoVerNet`, `HoVerNetPlus`, `IDaRS`, `MapDe`, `MicroNet`, `NuClick`, `SCCNN` can now be imported as \`from tiatoolbox.models import HoVerNet, HoVerNetPlus, IDaRS, MapDe, MicroNet, NuClick, SCCNN
+- Improves `PatchExtractor` performance. Updates `WSIPatchDataset` to be consistent. #571
+- Updates documentation for `License` for clarity on source code and model weights license.
+
+### Changes to API
+
+- Updates SCCNN architecture to make it consistent with other models. #544
+
+### Bug Fixes and Other Changes
+
+- Fixes Parsing Missing Omero Version NGFF Metadata #568
+  - Fixes #535 raised by @benkamphaus
+- Fixes reading of DICOM WSIs at the correct level #564
+  - Fixes #529
+- Fixes `scipy`, `matplotlib`, `scikit-image` deprecated code
+- Fixes breaking changes in `DICOMWSIReader` to make it compatible with latest `wsidicom` version. #539, #580
+- Updates `shapely` dependency to version >=2.0.0 and fixes any breaking changes.
+- Fixes bug with `DictionaryStore.bquery` and `geometry=None`, i.e. only a where predicate given.
+  - Partly Fixes #532 raised by @blaginin
+- Fixes local tests for Windows/Linux
+- Fixes `flake8`, `deepsource` errors.
+- Uses `logger` instead of `warnings` and `print` statements to properly log runs.
+
+### Development related changes
+
+- Upgrades dependencies which are dependent on Python 3.7
+- Moves `requirements*.txt` files to `requirements` folder
+- Removes `tox`
+- Uses `pyproject.toml` for `bdist_wheel`, `pytest` and `isort`
+- Adds `joblib` and `numba` as dependencies.
+
 ## 1.3.3 (2023-03-02)
 
 ### Major Updates and Feature Improvements
@@ -18,7 +67,6 @@ None
 
 - Restricts dependency versions for compatibility
 
-
 ## 1.3.2 (2023-02-17)
 
 ### Major Updates and Feature Improvements
@@ -37,7 +85,6 @@ None
 
 - Restricts wsidicom version to \<0.7.0 for compatibility
 
-
 ## 1.3.1 (2022-12-20)
 
 ### Major Updates and Feature Improvements

LICENSE

@@ -1,6 +1,9 @@
 BSD 3-Clause License
 
-Copyright (c) 2022, Tissue Image Analytics (TIA) Centre
+Model weights are provided under a different license. For details, please see
+https://tia-toolbox.readthedocs.io/en/latest/pretrained.html
+
+Copyright (c) 2023, Tissue Image Analytics (TIA) Centre
 All rights reserved.
 
 Redistribution and use in source and binary forms, with or without

MANIFEST.in

@@ -3,7 +3,7 @@ include CONTRIBUTING.rst
 include HISTORY.md
 include LICENSE
 include README.md
-include requirements.txt
+include requirements/requirements.txt
 include make.bat
 include Makefile
 include conf.py

Makefile

@@ -42,7 +42,6 @@ clean-pyc: ## remove Python file artifacts
 	find . -name '__pycache__' -exec rm -fr {} +
 
 clean-test: ## remove test and coverage artifacts
-	rm -fr .tox/
 	rm -f .coverage
 	rm -fr htmlcov/
 	rm -fr .pytest_cache
@@ -62,9 +61,6 @@ lint: ## check style with flake8
 test: ## run tests quickly with the default Python
 	pytest
 
-test-all: ## run tests on every Python version with tox
-	tox
-
 coverage: ## check code coverage quickly with the default Python
 	pytest  --cov=tiatoolbox --cov-report=term --cov-report=html --cov-report=xml
 	$(BROWSER) htmlcov/index.html
@@ -88,3 +84,6 @@ dist: clean ## builds source and wheel package
 
 install: clean ## install the package to the active Python's site-packages
 	python setup.py install
+
+update-notebook-urls: # Recursively update the notebook URLs in ./examples
+	find ./examples -name "*.ipynb" | sort | xargs python pre-commit/notebook_urls.py

README.md

@@ -22,8 +22,8 @@
     <img src="https://readthedocs.org/projects/tia-toolbox/badge/?version=latest" alt="Documentation Status" />
   </a>
   <br>
-  <a href="https://github.com/TissueImageAnalytics/tiatoolbox/blob/develop/LICENSE">
-    <img alt="GitHub license" src="https://img.shields.io/github/license/TissueImageAnalytics/tiatoolbox"></a>
+  <a href="#license">
+    <img alt="GitHub license" src="https://img.shields.io/static/v1?label=license&message=BSD-3-Clause&color=green"></a>
   <br>
   <br>
   <a href="https://github.com/TissueImageAnalytics/tiatoolbox/actions/workflows/pip-install.yml">
@@ -85,7 +85,7 @@ tiatoolbox supports various features through command line. For more information,
 
 Prepare a computer as a convenient platform for further development of the Python package `tiatoolbox` and related programs as follows.
 
-1. Install [pre-requisite software](https://tia-toolbox.readthedocs.io/en/latest/installation.html)
+1. Install [pre-requisite software](https://tia-toolbox.readthedocs.io/en/latest/installation.html#prerequisites)
 1. Open a terminal window<br/>
 
 ```sh
@@ -107,16 +107,16 @@ Prepare a computer as a convenient platform for further development of the Pytho
 5. Create virtual environment for TIAToolbox using
 
 ```sh
-    $ conda env create -f requirements.dev.conda.yml # for linux/mac only.
+    $ conda create -n tiatoolbox-dev python=3.8 # select version of your choice
     $ conda activate tiatoolbox-dev
+    $ pip install -r requirements/requirements_dev.txt
 ```
 
 or
 
 ```sh
-    $ conda create -n tiatoolbox-dev python=3.8 # select version of your choice
+    $ conda env create -f requirements/requirements.dev.conda.yml # for linux/mac only.
     $ conda activate tiatoolbox-dev
-    $ pip install -r requirements_dev.txt
 ```
 
 6. To use the packages installed in the environment, run the command:
@@ -127,9 +127,9 @@ or
 
 ### License
 
-The source code TIA Toolbox (tiatoolbox) as hosted on GitHub is released under the [The 3-Clause BSD License].
+The source code TIA Toolbox (tiatoolbox) as hosted on GitHub is released under the [BSD-3-Clause license](https://github.com/TissueImageAnalytics/tiatoolbox/blob/develop/LICENSE). The full text of the licence is included in [LICENSE](https://raw.githubusercontent.com/TissueImageAnalytics/tiatoolbox/develop/LICENSE).
 
-The full text of the licence is included in [LICENSE](https://raw.githubusercontent.com/TissueImageAnalytics/tiatoolbox/develop/LICENSE).
+Models weights are dependent on the datasets that they were trained on. Please refer to the [documentation](https://tia-toolbox.readthedocs.io/en/latest/pretrained.html) for more details.
 
 ### Cite this repository
 
@@ -152,13 +152,3 @@ If you find TIAToolbox useful or use it in your research, please consider citing
     year = {2022}
 }
 ```
-
-### Auxiliary Files
-
-Auxiliary files, such as pre-trained model weights downloaded from the TIA Centre webpage (https://warwick.ac.uk/tia/), are provided under the [Creative Commons Attribution-NonCommercial-ShareAlike Version 4 (CC BY-NC-SA 4.0) license](https://creativecommons.org/licenses/by-nc-sa/4.0/).
-
-### Dual License
-
-If you would like to use any of the source code or auxiliary files (e.g. pre-trained model weights) under a different license agreement please contact the Tissue Image Analytics (TIA) Centre at the University of Warwick (tia@dcs.warwick.ac.uk).
-
-[the 3-clause bsd license]: https://opensource.org/licenses/BSD-3-Clause