[DOC] Add example notebook for using aeon distances with sklearn clusterers

[SalmanDeveloperz]

Reference Issues/PRs

Fixes #1241

What does this implement/fix? Explain your changes.

This pull request introduces a new Jupyter Notebook: sklearn_clustering_with_aeon_distances.ipynb. The notebook demonstrates how to integrate aeon's distance metrics with scikit-learn clustering algorithms. It includes:

Hierarchical Clustering: Using AgglomerativeClustering with metric="precomputed".
Density-Based Clustering: Using DBSCAN and OPTICS with metric="precomputed".
Spectral Clustering: Using SpectralClustering with affinity="precomputed" and the inverse of the distance matrix as the similarity matrix.
This addition enhances the clustering documentation, showing how to combine aeon’s distance metrics with widely-used scikit-learn clusterers.

Does your contribution introduce a new dependency? If yes, which one?

No new dependencies introduced.

Any other comments?

• The notebook has been tested locally, and all cells execute without errors.

• A reference to this notebook has been added to the clustering section of the documentation.

PR checklist

For all contributions

• I've added myself to the list of contributors. Alternatively, you can use the @all-contributors bot to do this for you after the PR has been merged.
• The PR title starts with either [ENH], [MNT], [DOC], [BUG], [REF], [DEP] or [GOV] indicating whether the PR topic is related to enhancement, maintenance, documentation, bugs, refactoring, deprecation or governance.

For new estimators and functions

• I've added the estimator/function to the online API documentation.
• (OPTIONAL) I've added myself as a __maintainer__ at the top of relevant files and want to be contacted regarding its maintenance. Unmaintained files may be removed. This is for the full file, and you should not add yourself if you are just making minor changes or do not want to help maintain its contents.

For developers with write access

• (OPTIONAL) I've updated aeon's CODEOWNERS to receive notifications about future changes to these files.

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[aeon-actions-bot]

Thank you for contributing to aeon

I did not find any labels to add based on the title. Please add the [ENH], [MNT], [BUG], [DOC], [REF], [DEP] and/or [GOV] tags to your pull requests titles. For now you can add the labels manually.
I have added the following labels to this PR based on the changes made: [ $\color{#45FD64}{\textsf{examples}}$ ]. Feel free to change these if they do not properly represent the PR.

The Checks tab will show the status of our automated tests. You can click on individual test runs in the tab or "Details" in the panel below to see more information if there is a failure.

If our pre-commit code quality check fails, any trivial fixes will automatically be pushed to your PR unless it is a draft.

Don't hesitate to ask questions on the aeon Slack channel if you have any.

PR CI actions

These checkboxes will add labels to enable/disable CI functionality for this PR. This may not take effect immediately, and a new commit may be required to run the new configuration.

• Run pre-commit checks for all files
• Run mypy typecheck tests
• Run all pytest tests and configurations
• Run all notebook example tests
• Run numba-disabled codecov tests
• Stop automatic pre-commit fixes (always disabled for drafts)
• Disable numba cache loading
• Push an empty commit to re-run CI checks

[SalmanDeveloperz]

Dear maintainers,
I have created a new notebook and would like to know where to add its reference in the following documentation:

Clustering Overview
Clustering with sklearn.cluster
Could you please guide me on the most appropriate sections in these files to include the reference to the new notebook? I want to ensure it integrates well with the existing content before committing the changes.
Looking forward to your advice.
Best Regards

[TonyBagnall]

Dear maintainers, I have created a new notebook and would like to know where to add its reference in the following documentation:

Clustering Overview Clustering with sklearn.cluster Could you please guide me on the most appropriate sections in these files to include the reference to the new notebook? I want to ensure it integrates well with the existing content before committing the changes. Looking forward to your advice. Best Regards

hi, thanks for this, we will take a look

[SebastianSchmidl]

Dear maintainers, I have created a new notebook and would like to know where to add its reference in the following documentation:

Clustering Overview Clustering with sklearn.cluster Could you please guide me on the most appropriate sections in these files to include the reference to the new notebook? I want to ensure it integrates well with the existing content before committing the changes. Looking forward to your advice. Best Regards

Please refer to the links in my comment of the corresponding issue.

[SalmanDeveloperz]

Please refer to the links in my comment of the corresponding issue.

Thank you for the guidance! I have added a reference in the clustering.ipynb notebook under the Clustering Notebooks section, as suggested.

Please let me know if there are additional updates or adjustments required!

[MatthewMiddlehurst]

this is failing the tests currently, please click on details to see why and resolve.

[SebastianSchmidl]

Could you add a link to your new notebook from the Clustering-with-sklearn.cluster section of the distances/sklearn_distances-notebook, too?

[SalmanDeveloperz]

Could you add a link to your new notebook from the Clustering-with-sklearn.cluster section of the distances/sklearn_distances-notebook, too?

Inserted the required reference in the Clustering-with-sklearn.cluster section of sklearn_distances.ipynb. Please advise if any further refinements are necessary.

[SalmanDeveloperz]

Hi @SebastianSchmidl,
I've carefully reviewed the CI failures and made the necessary corrections based on the aeon distances API reference and a thorough review of the implementation.
After multiple iterations and meticulous debugging, all CI checks have now passed successfully.
Looking forward to your feedback!

[SebastianSchmidl]

Nice work 👍

The example plots (matplotlib) are not included in the notebook output, yet. I do not know if this should happen automatically or not.
@MatthewMiddlehurst Are the notebooks built and committed back by our nightly CI builds?

[MatthewMiddlehurst]

No this is not done automatically, but may be a good idea for those with no output. It would be good to run the notebook so the outputs are displayed here i agree.

https://aeon-toolkit--2511.org.readthedocs.build/en/2511/examples/clustering/sklearn_clustering_with_aeon_distances.html

[SalmanDeveloperz]

Hi,
The PR is updated as requested. Please review and merge if no further changes are needed. Thanks :)
Best

[MatthewMiddlehurst]

I see no changes since my comment. You should run the notebook so it has output visible on the website. Make sure you check the readthedocs output under checks.

[SalmanDeveloperz]

Dear @MatthewMiddlehurst,

I sincerely apologize for the oversight in my previous updates. I have now resolved the issues by fixing the merge conflict and correcting the indentation in the notebook. All GitHub Actions checks have passed successfully, and the notebook outputs for both Spectral and Hierarchical Clustering are now visible as requested. I’ve also verified the readthedocs output under checks. Kindly review the changes, and please let me know if any further adjustments are needed. Thank you for your guidance and patience throughout this process :)
Best

[MatthewMiddlehurst]

None of these plots look informative to me, unfortunately. What is it you want to show with these figures?

Maybe look into how others visualise clustering results (this may be a bit different for time series). There may be some useful functions in visualisation you can use.

[MatthewMiddlehurst]

X = np.vstack(
   (
       np.random.normal(loc=[2, 2], scale=0.5, size=(50, 2)),
       np.random.normal(loc=[5, 5], scale=0.5, size=(50, 2)),
   )
)

Use time series data like you did previously. As mentioned there are unique challenges when it comes to visualising time series, we are not that interested in regular clustering here.

Reply via ReviewNB

[SalmanDeveloperz]

Thanks for the feedback. I have updated the example to use time series data instead of the randomly generated clusters. Please check.

[SalmanDeveloperz]

Hi,
Thank you for your valuable feedback, and I sincerely apologize for the current state of the plots. I’m so sorry for the confusion caused. I’m feeling a bit overwhelmed as my previous attempts to update the visualization have not met the intended goal, and the plots don’t seem informative as you’ve noted.

Before proceeding with further changes, I’d like to seek your guidance. My intention was to demonstrate the separation of time series clusters using Hierarchical Clustering with DTW distance, but the mean-based plots (and my recent shift to time series data) haven’t effectively shown this :(. I noticed the previous 2D data code produced unexpected results locally (in Jupyter and VS Code), though it initially satisfied the output requirement. I’m unsure how to best proceed to align with aeon’s focus on time series clustering.
Please let me know how I can align better with your expectations 🙂.
Thanks for your patience

[MatthewMiddlehurst]

Look at the plotting functions in aeon currently and you may find some ideas. If nothing works, perhaps look at how others plot clustered time series.

[MatthewMiddlehurst]

Hello, is this still being worked on?

[SalmanDeveloperz]

Hello, It required some time I am working in GSoC rn, will take into it shortly

[SalmanDeveloperz]

Updated the DBSCAN visualization to show representative time series for each cluster, Along with a brief explanation to clarify the relevance of the plot.

[MatthewMiddlehurst]

I'm sorry for the delay, it has been a busy period. IMO the plots are still nonsense. I can only assume you are not checking them yourself.

Please have a think about what would actually be useful to display to users considering the time series data type if you want to use such plots. Again, we have plotting functions which may be better.

The image used for the notebook is also probably not relevant taking another look.