🚀 feat(dataset): add 3D-ADAM dataset to Anomalib #2986

PaulMcHard · 2025-10-01T11:50:56Z

📝 Description

This PR adds the 3D-ADAM dataset as a Depth Dataset option in Anomalib
🛠️ Fixes 📖 [STORY] Integrating the 3D-ADAM Dataset into Anomalib #2982

✨ Changes

Select what type of change your PR is:

🚀 New feature (non-breaking change which adds functionality)
📚 Documentation update

✅ Checklist

Before you submit your pull request, please make sure you have completed the following steps:

📚 I have made the necessary updates to the documentation (if applicable).
🧪 I have written tests that support my changes and prove that my fix is effective or my feature works (if applicable).
🏷️ My PR title follows conventional commit format.

For more information about code review checklists, see the Code Review Checklist.

ashwinvaidya17

Thanks for the contribution! I still need to check it manually but I have a few minor comments.
Can you also add a test for this?
These might help
https://github.com/open-edge-platform/anomalib/tree/main/tests/unit/data/datamodule/image

anomalib/tests/helpers/data.py

Line 263 in f95cdf9

class DummyImageDatasetGenerator(DummyDatasetGenerator):

If not, I can create a follow-up PR for the tests.

src/anomalib/data/datasets/depth/adam_3d.py

src/anomalib/data/datamodules/depth/adam_3d.py

rajeshgangireddy

Thanks for the contribution. I left some comments.
It also looks like you need to update the imports at src/anomalib/data/__init__.py

src/anomalib/data/datamodules/depth/adam_3d.py

src/anomalib/data/datasets/depth/adam_3d.py

PaulMcHard · 2025-10-01T15:11:07Z

Thank you for the feedback, I have enabled pre-commit hooks and resolved the given comments, much appreciated.

…l dependencies (open-edge-platform#2956) * fix(data): enable pin_memory for DataLoader instances across the codebase This commit updates various DataLoader instances in the project to enable the option, enhancing performance for data loading on GPU. Changes were made in the following files: - : Updated train and test DataLoader configurations. - : Modified datamodule DataLoader to include . - : Added to evaluation DataLoader. - : Updated DataLoader for datasets to utilize . - : Enabled for reference dataset DataLoader. - : Adjusted inference DataLoader to include . These changes aim to optimize memory usage and improve data transfer speeds during model training and inference. Signed-off-by: samet-akcay <samet.akcay@intel.com> * refactor(models): streamline decoder retrieval in function This commit refactors the function in to utilize a dictionary mapping for decoder architectures, improving readability and maintainability. The previous conditional checks have been replaced with a more efficient approach, enhancing the overall structure of the code. Signed-off-by: samet-akcay <samet.akcay@intel.com> * fix(model): update logit_scale initialization to use torch.log for consistency This commit modifies the initialization of the logit_scale parameter in the CLIP model to utilize torch.log instead of np.log. This change ensures consistency in tensor operations and improves compatibility with PyTorch's computation graph. Signed-off-by: samet-akcay <samet.akcay@intel.com> * fix(model): update anomaly map generation to use torch tensors for calculations This commit modifies the anomaly map generation logic to utilize PyTorch tensors instead of NumPy arrays for various calculations. This change enhances compatibility with the PyTorch computation graph and improves performance by leveraging GPU acceleration. Key updates include the conversion of statistical calculations and tensor operations to use PyTorch functions, ensuring consistency in tensor handling throughout the code. Signed-off-by: samet-akcay <samet.akcay@intel.com> * refactor(model): enhance anomaly map generation with PyTorch for statistical calculations This commit refactors the anomaly map generation logic to replace NumPy-based statistical calculations with PyTorch equivalents, specifically using the distribution for computing tau. Additionally, it improves precision handling by allowing the use of float64 in high precision mode. The changes streamline the computation process and maintain compatibility with the PyTorch computation graph. Signed-off-by: samet-akcay <samet.akcay@intel.com> * chore(license): update license year Signed-off-by: samet-akcay <samet.akcay@intel.com> * fix(download): enhance URL validation and update download logic This commit improves the URL validation in the download function to ensure only http and https schemes are allowed. Additionally, it adds comments to clarify the safety of using under these conditions, enhancing code readability and security awareness. Signed-off-by: samet-akcay <samet.akcay@intel.com> * 🗑️ chore(imports): remove OptionalImport class for optional dependency handling Signed-off-by: samet-akcay <samet.akcay@intel.com> * 🐛 fix(imports): Replace OptionalImport with dummy classes for missing dependencies This commit removes the OptionalImport class and introduces dummy classes for various loggers and video utilities when their respective dependencies are not installed. The changes enhance error messaging by providing clear instructions for installation, improving user experience when encountering missing packages. Affected files: - video.py - comet.py - mlflow.py - tensorboard.py - wandb.py - ollama.py Signed-off-by: samet-akcay <samet.akcay@intel.com> --------- Signed-off-by: samet-akcay <samet.akcay@intel.com> Signed-off-by: Paul McHard <paulmchard@gmail.com>

Signed-off-by: Paul McHard <paulmchard@gmail.com>

ignore uv.lock in semgrep Signed-off-by: Ashwin Vaidya <ashwinnitinvaidya@gmail.com> Signed-off-by: Paul McHard <paulmchard@gmail.com>

… itself will not be modified (open-edge-platform#2966) fix in dryrun_find_featuremap_dims: run the model in eval() so the weights of the model itself will not be modified Signed-off-by: Paul McHard <paulmchard@gmail.com>

…#2979) 🐛 fix(ovinferencer): update cache folder name to 'openvino_cache' and ensure string path for set_property Signed-off-by: Paul McHard <paulmchard@gmail.com>

Signed-off-by: Paul McHard <paulmchard@gmail.com>

PaulMcHard · 2025-10-02T21:29:47Z

Please advise if there's any changes required to correct for the unit test which is timing out.

src/anomalib/data/datamodules/depth/adam_3d.py

Signed-off-by: Paul McHard <paulmchard@gmail.com>

docs/source/examples

rajeshgangireddy · 2025-10-06T09:01:47Z

I was testing this locally, the download file link is still not the correct one I guess.
I think we need to use url="https://huggingface.co/datasets/pmchard/3D-ADAM_anomalib/resolve/main/adam3d_cropped.zip",
This works.

Signed-off-by: Paul McHard <paulmchard@gmail.com>

PaulMcHard · 2025-10-06T11:07:54Z

All outstanding issues should now be resolved. Please let me know if there's anything else needing addressed.

ashwinvaidya17

Thanks for the update, two questions.

Why are the file permissions changed?
Why is the uv lock file updated? I can see that almost all lines have been removed.

tests/unit/data/datamodule/depth/test_adam_3d.py

docs/source/examples

Signed-off-by: Paul McHard <paulmchard@gmail.com>

PaulMcHard · 2025-10-06T14:03:56Z

Thanks for the update, two questions.

Why are the file permissions changed?

Why is the uv lock file updated? I can see that almost all lines have been removed.

Unsure as to why uv lock file or file permissions appear updated, no change on my end to code should have causes this, and appears normal in local and on fork. Hopefully addressed these issues and remaining comments in most recent commit.

Signed-off-by: Paul McHard <32065439+PaulMcHard@users.noreply.github.com>

PaulMcHard · 2025-10-06T14:35:23Z

uv.lock issue appeared to be coming from an issue with git lfs on my end locally, appears to be resolved now let me know if there's any other changes required.

@samet-akcay

* 🚀 feat(dataset): add 3D-ADAM dataset to Anomalib (#2986) * 🐞 fix(utils): revert `OptionalImport` class to handle missing optional dependencies (#2956) * fix(data): enable pin_memory for DataLoader instances across the codebase This commit updates various DataLoader instances in the project to enable the option, enhancing performance for data loading on GPU. Changes were made in the following files: - : Updated train and test DataLoader configurations. - : Modified datamodule DataLoader to include . - : Added to evaluation DataLoader. - : Updated DataLoader for datasets to utilize . - : Enabled for reference dataset DataLoader. - : Adjusted inference DataLoader to include . These changes aim to optimize memory usage and improve data transfer speeds during model training and inference. Signed-off-by: samet-akcay <samet.akcay@intel.com> * refactor(models): streamline decoder retrieval in function This commit refactors the function in to utilize a dictionary mapping for decoder architectures, improving readability and maintainability. The previous conditional checks have been replaced with a more efficient approach, enhancing the overall structure of the code. Signed-off-by: samet-akcay <samet.akcay@intel.com> * fix(model): update logit_scale initialization to use torch.log for consistency This commit modifies the initialization of the logit_scale parameter in the CLIP model to utilize torch.log instead of np.log. This change ensures consistency in tensor operations and improves compatibility with PyTorch's computation graph. Signed-off-by: samet-akcay <samet.akcay@intel.com> * fix(model): update anomaly map generation to use torch tensors for calculations This commit modifies the anomaly map generation logic to utilize PyTorch tensors instead of NumPy arrays for various calculations. This change enhances compatibility with the PyTorch computation graph and improves performance by leveraging GPU acceleration. Key updates include the conversion of statistical calculations and tensor operations to use PyTorch functions, ensuring consistency in tensor handling throughout the code. Signed-off-by: samet-akcay <samet.akcay@intel.com> * refactor(model): enhance anomaly map generation with PyTorch for statistical calculations This commit refactors the anomaly map generation logic to replace NumPy-based statistical calculations with PyTorch equivalents, specifically using the distribution for computing tau. Additionally, it improves precision handling by allowing the use of float64 in high precision mode. The changes streamline the computation process and maintain compatibility with the PyTorch computation graph. Signed-off-by: samet-akcay <samet.akcay@intel.com> * chore(license): update license year Signed-off-by: samet-akcay <samet.akcay@intel.com> * fix(download): enhance URL validation and update download logic This commit improves the URL validation in the download function to ensure only http and https schemes are allowed. Additionally, it adds comments to clarify the safety of using under these conditions, enhancing code readability and security awareness. Signed-off-by: samet-akcay <samet.akcay@intel.com> * 🗑️ chore(imports): remove OptionalImport class for optional dependency handling Signed-off-by: samet-akcay <samet.akcay@intel.com> * 🐛 fix(imports): Replace OptionalImport with dummy classes for missing dependencies This commit removes the OptionalImport class and introduces dummy classes for various loggers and video utilities when their respective dependencies are not installed. The changes enhance error messaging by providing clear instructions for installation, improving user experience when encountering missing packages. Affected files: - video.py - comet.py - mlflow.py - tensorboard.py - wandb.py - ollama.py Signed-off-by: samet-akcay <samet.akcay@intel.com> --------- Signed-off-by: samet-akcay <samet.akcay@intel.com> Signed-off-by: Paul McHard <paulmchard@gmail.com> * Added 3D-ADAM Dataset Signed-off-by: Paul McHard <paulmchard@gmail.com> * Added tests and example config Signed-off-by: Paul McHard <paulmchard@gmail.com> * 🔨 fix(semgrep): ignore uv.lock in semgrep (#2972) ignore uv.lock in semgrep Signed-off-by: Ashwin Vaidya <ashwinnitinvaidya@gmail.com> Signed-off-by: Paul McHard <paulmchard@gmail.com> * 🐞 fix(utils): Run CS Flow model in eval() so the weights of the model itself will not be modified (#2966) fix in dryrun_find_featuremap_dims: run the model in eval() so the weights of the model itself will not be modified Signed-off-by: Paul McHard <paulmchard@gmail.com> * 🐛 fix(ovinferencer): Fix openvino cache directory (#2979) 🐛 fix(ovinferencer): update cache folder name to 'openvino_cache' and ensure string path for set_property Signed-off-by: Paul McHard <paulmchard@gmail.com> * changes per PR and pre-commit checks Signed-off-by: Paul McHard <paulmchard@gmail.com> * resolve circular import issue Signed-off-by: Paul McHard <paulmchard@gmail.com> * Resolve DownloadInfo URL issue. Signed-off-by: Paul McHard <paulmchard@gmail.com> * Resolve DownloadInfo URL Signed-off-by: Paul McHard <paulmchard@gmail.com> * restore doc images ignored in error Signed-off-by: Paul McHard <paulmchard@gmail.com> * Removed prettier cache included in error, re-run pre-commit hooks Signed-off-by: Paul McHard <paulmchard@gmail.com> * explictly resolve EOF error on docs/source/examples Signed-off-by: Paul McHard <paulmchard@gmail.com> * resolve filename issue in adam3d test. Signed-off-by: Paul McHard <paulmchard@gmail.com> * Restore uv.lock Signed-off-by: Paul McHard <32065439+PaulMcHard@users.noreply.github.com> --------- Signed-off-by: samet-akcay <samet.akcay@intel.com> Signed-off-by: Paul McHard <paulmchard@gmail.com> Signed-off-by: Ashwin Vaidya <ashwinnitinvaidya@gmail.com> Signed-off-by: Paul McHard <32065439+PaulMcHard@users.noreply.github.com> Co-authored-by: Samet Akcay <samet.akcay@intel.com> Co-authored-by: Ashwin Vaidya <ashwin.vaidya@intel.com> Co-authored-by: Rune <60315601+monzelr@users.noreply.github.com> Co-authored-by: Rajesh Gangireddy <rajesh.gangireddy@intel.com> * revert changes to example Signed-off-by: Ashwin Vaidya <ashwin.vaidya@intel.com> * revert file permissions Signed-off-by: Ashwin Vaidya <ashwin.vaidya@intel.com> * 🔧 refactor(data): Polish 3dAdam dataset (#3002) * Add changelog + fix dataset location Signed-off-by: Ashwin Vaidya <ashwin.vaidya@intel.com> * revert changes to example Signed-off-by: Ashwin Vaidya <ashwin.vaidya@intel.com> * revert file permissions Signed-off-by: Ashwin Vaidya <ashwin.vaidya@intel.com> * Fix tests Signed-off-by: Ashwin Vaidya <ashwin.vaidya@intel.com> --------- Signed-off-by: Ashwin Vaidya <ashwin.vaidya@intel.com> * revert examples Signed-off-by: Ashwin Vaidya <ashwin.vaidya@intel.com> * revert file permissions Signed-off-by: Ashwin Vaidya <ashwin.vaidya@intel.com> * Add imagecodecs dependency Signed-off-by: Ashwin Vaidya <ashwin.vaidya@intel.com> * Apply suggestion from @samet-akcay Co-authored-by: Samet Akcay <samet.akcay@intel.com> Signed-off-by: Ashwin Vaidya <ashwinnitinvaidya@gmail.com> --------- Signed-off-by: samet-akcay <samet.akcay@intel.com> Signed-off-by: Paul McHard <paulmchard@gmail.com> Signed-off-by: Ashwin Vaidya <ashwinnitinvaidya@gmail.com> Signed-off-by: Paul McHard <32065439+PaulMcHard@users.noreply.github.com> Signed-off-by: Ashwin Vaidya <ashwin.vaidya@intel.com> Co-authored-by: Paul McHard <32065439+PaulMcHard@users.noreply.github.com> Co-authored-by: Samet Akcay <samet.akcay@intel.com> Co-authored-by: Rune <60315601+monzelr@users.noreply.github.com> Co-authored-by: Rajesh Gangireddy <rajesh.gangireddy@intel.com>

rajeshgangireddy self-requested a review October 1, 2025 12:23

ashwinvaidya17 reviewed Oct 1, 2025

View reviewed changes

src/anomalib/data/datasets/depth/adam_3d.py Outdated Show resolved Hide resolved

src/anomalib/data/datamodules/depth/adam_3d.py Show resolved Hide resolved

src/anomalib/data/datamodules/depth/adam_3d.py Outdated Show resolved Hide resolved

rajeshgangireddy reviewed Oct 1, 2025

View reviewed changes

PaulMcHard requested a review from samet-akcay as a code owner October 1, 2025 14:09

samet-akcay and others added 7 commits October 1, 2025 16:52

Added 3D-ADAM Dataset

b1b5406

Signed-off-by: Paul McHard <paulmchard@gmail.com>

Added tests and example config

8a318f9

Signed-off-by: Paul McHard <paulmchard@gmail.com>

🔨 fix(semgrep): ignore uv.lock in semgrep (open-edge-platform#2972)

e576472

ignore uv.lock in semgrep Signed-off-by: Ashwin Vaidya <ashwinnitinvaidya@gmail.com> Signed-off-by: Paul McHard <paulmchard@gmail.com>

🐛 fix(ovinferencer): Fix openvino cache directory (open-edge-platform…

6f99ebc

…#2979) 🐛 fix(ovinferencer): update cache folder name to 'openvino_cache' and ensure string path for set_property Signed-off-by: Paul McHard <paulmchard@gmail.com>

changes per PR and pre-commit checks

f1db4b7

Signed-off-by: Paul McHard <paulmchard@gmail.com>

PaulMcHard force-pushed the main branch from cf832e9 to f1db4b7 Compare October 1, 2025 15:52

PaulMcHard and others added 3 commits October 1, 2025 16:53

Merge branch 'main' into main

15cffd3

resolve circular import issue

cf3b244

Signed-off-by: Paul McHard <paulmchard@gmail.com>

Merge branch 'main' of https://github.com/PaulMcHard/3D-ADAM_anomalib

cdd87d7

rajeshgangireddy reviewed Oct 3, 2025

View reviewed changes

src/anomalib/data/datamodules/depth/adam_3d.py Show resolved Hide resolved

PaulMcHard and others added 2 commits October 3, 2025 14:21

Merge branch 'open-edge-platform:main' into main

84bec1a

Resolve DownloadInfo URL issue.

530cef9

Signed-off-by: Paul McHard <paulmchard@gmail.com>

ashwinvaidya17 reviewed Oct 6, 2025

View reviewed changes

docs/source/examples Outdated Show resolved Hide resolved

PaulMcHard and others added 6 commits October 6, 2025 11:09

Resolve DownloadInfo URL

01574d2

Signed-off-by: Paul McHard <paulmchard@gmail.com>

Merge branch 'open-edge-platform:main' into main

302159d

restore doc images ignored in error

73df743

Signed-off-by: Paul McHard <paulmchard@gmail.com>

t:Merge branch 'main' of https://github.com/PaulMcHard/3D-ADAM_anomalib

bc97595

Removed prettier cache included in error, re-run pre-commit hooks

0175ec1

Signed-off-by: Paul McHard <paulmchard@gmail.com>

explictly resolve EOF error on docs/source/examples

fdac4af

Signed-off-by: Paul McHard <paulmchard@gmail.com>

Merge branch 'main' into main

fe04f7d

ashwinvaidya17 reviewed Oct 6, 2025

View reviewed changes

tests/unit/data/datamodule/depth/test_adam_3d.py Outdated Show resolved Hide resolved

docs/source/examples Show resolved Hide resolved

PaulMcHard added 2 commits October 6, 2025 14:59

resolve filename issue in adam3d test.

de13783

Signed-off-by: Paul McHard <paulmchard@gmail.com>

Merge branch 'main' of https://github.com/PaulMcHard/3D-ADAM_anomalib

67a87d1

Restore uv.lock

2ca11c1

Signed-off-by: Paul McHard <32065439+PaulMcHard@users.noreply.github.com>

Merge branch 'open-edge-platform:main' into main

cbc3e93

ashwinvaidya17 self-requested a review October 7, 2025 07:10

ashwinvaidya17 changed the base branch from main to feature/dataset/3dadam October 7, 2025 07:38

ashwinvaidya17 merged commit 1c99f8b into open-edge-platform:feature/dataset/3dadam Oct 7, 2025
13 of 14 checks passed

🚀 feat(dataset): add 3D-ADAM dataset to Anomalib #2986

🚀 feat(dataset): add 3D-ADAM dataset to Anomalib #2986

Uh oh!

Conversation

PaulMcHard commented Oct 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

📝 Description

✨ Changes

✅ Checklist

Uh oh!

ashwinvaidya17 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

rajeshgangireddy left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

PaulMcHard commented Oct 1, 2025

Uh oh!

PaulMcHard commented Oct 2, 2025

Uh oh!

Uh oh!

Uh oh!

rajeshgangireddy commented Oct 6, 2025

Uh oh!

PaulMcHard commented Oct 6, 2025

Uh oh!

ashwinvaidya17 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

PaulMcHard commented Oct 6, 2025

Uh oh!

PaulMcHard commented Oct 6, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

PaulMcHard commented Oct 1, 2025 •

edited

Loading