Skip to content

Fix Image Feature Extraction Observation by Removing Double Normalization and Using Only GPU Operations #1323

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

glvov-bdai
Copy link
Collaborator

@glvov-bdai glvov-bdai commented Oct 28, 2024

Description

Previously, image_features requested normalized images prior to normalizing as part of the preprocessing. As pointed out to me by Jinghuan Shang (Theia Vision Transformer first author), this lead to images being normalized more than once which is ineffective and incorrect. Furthermore, Jinghuan pointed out where Transformers needlessly converted from GPU to CPU for normalization, where these operations could be converted to be GPU only.

ALSO: Fixes device and setup.py

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)

Checklist

  • I have run the pre-commit checks with ./isaaclab.sh --format
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • I have updated the changelog and the corresponding version in the extension's config/extension.toml file
  • I have added my name to the CONTRIBUTORS.md or my name already exists there

Signed-off-by: glvov-bdai <glvov@theaiinstitute.com>
Copy link
Collaborator

@jsmith-bdai jsmith-bdai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM thanks!

Signed-off-by: glvov-bdai <glvov@theaiinstitute.com>
Signed-off-by: glvov-bdai <glvov@theaiinstitute.com>
@Mayankm96
Copy link
Contributor

Shall we close this MR in favor of #1340 ?

@glvov-bdai
Copy link
Collaborator Author

glvov-bdai commented Oct 30, 2024

Yup works for me. Closed in favor of #1340

@glvov-bdai glvov-bdai closed this Oct 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants