Skip to content

Conversation

saksham23467
Copy link

@saksham23467 saksham23467 commented Aug 1, 2025

Description

  • This PR fixes error handling for deleted repositories (404 errors) in the GitHub repository move detection system.
  • Previously, when a repository returned a 404 (deleted), the system would mark it as IGNORE but still raise an exception, causing Celery tasks to fail.
  • Now deleted repositories are handled gracefully without causing task failures.

Changes Made

  • Fixed 404 error handling in ping_github_for_repo_move() to return gracefully instead of raising exceptions
  • Added comprehensive task-level error handling with try-catch blocks
  • Enhanced API call retry logic with better error handling and validation
  • Added validation for repository URL parsing and API responses
  • Improved logging throughout the process for better monitoring
  • Deleted repositories are now properly marked as IGNORE in the database

Testing

  • The fix addresses the specific error case where repositories like https://github.com/anaconda-platform/transformers-feedstock return 404 errors
  • Error handling now prevents Celery task failures for deleted repositories
  • System continues processing other repositories even when encountering deleted ones

This PR fixes #3166

Notes for Reviewers

  • The changes are backward compatible and don't affect existing functionality
  • Enhanced logging makes it easier to monitor repository status changes
  • Error handling follows existing patterns in the codebase

Signed commits

  • Yes, I signed my commits.

Signed-off-by: saksham23467 <142910439+saksham23467@users.noreply.github.com>
…etection

- Fix 404 error handling to gracefully handle deleted repositories instead of raising exceptions
- Add proper task-level error handling with try-catch blocks
- Enhance API call retry logic with better error handling
- Add validation for repository URL parsing and API responses
- Improve logging throughout the process for better monitoring
- Mark deleted repositories as IGNORE in database instead of failing tasks

Signed-off-by: saksham23467 <142910439+saksham23467@users.noreply.github.com>
@sgoggins sgoggins added the add-feature Adds new features label Aug 12, 2025
@sgoggins
Copy link
Member

@saksham23467 : This looks promising.

@saksham23467
Copy link
Author

@saksham23467 : This looks promising.

Thanks for reviewing! If everything looks good, could you approve the PR so we can move it forward?

@sgoggins
Copy link
Member

sgoggins commented Sep 9, 2025

@IsaacMilarky /
@Ulincsys : Could you review this?

Copy link
Contributor

@Ulincsys Ulincsys left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are extra changes in this PR that should not be included

return results

@register_metric()
def clones(repo_group_id, repo_id=None, begin_date=None, end_date=None):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like the Clones metric API you implemented for #3247 got included in this PR.

Feature implementations and bugfixes for existing infrastructure should be made into separate PRs. Please rebase and update the PR to only include the error handling changes.

try:
owner, name = get_owner_repo(repo.repo_git)
url = f"https://api.github.com/repos/{owner}/{name}"
except Exception as e:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where possible and appropriate, please try to catch a specific exception class instead of a bare Exception.

@Ulincsys Ulincsys mentioned this pull request Sep 15, 2025
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
add-feature Adds new features
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Core:detect_github_repo_move_core error on repos that have been deleted
3 participants