Fix pending files retry and scheduler reliability (v2.5.51-v2.5.57)#31
Merged
ttlequals0 merged 8 commits intomainfrom Jan 4, 2026
Merged
Fix pending files retry and scheduler reliability (v2.5.51-v2.5.57)#31ttlequals0 merged 8 commits intomainfrom
ttlequals0 merged 8 commits intomainfrom
Conversation
Root cause: redis.from_url() returns low-level Connection object, not full Redis client. When connection was reset by peer, code crashed with: 'Connection' object has no attribute 'register_connect_callback' Solution: - Added robust Redis connection handling in progress_utils.py - Connection pooling with health_check_interval=30 - Auto-retry on connection failures (3 attempts, 1s delay) - socket_keepalive=True to detect stale connections - retry_on_timeout=True for transient failures New utility functions: - get_redis_client(): Returns properly configured Redis client - get_redis_info(): Safely gets Redis server info with retry - with_redis_retry(): Decorator for Redis operation retry logic Updated maintenance_service.py and app.py to use new utilities instead of problematic redis.from_url() pattern. Files affected: - pixelprobe/progress_utils.py - pixelprobe/services/maintenance_service.py - app.py - version.py - CHANGELOG.MD
HEIC files from iOS 18 devices were incorrectly flagged as corrupted due to older libheif versions not supporting shared auxiliary images feature. - Detect libheif "auxiliary image" errors in ImageMagick stderr - Detect "cannot identify" errors on HEIC files in PIL - Treat these as warnings instead of corruption flags - Files now show as "HEIC validation skipped: libheif version limitation" Ref: github.com/strukturag/libheif/issues/1190 Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
ttlequals0
changed the title
Add safe_task_ready() and safe_task_get() helper functions with retry logic to handle Redis connection errors in Celery's internal backend. The previous fix (v2.5.51) addressed application-level Redis connections but Celery's task.ready() and task.get() methods use their own internal Redis connection which can still fail with connection reset errors. This fix wraps those Celery methods with retry logic and graceful error handling to prevent integrity scan crashes.
Root cause: v2.5.53 wrappers had insufficient retry logic (3 retries, 0.5s delay). When Redis connection pool gets corrupted, ALL connections are bad. Solution: - Enhanced safe_task_ready/safe_task_get with 5 retries and exponential backoff - Added reset_redis_pool() to force-disconnect and recreate pool on first failure - Added safe_check_task_state() wrapper for scan_routes.py AsyncResult access - Added Celery transport options for connection stability Files: progress_utils.py, maintenance_service.py, scan_routes.py, celery_config.py
Update user-facing text for consistency: - Scan reports filter dropdown - Schedule type dropdowns (2 locations) - All notification messages (8 locations) - Scan type display mapping
…hanged files (v2.5.56) - Revert scheduler to v2.5.19 behavior - remove is_celery_worker check - Celery prefork model incompatible with APScheduler threads - Allow any process to acquire scheduler lock (gunicorn works correctly) - Fix integrity scan reports missing changed files - Assign local changed_files to self.changed_files_list after processing 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
v2.5.56 removed is_celery_worker definition but left a reference at line 1091, causing NameError on startup. This fix removes the orphaned reference from the file-lock fallback condition. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Delete tools/migrate_db.py, migrate_db_safe.py, run_migration.py - Update tools/README.md, MIGRATION_GUIDE.md, docs/maintenance/TOOLS_AND_SCRIPTS.md - Migrations now run automatically via app_startup_migration.py on startup - Update CHANGELOG.md with cleanup notes for v2.5.57
ttlequals0
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR fixes multiple issues related to Redis connection handling, scheduled scans, and cleans up obsolete migration code.
v2.5.57 - Startup Crash Fix & Cleanup
migrate_db.py,migrate_db_safe.py,run_migration.py)v2.5.56 - Scheduler & Integrity Scan Fixes
v2.5.55 - UI Terminology Update
v2.5.54 - Enhanced Redis Resilience
v2.5.53 - Celery Redis Wrappers
v2.5.52 - HEIC False Positive Fix
v2.5.51 - Redis Connection Fix
ConnectionResetErrorduring file changes checkFiles Changed
Core Fixes:
app.py- Scheduler lock handling, version endpoint Redis utilitiespixelprobe/services/maintenance_service.py- Redis wrappers, changed files list fixpixelprobe/progress_utils.py- Robust Redis client utilitiespixelprobe/api/scan_routes.py- Safe task state checkingcelery_config.py- Transport options for resiliencemedia_checker.py- HEIC libheif error detectionUI Updates:
templates/index.html- Terminology updatesstatic/js/app.js- Notification message updatesCleanup (v2.5.57):
tools/migrate_db.py,tools/migrate_db_safe.py,tools/run_migration.pytools/README.md,tools/MIGRATION_GUIDE.md,docs/maintenance/TOOLS_AND_SCRIPTS.mdTest plan