add probe signals for beat

[rkuo-danswer]

Description

Fixes https://linear.app/danswer/issue/DAN-2007/k8s-probes-for-celery-beat

How Has This Been Tested?

[Describe the tests you ran to verify your changes]

Backporting (check the box to trigger backport action)

Note: You have to check that the action passes, otherwise resolve the conflicts manually and tag the patches.

• This PR should be backported (make sure to check that the backport attempt succeeds)
• [Optional] Override Linear Check

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
internal-search	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	May 22, 2025 9:56pm

[greptile-apps]

PR Summary

This PR implements Kubernetes health monitoring for Celery Beat and workers through file-based liveness and readiness probes.

• Added make_probe_path in celery_utils.py to generate standardized probe file paths with hostname validation
• Implemented liveness probe in beat.py that updates during scheduler ticks and readiness probe after Redis initialization
• Added LivenessProbe bootstep class in app_base.py for periodic worker health checks
• Probe files are managed in /tmp with proper cleanup on worker shutdown
• Consider adding error handling for probe file operations in case of filesystem issues

_{3 file(s) reviewed, 4 comment(s)
Edit PR Review Bot Settings | Greptile}

[greptile-apps]

logic: Similar to liveness probe, readiness probe should handle potential IOErrors when touching the file

Suggested change

	hostname: str = cast(str, sender.hostname)
	path = _make_probe_path("readiness", hostname)
	path = make_probe_path("readiness", hostname)
	path.touch()
	logger.info(f"Readiness signal touched at {path}.")
	hostname: str = cast(str, sender.hostname)
	path = make_probe_path("readiness", hostname)
	try:
	path.touch()
	logger.info(f"Readiness signal touched at {path}.")
	except IOError:
	logger.exception(f"Failed to create readiness probe file at {path}")