-
Notifications
You must be signed in to change notification settings - Fork 2k
add probes #4762
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
add probes #4762
Conversation
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
PR Summary
Added Kubernetes health probes (startup, readiness, liveness) across all Celery worker deployments to improve container lifecycle management and monitoring.
- Configured shared probe settings in
values.yaml
with specific timeouts: startup (2m), readiness (2m15s), and liveness (5m) - Added Python-based health check script execution in
/deployment/helm/charts/onyx/templates/celery-worker-*.yaml
files - Each worker type (light, heavy, indexing, etc.) uses unique temporary files for probe status tracking
- Startup probe checks for
/app/onyx/main.py
existence while readiness/liveness probes execute custom health checks - Potential concern: Probe script paths and temp file permissions need runtime verification
8 file(s) reviewed, 12 comment(s)
Edit PR Review Bot Settings | Greptile
* k -n onyx get pvc | ||
* k -n onyx delete pvc vespa-storage-da-vespa-0 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
style: Consider using full kubectl command instead of shorthand 'k' in documentation for clarity
* k -n onyx get pvc | |
* k -n onyx delete pvc vespa-storage-da-vespa-0 | |
* kubectl -n onyx get pvc | |
* kubectl -n onyx delete pvc vespa-storage-da-vespa-0 |
* add probes * lint fixes --------- Co-authored-by: Richard Kuo (Onyx) <rkuo@onyx.app>
* add probes * lint fixes --------- Co-authored-by: Richard Kuo (Onyx) <rkuo@onyx.app>
* add probes * lint fixes --------- Co-authored-by: Richard Kuo (Onyx) <rkuo@onyx.app>
* add probes * lint fixes --------- Co-authored-by: Richard Kuo (Onyx) <rkuo@onyx.app>
Description
Fixes https://linear.app/danswer/issue/DAN-2009/add-k8s-probes-to-helm
How Has This Been Tested?
[Describe the tests you ran to verify your changes]
Backporting (check the box to trigger backport action)
Note: You have to check that the action passes, otherwise resolve the conflicts manually and tag the patches.