-
Notifications
You must be signed in to change notification settings - Fork 905
set up postgres monitoring with grafana and prometheus #3268
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
Reviewers: if you want i can comment out the new config in the dockerfile to match the convention we use for flower |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Excellent!
current status (as a draft): postgres monitoring seems to work (just do Flower also (apparrently) ships with a /metrics endpoint that prometheus can ingest so im working on getting that hooked up. feel free to play with this if anyone is interested! |
…orter Signed-off-by: Adrian Edwards <adredwar@redhat.com>
Seems to be working under both podman and docker environments |
This also includes both postgres and celery monitoring dashboards. To use this you need to add a .env file in the monitoring folder containing augur db credentials ( this is intended to run independently of augur so you can take augur down and back up and the monitoring should keep going and simply reflect a gap in the data |
Signed-off-by: Adrian Edwards <adredwar@redhat.com>
Signed-off-by: Adrian Edwards <adredwar@redhat.com>
Signed-off-by: Adrian Edwards <adredwar@redhat.com>
Signed-off-by: Adrian Edwards <adredwar@redhat.com>
…n the host Signed-off-by: Adrian Edwards <adredwar@redhat.com>
Signed-off-by: Adrian Edwards <adredwar@redhat.com>
Signed-off-by: Adrian Edwards <adredwar@redhat.com>
Signed-off-by: Adrian Edwards <adredwar@redhat.com>
Signed-off-by: Adrian Edwards <adredwar@redhat.com>
Signed-off-by: Adrian Edwards <adredwar@redhat.com>
Signed-off-by: Adrian Edwards <adredwar@redhat.com>
Signed-off-by: Adrian Edwards <adredwar@redhat.com>
Signed-off-by: Adrian Edwards <adredwar@redhat.com>
@sgoggins i think my force pushing dismissed your review |
Description
this adds some containers to the stack to enable better monitoring of the postgres DB
This is intended to be something similar to flower in that its mainly (for now) intended for development use.
Grafana in particular takes a hot sec to start up (probably about a minute) before its accessible via the web interface (it has db migrations to do apparrently - the logs for this should be hidden by the instruction to only show error logs)
Usage:
Observe: data
The usecases for this is to generally provide more observability into what is going on with postgres. This may specifically help with debugging facade things, especially with respect to how many deadlock events are happening, how frequently postgres is checkpointing, how many idle processes/connections there are, etc.
I also hope that, in the future we can add some openTelemetry type reporting into augur so that augur itself can be monitored through a dashboard like this (maybe we can pipe Celery data into this?)
Signed commits