-
Notifications
You must be signed in to change notification settings - Fork 21
Open
Labels
Description
Describe the bug
After updating and restarting almost every package, a race between the relational database and Icinga DB resulted in a failing query which is currently not being retried.
FATAL icingadb read tcp [::1]:6464->[::1]:5432: read: connection reset by peer
can't perform "SELECT id FROM service_state WHERE is_overdue='y'"
github.com/icinga/icinga-go-library/database.CantPerformQuery
github.com/icinga/icinga-go-library@v0.7.2/database/utils.go:19
github.com/icinga/icingadb/pkg/icingadb/overdue.Sync.initSync
github.com/icinga/icingadb/pkg/icingadb/overdue/sync.go:89
github.com/icinga/icingadb/pkg/icingadb/overdue.Sync.Sync.func2
github.com/icinga/icingadb/pkg/icingadb/overdue/sync.go:53
golang.org/x/sync/errgroup.(*Group).add.func1
golang.org/x/sync@v0.15.0/errgroup/errgroup.go:128
runtime.goexit
runtime/asm_amd64.s:1699
can't sync overdue indicators
github.com/icinga/icingadb/pkg/icingadb/overdue.Sync.Sync
github.com/icinga/icingadb/pkg/icingadb/overdue/sync.go:57
main.run.func6.3
github.com/icinga/icingadb/cmd/icingadb/main.go:215
golang.org/x/sync/errgroup.(*Group).add.func1
golang.org/x/sync@v0.15.0/errgroup/errgroup.go:128
runtime.goexit
runtime/asm_amd64.s:1699
icingadb/pkg/icingadb/overdue/sync.go
Lines 85 to 90 in a979409
| var rows []v1.IdMeta | |
| query := fmt.Sprintf("SELECT id FROM %s_state WHERE is_overdue='y'", objectType) | |
| if err := s.db.SelectContext(ctx, &rows, query); err != nil { | |
| return database.CantPerformQuery(err, query) | |
| } |
Expected behavior
Same retry logic as everywhere else.
Your Environment
- Icinga DB version: 1.4.0
Additional context
In general, we should scan the code for SQL queries not wrapped in our retry logic. I would guess there are some more which can be wrapped safely.