Skip to content

Replica in archive recovery mode #2964

@pierreozoux

Description

@pierreozoux

This morning, for the first time, and an unnown reason, I saw this log:

2025-10-16 06:01:10,984 INFO: no action. I am (entre-vos-mains--de-pg-0), a secondary, and following a leader (entre-vos-mains--de-pg-1)
2025-10-16 06:01:20,972 INFO: no action. I am (entre-vos-mains--de-pg-0), a secondary, and following a leader (entre-vos-mains--de-pg-1)
2025-10-16 06:01:30,977 INFO: no action. I am (entre-vos-mains--de-pg-0), a secondary, and following a leader (entre-vos-mains--de-pg-1)
2025-10-16 06:01:41,011 INFO: no action. I am (entre-vos-mains--de-pg-0), a secondary, and following a leader (entre-vos-mains--de-pg-1)
2025-10-16 06:01:50,975 INFO: no action. I am (entre-vos-mains--de-pg-0), a secondary, and following a leader (entre-vos-mains--de-pg-1)
2025-10-16 06:02:01,053 INFO: no action. I am (entre-vos-mains--de-pg-0), a secondary, and following a leader (entre-vos-mains--de-pg-1)
2025-10-16 06:02:11,008 INFO: no action. I am (entre-vos-mains--de-pg-0), a secondary, and following a leader (entre-vos-mains--de-pg-1)
2025-10-16 06:02:20,977 INFO: no action. I am (entre-vos-mains--de-pg-0), a secondary, and following a leader (entre-vos-mains--de-pg-1)
2025-10-16 06:02:30,978 INFO: no action. I am (entre-vos-mains--de-pg-0), a secondary, and following a leader (entre-vos-mains--de-pg-1)
2025-10-16 06:02:40,982 INFO: no action. I am (entre-vos-mains--de-pg-0), a secondary, and following a leader (entre-vos-mains--de-pg-1)
2025-10-16 06:02:51,482 INFO: no action. I am (entre-vos-mains--de-pg-0), a secondary, and following a leader (entre-vos-mains--de-pg-1)
2025-10-16 06:03:01,482 INFO: no action. I am (entre-vos-mains--de-pg-0), a secondary, and following a leader (entre-vos-mains--de-pg-1)
2025-10-16 06:03:11,487 INFO: Got response from entre-vos-mains--de-pg-1 http://100.64.24.113:8008/patroni: {"state": "running", "postmaster_start_time": "2025-10-09 05:03:13.898519+00:00", "role": "master", "server_version": 150010, "xlog": {"location": 487311174984}, "timeline": 52, "replication": [{"usename": "standby", "application_name": "entre-vos-mains--de-pg-0", "client_addr": "100.64.29.244", "state": "streaming", "sync_state": "async", "sync_priority": 0}], "dcs_last_seen": 1760594590, "database_system_identifier": "7276070543149191243", "patroni": {"version": "3.3.3", "scope": "entre-vos-mains--de-pg", "name": "entre-vos-mains--de-pg-1"}}
2025-10-16 06:03:11,489 WARNING: Primary (entre-vos-mains--de-pg-1) is still alive
server signaled
2025-10-16 06:03:11,500 INFO: following a different leader because i am not the healthiest node
2025-10-16 06:03:20,987 INFO: Got response from entre-vos-mains--de-pg-1 http://100.64.24.113:8008/patroni: {"state": "running", "postmaster_start_time": "2025-10-09 05:03:13.898519+00:00", "role": "master", "server_version": 150010, "xlog": {"location": 487311174984}, "timeline": 52, "dcs_last_seen": 1760594600, "database_system_identifier": "7276070543149191243", "patroni": {"version": "3.3.3", "scope": "entre-vos-mains--de-pg", "name": "entre-vos-mains--de-pg-1"}}
2025-10-16 06:03:20,988 WARNING: Primary (entre-vos-mains--de-pg-1) is still alive
2025-10-16 06:03:21,041 INFO: following a different leader because i am not the healthiest node
2025-10-16 06:03:30,985 INFO: Got response from entre-vos-mains--de-pg-1 http://100.64.24.113:8008/patroni: {"state": "running", "postmaster_start_time": "2025-10-09 05:03:13.898519+00:00", "role": "master", "server_version": 150010, "xlog": {"location": 487311252208}, "timeline": 52, "dcs_last_seen": 1760594610, "database_system_identifier": "7276070543149191243", "patroni": {"version": "3.3.3", "scope": "entre-vos-mains--de-pg", "name": "entre-vos-mains--de-pg-1"}}

Patronictl list gave for the replica:

in archive recovery

I wonder what happened.

restarting the pod worked.

I noticed it with increase cpu usage on our object storage ingress.

I think it would be nice to show the CRD status to something like degraded in this case.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions