Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ Currently supported PostgreSQL versions: 9.3 to 17.
tools_integration
security
ha_multi_dc
multisite
faq
releases
CONTRIBUTING
Expand Down
117 changes: 59 additions & 58 deletions docs/multisite.rst
Original file line number Diff line number Diff line change
Expand Up @@ -91,48 +91,49 @@ The configuration is very similar to the usual Patroni config. In fact, the key

An example configuration for two Patroni sites:

```
multisite:
name: dc1
namespace: /multisite/
etcd3: # <DCS>
hosts:
# dc1
- 10.0.1.1:2379
- 10.0.1.2:2379
- 10.0.1.3:2379
# dc2
- 10.0.2.1:2379
- 10.0.2.2:2379
- 10.0.2.3:2379
# dc 3
- 10.0.0.1:2379
host: 10.0.1.1,10.0.1.2,10.0.1.3 # How the leader of the other site(s) can connect to the primary on this site
port: 5432
# Multisite failover timeouts
ttl: 90
retry_timeout: 40
```
.. code:: YAML

multisite:
name: dc1
namespace: /multisite/
etcd3: # <DCS>
hosts:
# dc1
- 10.0.1.1:2379
- 10.0.1.2:2379
- 10.0.1.3:2379
# dc2
- 10.0.2.1:2379
- 10.0.2.2:2379
- 10.0.2.3:2379
# dc 3
- 10.0.0.1:2379
host: 10.0.1.1,10.0.1.2,10.0.1.3 # How the leader of the other site(s) can connect to the primary on this site
port: 5432
# Multisite failover timeouts
ttl: 90
retry_timeout: 40


Details of the configuration parameters
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

`name`
: The name of the site. All nodes that share the same value are considered to be a part of the same site, thus it must be different for each site.
`namespace`
: Optional path within DCS where Patroni stores the multisite state. If used, it should be different from the namespace used by the base config, but the same on all sites.
`<DCS>` (in the example `etcd3`)
: The DCS implementation in use. Possible values are `etcd`, `etcd3`, `zookeeper`, `consul`, `exhibitor`, `kubernetes`, or `raft` (the latter is deprecated).
`<DCS>.hosts`
: a list of IP addresses of nodes forming the global DCS cluster, including the extra (tiebreaking) node(s)
`host`
: Comma-separated list of IPs of the Patroni nodes that can become a primary on the present site
`port`
: Postgres port, through which other sites' members can connect to this site. It can be specified once if all nodes use the same port, or as a comma-separated list matching the different port numbers, in the order used in the `host` key.
`ttl`
: Time to live of site leader lock. If the site is unable to elect a functioning leader within this timeout, a different site can take over the leader role. Must be a few times longer than the usual `ttl` value in order to prevent unnecessary site failovers.
`retry_timeout`
: How long the global etcd cluster can be inaccessible before the cluster is demoted. Must be a few times longer than the usual `retry_timeout` value in order to prevent unnecessary site failovers.
``name``
The name of the site. All nodes that share the same value are considered to be a part of the same site, thus it must be different for each site.
``namespace``
Optional path within DCS where Patroni stores the multisite state. If used, it should be different from the namespace used by the base config, but the same on all sites.
``<DCS>`` (in the example ``etcd3``)
The DCS implementation in use. Possible values are ``etcd``, ``etcd3``, ``zookeeper``, ``consul``, ``exhibitor``, ``kubernetes``, or ``raft`` (the latter is deprecated).
``<DCS>.hosts``
a list of IP addresses of nodes forming the global DCS cluster, including the extra (tiebreaking) node(s)
``host``
Comma-separated list of IPs of the Patroni nodes that can become a primary on the present site
``port``
Postgres port, through which other sites' members can connect to this site. It can be specified once if all nodes use the same port, or as a comma-separated list matching the different port numbers, in the order used in the ``host`` key.
``ttl``
Time to live of site leader lock. If the site is unable to elect a functioning leader within this timeout, a different site can take over the leader role. Must be a few times longer than the usual ``ttl`` value in order to prevent unnecessary site failovers.
``retry_timeout``
How long the global etcd cluster can be inaccessible before the cluster is demoted. Must be a few times longer than the usual ``retry_timeout`` value in order to prevent unnecessary site failovers.

Passwords in the YAML configuration
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Expand Down Expand Up @@ -184,23 +185,23 @@ Applications should be ready to try to connect to the new primary. See 'Connect
Glossary
++++++++

DCS
: distributed configuration store
site
: a Patroni cluster with any number of nodes, and the respective DCS - usually corresponding to a data centre
primary
: the writable PostgreSQL node, from which the other nodes replicate their data (either directly or in a cascading fashion)
leader
: the node which other nodes inside the same site replicate from - the leader can be a replica itself, in which case it's called a _standby leader_
site switchover
: a (manual) leader site switch performed when both sites are functioning fine
site failover
: when the main site goes down (meaning there is no Patroni leader and none of the remaining nodes (if any left) can become a leader), the standby leader will be promoted, becoming a leader proper, and the Postgres instance running there becoming the primary
leader site
: the site where the PostgreSQL primary instance is
standby site
: a site replicating from the leader site, and a potential target for site switchover/failover
DCS quorum
: more than half of the DCS nodes are available (and can take part in a leader race)
multisite leader lock
: just like under normal Patroni operation, the leader puts/updates an entry in DCS, thus notifying other sites that there is a functioning Postgres primary running. The entry mentioned is the multisite leader lock.
**DCS**
distributed configuration store
**site**
a Patroni cluster with any number of nodes, and the respective DCS - usually corresponding to a data centre
**primary**
the writable PostgreSQL node, from which the other nodes replicate their data (either directly or in a cascading fashion)
**leader**
the node which other nodes inside the same site replicate from - the leader can be a replica itself, in which case it's called a *standby leader*
**site switchover**
a (manual) leader site switch performed when both sites are functioning fine
**site failover**
when the main site goes down (meaning there is no Patroni leader and none of the remaining nodes (if any left) can become a leader), the standby leader will be promoted, becoming a leader proper, and the Postgres instance running there becoming the primary
**leader site**
the site where the PostgreSQL primary instance is
**standby site**
a site replicating from the leader site, and a potential target for site switchover/failover
**DCS quorum**
more than half of the DCS nodes are available (and can take part in a leader race)
**multisite leader lock**
just like under normal Patroni operation, the leader puts/updates an entry in DCS, thus notifying other sites that there is a functioning Postgres primary running. The entry mentioned is the multisite leader lock.
6 changes: 3 additions & 3 deletions patroni/api.py
Original file line number Diff line number Diff line change
Expand Up @@ -681,7 +681,7 @@ def do_GET_metrics(self) -> None:
metrics.append("# HELP patroni_multisite_switches Number of times multisite leader has been switched")
metrics.append("# TYPE patroni_multisite_switches counter")
metrics.append("patroni_multisite_switches{0} {1}"
.format(labels, patroni.multisite.site_switches))
.format(labels, patroni.multisite.site_switches)) # noqa: E501 # pyright: ignore [reportUnknownMemberType, reportUnknownArgumentType, reportAttributeAccessIssue]

self.write_response(200, '\n'.join(metrics) + '\n', content_type='text/plain')

Expand Down Expand Up @@ -1199,13 +1199,13 @@ def do_POST_switchover(self) -> None:
self.do_POST_failover(action='switchover')

@check_access
def do_POST_site_switchover(self):
def do_POST_site_switchover(self) -> None:
request = self._read_json_content()
(status_code, data) = (400, '')
if not request:
return
if not self.server.patroni.multisite.is_active:
return self._write_response(400, 'Cluster is not in multisite mode')
return self.write_response(400, 'Cluster is not in multisite mode')

scheduled_at = request.get('scheduled_at')
target_site = request.get('target_site')
Expand Down
7 changes: 4 additions & 3 deletions patroni/ha.py
Original file line number Diff line number Diff line change
Expand Up @@ -282,7 +282,7 @@ def is_standby_cluster(self) -> bool:
return self.patroni.multisite.is_active and not self.patroni.multisite.is_leader_site() \
or global_config.is_standby_cluster

def get_standby_cluster_config(self):
def get_standby_cluster_config(self) -> Any:
if self.patroni.multisite.is_active:
return self.patroni.multisite.get_active_standby_config()
return global_config.get_standby_cluster_config()
Expand Down Expand Up @@ -359,7 +359,8 @@ def acquire_lock(self) -> bool:
multisite_ret = self.patroni.multisite.resolve_leader()
if multisite_ret:
logger.error("Releasing leader lock because multi site status is: " + multisite_ret)
self.dcs.delete_leader()
# self.dcs.delete_leader()
self._delete_leader()
return False
return ret

Expand Down Expand Up @@ -1604,7 +1605,7 @@ def on_shutdown(checkpoint_location: int, prev_location: int) -> None:
status['released'] = True

if mode == 'multisite':
on_shutdown = self.patroni.multisite.on_shutdown # noqa: F811
on_shutdown = self.patroni.multisite.on_shutdown # pyright: ignore [reportAssignmentType] # noqa: F811

def before_shutdown() -> None:
if self.state_handler.mpp_handler.is_coordinator():
Expand Down
Loading
Loading