Setting up core dumps and debuginfo packages with EDB TPA

[RAKAP02]

Summary

An example of setting up core dumps and debuginfo packages with EPAS v17 TPA

Where would you like to see this added?

https://www.enterprisedb.com/docs/tpa/latest/reference/sysctl_values/

Rationale

S-1 Config YAML

cluster_vars:
  ...
  postgres_coredump_filter: '0x33'
  postgres_flavour: epas
  epas_redwood_compat: false
  postgres_version: '17'
  ...

System-Level Verification

$ sudo sysctl kernel.core_pattern
kernel.core_pattern = core.%e.%p.%t
$ systemctl show -p LimitCORE postgres
LimitCORE=infinity
$ sudo sysctl kernel.core_pipe_limit
kernel.core_pipe_limit = 0

Simulating a Crash

$ ps auxf | grep postgres
admin      19589  0.0  0.0   3240   648 pts/0    S+   10:21   0:00              \_ grep postgres
enterpr+   18528  0.0  4.9 357628 48580 ?        Ss   08:04   0:00 /usr/lib/edb-as/17/bin/edb-postgres -D /opt/postgres/data -c config_file=/opt/postgres/data/postgresql.conf
$ sudo kill -SIGSEGV 18528 

Core Dump File Confirmation

sudo ls -als /opt/postgres/data/
total 283704
     4 drwx------ 21 enterprisedb enterprisedb      4096 Apr  9 10:21 .
     4 drwxr-xr-x  4 root         root              4096 Apr  9 08:03 ..
     4 -rw-------  1 enterprisedb enterprisedb         3 Apr  9 08:03 PG_VERSION
     4 drwx------  6 enterprisedb enterprisedb      4096 Apr  9 08:04 base
     4 drwx------  2 enterprisedb enterprisedb      4096 Apr  9 08:03 conf.d
283556 -rw-------  1 enterprisedb enterprisedb 291000320 Apr  9 10:21 core.edb-postgres.18528.1744194105

Stack Trace Extraction Using GDB

$ sudo gdb -q --batch -ex "bt full" /usr/lib/edb-as/17/bin/edb-postgres  /opt/postgres/data/core.edb-postgres.18528.1744194105 > /tmp/stacktrace.core.edb-postgres.18528.1744194105.txt 2>&1
$ cat /tmp/stacktrace.core.edb-postgres.18528.1744194105.txt
warning: Can't open file /dev/zero (deleted) during file-backed mapping note processing
warning: Can't open file /SYSV00180001 (deleted) during file-backed mapping note processing
[New LWP 18528]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `/usr/lib/edb-as/17/bin/edb-postgres -D /opt/postgres/data -c config_file=/opt/p'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00007f36a44a0db6 in epoll_wait (epfd=10, events=0x564ccae4d328, maxevents=4, timeout=60000) at ../sysdeps/unix/sysv/linux/epoll_wait.c:30
30  ../sysdeps/unix/sysv/linux/epoll_wait.c: No such file or directory.
#0  0x00007f36a44a0db6 in epoll_wait (epfd=10, events=0x564ccae4d328, maxevents=4, timeout=60000) at ../sysdeps/unix/sysv/linux/epoll_wait.c:30
No locals.
#1  0x0000564cca67cc7d in WaitEventSetWait ()
No symbol table info available.
#2  0x0000564cca609d66 in ?? ()
No symbol table info available.
#3  0x0000564cca60cc78 in PostmasterMain ()
No symbol table info available.
#4  0x0000564cca2cbfac in main ()
No symbol table info available.

Installed Packages:

$ sudo apt list *as17* --installed
Listing... Done
edb-as17-libpq-dev/bullseye,now 17.4.0-1.bullseye amd64 [installed,automatic]
edb-as17-libpq5/bullseye,now 17.4.0-1.bullseye amd64 [installed,automatic]
edb-as17-server-client/bullseye,now 17.4.0-1.bullseye amd64 [installed]
edb-as17-server-core/bullseye,now 17.4.0-1.bullseye amd64 [installed]
edb-as17-server-dev/bullseye,now 17.4.0-1.bullseye amd64 [installed]
edb-as17-server-doc/bullseye,now 17.4.0-1.bullseye all [installed]
edb-as17-server-plperl/bullseye,now 17.4.0-1.bullseye amd64 [installed]
edb-as17-server-plpython3/bullseye,now 17.4.0-1.bullseye amd64 [installed]
edb-as17-server-pltcl/bullseye,now 17.4.0-1.bullseye amd64 [installed]
edb-as17-server/bullseye,now 17.4.0-1.bullseye amd64 [installed]
edb-bdr5-epas17-dbgsym/bullseye,now 4:5.7.0-1.bullseye amd64 [installed]
edb-bdr5-epas17/bullseye,now 4:5.7.0-1.bullseye amd64 [installed]

Note: If you are planning to test on a Docker platform, please note that TPA does not configure core dump settings automatically for containers. Additional configuration might be required at the container runtime level.Ensure that postgres_coredump_filter is correctly set (e.g., '0x33') to allow meaningful dumps. Using the default '0xff' will suppress all core dumps.