google compute startup script starting postgresql

0

votes

I have a startup script set by an instance template that initializes the server for google compute. After installing postgres, I manually call for it to start using :

/etc/init.d/postgresql start

This completes successfully, but the server is not listening on 5432 when run by the startup script (postgres isn't started, although that service start call completes successfully). After startup completes, and I log in, I can do it successfully. Anyone know why that won't work within the startup script ? I need to load up data during startup so I need to startup postgres during initialization.

google-cloud-platformgoogle-compute-engine

Are you using CloudSQL for PostgreSQL or installing PostgreSQL on top of your GCE VM, or are you just trying to connect to a PostgreSQL server only? Also, can you share the startup script that you been using and the output of serial console? – Taher

Ok so I think I figured it out. Was using an older debian that I thought was stable but was apparently a bit too old. Using a newer image in my template, postgres came up as expected. Will close this now. – aao

Hi aao, can you post your solution as an answer to this question which would be helpful for the community and also will have better visibility to someone who has a similar issue in future? – Taher

1

votes

I Solved using newer debian image

0

votes

I had the same problem as you (installing postgresql in GCE startup-script results in the package being installed, but the server is not running), and I think I figured out the root cause.

Normally, the postgresql-11 package is supposed to start the PostgreSQL server after installation. Here is a snippet from its postinst script:

if [ "$1" = configure ]; then
    . /usr/share/postgresql-common/maintscripts-functions

    configure_version $VERSION "$2"
fi

Taking a look at /usr/share/postgresql-common/maintscripts-functions, we see:

configure_version() {
    ...
    # reload systemd to let the generator pick up the new unit
    if [ -d /run/systemd/system ]; then
        systemctl daemon-reload
    fi
    invoke-rc.d postgresql start $VERSION # systemd: argument ignored, starts all versions
}

My debian installation comes with init-system-helpers version "1.56+nmu1", which contains this bit of code in invoke-rc.d:

# avoid deadlocks during bootup and shutdown from units/hooks
# which call "invoke-rc.d service reload" and similar, since
# the synchronous wait plus systemd's normal behaviour of
# transactionally processing all dependencies first easily
# causes dependency loops
if ! systemctl --quiet is-active multi-user.target; then
  sctl_args="--job-mode=ignore-dependencies"
fi
case $saction in
  start|restart|try-restart)
      [ "$_state" != "LoadState=masked" ] || exit 0
      systemctl $sctl_args "${saction}" "${UNIT}" && exit 0
      ;;

The debian postgresql-11 package makes use of templated systemd units. The main one is called postgresql.service but this is a dummy service that doesn't actually do anything. The PostgreSQL server is actually started by a templated unit named postgresql@11-main which is usually started alongside the main service because it has ReloadPropagatedFrom=postgresql.service.

Note that when this issue occurs, the main unit is started but the templated one is not:

$ sudo systemctl status postgresql
● postgresql.service - PostgreSQL RDBMS
   Loaded: loaded (/lib/systemd/system/postgresql.service; enabled; vendor preset: enabled)
   Active: active (exited) since Fri 2021-04-02 05:40:48 UTC; 32min ago
 Main PID: 1663 (code=exited, status=0/SUCCESS)
    Tasks: 0 (limit: 4665)
   Memory: 0B
   CGroup: /system.slice/postgresql.service

Apr 02 05:40:48 hubnext-west-r21r systemd[1]: Starting PostgreSQL RDBMS...
Apr 02 05:40:48 hubnext-west-r21r systemd[1]: Started PostgreSQL RDBMS.

$ sudo systemctl status postgresql@11-main
● [email protected] - PostgreSQL Cluster 11-main
   Loaded: loaded (/lib/systemd/system/[email protected]; enabled-runtime; vendor preset: enabled)
   Active: inactive (dead)

That's because when --job-mode=ignore-dependencies is specified, this link is ignored.

The GcE startup script runs as a systemd unit, which starts before multi-user.target is up:

$ find /etc/systemd | grep startup
/etc/systemd/system/multi-user.target.wants/google-startup-scripts.service

Therefore, invoke-rc.d notices that systemctl --quiet is-active multi-user.target is false and adds --job-mode=ignore-dependencies, which results in the PostgreSQL server not starting.

One possible workaround is explicitly running systemd start [email protected] from your startup script after installing postgres.

By the way, I noticed that a recent commit (Nov 2020) changed this invoke-rc.d behavior so that it no longer uses --job-mode=ignore-dependencies. That would help avoid this issue.

google compute startup script starting postgresql

2 Answers