Discussion:
Hints for upgrading systemd on a running system
(too old to reply)
Reindl Harald
2018-02-20 19:06:05 UTC
Permalink
Raw Message
Dear systemd folks,
We finally are going to upgrade from a very old systemd version 27 from
2011 to the current systemd v237. (Historical reasons.)
hopefully you have a working backup
Anyway, I already was told about `systemctl daemon-reexec`, and we got
it working.
the "reexec" is misleading because it's not possible to terminate PID1
and start it again on a running system with a new binary
After that, looking at the output of `systemctl`, there are many units
from the old version, which were removed in the meantime.
i doubt that anybody has done such a version jump at all and even if the
environment won't match anyways - you need to reboot and hope it works
as well as be prepared for the case it won't boot - it's that easy

make at least sure that initrd has been updated with the new systemd!
$ systemctl --state=not-found
  UNIT                                 LOAD      ACTIVE   SUB DESCRIPTION
● dev-hugepages.automount              not-found active   waiting
dev-hugepages.automount
● dev-mqueue.automount                 not-found active   waiting
dev-mqueue.automount
● sys-kernel-debug.automount           not-found active   waiting
sys-kernel-debug.automount
● sys-kernel-security.automount        not-found active   waiting
sys-kernel-security.automount
● auditd.service                       not-found inactive dead
auditd.service
● console-kit-log-system-start.service not-found active   exited
console-kit-log-system-start.service
● display-manager.service              not-found inactive dead
display-manager.service
● hwclock-load.service                 not-found active   exited
hwclock-load.service
● plymouth-quit-wait.service           not-found inactive dead
plymouth-quit-wait.service
● plymouth-start.service               not-found inactive dead
plymouth-start.service
● remount-rootfs.service               not-found active   exited
remount-rootfs.service
● syslog.service                       not-found inactive dead
syslog.service
● systemd-kmsg-syslogd.service         not-found active   running
systemd-kmsg-syslogd.service
● systemd-remount-api-vfs.service      not-found active   exited
systemd-remount-api-vfs.service
● systemd-sysusers.service             not-found inactive dead
systemd-sysusers.service
● udev-retry.service                   not-found active   exited
udev-retry.service
● udev-settle.service                  not-found active   exited
udev-settle.service
● systemd-logger.socket                not-found active   listening
systemd-logger.socket
● systemd-shutdownd.socket             not-found active   listening
systemd-shutdownd.socket
● cryptsetup.target                    not-found active   active
cryptsetup.target
● syslog.target                        not-found active   active
syslog.target
LOAD   = Reflects whether the unit definition was properly loaded.
ACTIVE = The high-level unit activation state, i.e. generalization of SUB.
SUB    = The low-level unit activation state, values depend on unit type.
21 loaded units listed. Pass --all to see loaded but inactive units, too.
To show all installed unit files use 'systemctl list-unit-files'.
```
Do I need to stop those manually beforehand, or is there another way to
clean up?
Is the recommended update procedure documented somewhere?
Michael Biebl
2018-02-20 19:09:37 UTC
Permalink
Raw Message
Do I need to stop those manually beforehand, or is there another way to
clean up?
reboot.
--
Why is it that all of the instruments seeking intelligent life in the
universe are pointed away from Earth?
Mantas Mikulėnas
2018-02-20 21:04:33 UTC
Permalink
Raw Message
Post by Reindl Harald
Dear systemd folks,
We finally are going to upgrade from a very old systemd version 27 from
2011 to the current systemd v237. (Historical reasons.)
hopefully you have a working backup
Anyway, I already was told about `systemctl daemon-reexec`, and we got
it working.
the "reexec" is misleading because it's not possible to terminate PID1
and start it again on a running system with a new binary
But you don't *have to* terminate pid1 to change the binary – you can just
exec the new one. Hence the "reexec".

It's possible. That's how initramfs works.
Post by Reindl Harald
--
Mantas Mikulėnas <***@gmail.com>
Sent from my phone
Reindl Harald
2018-02-20 21:12:38 UTC
Permalink
Raw Message
Dear systemd folks,
We finally are going to upgrade from a very old systemd version
27 from
2011 to the current systemd v237. (Historical reasons.)
hopefully you have a working backup
Anyway, I already was told about `systemctl daemon-reexec`, and
we got
it working.
the "reexec" is misleading because it's not possible to terminate PID1
and start it again on a running system with a new binary
But you don't *have to* terminate pid1 to change the binary – you can
just exec the new one. Hence the "reexec".
It's possible. That's how initramfs works
a lot of other threads here made it clear that there is not much
difference between "daemon-reload" and "daemon-reexec" at all in the
past and i strongly doubt that it's technically possible to seamless
switch from version 27 (the first Fedora GA AFAIK hat version 44) to 237
and keep the system happily running as like nothing has changed without
reboot
Lennart Poettering
2018-02-20 22:12:37 UTC
Permalink
Raw Message
Dear systemd folks,
We finally are going to upgrade from a very old systemd version 27 from 2011
to the current systemd v237. (Historical reasons.)
Anyway, I already was told about `systemctl daemon-reexec`, and we got it
working.
While we try to ensure that live upgrades of PID 1 like that work
quite well, this is generally tested only for small steps. Jumping 6
years ahead in one go is not something people typically test.
After that, looking at the output of `systemctl`, there are many units from
the old version, which were removed in the meantime.
```
$ systemctl --state=not-found
UNIT LOAD ACTIVE SUB DESCRIPTION
● dev-hugepages.automount not-found active waiting
dev-hugepages.automount
● dev-mqueue.automount not-found active waiting
dev-mqueue.automount
● sys-kernel-debug.automount not-found active waiting
sys-kernel-debug.automount
● sys-kernel-security.automount not-found active waiting
sys-kernel-security.automount
● auditd.service not-found inactive dead
auditd.service
● console-kit-log-system-start.service not-found active exited
console-kit-log-system-start.service
● display-manager.service not-found inactive dead
display-manager.service
● hwclock-load.service not-found active exited
hwclock-load.service
● plymouth-quit-wait.service not-found inactive dead
plymouth-quit-wait.service
● plymouth-start.service not-found inactive dead
plymouth-start.service
● remount-rootfs.service not-found active exited
remount-rootfs.service
● syslog.service not-found inactive dead
syslog.service
● systemd-kmsg-syslogd.service not-found active running
systemd-kmsg-syslogd.service
● systemd-remount-api-vfs.service not-found active exited
systemd-remount-api-vfs.service
● systemd-sysusers.service not-found inactive dead
systemd-sysusers.service
● udev-retry.service not-found active exited
udev-retry.service
● udev-settle.service not-found active exited
udev-settle.service
● systemd-logger.socket not-found active listening
systemd-logger.socket
● systemd-shutdownd.socket not-found active listening
systemd-shutdownd.socket
● cryptsetup.target not-found active active
cryptsetup.target
● syslog.target not-found active active
syslog.target
My recommendation: simply reboot. That should clean up everything
properly.

Note that PID 1 itself is probably pretty Ok with such a massive
update in one step, but the unit files have been rearranged quite a
bit since then. Downstream distributions generally expect you to
reboot even between single-step distro updates, but this becomes much
more of a necessity if you jump even further.

Note that systemd upstream currently requires kernel 3.13 at least
which was released in 2014. Hence, if you update from a 2011 system
you have to reboot anyway, already to update the kernel...
Do I need to stop those manually beforehand, or is there another way to
clean up?
Is the recommended update procedure documented somewhere?
Usually distributions document that invididually as systemd is just
one component of many that make up the distribution.

Lennart
--
Lennart Poettering, Red Hat
Lennart Poettering
2018-02-22 09:55:25 UTC
Permalink
Raw Message
Post by Lennart Poettering
Note that PID 1 itself is probably pretty Ok with such a massive
update in one step, but the unit files have been rearranged quite a
bit since then. Downstream distributions generally expect you to
reboot even between single-step distro updates, but this becomes much
more of a necessity if you jump even further.
But if reboot wouldn’t be an option, is there a way to get rid of not-found
services?
stop them.

How did you upgrade your kernel without rebooting?

Note that there's also stuff such as dbus-daemon which cannot be
updated without rebooting.

Lennart
--
Lennart Poettering, Red Hat
Lennart Poettering
2018-02-20 22:30:50 UTC
Permalink
Raw Message
Post by Reindl Harald
Dear systemd folks,
We finally are going to upgrade from a very old systemd version 27 from
2011 to the current systemd v237. (Historical reasons.)
hopefully you have a working backup
Anyway, I already was told about `systemctl daemon-reexec`, and we got
it working.
the "reexec" is misleading because it's not possible to terminate PID1 and
start it again on a running system with a new binary
Hmm? "systemctl daemon-reexec" is supposed to support upgrades between
systemd versions. Doing massive jumps v27 → v237 in one go isn't
precisely well tested, but smaller ones should work quite well.

Lennart
--
Lennart Poettering, Red Hat
Loading...