Discussion:
Dependencies on DBus activated services during shutdown
Add Reply
Michal Koutný
2018-01-08 15:07:48 UTC
Reply
Permalink
Raw Message
Hello,
I'd like to ask your opinion on the following situation.

B.service exposes its API through D-Bus. A.service uses this API and
thus it has a dependency on B.service. This is implicit though -- and
we're happy we can rely on D-Bus activation and needn't to list all
dependencies explicitly.

As it comes, A.service needs B.service for proper termination. During
the shutdown transaction there is unspecified ordering of the two (since
the dependency is implicit only) and B.service is stopped before A.service.

A.service would attempt to D-Bus-activate B.service but that is rejected
because dbus-daemon will eventually stop too. Note this doesn't mean
dbus-daemon is already handling SIGTERM, it's because a dbus-daemon stop
job is pending [1]. A.service may thus cannot terminate properly.

I know this could be circumvented by explicitly specifying
After=b.service for the A.service but denies the elegance of the lazy
(implicit) activation.

Are there any better ways how to deal with this?

Thanks,
Michal KoutnÜ

P.S. FTR, in my case A.service=libvirtd.service and
B.service=systemd-machined.service.

[1] https://github.com/systemd/systemd/blob/master/src/core/dbus.c#L169
Jérémy Rosen
2018-01-08 15:53:04 UTC
Reply
Permalink
Raw Message
well, afaict from your description the information "A needs B" is
totally implicit, there is no place where this info is available, so no
way to pass that info to systemd.

That means that the only way to fix that without explicitely telling
someone about the dependency is to allow dbus to start units while its
shutdown is pending

this seems to be explicitely forbidden, so my guess is that there are
very good reasons for that and there is little chances that there is a
workaround...

so, i'm not an expert, but I would be very suprised if there was a way
to do that... an explicit "allow startup during termination" flag in the
auto-activation .service file could be a way around but I don't think
such file exists...


sorry
Post by Michal Koutný
Hello,
I'd like to ask your opinion on the following situation.
B.service exposes its API through D-Bus. A.service uses this API and
thus it has a dependency on B.service. This is implicit though -- and
we're happy we can rely on D-Bus activation and needn't to list all
dependencies explicitly.
As it comes, A.service needs B.service for proper termination. During
the shutdown transaction there is unspecified ordering of the two (since
the dependency is implicit only) and B.service is stopped before A.service.
A.service would attempt to D-Bus-activate B.service but that is rejected
because dbus-daemon will eventually stop too. Note this doesn't mean
dbus-daemon is already handling SIGTERM, it's because a dbus-daemon stop
job is pending [1]. A.service may thus cannot terminate properly.
I know this could be circumvented by explicitly specifying
After=b.service for the A.service but denies the elegance of the lazy
(implicit) activation.
Are there any better ways how to deal with this?
Thanks,
Michal KoutnÜ
P.S. FTR, in my case A.service=libvirtd.service and
B.service=systemd-machined.service.
[1] https://github.com/systemd/systemd/blob/master/src/core/dbus.c#L169
_______________________________________________
systemd-devel mailing list
https://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
SMILE <http://www.smile.eu/>

20 rue des Jardins
92600 AsniÚres-sur-Seine


*Jérémy ROSEN*
Architecte technique
Responsable de l'expertise Smile-ECS

email ***@smile.fr <mailto:***@smile.fr>
phone +33141402967
url http://www.smile.eu

Twitter <https://twitter.com/GroupeSmile> Facebook
<https://www.facebook.com/smileopensource> LinkedIn
<https://www.linkedin.com/company/smile> Github
<https://github.com/Smile-SA>


Découvrez l’univers Smile, rendez-vous sur smile.eu
<http://smile.eu/?utm_source=signature&utm_medium=email&utm_campaign=signature>

eco Pour la planÚte, n'imprimez ce mail que si c'est nécessaire
Andrei Borzenkov
2018-01-08 19:04:48 UTC
Reply
Permalink
Raw Message
Post by Jérémy Rosen
well, afaict from your description the information "A needs B" is
totally implicit, there is no place where this info is available, so no
way to pass that info to systemd.
If systemd could infer that A requested B to be started it could also
add implicit ordering between A and B. I am not sure whether this
information is available when service is activated though.
Michal Koutný
2018-01-08 19:27:46 UTC
Reply
Permalink
Raw Message
Post by Andrei Borzenkov
If systemd could infer that A requested B to be started it could also
add implicit ordering between A and B.
Yes, this would be a way to the complete solution where all dependencies
are tracked (however, I'm not sure it's achievable). ActivationRequest
signal doesn't pass the requester identification and this would probably
track just the dependencies when the B.service isn't active yet.

Michal
Andrei Borzenkov
2018-01-08 19:35:30 UTC
Reply
Permalink
Raw Message
Post by Michal Koutný
Post by Andrei Borzenkov
If systemd could infer that A requested B to be started it could also
add implicit ordering between A and B.
Yes, this would be a way to the complete solution where all dependencies
are tracked
On the second thought you still need to handle situation when B is
already active when A gets started. Then you have no implicit
dependencies at all (no socket activation happens), but A still depends
on B during shutdown. So I'm afraid explicitly adding this information
to A is the only way.
Post by Michal Koutný
(however, I'm not sure it's achievable). ActivationRequest
signal doesn't pass the requester identification and this would probably
track just the dependencies when the B.service isn't active yet.
Michal
Simon McVittie
2018-01-08 16:52:04 UTC
Reply
Permalink
Raw Message
Post by Michal Koutný
As it comes, A.service needs B.service for proper termination. During
the shutdown transaction there is unspecified ordering of the two (since
the dependency is implicit only) and B.service is stopped before A.service.
Does A.service even need B.service during shutdown if it has not
previously interacted with (=> started) B.service?

Or is the issue that B.service handles requests from A.service, then shuts
down when unused (leaving "background" state that it's responsible for),
and during shutdown, A.service needs to wake B.service back up to undo
those requests?
Post by Michal Koutný
I know this could be circumvented by explicitly specifying
After=b.service for the A.service
To be honest that doesn't seem too bad to me.
Post by Michal Koutný
That means that the only way to fix that without explicitely telling someone
about the dependency is to allow dbus to start units while its shutdown is
pending [...] this seems to be explicitely forbidden
I didn't write that special case, but I agree with it. Starting D-Bus
services while the sword of Damocles is hanging over dbus-daemon's
head does not sound like a route to guaranteed success. As soon as
dbus-daemon gets SIGTERM, they'll find that their D-Bus AF_UNIX socket
is rather less useful than it was a moment ago...

If A.service can be made to shut down correctly without B.service,
then that seems good in any case. (What happens if B.service crashes?)

Failing that, if A.service genuinely needs D-Bus during its shutdown,
it probably also makes sense to serialize it After=dbus.service (not
just dbus.socket) so that dbus-daemon will be kept alive until A.service
actually exits.

If systemd had an asymmetric StopBefore=dbus.service, that could be used
here (and also for <https://bugs.freedesktop.org/show_bug.cgi?id=89847>);
but it doesn't (and I can see why the systemd maintainers want to preserve
the symmetry of startup/shutdown order), and After=dbus.service is
the next best thing. I agree with Lennart's assertion on that bug that
having ExecStop pretend to stop dbus-daemon, but not actually do it,
is the wrong answer.

smcv
Michal Koutný
2018-01-08 19:58:57 UTC
Reply
Permalink
Raw Message
Post by Simon McVittie
Does A.service even need B.service during shutdown if it has not
previously interacted with (=> started) B.service?
That's not correct implication, due to concurrency, B.service may be
stopped before A.service.
Post by Simon McVittie
Post by Michal Koutný
That means that the only way to fix that without explicitely telling someone
about the dependency is to allow dbus to start units while its shutdown is
pending [...] this seems to be explicitely forbidden
I didn't write that special case, but I agree with it. Starting D-Bus
services while the sword of Damocles is hanging over dbus-daemon's
head does not sound like a route to guaranteed success. As soon as
dbus-daemon gets SIGTERM, they'll find that their D-Bus AF_UNIX socket
is rather less useful than it was a moment ago...
Nice explanation. I forgot to mention that A.service already has
dependency on dbus.service (which is unnecessary by the elegance
criterium). So there's a guarantee two sword will not fall down until
A.service terminates.
Post by Simon McVittie
If A.service can be made to shut down correctly without B.service,
then that seems good in any case. (What happens if B.service crashes?)
The A's best effort would make sense in the crash case, however, I don't
think it's the right place .
Post by Simon McVittie
Failing that, if A.service genuinely needs D-Bus during its shutdown,
it probably also makes sense to serialize it After=dbus.service (not
just dbus.socket) so that dbus-daemon will be kept alive until A.service
actually exits.
That's actually a valid idea (see above).
Post by Simon McVittie
To be honest that doesn't seem too bad to me.
Yes, the lesser viability of other options I see, the more I like it as
well.

The alternative of changing the condition from
manager_unit_inactive_or_pending to unit_inactive would AFAIU lead to a
conflict with the shutdown target if B.service was reactivated. (And if
not, it could potentially make shutdown transaction infinite.)

Thanks,
Michal
Lennart Poettering
2018-01-24 11:48:09 UTC
Reply
Permalink
Raw Message
Post by Michal Koutný
Hello,
I'd like to ask your opinion on the following situation.
B.service exposes its API through D-Bus. A.service uses this API and
thus it has a dependency on B.service. This is implicit though -- and
we're happy we can rely on D-Bus activation and needn't to list all
dependencies explicitly.
As it comes, A.service needs B.service for proper termination. During
the shutdown transaction there is unspecified ordering of the two (since
the dependency is implicit only) and B.service is stopped before A.service.
If you need it for proper termination you should declare the dep
really.

My general recommendation would be though to write daemons in a
fashion that they do not require any other daemons to shut down in an
orderly fashion. It's a matter of general robustness I guess: if your
daemon is written in a way that it basically can die abruptly any time
without leaving unclean state around then this also means that
shutting it down always works and has no dependencies. That said, I am
full aware that it might not be possible to write all daemons in this
fashion...

Lennart
--
Lennart Poettering, Red Hat
Loading...