Simon McVittie
2015-01-30 08:30:19 UTC
[For those who are there, I'll be at the system hackfest today and at
FOSDEM this weekend, so if you are interested in these topics, please
talk to me about them; I'll try to summarize discussion to these lists.
For those not there, I'll try to keep up with responses via email and
raise any interesting points in person.]
I've recently been looking into D-Bus' interaction with X session
startup, systemd --user, upstart --user, and the definition of a
session. Every few months or years there is an attempt to sort out
whether D-Bus has a user bus or a session bus or what. Here is another,
with particular reference to systemd and potentially kdbus.
Terminology
===========
login session
-------------
A *login session* is as follows: when you log in to *dm, that starts a
*graphical login session* which lasts until you log out. When you log in
on a virtual console or via ssh or something, that starts a
*non-graphical login session*. Background processes like cron or screen
might also be non-graphical login sessions. A non-graphical login
session might become graphical by running startx or similar.
In a systemd-logind environment, login sessions have an ID which is
made available to in-session processes in the XDG_SESSION_ID environment
variable. If the Linux "auditing" feature is enabled, the XDG_SESSION_ID
equals the audit session ID (/proc/self/sessionid). If not,
there is no audit session ID, so systemd-logind makes up a sequential
XDG_SESSION_ID and uses that.
seat
----
A seat is a set of displays, input devices etc. attached to a machine.
Most machines have one seat "seat0", which occupies all the devices.
By attaching additional devices via something like
http://plugable.com/products/ud-160-m you can construct a
multi-seat machine.
user-session
------------
I don't think there is a standard term for this so I'm making one up.
The XDG_RUNTIME_DIR specification says that there is at most one
XDG_RUNTIME_DIR per uid per machine, that it is created at the beginning
of the user's first login session, and that it is removed at the end of
a login session if no other login sessions remain for that uid.
systemd-logind implements those semantics, and also runs a `systemd --user`
for the lifetime of the user-session. Ubuntu previously used
libpam-xdg-runtime to provide compatible semantics without systemd.
The resulting situation looks something like this (assume all the
sessions shown are under the same uid):
--------\ ========\\
login | user- ||
session | --------\ session || <- overlapping login session c42
23 | login | || shares the user-session with 23
--------/ session | || <- 23 ends but c42 keeps u.-session
c42 | || alive
--------\ | || <- 57 begins, shares u.-sess.
login | --------/ || <- c42 ends, 57 keeps user-session
session | || alive
57 | ||
--------/ ========// no more l.sessions, u.-session ends
--------\ ========\\ new l.session, u.-session starts
login | user- ||
session | session ||
... . ... ..
user-sessions do not have any meaningful identifier apart from their
owner's uid (strictly speaking, the owner's "loginuid", i.e. the user
who initially logged in, even if they su). They do not need another
identifier, because by definition there is only one per uid at a time.
Impact on D-Bus
===============
There are two models for how the D-Bus session bus, which is designed to
be per "high-level session" (whatever that means), can work in a world
with user-sessions.
Last time this was discussed, Havoc had the useful insight that this is
not really a question about D-Bus, it's a question about how you build
OSs, and in particular how you model sessions.
(Up to) one high-level session per login
----------------------------------------
The first model, which is how it traditionally worked (before there was
such a thing as a user-session), is that each graphical login session is
declared to be a "session", and non-graphical login sessions are pretty
much swept under the carpet and forgotten about.
In graphical sessions, vaguely modern Unix OSs typically know how to
start up a dbus-daemon during the creation of a graphical session (e.g.
in Debian and derivatives it's started by /etc/X11/Xsession.d, and
Fedora derivatives have a similar setup under a different name). If they
don't, modern desktop environments also know how to start a dbus-daemon
if they need one (e.g. gnome-session does this for GNOME), and if *that*
doesn't start one (the "I use Firefox under fvwm" use-case), we have a
slightly shaky but functional "autolaunch" mechanism based on X11
properties.
In principle, a PAM module or something could ensure that we have a
dbus-daemon per login session, even tty/ssh/cron login sessions
(which all go through PAM). In practice, nobody has ever cared enough to
implement this, so we're left with D-Bus autolaunch, which can't
actually work for tty sessions and had bad side-effects from its
attempts to do so, so I disabled it 3 years ago in favour of
recommending that users requiring a D-Bus session should start their own
and manage its lifetime themselves e.g. with dbus-run-session(1).
If people want to put work into this model, we could do a lot better
than we do now; for instance, the bus socket could be
unix:path=${XDG_RUNTIME_DIR}/sessions/${XDG_SESSION_ID}/bus (but with
the necessary escaping) on systems where those variables are set, rather
than messing about with $TMPDIR.
However, the people doing the work have a somewhat strong correlation
with the people who want a different model, for which read on...
Digression: (at least some) users and developers don't want that
----------------------------------------------------------------
One major issue with one high-level session (and hence one session bus)
per login session is that it places an artifical barrier between login
sessions. It's difficult for cron jobs, ssh logins, tty logins to share
non-GUI backend services such as dconf, Rygel, Telepathy: on my NAS box,
which runs Rygel with no GUI to export my music in a way the
notoriously picky PS3 system software can understand, I had to
construct a D-Bus session to let Rygel and Tracker components talk
to each other. Happily, I had already written dbus-run-session(1),
but I shouldn't need to do this rubbish to let non-GUI daemons talk
to each other - it should just work.
Similarly, if you leave a screen/tmux session detached and running
in the background, systemd is fine with that: its view of the world
is that there are processes left over from your previous login session,
keeping the login session alive in "closing" state, with the consequence
that the user-session remains alive too (unless you have configured
systemd-logind to kill leftover processes, as required by the sysadmins
of computer labs and other massively multi-user machines, in which case
you don't get to run screen sessions and that's by design).
However, the login-session-centric model can't cope with that:
the dbus-daemon dies with the X server, the D-Bus services die with
the dbus-daemon, and now the duplicity(1) process in your screen session
can't use gvfs to upload your backups to a server (or whatever).
Another issue is that it is an extremely poor match for how
`systemd --user` works (although apparently a better match for
how Ubuntu uses `upstart --user`, which I hope to discuss with Ubuntu
people at the systemd hackfest today / at FOSDEM) - you can't use
systemd to manage your login-session-bound D-Bus services, because they
all share a systemd, which doesn't know which login session wants which
service.
I think it's telling that, in recent discussion with Ubuntu developers
about their use of one `upstart --user` per login session, they
clarified that they only actually do this for *graphical* login
sessions, and they provide tools to let a developer copy the environment
variables from what, conceptually, should be another session,
effectively "joining" that session for a subtree of processes. I think
this proves the point that Robert McQueen made on the subject back in 2006:
"It's already been discussed multiple times on the list and
apparently a user bus is too confusing or not really useful unless
you make it a user bus shared across a network cluster. I don't
agree, and I think that the consequence of this stance will be a
proliferation of hacks like the one I just sketched out (not to
mention even grosser hacks like Ubuntu's ACPI scripts grepping the
environment of the user's processes in /proc to find their session
bus to tell their screensaver to lock)."
One high-level session per user-session
---------------------------------------
As a result of the problems noted above, various people, most vocally
the systemd and GNOME developers, have advocated a different model
for how to model sessions in the OS - what I called "user-sessions"
above. A typical implementation of that idea forbids multiple graphical
login sessions altogether: if you type your username and password into
gdm while already logged in on another virtual console, it will just
vt-switch to your existing graphical login session and unlock it.
Under X11, that might well be the best we can do, because typical
X11 applications can't cope with being asked to put windows on more
than one $DISPLAY (although I've heard rumours that Emacs can, which
would make Emacs an ideal candidate to be a user-session service).
Under Wayland or similar future cleverness, hopefully there'll be
some way for a new login on a new seat to "steal" all the windows
from the inactive seat, or some way to merge both seats into
one big virtual display if the same person is using both (AIUI this
is what GNOME designers want to do), or some other clever solution.
What this means for D-Bus
-------------------------
This leaves the problem that, if you have chosen the
user-session-centric model, current D-Bus doesn't actually work very
well: we have dbus-launch and dbus-run-session for login-session-centric
D-Bus, but if you want to make D-Bus user-session-centric, you need to
resort to third-party user bus units like user-session-units, which
float around Github and somehow never make it upstream. I'm now on my
third unrelated work project that contains a copypaste of dbus.socket
and dbus.service, and I'm sure my colleagues have others - this is not
the flying car future I signed up for. So I would like to get this stuff
upstream into dbus.git.
So that the people who are happy with the complexities of the current
arrangement can remain happy, here is how I intend it to work:
* ./configure --disable-user-bus: you get a login-session-centric world
* ./configure --enable-user-bus: you get a user-session-centric world
* configure either way and selectively install/delete files: you can
either have a login-session-centric or user-session-centric world,
depending what you install. (This is so I can have dbus-user-bus and
dbus-x11 binary packages in Debian, where the answer to "do A or do
B?" is always "both!", without having to compile everything twice;
the GNOME metapackage could eventually depend on dbus-user-bus.)
I think in practice that design will mean that by default, libdbus tries
to connect to both user-session- and login-session-centric locations for
the bus (but the "wrong one" for this OS will fail and the "right one"
will succeed), while dbus-user-bus and dbus-x11 would arrange for the
bus to listen in the appropriate place for one option or the other so
that the appropriate version succeeds.
Conveniently, this is what I've already implemented (in two
different ways): "try the user-session-centric path first, and if that
fails, fall back to login-session-centric autolaunch". There is the
slight nit that the server side of user-session-centric operation
requires systemd, although that could be argued to be a feature: the
other reasonable implementation is libpam-xdg-runtime, which AIUI is
unmaintained now that Ubuntu has switched to systemd-logind, and the
people who are avoiding systemd on general principles (hello Devuan!)
are probably the sort of people who are sufficiently change-averse to
prefer D-Bus to work the way it worked in 2005 (if they install it at
all). I'd be willing to consider implementations that work with other
service managers if their authors are willing to help maintain them.
The glorious(?) kdbus future
============================
kdbus is a proposed Linux kernel module providing
somewhat-D-Bus-compatible message-passing semantics (I'm being vague
because it's been 4 months since I last looked at it in detail - I hope
to return to it soon). To provide complete dbus-daemon-like
functionality, you also need a user-space process to create the bus and
manage activations; the
reference implementation, and so far the only implementation, is in
systemd.
The systemd developers who are working on the user-space counterpart
for kdbus have indicated that they have zero interest in supporting a
login-session-centric setup for session kdbus. In practice, I think
this means that the proponents of a login-session-centric world either
can't use kdbus in their sessions, or need to convince the relevant
developers that the added complexity of their model is worth it, or need
to write their own user-space for kdbus sessions and
disable systemd's. (It's not all bad - they can still use kdbus for
the system bus, which is already an improvement.)
Remaining issue: environment variables
======================================
Sadly, not all the issues have been resolved yet. The biggest is
environment variables: on existing systems there is an expectation
that environment variables set by *dm, PAM, or /etc/X11/Xsession.d
(or the non-Debian equivalent) will be propagated into every process
in the session. In a brief survey of about half the Xsession.d scripts
in Debian, I identified these environment variables:
* PULSE_SERVER
* ESPEAKER
* XDG_DATA_HOME, XDG_DATA_DIRS
* each variable printed by locale(1)
* VDPAU_DRIVER
* GTK_IM_MODULE, QT_IM_MODULE, QT4_IM_MODULE, CLUTTER_IM_MODULE,
XMODIFIERS
* GTK_MODULES
plus the obvious ones set by *dm, such as DISPLAY, or by PAM. Similarly,
a user's ~/.xsession can set arbitrary variables - mine sets CCACHE_DIR,
EDITOR, MPD_HOST and XDG_CONFIG_DIRS, among others.
The naive implementation would be to run a tool at the end of
/etc/X11/Xsession.d that uploads all of these into `dbus-daemon
--session` (if used) and into `systemd --user` (if used). However, not
all environment variables set in such scripts are suitable for that use:
notably, XDG_SESSION_ID from the login session should not be copied into
activated processes that exist outside any login session.
As a short-term solution, I'm tempted to write that tool, but make it
only upload a whitelisted set of variables automatically, and say "if
you install dbus-user-bus, you are expected to run this tool from your
~/.xsession if you need it".
Thoughts?
S
FOSDEM this weekend, so if you are interested in these topics, please
talk to me about them; I'll try to summarize discussion to these lists.
For those not there, I'll try to keep up with responses via email and
raise any interesting points in person.]
I've recently been looking into D-Bus' interaction with X session
startup, systemd --user, upstart --user, and the definition of a
session. Every few months or years there is an attempt to sort out
whether D-Bus has a user bus or a session bus or what. Here is another,
with particular reference to systemd and potentially kdbus.
Terminology
===========
login session
-------------
A *login session* is as follows: when you log in to *dm, that starts a
*graphical login session* which lasts until you log out. When you log in
on a virtual console or via ssh or something, that starts a
*non-graphical login session*. Background processes like cron or screen
might also be non-graphical login sessions. A non-graphical login
session might become graphical by running startx or similar.
In a systemd-logind environment, login sessions have an ID which is
made available to in-session processes in the XDG_SESSION_ID environment
variable. If the Linux "auditing" feature is enabled, the XDG_SESSION_ID
equals the audit session ID (/proc/self/sessionid). If not,
there is no audit session ID, so systemd-logind makes up a sequential
XDG_SESSION_ID and uses that.
seat
----
A seat is a set of displays, input devices etc. attached to a machine.
Most machines have one seat "seat0", which occupies all the devices.
By attaching additional devices via something like
http://plugable.com/products/ud-160-m you can construct a
multi-seat machine.
user-session
------------
I don't think there is a standard term for this so I'm making one up.
The XDG_RUNTIME_DIR specification says that there is at most one
XDG_RUNTIME_DIR per uid per machine, that it is created at the beginning
of the user's first login session, and that it is removed at the end of
a login session if no other login sessions remain for that uid.
systemd-logind implements those semantics, and also runs a `systemd --user`
for the lifetime of the user-session. Ubuntu previously used
libpam-xdg-runtime to provide compatible semantics without systemd.
The resulting situation looks something like this (assume all the
sessions shown are under the same uid):
--------\ ========\\
login | user- ||
session | --------\ session || <- overlapping login session c42
23 | login | || shares the user-session with 23
--------/ session | || <- 23 ends but c42 keeps u.-session
c42 | || alive
--------\ | || <- 57 begins, shares u.-sess.
login | --------/ || <- c42 ends, 57 keeps user-session
session | || alive
57 | ||
--------/ ========// no more l.sessions, u.-session ends
--------\ ========\\ new l.session, u.-session starts
login | user- ||
session | session ||
... . ... ..
user-sessions do not have any meaningful identifier apart from their
owner's uid (strictly speaking, the owner's "loginuid", i.e. the user
who initially logged in, even if they su). They do not need another
identifier, because by definition there is only one per uid at a time.
Impact on D-Bus
===============
There are two models for how the D-Bus session bus, which is designed to
be per "high-level session" (whatever that means), can work in a world
with user-sessions.
Last time this was discussed, Havoc had the useful insight that this is
not really a question about D-Bus, it's a question about how you build
OSs, and in particular how you model sessions.
(Up to) one high-level session per login
----------------------------------------
The first model, which is how it traditionally worked (before there was
such a thing as a user-session), is that each graphical login session is
declared to be a "session", and non-graphical login sessions are pretty
much swept under the carpet and forgotten about.
In graphical sessions, vaguely modern Unix OSs typically know how to
start up a dbus-daemon during the creation of a graphical session (e.g.
in Debian and derivatives it's started by /etc/X11/Xsession.d, and
Fedora derivatives have a similar setup under a different name). If they
don't, modern desktop environments also know how to start a dbus-daemon
if they need one (e.g. gnome-session does this for GNOME), and if *that*
doesn't start one (the "I use Firefox under fvwm" use-case), we have a
slightly shaky but functional "autolaunch" mechanism based on X11
properties.
In principle, a PAM module or something could ensure that we have a
dbus-daemon per login session, even tty/ssh/cron login sessions
(which all go through PAM). In practice, nobody has ever cared enough to
implement this, so we're left with D-Bus autolaunch, which can't
actually work for tty sessions and had bad side-effects from its
attempts to do so, so I disabled it 3 years ago in favour of
recommending that users requiring a D-Bus session should start their own
and manage its lifetime themselves e.g. with dbus-run-session(1).
If people want to put work into this model, we could do a lot better
than we do now; for instance, the bus socket could be
unix:path=${XDG_RUNTIME_DIR}/sessions/${XDG_SESSION_ID}/bus (but with
the necessary escaping) on systems where those variables are set, rather
than messing about with $TMPDIR.
However, the people doing the work have a somewhat strong correlation
with the people who want a different model, for which read on...
Digression: (at least some) users and developers don't want that
----------------------------------------------------------------
One major issue with one high-level session (and hence one session bus)
per login session is that it places an artifical barrier between login
sessions. It's difficult for cron jobs, ssh logins, tty logins to share
non-GUI backend services such as dconf, Rygel, Telepathy: on my NAS box,
which runs Rygel with no GUI to export my music in a way the
notoriously picky PS3 system software can understand, I had to
construct a D-Bus session to let Rygel and Tracker components talk
to each other. Happily, I had already written dbus-run-session(1),
but I shouldn't need to do this rubbish to let non-GUI daemons talk
to each other - it should just work.
Similarly, if you leave a screen/tmux session detached and running
in the background, systemd is fine with that: its view of the world
is that there are processes left over from your previous login session,
keeping the login session alive in "closing" state, with the consequence
that the user-session remains alive too (unless you have configured
systemd-logind to kill leftover processes, as required by the sysadmins
of computer labs and other massively multi-user machines, in which case
you don't get to run screen sessions and that's by design).
However, the login-session-centric model can't cope with that:
the dbus-daemon dies with the X server, the D-Bus services die with
the dbus-daemon, and now the duplicity(1) process in your screen session
can't use gvfs to upload your backups to a server (or whatever).
Another issue is that it is an extremely poor match for how
`systemd --user` works (although apparently a better match for
how Ubuntu uses `upstart --user`, which I hope to discuss with Ubuntu
people at the systemd hackfest today / at FOSDEM) - you can't use
systemd to manage your login-session-bound D-Bus services, because they
all share a systemd, which doesn't know which login session wants which
service.
I think it's telling that, in recent discussion with Ubuntu developers
about their use of one `upstart --user` per login session, they
clarified that they only actually do this for *graphical* login
sessions, and they provide tools to let a developer copy the environment
variables from what, conceptually, should be another session,
effectively "joining" that session for a subtree of processes. I think
this proves the point that Robert McQueen made on the subject back in 2006:
"It's already been discussed multiple times on the list and
apparently a user bus is too confusing or not really useful unless
you make it a user bus shared across a network cluster. I don't
agree, and I think that the consequence of this stance will be a
proliferation of hacks like the one I just sketched out (not to
mention even grosser hacks like Ubuntu's ACPI scripts grepping the
environment of the user's processes in /proc to find their session
bus to tell their screensaver to lock)."
One high-level session per user-session
---------------------------------------
As a result of the problems noted above, various people, most vocally
the systemd and GNOME developers, have advocated a different model
for how to model sessions in the OS - what I called "user-sessions"
above. A typical implementation of that idea forbids multiple graphical
login sessions altogether: if you type your username and password into
gdm while already logged in on another virtual console, it will just
vt-switch to your existing graphical login session and unlock it.
Under X11, that might well be the best we can do, because typical
X11 applications can't cope with being asked to put windows on more
than one $DISPLAY (although I've heard rumours that Emacs can, which
would make Emacs an ideal candidate to be a user-session service).
Under Wayland or similar future cleverness, hopefully there'll be
some way for a new login on a new seat to "steal" all the windows
from the inactive seat, or some way to merge both seats into
one big virtual display if the same person is using both (AIUI this
is what GNOME designers want to do), or some other clever solution.
What this means for D-Bus
-------------------------
This leaves the problem that, if you have chosen the
user-session-centric model, current D-Bus doesn't actually work very
well: we have dbus-launch and dbus-run-session for login-session-centric
D-Bus, but if you want to make D-Bus user-session-centric, you need to
resort to third-party user bus units like user-session-units, which
float around Github and somehow never make it upstream. I'm now on my
third unrelated work project that contains a copypaste of dbus.socket
and dbus.service, and I'm sure my colleagues have others - this is not
the flying car future I signed up for. So I would like to get this stuff
upstream into dbus.git.
So that the people who are happy with the complexities of the current
arrangement can remain happy, here is how I intend it to work:
* ./configure --disable-user-bus: you get a login-session-centric world
* ./configure --enable-user-bus: you get a user-session-centric world
* configure either way and selectively install/delete files: you can
either have a login-session-centric or user-session-centric world,
depending what you install. (This is so I can have dbus-user-bus and
dbus-x11 binary packages in Debian, where the answer to "do A or do
B?" is always "both!", without having to compile everything twice;
the GNOME metapackage could eventually depend on dbus-user-bus.)
I think in practice that design will mean that by default, libdbus tries
to connect to both user-session- and login-session-centric locations for
the bus (but the "wrong one" for this OS will fail and the "right one"
will succeed), while dbus-user-bus and dbus-x11 would arrange for the
bus to listen in the appropriate place for one option or the other so
that the appropriate version succeeds.
Conveniently, this is what I've already implemented (in two
different ways): "try the user-session-centric path first, and if that
fails, fall back to login-session-centric autolaunch". There is the
slight nit that the server side of user-session-centric operation
requires systemd, although that could be argued to be a feature: the
other reasonable implementation is libpam-xdg-runtime, which AIUI is
unmaintained now that Ubuntu has switched to systemd-logind, and the
people who are avoiding systemd on general principles (hello Devuan!)
are probably the sort of people who are sufficiently change-averse to
prefer D-Bus to work the way it worked in 2005 (if they install it at
all). I'd be willing to consider implementations that work with other
service managers if their authors are willing to help maintain them.
The glorious(?) kdbus future
============================
kdbus is a proposed Linux kernel module providing
somewhat-D-Bus-compatible message-passing semantics (I'm being vague
because it's been 4 months since I last looked at it in detail - I hope
to return to it soon). To provide complete dbus-daemon-like
functionality, you also need a user-space process to create the bus and
manage activations; the
reference implementation, and so far the only implementation, is in
systemd.
The systemd developers who are working on the user-space counterpart
for kdbus have indicated that they have zero interest in supporting a
login-session-centric setup for session kdbus. In practice, I think
this means that the proponents of a login-session-centric world either
can't use kdbus in their sessions, or need to convince the relevant
developers that the added complexity of their model is worth it, or need
to write their own user-space for kdbus sessions and
disable systemd's. (It's not all bad - they can still use kdbus for
the system bus, which is already an improvement.)
Remaining issue: environment variables
======================================
Sadly, not all the issues have been resolved yet. The biggest is
environment variables: on existing systems there is an expectation
that environment variables set by *dm, PAM, or /etc/X11/Xsession.d
(or the non-Debian equivalent) will be propagated into every process
in the session. In a brief survey of about half the Xsession.d scripts
in Debian, I identified these environment variables:
* PULSE_SERVER
* ESPEAKER
* XDG_DATA_HOME, XDG_DATA_DIRS
* each variable printed by locale(1)
* VDPAU_DRIVER
* GTK_IM_MODULE, QT_IM_MODULE, QT4_IM_MODULE, CLUTTER_IM_MODULE,
XMODIFIERS
* GTK_MODULES
plus the obvious ones set by *dm, such as DISPLAY, or by PAM. Similarly,
a user's ~/.xsession can set arbitrary variables - mine sets CCACHE_DIR,
EDITOR, MPD_HOST and XDG_CONFIG_DIRS, among others.
The naive implementation would be to run a tool at the end of
/etc/X11/Xsession.d that uploads all of these into `dbus-daemon
--session` (if used) and into `systemd --user` (if used). However, not
all environment variables set in such scripts are suitable for that use:
notably, XDG_SESSION_ID from the login session should not be copied into
activated processes that exist outside any login session.
As a short-term solution, I'm tempted to write that tool, but make it
only upload a whitelisted set of variables automatically, and say "if
you install dbus-user-bus, you are expected to run this tool from your
~/.xsession if you need it".
Thoughts?
S