Discussion:
User configs, BTRFS subvolume for /home and automatic service start during boot?
(too old to reply)
Thorsten Schöning
2018-05-03 19:07:49 UTC
Permalink
Raw Message
Hi all,

I'm using Ubuntu 16.04 LTS server with BTRFS, which created one
subvolume for / and another one for /home. While I do like this setup,
it makes me trouble with how I deploy our own software developed for
various customers. For many reasons that software is stored under
/home[/tenant1|tenant2|...] currently, including systemd service files
nowadays. The good thing is that enabling those files is as easy as
systemctl enable ...
with the absolute path. The bad thing is that because /home is another
file system, during boot systemd can't find my service files and
therefore can't start my services[1].

I'm looking for alternatives[2] now and thought of user configs. The
good thing is that my service files seem to be properly available to
systemd if I login e.g. using SSH, the status of the service is loaded
and I'm able to start it manually. But I have trouble getting systemd
to automatically start the service during boot and even during SSH
login.

I'm pretty sure to already have use the suggested "loginctl enable-linger ...",
but might have some error somewhere of course. But it might as well be
that the subvolume /home itself is making trouble here again. In the
end, there's the following sentence in the docs which doesn't
The file system where the linked unit files are located must be
accessible when systemd is started (e.g. anything underneath /home
or /var is not allowed, unless those directories are located on the
root file system).
So, are user configs for systemd supported to be placed on a non-root
file system AND be started during boot time? I don't have any users
logging in in the end.

Thanks!

[1]: https://serverfault.com/questions/910173/why-cant-systemd-find-service-files-within-a-btrfs-subvolume-within-a-btrfs-roo
[2]: https://serverfault.com/questions/910274/how-to-make-initrd-of-ubuntu-16-04-mount-home-automatically-in-addition-to

Mit freundlichen Grüßen,

Thorsten Schöning
--
Thorsten Schöning E-Mail: ***@AM-SoFT.de
AM-SoFT IT-Systeme http://www.AM-SoFT.de/

Telefon...........05151- 9468- 55
Fax...............05151- 9468- 88
Mobil..............0178-8 9468- 04

AM-SoFT GmbH IT-Systeme, Brandenburger Str. 7c, 31789 Hameln
AG Hannover HRB 207 694 - Geschäftsführer: Andreas Muchow
Andrei Borzenkov
2018-05-04 04:57:28 UTC
Permalink
Raw Message
Post by Thorsten Schöning
I'm looking for alternatives[2] now and thought of user configs. The
good thing is that my service files seem to be properly available to
systemd if I login e.g. using SSH, the status of the service is loaded
and I'm able to start it manually. But I have trouble getting systemd
to automatically start the service during boot and even during SSH
login.
I'm pretty sure to already have use the suggested "loginctl enable-linger ...",
but might have some error somewhere of course. But it might as well be
that the subvolume /home itself is making trouble here again. In the
end, there's the following sentence in the docs which doesn't
distinguish between system wide and user configs:>
So, are user configs for systemd supported to be placed on a non-root
file system
I would say "yes", at least I do not see anything that prevents it.
Post by Thorsten Schöning
AND be started during boot time?
User services may be started by user instance which is started normally
by the first user session (which is not necessarily user login) and
stopped when the last user session ends. Or you can enable linger in
which case user instance will be started on boot.

Which services will actually be started depends on this user's
configuration, which services are enabled. It sounds like you may not
have enabled them or at least not enabled for the correct top-level target.
Post by Thorsten Schöning
I don't have any users
logging in in the end.
Thorsten Schöning
2018-05-04 06:14:18 UTC
Permalink
Raw Message
Guten Tag Andrei Borzenkov,
Post by Andrei Borzenkov
Or you can enable linger in
which case user instance will be started on boot.
That's the part not working in my setup for some reason.
Post by Andrei Borzenkov
Which services will actually be started depends on this user's
configuration, which services are enabled. It sounds like you may not
have enabled them or at least not enabled for the correct top-level target.
I'm pretty sure they are enabled, all the documented directories,
files and links are available AND the services are started if I do it
manually using SSH and "systemctl start ...". The service itself is
Post by Andrei Borzenkov
[Unit]
Description=Test /home being a subvolume in BTRFS and containing a service.
[Install]
WantedBy=multi-user.target
[Service]
Type=oneshot
ExecStart=/bin/bash -c "echo Works: `date` >> /home/tschoening/btrfs_home_test/ping.txt"
Do you see any error? It is enabled using "systemctl --user enable ..."
using the absolut path and can be diabled the same way and started
manually. It's only not started automaticalyl, neither if I login
using SSH, nor at boot time.

So am I missing something in the service config or regarding
"enable-linger"?

Mit freundlichen Grüßen,

Thorsten Schöning
--
Thorsten Schöning E-Mail: ***@AM-SoFT.de
AM-SoFT IT-Systeme http://www.AM-SoFT.de/

Telefon...........05151- 9468- 55
Fax...............05151- 9468- 88
Mobil..............0178-8 9468- 04

AM-SoFT GmbH IT-Systeme, Brandenburger Str. 7c, 31789 Hameln
AG Hannover HRB 207 694 - Geschäftsführer: Andreas Muchow
Andrei Borzenkov
2018-05-04 10:07:11 UTC
Permalink
Raw Message
Post by Thorsten Schöning
Guten Tag Andrei Borzenkov,
Post by Andrei Borzenkov
Or you can enable linger in
which case user instance will be started on boot.
That's the part not working in my setup for some reason.
Post by Andrei Borzenkov
Which services will actually be started depends on this user's
configuration, which services are enabled. It sounds like you may not
have enabled them or at least not enabled for the correct top-level target.
I'm pretty sure they are enabled, all the documented directories,
files and links are available AND the services are started if I do it
manually
How does it confirm that units are enabled?
Post by Thorsten Schöning
using SSH and "systemctl start ...". The service itself is
Post by Andrei Borzenkov
[Unit]
Description=Test /home being a subvolume in BTRFS and containing a service.
[Install]
WantedBy=multi-user.target
[Service]
Type=oneshot
ExecStart=/bin/bash -c "echo Works: `date` >> /home/tschoening/btrfs_home_test/ping.txt"
Do you see any error?
There is no multi-user.target in user instance (unless you added it
yourself), user instance starts default.target. Which is incidentally
true also for system instance unless explicitly overridden, usually on
kernel command line.
Post by Thorsten Schöning
It is enabled using "systemctl --user enable ..."
using the absolut path
That just makes it depedency of multi-user.target which itself does not exist.
Post by Thorsten Schöning
and can be diabled the same way and started
manually. It's only not started automaticalyl, neither if I login
using SSH, nor at boot time.
So am I missing something in the service config or regarding
"enable-linger"?
Mit freundlichen Grüßen,
Thorsten Schöning
--
AM-SoFT IT-Systeme http://www.AM-SoFT.de/
Telefon...........05151- 9468- 55
Fax...............05151- 9468- 88
Mobil..............0178-8 9468- 04
AM-SoFT GmbH IT-Systeme, Brandenburger Str. 7c, 31789 Hameln
AG Hannover HRB 207 694 - Geschäftsführer: Andreas Muchow
_______________________________________________
systemd-devel mailing list
https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Thorsten Schöning
2018-05-04 14:13:35 UTC
Permalink
Raw Message
Guten Tag Andrei Borzenkov,
Post by Andrei Borzenkov
There is no multi-user.target in user instance (unless you added it
yourself), user instance starts default.target.
Thanks, that was the problem, after changing the target everything
seems to work as expected. I didn't spot that difference to the docs I
was using.

Is there some counterpart for "systemctl --user ..." allowing me to
provide a username to work on instead of the "current user"? The only
thing in this direction I see is "--root" and providing a path to some
user's home dir.

Mit freundlichen Grüßen,

Thorsten Schöning
--
Thorsten Schöning E-Mail: ***@AM-SoFT.de
AM-SoFT IT-Systeme http://www.AM-SoFT.de/

Telefon...........05151- 9468- 55
Fax...............05151- 9468- 88
Mobil..............0178-8 9468- 04

AM-SoFT GmbH IT-Systeme, Brandenburger Str. 7c, 31789 Hameln
AG Hannover HRB 207 694 - Geschäftsführer: Andreas Muchow
Lennart Poettering
2018-05-04 16:09:44 UTC
Permalink
Raw Message
Post by Thorsten Schöning
Guten Tag Andrei Borzenkov,
Post by Andrei Borzenkov
There is no multi-user.target in user instance (unless you added it
yourself), user instance starts default.target.
Thanks, that was the problem, after changing the target everything
seems to work as expected. I didn't spot that difference to the docs I
was using.
Is there some counterpart for "systemctl --user ..." allowing me to
provide a username to work on instead of the "current user"? The only
thing in this direction I see is "--root" and providing a path to some
user's home dir.
No, that's not going to work. You might be able to use "sudo" or "su"
for that however.

Lennart
--
Lennart Poettering, Red Hat
Lennart Poettering
2018-05-04 16:07:42 UTC
Permalink
Raw Message
Post by Thorsten Schöning
Hi all,
I'm using Ubuntu 16.04 LTS server with BTRFS, which created one
subvolume for / and another one for /home. While I do like this setup,
it makes me trouble with how I deploy our own software developed for
various customers. For many reasons that software is stored under
/home[/tenant1|tenant2|...] currently, including systemd service files
nowadays. The good thing is that enabling those files is as easy as
systemctl enable ...
with the absolute path. The bad thing is that because /home is another
file system, during boot systemd can't find my service files and
therefore can't start my services[1].
Hmm, you said /home was a subvolume? That suggests they are on the
same fs? Not following here?
Post by Thorsten Schöning
I'm looking for alternatives[2] now and thought of user configs. The
good thing is that my service files seem to be properly available to
systemd if I login e.g. using SSH, the status of the service is loaded
and I'm able to start it manually. But I have trouble getting systemd
to automatically start the service during boot and even during SSH
login.
I'm pretty sure to already have use the suggested "loginctl enable-linger ...",
but might have some error somewhere of course. But it might as well be
that the subvolume /home itself is making trouble here again. In the
end, there's the following sentence in the docs which doesn't
The file system where the linked unit files are located must be
accessible when systemd is started (e.g. anything underneath /home
or /var is not allowed, unless those directories are located on the
root file system).
So, are user configs for systemd supported to be placed on a non-root
file system AND be started during boot time? I don't have any users
logging in in the end.
The comment above only applies to the systemd system instance,
i.e. PID 1. It does not apply to systemd user instances. I figure we
should clarify that in the docs...

Lennart
--
Lennart Poettering, Red Hat
Thorsten Schöning
2018-05-04 16:59:57 UTC
Permalink
Raw Message
Guten Tag Lennart Poettering,
Post by Lennart Poettering
Hmm, you said /home was a subvolume? That suggests they are on the
same fs? Not following here?
# <file system> <mount point> <type> <options> <dump> <pass>
# / was on /dev/sda1 during installation
# /home was on /dev/sda1 during installation
So the same BTRFS file system containing two sub volumes. That seems
to work for user configs, my problem was a wrong "WantedBy".

Mit freundlichen Grüßen,

Thorsten Schöning
--
Thorsten Schöning E-Mail: ***@AM-SoFT.de
AM-SoFT IT-Systeme http://www.AM-SoFT.de/

Telefon...........05151- 9468- 55
Fax...............05151- 9468- 88
Mobil..............0178-8 9468- 04

AM-SoFT GmbH IT-Systeme, Brandenburger Str. 7c, 31789 Hameln
AG Hannover HRB 207 694 - Geschäftsführer: Andreas Muchow
Lennart Poettering
2018-05-04 17:03:59 UTC
Permalink
Raw Message
Post by Thorsten Schöning
Guten Tag Lennart Poettering,
Post by Lennart Poettering
Hmm, you said /home was a subvolume? That suggests they are on the
same fs? Not following here?
# <file system> <mount point> <type> <options> <dump> <pass>
# / was on /dev/sda1 during installation
# /home was on /dev/sda1 during installation
So the same BTRFS file system containing two sub volumes. That seems
to work for user configs, my problem was a wrong "WantedBy".
Note that btrfs subvolumes are just special directories, hence you can
also do without an explicit entry for the home subvol, and simply
create it below the root subvol.

Lennart
--
Lennart Poettering, Red Hat
Thorsten Schöning
2018-05-04 17:35:24 UTC
Permalink
Raw Message
Guten Tag Lennart Poettering,
Post by Lennart Poettering
Note that btrfs subvolumes are just special directories, hence you can
also do without an explicit entry for the home subvol, and simply
create it below the root subvol.
Thanks for the hint, I'll think about that. The provided fstab was
from an Ubuntu default installation and I would like to keep that as
much as possible.

Mit freundlichen Grüßen,

Thorsten Schöning
--
Thorsten Schöning E-Mail: ***@AM-SoFT.de
AM-SoFT IT-Systeme http://www.AM-SoFT.de/

Telefon...........05151- 9468- 55
Fax...............05151- 9468- 88
Mobil..............0178-8 9468- 04

AM-SoFT GmbH IT-Systeme, Brandenburger Str. 7c, 31789 Hameln
AG Hannover HRB 207 694 - Geschäftsführer: Andreas Muchow
Julian Andres Klode
2018-05-04 18:55:48 UTC
Permalink
Raw Message
Post by Lennart Poettering
Post by Thorsten Schöning
Guten Tag Lennart Poettering,
Post by Lennart Poettering
Hmm, you said /home was a subvolume? That suggests they are on the
same fs? Not following here?
# <file system> <mount point> <type> <options> <dump> <pass>
# / was on /dev/sda1 during installation
# /home was on /dev/sda1 during installation
So the same BTRFS file system containing two sub volumes. That seems
to work for user configs, my problem was a wrong "WantedBy".
Note that btrfs subvolumes are just special directories, hence you can
also do without an explicit entry for the home subvol, and simply
create it below the root subvol.
That's not the best advise IMO. Subvolumes inside subvolumes have
weird behavior that sometimes bites you very hard. For example:
You can't restore an older / snapshot because it won't contain
your /home because it's a subvolume of the current /, but not of the
snapshot. It's just going to be a pain if you forget that.

I had this problem with lxd - it keeps containers in a btrfs subvolume
below /, and then I restored / to an older snapshot and all my containers
were gone / broken - I had to recreate them, as I only noticed later and
already had deleted my broken old "/"; so I switched to a mount for
them:

/dev/mapper/ubuntu--vg-root /var/snap/lxd/common/lxd/storage-pools/default btrfs defaults,subvol=@lxd 0 2

For Ubuntu, it's worth noting that the @ subvolume that's mounted to / is automatically
snapshotted when upgrading distributions; or even on normal upgrades with apt-btrfs-snapshot.
--
debian developer - deb.li/jak | jak-linux.org - free software dev
ubuntu core developer i speak de, en
Loading...