[systemd-devel] file-hierarchy. user persistent data dir.

[Ivan Romanov]

Hello.

Now specification defines system variable persistent data dir (/var) but not
defines similar directory for user data. I developer of Psi+ (IM client) when
I was implementing XDG Base Directory Specification was questions where should
be saved user conversation history. How I understand ~/.config is used for any
settings of application, ~/.local/share is analogue of /usr/share, ~/.cache
for data which always can be regenerated and can be freely removed. So no one
from these dirs is not suitable for saving user data which is generating by
application and needest only in this application on this machine (user
conversation history for example).

So I would like to see Persistent Variable User Data Directory in
specification.

[Mantas Mikulėnas]

Afaik, XDG defines $XDG_DATA_HOME or ~/.local/share to be the location for
such data, and various other chat clients use it for logs (it doesn't
exactly match /usr/share because XDG doesn't care for the rest of ~/.local).
--
Mantas Mikulėnas <***@gmail.com>
// sent from phone

Post by Ivan Romanov
Hello.
Now specification defines system variable persistent data dir (/var) but not
defines similar directory for user data. I developer of Psi+ (IM client)
when
I was implementing XDG Base Directory Specification was questions where should
be saved user conversation history. How I understand ~/.config is used for any
settings of application, ~/.local/share is analogue of /usr/share, ~/.cache
for data which always can be regenerated and can be freely removed. So no one
from these dirs is not suitable for saving user data which is generating by
application and needest only in this application on this machine (user
conversation history for example).
So I would like to see Persistent Variable User Data Directory in
specification.
_______________________________________________
systemd-devel mailing list
http://lists.freedesktop.org/mailman/listinfo/systemd-devel

[Lennart Poettering]

Post by Mantas Mikulėnas
Afaik, XDG defines $XDG_DATA_HOME or ~/.local/share to be the location for
such data, and various other chat clients use it for logs (it doesn't
exactly match /usr/share because XDG doesn't care for the rest of ~/.local).

Oh, please don't!

Lennart

[Lennart Poettering]

Post by Ivan Romanov
Hello.
Now specification defines system variable persistent data dir (/var) but not
defines similar directory for user data. I developer of Psi+ (IM client) when
I was implementing XDG Base Directory Specification was questions where should
be saved user conversation history. How I understand ~/.config is used for any
settings of application, ~/.local/share is analogue of /usr/share, ~/.cache
for data which always can be regenerated and can be freely removed. So no one
from these dirs is not suitable for saving user data which is generating by
application and needest only in this application on this machine (user
conversation history for example).
So I would like to see Persistent Variable User Data Directory in
specification.

I have discussed this a couple of times with various people from GNOME,
XDG and otherwise, and we came to the conclusion to avoid this
distinction between configuration and state for the user, and simply ask
applications to put both into ~/.config. Which is actually what
file-hierarchy(7) even says.

The rationale is that the distinction is too specialized for
applications, they never got that right, and since we cannot protect
this anyway we probably shouldn't simplify things here, and tell app
developers to put whatever they like that is variable and changing in
that directory. So there are there dirs:

~/.config: state and configuration, the counterpart for both /etc/ *and*
/var/lib/ in the home directory

~/.local: static vendor resources of additional packages installed into
the home directory. Should be considered read-only except when
new packages are installed or removed from the home
directory. The counterpart of /usr/ in the home directory.

~/.cache: unessential state, that can be flushed if the user desires so
without ill effect. The cunterpart of /var/cache in the home
directory.

Also see the discussion around this:

http://lists.freedesktop.org/archives/xdg/2014-February/013189.html

Not of course, that this is not what many apps do. Because the XDG
basedir spec is badly writtem and more confusing than explaining, what
people ultimately did is use ~/.local for state. I can only recomend not
to do that though.

We are working on making user app sandboxing work in the context of
systemd, one part of that will be that sandboxed apps will only get
write access to ~/.config and ~/.cache, but only read-only access to
~/.local, if at all.

Lennart

[Ivan Romanov]

Thanks for such detailed answer.

Post by Lennart Poettering

Post by Ivan Romanov
Hello.
Now specification defines system variable persistent data dir (/var) but
not defines similar directory for user data. I developer of Psi+ (IM
client) when I was implementing XDG Base Directory Specification was
questions where should be saved user conversation history. How I
understand ~/.config is used for any settings of application,
~/.local/share is analogue of /usr/share, ~/.cache for data which always
can be regenerated and can be freely removed. So no one from these dirs
is not suitable for saving user data which is generating by application
and needest only in this application on this machine (user conversation
history for example).
So I would like to see Persistent Variable User Data Directory in
specification.

I have discussed this a couple of times with various people from GNOME,
XDG and otherwise, and we came to the conclusion to avoid this
distinction between configuration and state for the user, and simply ask
applications to put both into ~/.config. Which is actually what
file-hierarchy(7) even says.
The rationale is that the distinction is too specialized for
applications, they never got that right, and since we cannot protect
this anyway we probably shouldn't simplify things here, and tell app
developers to put whatever they like that is variable and changing in
~/.config: state and configuration, the counterpart for both /etc/ *and*
/var/lib/ in the home directory
~/.local: static vendor resources of additional packages installed into
the home directory. Should be considered read-only except when
new packages are installed or removed from the home
directory. The counterpart of /usr/ in the home directory.
~/.cache: unessential state, that can be flushed if the user desires so
without ill effect. The cunterpart of /var/cache in the home
directory.
http://lists.freedesktop.org/archives/xdg/2014-February/013189.html
Not of course, that this is not what many apps do. Because the XDG
basedir spec is badly writtem and more confusing than explaining, what
people ultimately did is use ~/.local for state. I can only recomend not
to do that though.

So I think new specification should explain this with more obviously way.
To resolve the ambiguity.

Post by Lennart Poettering
We are working on making user app sandboxing work in the context of
systemd, one part of that will be that sandboxed apps will only get
write access to ~/.config and ~/.cache, but only read-only access to
~/.local, if at all.
Lennart

[Simon McVittie]

Post by Lennart Poettering
~/.config: state and configuration, the counterpart for both /etc/ *and*
/var/lib/ in the home directory
~/.local: static vendor resources of additional packages installed into
the home directory. Should be considered read-only except when
new packages are installed or removed from the home
directory. The counterpart of /usr/ in the home directory.

This is not, in practice, what happens: I certainly have a lot of state
(/var/lib-like) in my ~/.local/share.

It isn't clear to me that there's consensus that all applications that
do this have always been wrong, and so is Lionel Dricot's somewhat
widely-cited guide
<http://ploum.net/207-modify-your-application-to-use-xdg-folders/>. If
I'm the only one who still thinks like that, fine, change the policy;
but the number of applications putting state in my ~/.local/share seems
like anecdotal evidence that I'm not the only one with that mental model :-)

Also, on Windows, GLib's g_get_user_data_dir() returns
CSIDL_LOCAL_APPDATA, which is primarily a state directory (the same
thing returned by g_get_user_config_dir(), in fact). Is that wrong?

The major use-cases I've seen for splitting configuration vs. state are:

* keep your config in a VCS, don't keep your state/data in a VCS
* reset config to "factory defaults" without losing state/data

If you don't consider those use-cases to be interesting or useful, fine
(I consider the second one to be something I'd never want to do), but
please do consider them before breaking them.

My proposal, which I think matches how Ted Gould said Ubuntu's
sandboxing works:

* An application with ID foo[1] may write to
${XDG_CONFIG_HOME:~/.config}/foo during normal operation;
this location
is intended for [?]

(my point of view: configuration)
(Lennart's point of view: configuration, state, and miscellaneous
user data)

* foo may write to ${XDG_DATA_HOME:~/.local/share}/foo during normal
operation; this location is intended for [?]

(my point of view: state, miscellaneous user data, and
read-only resources)
(Lennart's point of view: read-only resources)

* foo may write to ${XDG_CACHE_HOME:~/.cache}/foo during normal
operation, but must assume that files in there may be deleted

* foo must not assume that it can write to
${XDG_DATA_HOME:~/.local/share}/bar (for bar != foo) during normal
operation, only when it is installed or uninstalled (e.g. an app
installer can write to ~/.local/share/applications/foo.desktop
but foo itself cannot)

I think that keeps the ability to sandbox, while preserving the layout
that various apps have come to rely on, and not making life worse for
people who do distinguish between config and other data?

Regards,
S

[1] where foo is from some namespace outside the scope of basedir:
reversed domain name, or GUID, or name in /usr/bin, or distro package
name, or whatever. The exact definition is not important here.

[Michael Biebl]

Post by Simon McVittie

Post by Lennart Poettering
~/.config: state and configuration, the counterpart for both /etc/ *and*
/var/lib/ in the home directory
~/.local: static vendor resources of additional packages installed into
the home directory. Should be considered read-only except when
new packages are installed or removed from the home
directory. The counterpart of /usr/ in the home directory.

This is not, in practice, what happens: I certainly have a lot of state
(/var/lib-like) in my ~/.local/share.

Indeed. When looking through my ~/.local/share there are a lot of state files.

Actually, the majority of applications that I use treat ~/.local as
app state directory, only a very seem to use ~/.config for that.

This clearly needs clarification in the XDG spec.

[Colin Guthrie]

Post by Michael Biebl

Post by Simon McVittie

Post by Lennart Poettering
~/.config: state and configuration, the counterpart for both /etc/ *and*
/var/lib/ in the home directory
~/.local: static vendor resources of additional packages installed into
the home directory. Should be considered read-only except when
new packages are installed or removed from the home
directory. The counterpart of /usr/ in the home directory.

This is not, in practice, what happens: I certainly have a lot of state
(/var/lib-like) in my ~/.local/share.

Indeed. When looking through my ~/.local/share there are a lot of state files.
Actually, the majority of applications that I use treat ~/.local as
app state directory, only a very seem to use ~/.config for that.
This clearly needs clarification in the XDG spec.

I think ~/.local/share sounds like it *should* be similar to
/usr/{lib,share}/ and there should be a different tree for state, e.g.
~/.local/var, but as people have pointed out this is not the behaviour
in practice.

The names "usr" and "var" don't really convey much these days so perhaps
~/.local/share should be deprecated and symlinked to ~/.local/resources
which is defined for read-only resources and ~/.local/state becomes a
new folder to store state.

Actually ~/.local/share should probably be symlinked to ~/.local/state
initially, (to preserve old behaviour) and thus allow read/write access
under sandboxing and not break any existing applications.

It would IMO have been nicer if .config lived inside .local as
~/.local/config that way the whole .local tree is what matters. Of
course "local" is still a shit name IMO but then I always think the
bikeshed should have stripes.

So perhaps we should define ~/.local as the top level tree with the
subfolders "config", "state", "resources". Each is clearly defined,
~/.config can be changed to be a symlink to ~/.local/config and
~/.local/share can be a symlink to "state" (although the name "share"
suggests it should really be symlinked to "resources", it's usage "in
the wild" means that symlinking it to "state" is safer until the changes
trickle through). [Side note: I do have some files in ~/.local/lib too
related to python]

Either way, the underspecification in XDG (I have other complaints
regarding the cache folder handling too but that's another rant) is not
ideal and definitive clarification would help.

It should definitely be discussed with the GNOME and KDE (+$other) guys
tho' (as session managers would have to ensure the proper work was done
to convert users' directory layouts, so this be handled carefully).

Col
--
Colin Guthrie
gmane(at)colin.guthr.ie
http://colin.guthr.ie/

Day Job:
Tribalogic Limited http://www.tribalogic.net/
Open Source:
Mageia Contributor http://www.mageia.org/
PulseAudio Hacker http://www.pulseaudio.org/
Trac Hacker http://trac.edgewall.org/

[Lennart Poettering]

Post by Simon McVittie

Post by Lennart Poettering
~/.config: state and configuration, the counterpart for both /etc/ *and*
/var/lib/ in the home directory
~/.local: static vendor resources of additional packages installed into
the home directory. Should be considered read-only except when
new packages are installed or removed from the home
directory. The counterpart of /usr/ in the home directory.

This is not, in practice, what happens: I certainly have a lot of state
(/var/lib-like) in my ~/.local/share.

Yes, of course. I am fully aware of that and I already mentioned that
even.

Post by Simon McVittie
* keep your config in a VCS, don't keep your state/data in a VCS
* reset config to "factory defaults" without losing state/data
If you don't consider those use-cases to be interesting or useful, fine
(I consider the second one to be something I'd never want to do), but
please do consider them before breaking them.

Actually, I so far kept my home directory in a VCS too.

However, I am also aware that the destinction between state and
configuration is very vague and nothing I want to dump on people's heads
for desktop apps (that we do this for system stuff is difficult
enough). The distinction is already completeld thwarted anyway by the
fact that gconf/dconf-like systems are making this scheme impossible
anyway, as they already are the single location for both configuration
and state (also, you cannot check them into VCS anyway).

So yupp, I think neither the VCS issue nor the destinction between
configuration and state are really reasons strong enough to keep
this. I think ~/.config should be little more than a file-system
counterpart of dconf, and this means, VCS-compatibility, and the
distinction between state and configuration are not at the center.

The thing with all of this is that we should make a spec so simple that
people can make sense of (for example, Mozilla made clear they find the
spec so stupid they won't support it, more or less). And yeah, if we can
tell people that there's just one directory they really have to care
about, which is ~/.config, then things get relatively easy to
tell people about.

I am also not particularly concerned that many apps don't follow these
guidelines right now. This is fully OK, after all $HOME is writable for
everybody, currently.

However, as part of bringing sandboxing to Linux we can clean up this
mess, and can and have to enforce much stricter rules here. The
sandboxing will be something that will break a number of different areas
where people made assumptions, no doubt, this is just going to be one of
them.

Summary: current apps can do whatever they want in $HOME, including
ignoring the spec entirely. Sandboxed apps won't. But if you have want to
make your app work in a sandbox, then you have to make a couple of
changes anyway, this once is just one of them.

Lennart