Discussion:
[lennart@kemper.freedesktop.org: [systemd-commits] src/pam-module.c]
(too old to reply)
Lennart Poettering
2010-12-24 09:56:53 UTC
Permalink
Heya,

just wanted to let everybody know that I reverted the logic in
pam_sytemd which automatically creates a cgroup in the 'cpu' hierarchy
for every user/session, since it makes it impossible to use RT
scheduling from user/session processes then. For a longer explanation
see the patch I commited (attachment).

This probably needs to be fixed in the kernel before we can enable this
by default again in systemd.

This is related to the issue reported in LWN about JACK+cgroups. Also,
this issue breaks my 4 line bash patch as known from slashdot, and I am
wondering if the auto-grouping kernel patch is affected by this too.

Lennart
--
Lennart Poettering - Red Hat, Inc.
f***@gmail.com
2010-12-27 03:05:51 UTC
Permalink
Hi Lennart,
Post by Lennart Poettering
This is related to the issue reported in LWN about JACK+cgroups. Also,
this issue breaks my 4 line bash patch as known from slashdot,
AIUI, This is not a kernel problem, and is related with
libcgroup<http://sourceforge.net/projects/libcg/>
.

Quotes from LWN(lwn.net/Articles/420407):
"""
if a process is run in a control group with no access to realtime
scheduling, that process will not be able to put itself into a realtime
scheduling class regardless of any resource limit settings.

The kernel, by default, grants realtime access to the "root" control group -
the one which contains all processes in the absence of some policy to the
contrary.
"""

"""
If, however, (1) the libcgroup package has been installed, and (2)
that package has been configured to put all user processes into a
default (non-root) group, the situation changes.

The libcgroup default group does not have realtime access, so processes
expecting to be able to run in a realtime scheduling class will be
disappointed.
"""

IMHO, It should be the job of some thing(like systemd) directly play with
cgroup.
It seems systemd doesn't use libcgroup, am I right? Then, It should be fine
if systemd grants realtime access to cgroup for each user session itself.
--
Regards,

- cee1
Lennart Poettering
2011-01-01 20:45:40 UTC
Permalink
Post by f***@gmail.com
Hi Lennart,
Post by Lennart Poettering
This is related to the issue reported in LWN about JACK+cgroups. Also,
this issue breaks my 4 line bash patch as known from slashdot,
AIUI, This is not a kernel problem, and is related with
libcgroup<http://sourceforge.net/projects/libcg/>
Nah, the way I see it it is a kernel problem. The right fix for the
problem in my eyes would be if the kernel cpu controller would be split
into two: one that manages rt time slices (and is never used by
default), and one that manages CPU shares for normally scheduled
processes (which is what sytsemd would use).

My take on this:

http://article.gmane.org/gmane.comp.audio.jackit/23071
Post by f***@gmail.com
IMHO, It should be the job of some thing(like systemd) directly play
with cgroup. It seems systemd doesn't use libcgroup, am I right?
Then, It should be fine if systemd grants realtime access to cgroup
for each user session itself.
The problem is that in systemd we'd like to manage CPU shares for normal
processes, but cannot realistically manage CPU shares for RT
processes. However as the kernel cpu cgroup controller is written right
now, you either have to do both, or neither. And that I believe is very
much broken and should not be tried to worked around but fixed on the
kernel side.

Lennart
--
Lennart Poettering - Red Hat, Inc.
Andrey Borzenkov
2011-01-29 09:01:14 UTC
Permalink
On Fri, Dec 24, 2010 at 12:56 PM, Lennart Poettering
Post by Lennart Poettering
Heya,
just wanted to let everybody know that I reverted the logic in
pam_sytemd which automatically creates a cgroup in the 'cpu' hierarchy
for every user/session, since it makes it impossible to use RT
scheduling from user/session processes then.
But this is not limited to user session - right now rtkit is not
usable with systemd.

https://bugzilla.redhat.com/show_bug.cgi?id=655321

And the same on Mandriva

Jan 27 22:54:17 cooker rtkit-daemon[3192]: Failed to make ourselves
RT: Operation not permitted
Jan 27 22:54:17 cooker rtkit-daemon[3192]: Failed to make ourselves
RT: Operation not permitted
Lennart Poettering
2011-02-03 20:15:45 UTC
Permalink
Post by Andrey Borzenkov
On Fri, Dec 24, 2010 at 12:56 PM, Lennart Poettering
Post by Lennart Poettering
Heya,
just wanted to let everybody know that I reverted the logic in
pam_sytemd which automatically creates a cgroup in the 'cpu' hierarchy
for every user/session, since it makes it impossible to use RT
scheduling from user/session processes then.
But this is not limited to user session - right now rtkit is not
usable with systemd.
https://bugzilla.redhat.com/show_bug.cgi?id=655321
And the same on Mandriva
Jan 27 22:54:17 cooker rtkit-daemon[3192]: Failed to make ourselves
RT: Operation not permitted
Jan 27 22:54:17 cooker rtkit-daemon[3192]: Failed to make ourselves
RT: Operation not permitted
Yupp, there's a ControlGroups=cpu:/ missing in the rtkit service file. I
have forgotten to commit that. Will fix this soon.

Lennart
--
Lennart Poettering - Red Hat, Inc.
Lennart Poettering
2011-02-08 08:04:26 UTC
Permalink
Post by Lennart Poettering
Post by Andrey Borzenkov
On Fri, Dec 24, 2010 at 12:56 PM, Lennart Poettering
Post by Lennart Poettering
Heya,
just wanted to let everybody know that I reverted the logic in
pam_sytemd which automatically creates a cgroup in the 'cpu' hierarchy
for every user/session, since it makes it impossible to use RT
scheduling from user/session processes then.
But this is not limited to user session - right now rtkit is not
usable with systemd.
https://bugzilla.redhat.com/show_bug.cgi?id=655321
And the same on Mandriva
Jan 27 22:54:17 cooker rtkit-daemon[3192]: Failed to make ourselves
RT: Operation not permitted
Jan 27 22:54:17 cooker rtkit-daemon[3192]: Failed to make ourselves
RT: Operation not permitted
Yupp, there's a ControlGroups=cpu:/ missing in the rtkit service file. I
have forgotten to commit that. Will fix this soon.
Fixed now in rtkit and systemd git.

Lennart
--
Lennart Poettering - Red Hat, Inc.
Andrey Borzenkov
2011-02-08 08:27:47 UTC
Permalink
On Tue, Feb 8, 2011 at 11:04 AM, Lennart Poettering
Post by Lennart Poettering
Post by Lennart Poettering
Post by Andrey Borzenkov
On Fri, Dec 24, 2010 at 12:56 PM, Lennart Poettering
Post by Lennart Poettering
Heya,
just wanted to let everybody know that I reverted the logic in
pam_sytemd which automatically creates a cgroup in the 'cpu' hierarchy
for every user/session, since it makes it impossible to use RT
scheduling from user/session processes then.
But this is not limited to user session - right now rtkit is not
usable with systemd.
https://bugzilla.redhat.com/show_bug.cgi?id=655321
And the same on Mandriva
Jan 27 22:54:17 cooker rtkit-daemon[3192]: Failed to make ourselves
RT: Operation not permitted
Jan 27 22:54:17 cooker rtkit-daemon[3192]: Failed to make ourselves
RT: Operation not permitted
Yupp, there's a ControlGroups=cpu:/ missing in the rtkit service file. I
have forgotten to commit that. Will fix this soon.
Fixed now in rtkit and systemd git.
The last commit on http://git.0pointer.de/?p=rtkit.git is 6 months
old. Is it the wrong URL?

Also the only new commit in systemd is

commit b20c6be697ded108e3c3bd5b8812fee13326eefc
Author: Lennart Poettering <***@poettering.net>
Date: Fri Feb 4 12:46:38 2011 +0100

pam: optionally reset cgroup memberships for login sessions

It does not look like it applies to rtkit?
Lennart Poettering
2011-02-08 09:03:41 UTC
Permalink
Post by Andrey Borzenkov
Post by Lennart Poettering
Post by Lennart Poettering
Yupp, there's a ControlGroups=cpu:/ missing in the rtkit service file. I
have forgotten to commit that. Will fix this soon.
Fixed now in rtkit and systemd git.
The last commit on http://git.0pointer.de/?p=rtkit.git is 6 months
old. Is it the wrong URL?
Nah, I just wrote the mail first and only then pushed. I didn't expect
you to check git so quickly ;-).

The commit is now there.
Post by Andrey Borzenkov
Also the only new commit in systemd is
commit b20c6be697ded108e3c3bd5b8812fee13326eefc
Date: Fri Feb 4 12:46:38 2011 +0100
pam: optionally reset cgroup memberships for login sessions
It does not look like it applies to rtkit?
Actually it does.

The rtkit patch ensures rtkit itself can get RT privs. This systemd
patch ensures apps (such as PA) started within a systemd session can get
RT privs. Without neither patch neither side can get RT privs. To work
properly both sides need to be able to get RT privs.

Lennart
--
Lennart Poettering - Red Hat, Inc.
Andrey Borzenkov
2011-02-08 09:29:59 UTC
Permalink
On Tue, Feb 8, 2011 at 12:03 PM, Lennart Poettering
Post by Lennart Poettering
Post by Andrey Borzenkov
Post by Lennart Poettering
Post by Lennart Poettering
Yupp, there's a ControlGroups=cpu:/ missing in the rtkit service file. I
have forgotten to commit that. Will fix this soon.
Fixed now in rtkit and systemd git.
The last commit on http://git.0pointer.de/?p=rtkit.git is 6 months
old. Is it the wrong URL?
Nah, I just wrote the mail first and only then pushed. I didn't expect
you to check git so quickly ;-).
The commit is now there.
Post by Andrey Borzenkov
Also the only new commit in systemd is
commit b20c6be697ded108e3c3bd5b8812fee13326eefc
Date:   Fri Feb 4 12:46:38 2011 +0100
    pam: optionally reset cgroup memberships for login sessions
It does not look like it applies to rtkit?
Actually it does.
The rtkit patch ensures rtkit itself can get RT privs. This systemd
patch ensures apps (such as PA) started within a systemd session can get
RT privs. Without neither patch neither side can get RT privs. To work
properly both sides need to be able to get RT privs.
Do I need this patch to *strart* rtkit?

{pts/0}% sudo systemctl status rtkit-daemon.service
rtkit-daemon.service - RealtimeKit Scheduling Policy Service
Loaded: loaded (/lib/systemd/system/rtkit-daemon.service)
Active: active (running) since Tue, 08 Feb 2011 12:22:30 +0300; 5s ago
Main PID: 13399 (rtkit-daemon)
Status: "Running."
CGroup: name=systemd:/system/rtkit-daemon.service
└ 13399 /usr/lib64/rtkit-daemon

Feb 8 12:22:30 cooker rtkit-daemon[13399]: Failed to make ourselves
RT: Operation not permitted


{pts/1}% systemctl --no-pager --property=ControlGroups show rtkit-daemon.service
ControlGroups=name=systemd:/system/rtkit-daemon.service cpu:/

BTW property name is incinsistent between unit definition and
systemctl output - ControlGroup vs. ControlGroups. Is it intentional?
Lennart Poettering
2011-02-08 10:15:11 UTC
Permalink
Post by Andrey Borzenkov
Post by Lennart Poettering
The rtkit patch ensures rtkit itself can get RT privs. This systemd
patch ensures apps (such as PA) started within a systemd session can get
RT privs. Without neither patch neither side can get RT privs. To work
properly both sides need to be able to get RT privs.
Do I need this patch to *strart* rtkit?
Hmm, yes? The cgroup fix needs to be applied when you start rtkit.
Post by Andrey Borzenkov
{pts/0}% sudo systemctl status rtkit-daemon.service
rtkit-daemon.service - RealtimeKit Scheduling Policy Service
Loaded: loaded (/lib/systemd/system/rtkit-daemon.service)
Active: active (running) since Tue, 08 Feb 2011 12:22:30 +0300; 5s ago
Main PID: 13399 (rtkit-daemon)
Status: "Running."
CGroup: name=systemd:/system/rtkit-daemon.service
└ 13399 /usr/lib64/rtkit-daemon
Feb 8 12:22:30 cooker rtkit-daemon[13399]: Failed to make ourselves
RT: Operation not permitted
{pts/1}% systemctl --no-pager --property=ControlGroups show rtkit-daemon.service
ControlGroups=name=systemd:/system/rtkit-daemon.service cpu:/
Uh, oh. Are you suggesting that rtkit does not actually run in the cpu:/
cgroup? Can you verify this with "ps xawf -eo pid,args,cgroup"?
Post by Andrey Borzenkov
BTW property name is incinsistent between unit definition and
systemctl output - ControlGroup vs. ControlGroups. Is it intentional?
Oh, good point. Fixed now.

Lennart
--
Lennart Poettering - Red Hat, Inc.
Andrey Borzenkov
2011-02-08 10:30:21 UTC
Permalink
On Tue, Feb 8, 2011 at 1:15 PM, Lennart Poettering
Post by Lennart Poettering
Post by Lennart Poettering
The rtkit patch ensures rtkit itself can get RT privs. This systemd
patch ensures apps (such as PA) started within a systemd session can get
RT privs. Without neither patch neither side can get RT privs. To work
properly both sides need to be able to get RT privs.
Do  I need this patch to *strart* rtkit?
Hmm, yes? The cgroup fix needs to be applied when you start rtkit.
But there is no login session at this point; is PAM involved at all?
At least "pam" does not appear anywhere in rtkit sources ... and we
must be able to use systemd with pam_systemd as well, must not we?
Post by Lennart Poettering
{pts/0}% sudo systemctl status rtkit-daemon.service
rtkit-daemon.service - RealtimeKit Scheduling Policy Service
          Loaded: loaded (/lib/systemd/system/rtkit-daemon.service)
          Active: active (running) since Tue, 08 Feb 2011 12:22:30 +0300; 5s ago
        Main PID: 13399 (rtkit-daemon)
          Status: "Running."
          CGroup: name=systemd:/system/rtkit-daemon.service
                  └ 13399 /usr/lib64/rtkit-daemon
Feb  8 12:22:30 cooker rtkit-daemon[13399]: Failed to make ourselves
RT: Operation not permitted
{pts/1}% systemctl --no-pager --property=ControlGroups show rtkit-daemon.service
ControlGroups=name=systemd:/system/rtkit-daemon.service cpu:/
Uh, oh. Are you suggesting that rtkit does not actually run in the cpu:/
cgroup? Can you verify this with "ps xawf -eo pid,args,cgroup"?
{pts/1}% ps xawf -eo pid,args,cgroup | grep rtkit
3781 /usr/lib64/rtkit-daemon name=systemd:/system/rtkit-daemon.service

This is 0.9 with your patch on top. systemd patch not yet applied.
Andrey Borzenkov
2011-02-08 10:31:03 UTC
Permalink
Post by Andrey Borzenkov
On Tue, Feb 8, 2011 at 1:15 PM, Lennart Poettering
Post by Lennart Poettering
Post by Lennart Poettering
The rtkit patch ensures rtkit itself can get RT privs. This systemd
patch ensures apps (such as PA) started within a systemd session can get
RT privs. Without neither patch neither side can get RT privs. To work
properly both sides need to be able to get RT privs.
Do  I need this patch to *strart* rtkit?
Hmm, yes? The cgroup fix needs to be applied when you start rtkit.
But there is no login session at this point; is PAM involved at all?
At least "pam" does not appear anywhere in rtkit sources ... and we
must be able to use systemd with pam_systemd as well, must not we?
*without* pam_systemd that is ...
Post by Andrey Borzenkov
Post by Lennart Poettering
{pts/0}% sudo systemctl status rtkit-daemon.service
rtkit-daemon.service - RealtimeKit Scheduling Policy Service
          Loaded: loaded (/lib/systemd/system/rtkit-daemon.service)
          Active: active (running) since Tue, 08 Feb 2011 12:22:30 +0300; 5s ago
        Main PID: 13399 (rtkit-daemon)
          Status: "Running."
          CGroup: name=systemd:/system/rtkit-daemon.service
                  └ 13399 /usr/lib64/rtkit-daemon
Feb  8 12:22:30 cooker rtkit-daemon[13399]: Failed to make ourselves
RT: Operation not permitted
{pts/1}% systemctl --no-pager --property=ControlGroups show rtkit-daemon.service
ControlGroups=name=systemd:/system/rtkit-daemon.service cpu:/
Uh, oh. Are you suggesting that rtkit does not actually run in the cpu:/
cgroup? Can you verify this with "ps xawf -eo pid,args,cgroup"?
{pts/1}% ps xawf -eo pid,args,cgroup | grep rtkit
 3781 /usr/lib64/rtkit-daemon     name=systemd:/system/rtkit-daemon.service
This is 0.9 with your patch on top. systemd patch not yet applied.
Lennart Poettering
2011-02-08 10:36:38 UTC
Permalink
Post by Andrey Borzenkov
On Tue, Feb 8, 2011 at 1:15 PM, Lennart Poettering
Post by Lennart Poettering
Post by Lennart Poettering
The rtkit patch ensures rtkit itself can get RT privs. This systemd
patch ensures apps (such as PA) started within a systemd session can get
RT privs. Without neither patch neither side can get RT privs. To work
properly both sides need to be able to get RT privs.
Do  I need this patch to *strart* rtkit?
Hmm, yes? The cgroup fix needs to be applied when you start rtkit.
But there is no login session at this point; is PAM involved at all?
At least "pam" does not appear anywhere in rtkit sources ... and we
must be able to use systemd with pam_systemd as well, must not we?
Hmm?

The patch to rtkit needs to be applied before rtkit is started. After
applying, building and installing rtkit you need to reload the systemd
configuration.

The patch to systemd needs to be applied before you login. After
applying, building and installing systemd it should be sufficient to
relogin, since that will already load the updated PAM module.
Post by Andrey Borzenkov
Post by Lennart Poettering
{pts/1}% systemctl --no-pager --property=ControlGroups show rtkit-daemon.service
ControlGroups=name=systemd:/system/rtkit-daemon.service cpu:/
Uh, oh. Are you suggesting that rtkit does not actually run in the cpu:/
cgroup? Can you verify this with "ps xawf -eo pid,args,cgroup"?
{pts/1}% ps xawf -eo pid,args,cgroup | grep rtkit
3781 /usr/lib64/rtkit-daemon
name=systemd:/system/rtkit-daemon.service
This looks pretty much correct, rtkit is in the root cpu cgroup.

Lennart
--
Lennart Poettering - Red Hat, Inc.
Andrey Borzenkov
2011-02-08 11:22:25 UTC
Permalink
On Tue, Feb 8, 2011 at 1:36 PM, Lennart Poettering
Post by Lennart Poettering
Post by Andrey Borzenkov
On Tue, Feb 8, 2011 at 1:15 PM, Lennart Poettering
Post by Lennart Poettering
Post by Lennart Poettering
The rtkit patch ensures rtkit itself can get RT privs. This systemd
patch ensures apps (such as PA) started within a systemd session can get
RT privs. Without neither patch neither side can get RT privs. To work
properly both sides need to be able to get RT privs.
Do  I need this patch to *strart* rtkit?
Hmm, yes? The cgroup fix needs to be applied when you start rtkit.
But there is no login session at this point; is PAM involved at all?
At least "pam" does not appear anywhere in rtkit sources ... and we
must be able to use systemd with pam_systemd as well, must not we?
Hmm?
The patch to rtkit needs to be applied before rtkit is started. After
applying, building and installing rtkit you need to reload the systemd
configuration.
The patch to systemd needs to be applied before you login. After
applying, building and installing systemd it should be sufficient to
relogin, since that will already load the updated PAM module.
We apparently misunderstand each other.

I speak about failure of rtkit-daemon to put itself in RT scheduling
group on startup. At this point there is no login at all.

Anyway, I rebuild systemd with your PAM patch and restarted system and
as expected nothing changed:

Feb 8 14:14:51 cooker rtkit-daemon[3165]: Failed to make ourselves
RT: Operation not permitted
Post by Lennart Poettering
Post by Andrey Borzenkov
Post by Lennart Poettering
{pts/1}% systemctl --no-pager --property=ControlGroups show rtkit-daemon.service
ControlGroups=name=systemd:/system/rtkit-daemon.service cpu:/
Uh, oh. Are you suggesting that rtkit does not actually run in the cpu:/
cgroup? Can you verify this with "ps xawf -eo pid,args,cgroup"?
{pts/1}% ps xawf -eo pid,args,cgroup | grep rtkit
 3781 /usr/lib64/rtkit-daemon
 name=systemd:/system/rtkit-daemon.service
This looks pretty much correct, rtkit is in the root cpu cgroup.
So - can this message on startup ("rtkit-daemon[3165]: Failed to make
ourselves RT: Operation not permitted") be ignored? If yes - this
message should not be logged as error. If no - your changes so far did
not fix it.
Lennart Poettering
2011-02-08 11:42:49 UTC
Permalink
Post by Andrey Borzenkov
On Tue, Feb 8, 2011 at 1:36 PM, Lennart Poettering
Post by Lennart Poettering
Post by Andrey Borzenkov
On Tue, Feb 8, 2011 at 1:15 PM, Lennart Poettering
Post by Lennart Poettering
Post by Lennart Poettering
The rtkit patch ensures rtkit itself can get RT privs. This systemd
patch ensures apps (such as PA) started within a systemd session can get
RT privs. Without neither patch neither side can get RT privs. To work
properly both sides need to be able to get RT privs.
Do  I need this patch to *strart* rtkit?
Hmm, yes? The cgroup fix needs to be applied when you start rtkit.
But there is no login session at this point; is PAM involved at all?
At least "pam" does not appear anywhere in rtkit sources ... and we
must be able to use systemd with pam_systemd as well, must not we?
Hmm?
The patch to rtkit needs to be applied before rtkit is started. After
applying, building and installing rtkit you need to reload the systemd
configuration.
The patch to systemd needs to be applied before you login. After
applying, building and installing systemd it should be sufficient to
relogin, since that will already load the updated PAM module.
We apparently misunderstand each other.
I speak about failure of rtkit-daemon to put itself in RT scheduling
group on startup. At this point there is no login at all.
Anyway, I rebuild systemd with your PAM patch and restarted system and
Feb 8 14:14:51 cooker rtkit-daemon[3165]: Failed to make ourselves
RT: Operation not permitted
Hmpf. so you are suggesting that although rtkit itself is in the cpu:/
cgroup it cannot make itself RT? That is really weird. What is the
contents of /sys/fs/cgroup/cpu/cpu.rt_*_us?

Lennart
--
Lennart Poettering - Red Hat, Inc.
Andrey Borzenkov
2011-02-08 11:52:21 UTC
Permalink
On Tue, Feb 8, 2011 at 2:42 PM, Lennart Poettering
Post by Lennart Poettering
Post by Andrey Borzenkov
On Tue, Feb 8, 2011 at 1:36 PM, Lennart Poettering
Post by Lennart Poettering
Post by Andrey Borzenkov
On Tue, Feb 8, 2011 at 1:15 PM, Lennart Poettering
Post by Lennart Poettering
Post by Lennart Poettering
The rtkit patch ensures rtkit itself can get RT privs. This systemd
patch ensures apps (such as PA) started within a systemd session can get
RT privs. Without neither patch neither side can get RT privs. To work
properly both sides need to be able to get RT privs.
Do  I need this patch to *strart* rtkit?
Hmm, yes? The cgroup fix needs to be applied when you start rtkit.
But there is no login session at this point; is PAM involved at all?
At least "pam" does not appear anywhere in rtkit sources ... and we
must be able to use systemd with pam_systemd as well, must not we?
Hmm?
The patch to rtkit needs to be applied before rtkit is started. After
applying, building and installing rtkit you need to reload the systemd
configuration.
The patch to systemd needs to be applied before you login. After
applying, building and installing systemd it should be sufficient to
relogin, since that will already load the updated PAM module.
We apparently misunderstand each other.
I speak about failure of rtkit-daemon to put itself in RT scheduling
group on startup. At this point there is no login at all.
Anyway, I rebuild systemd with your PAM patch and restarted system and
Feb  8 14:14:51 cooker rtkit-daemon[3165]: Failed to make ourselves
RT: Operation not permitted
Hmpf. so you are suggesting that although rtkit itself is in the cpu:/
No. I do not. You stated it :)

As indicated by ps output, rtkit-daemon is *not* in cpu group

{pts/1}% ps xawf -eo pid,args,cgroup | grep rtkit
3165 /usr/lib64/rtkit-daemon name=systemd:/system/rtkit-daemon.service

Compare this with any other service like

{pts/1}% ps xawf -eo pid,args,cgroup | grep networkmanager
1277 /usr/sbin/NetworkManager --
cpu:/system/networkmanager.service;name=systemd:/system/networkmanager.service
2550 \_ /sbin/dhclient -d -4 -s
cpu:/system/networkmanager.service;name=systemd:/system/networkmanager.service

BTW rtkit is the only service having ControlGroup at all ...

{pts/1}% grep -r ControlGroup /lib/systemd/system
/lib/systemd/system/rtkit-daemon.service:ControlGroup=cpu:/
Post by Lennart Poettering
cgroup it cannot make itself RT? That is really weird. What is the
contents of /sys/fs/cgroup/cpu/cpu.rt_*_us?
{pts/1}% cat /sys/fs/cgroup/cpu/cpu.rt_*_us
1000000
950000
Lennart Poettering
2011-02-08 12:11:25 UTC
Permalink
Post by Andrey Borzenkov
Post by Lennart Poettering
Post by Andrey Borzenkov
I speak about failure of rtkit-daemon to put itself in RT scheduling
group on startup. At this point there is no login at all.
Anyway, I rebuild systemd with your PAM patch and restarted system and
Feb  8 14:14:51 cooker rtkit-daemon[3165]: Failed to make ourselves
RT: Operation not permitted
Hmpf. so you are suggesting that although rtkit itself is in the cpu:/
No. I do not. You stated it :)
As indicated by ps output, rtkit-daemon is *not* in cpu group
{pts/1}% ps xawf -eo pid,args,cgroup | grep rtkit
3165 /usr/lib64/rtkit-daemon name=systemd:/system/rtkit-daemon.service
Compare this with any other service like
{pts/1}% ps xawf -eo pid,args,cgroup | grep networkmanager
1277 /usr/sbin/NetworkManager --
cpu:/system/networkmanager.service;name=systemd:/system/networkmanager.service
2550 \_ /sbin/dhclient -d -4 -s
cpu:/system/networkmanager.service;name=systemd:/system/networkmanager.service
BTW rtkit is the only service having ControlGroup at all ...
Well, yes. ps supresses the output if a process is in the root group of
a hierarchy, because by default all processes are in the default
group. To get the full list of cgorup memberships you could look into
/proc/$(pidof rtkit-daemon)/cgroup.
Post by Andrey Borzenkov
Post by Lennart Poettering
cgroup it cannot make itself RT? That is really weird. What is the
contents of /sys/fs/cgroup/cpu/cpu.rt_*_us?
{pts/1}% cat /sys/fs/cgroup/cpu/cpu.rt_*_us
1000000
950000
Weird, weird, weird. So, systemd is in the right cgroup with correct
settings but still doesn't have the privs to make itself RT. There's
something very weird here.

Lennart
--
Lennart Poettering - Red Hat, Inc.
Andrey Borzenkov
2011-02-08 13:09:09 UTC
Permalink
On Tue, Feb 8, 2011 at 3:11 PM, Lennart Poettering
Post by Lennart Poettering
Well, yes. ps supresses the output if a process is in the root group of
a hierarchy, because by default all processes are in the default
group. To get the full list of cgorup memberships you could look into
/proc/$(pidof rtkit-daemon)/cgroup.
Yes, you are right.
Post by Lennart Poettering
Post by Andrey Borzenkov
Post by Lennart Poettering
cgroup it cannot make itself RT? That is really weird. What is the
contents of /sys/fs/cgroup/cpu/cpu.rt_*_us?
{pts/1}% cat /sys/fs/cgroup/cpu/cpu.rt_*_us
1000000
950000
Weird, weird, weird. So, systemd is in the right cgroup with correct
settings but still doesn't have the privs to make itself RT. There's
something very weird here.
Yes, but apparently it is not directly related to systemd. I tried to
reboot with sysvinit and got the same errors from rtkit-daemon. What
is the right place to discuss rtkit? I'd still appreciate help in
tracking down.

Thank you and sorry for noise here.
Lennart Poettering
2011-02-27 22:11:02 UTC
Permalink
Post by Andrey Borzenkov
On Tue, Feb 8, 2011 at 3:11 PM, Lennart Poettering
Post by Lennart Poettering
Well, yes. ps supresses the output if a process is in the root group of
a hierarchy, because by default all processes are in the default
group. To get the full list of cgorup memberships you could look into
/proc/$(pidof rtkit-daemon)/cgroup.
Yes, you are right.
Post by Lennart Poettering
Post by Andrey Borzenkov
Post by Lennart Poettering
cgroup it cannot make itself RT? That is really weird. What is the
contents of /sys/fs/cgroup/cpu/cpu.rt_*_us?
{pts/1}% cat /sys/fs/cgroup/cpu/cpu.rt_*_us
1000000
950000
Weird, weird, weird. So, systemd is in the right cgroup with correct
settings but still doesn't have the privs to make itself RT. There's
something very weird here.
Yes, but apparently it is not directly related to systemd. I tried to
reboot with sysvinit and got the same errors from rtkit-daemon. What
is the right place to discuss rtkit? I'd still appreciate help in
tracking down.
rtkit doesn't have a ML. So it's somewhere between here, the PA mailing
list and me personally. Mostly the latter.

Lennart
--
Lennart Poettering - Red Hat, Inc.
Andrey Borzenkov
2011-02-09 20:55:30 UTC
Permalink
On Tue, Feb 8, 2011 at 3:11 PM, Lennart Poettering
Post by Lennart Poettering
Weird, weird, weird. So, systemd is in the right cgroup with correct
settings but still doesn't have the privs to make itself RT. There's
something very weird here.
Kernel bug.

commit f44937718ce3b8360f72f6c68c9481712517a867
Author: Mike Galbraith <***@gmx.de>
Date: Thu Jan 13 04:54:50 2011 +0100

sched, autogroup: Fix CONFIG_RT_GROUP_SCHED sched_setscheduler() failure

If CONFIG_RT_GROUP_SCHED is set, __sched_setscheduler() fails due
to autogroup
not allocating rt_runtime. Free unused/unusable rt_se and rt_rq,
redirect RT
tasks to the root task group, and tell __sched_setscheduler() that it's ok.
f***@gmail.com
2011-02-11 02:30:06 UTC
Permalink
Hi Lennart, Andrey,

Some questions about the discuss:
1. Does systemd re-enable sorting user sessions into their own cgroups in
the 'cpu' hierarchy?
2. AIUI, rtkit is a daemon used for doing RT-related privilege operations.
It doesn't spawn RT-threads. Am I right?
3. Does rtkit have related systemd service file? Does it run at its own
cgroup with controller=cpu? If yes, how does the cgroup receive
rt-bandwidth(period time, run time)?
Post by Andrey Borzenkov
On Tue, Feb 8, 2011 at 3:11 PM, Lennart Poettering
Post by Lennart Poettering
Weird, weird, weird. So, systemd is in the right cgroup with correct
settings but still doesn't have the privs to make itself RT. There's
something very weird here.
Kernel bug.
commit f44937718ce3b8360f72f6c68c9481712517a867
Date: Thu Jan 13 04:54:50 2011 +0100
sched, autogroup: Fix CONFIG_RT_GROUP_SCHED sched_setscheduler() failure
If CONFIG_RT_GROUP_SCHED is set, __sched_setscheduler() fails due
to autogroup
not allocating rt_runtime. Free unused/unusable rt_se and rt_rq,
redirect RT
tasks to the root task group, and tell __sched_setscheduler() that it's ok.
_______________________________________________
systemd-devel mailing list
http://lists.freedesktop.org/mailman/listinfo/systemd-devel
--
Regards,

- cee1
Lennart Poettering
2011-02-13 21:30:46 UTC
Permalink
Post by f***@gmail.com
Hi Lennart, Andrey,
1. Does systemd re-enable sorting user sessions into their own cgroups in
the 'cpu' hierarchy?
No, we cannot reenable that before the cpu cgroup controller gets fixed.

On current kernels doing this kind of sorting means practically that all
processes we sort into a 'cpu' cgroup lose their capability to use RT
scheduling.

I am not aware of any typical daemon we ship that would use RT
scheduling hence we are keeping the default 'cpu' cgroup sorting for
system daemons enabled. However user applications are more likely to use
RT (for example PA does) and hence we have disabled this for sessions
for now.
Post by f***@gmail.com
2. AIUI, rtkit is a daemon used for doing RT-related privilege operations.
It doesn't spawn RT-threads. Am I right?
It does use RT privs for the implementatin of the canary watchdog.
Post by f***@gmail.com
3. Does rtkit have related systemd service file? Does it run at its own
cgroup with controller=cpu? If yes, how does the cgroup receive
rt-bandwidth(period time, run time)?
rtkit comes with a systemd unit file. In rtkit git this unit file
ensures that rtkit is explicitly moved into the root "cpu" cgroup so
that it can make use of RT scheduling -- if you so will rtkit is the one
exception to what i mentioned above regarding no system daemons using
RT.

Lennart
--
Lennart Poettering - Red Hat, Inc.
f***@gmail.com
2011-02-14 03:12:38 UTC
Permalink
Hi Lennart,
Thanks for the explanation.

IMHO, To re-enable user session 'cpu' sorting:
a) Desktop distributions disable GROUP_RT in the kernel, then no
rt_bandwidth, all RT-apps can be fully administrated under rtkit.

Or b) cpu cgroup controller should default make sub-cgroups share
rt_bandwidth with their parent.
Internally, kernel uses two separate hierarchies for NORMAL and RT
processes, current logic should be changed as: a new cgroup with a cpu
controller will add a new level in 'NORMAL hierarchy', but not touch
'RThierarchy' unless set sched_rt_runtime_us.
Post by Lennart Poettering
Post by f***@gmail.com
Hi Lennart, Andrey,
1. Does systemd re-enable sorting user sessions into their own cgroups
in
Post by f***@gmail.com
the 'cpu' hierarchy?
No, we cannot reenable that before the cpu cgroup controller gets fixed.
On current kernels doing this kind of sorting means practically that all
processes we sort into a 'cpu' cgroup lose their capability to use RT
scheduling.
I am not aware of any typical daemon we ship that would use RT
scheduling hence we are keeping the default 'cpu' cgroup sorting for
system daemons enabled. However user applications are more likely to use
RT (for example PA does) and hence we have disabled this for sessions
for now.
Post by f***@gmail.com
2. AIUI, rtkit is a daemon used for doing RT-related privilege
operations.
Post by f***@gmail.com
It doesn't spawn RT-threads. Am I right?
It does use RT privs for the implementatin of the canary watchdog.
Post by f***@gmail.com
3. Does rtkit have related systemd service file? Does it run at its own
cgroup with controller=cpu? If yes, how does the cgroup receive
rt-bandwidth(period time, run time)?
rtkit comes with a systemd unit file. In rtkit git this unit file
ensures that rtkit is explicitly moved into the root "cpu" cgroup so
that it can make use of RT scheduling -- if you so will rtkit is the one
exception to what i mentioned above regarding no system daemons using
RT.
Lennart
--
Lennart Poettering - Red Hat, Inc.
--
Regards,

- cee1
Andrey Borzenkov
2011-02-14 07:58:46 UTC
Permalink
Post by f***@gmail.com
Hi Lennart,
Thanks for the explanation.
a) Desktop distributions disable GROUP_RT in the kernel, then no
rt_bandwidth, all RT-apps can be fully administrated under rtkit.
Or b) cpu cgroup controller should default make sub-cgroups share
rt_bandwidth with their parent.
RT is about determinism. You need to ensure that task will be able to
respond in fixed time. If you allow arbitrary, unknown in advance,
number of tasks share the same limited CPU share, you simply kill
determinism.

Personally I think that RT should be restricted to limited number of
tasks that are known in advance; then it is responsibility of
administrator to allocate their CPU share according to requirements.
Post by f***@gmail.com
Post by Lennart Poettering
I am not aware of any typical daemon we ship that would use RT
scheduling hence we are keeping the default 'cpu' cgroup sorting for
system daemons enabled. However user applications are more likely to use
RT (for example PA does) and hence we have disabled this for sessions
for now.
and for reasons outlined above I think that either PA should not
require RT to run, or we need dedicated system wide PA daemon that can
be made RT :)
Lennart Poettering
2011-02-14 11:28:03 UTC
Permalink
Post by Andrey Borzenkov
Post by f***@gmail.com
Hi Lennart,
Thanks for the explanation.
a) Desktop distributions disable GROUP_RT in the kernel, then no
rt_bandwidth, all RT-apps can be fully administrated under rtkit.
Or b) cpu cgroup controller should default make sub-cgroups share
rt_bandwidth with their parent.
RT is about determinism. You need to ensure that task will be able to
respond in fixed time. If you allow arbitrary, unknown in advance,
number of tasks share the same limited CPU share, you simply kill
determinism.
Well, RT on Linux is not deterministic really. It just means that
processes can monopolize the CPU for a while. The general approach in RT
on Linux is "remove scheduling latency where we find it". Such an
approach can never guarantee determinism. And I don't think anything
else would be realistic to do.
Post by Andrey Borzenkov
Personally I think that RT should be restricted to limited number of
tasks that are known in advance; then it is responsibility of
administrator to allocate their CPU share according to requirements.
Well, I think this might be an approach for technical appliances, but
not really for general purpose stuff such as audio where we need a very
weak definition of RT only.

Lennart
--
Lennart Poettering - Red Hat, Inc.
f***@gmail.com
2011-02-14 11:42:30 UTC
Permalink
Post by Andrey Borzenkov
RT is about determinism. You need to ensure that task will be able to
respond in fixed time. If you allow arbitrary, unknown in advance,
number of tasks share the same limited CPU share, you simply kill
determinism.
Personally I think that RT should be restricted to limited number of
tasks that are known in advance; then it is responsibility of
administrator to allocate their CPU share according to requirements.
Post by Lennart Poettering
I am not aware of any typical daemon we ship that would use RT
scheduling hence we are keeping the default 'cpu' cgroup sorting for
system daemons enabled. However user applications are more likely to use
RT (for example PA does) and hence we have disabled this for sessions
for now.
and for reasons outlined above I think that either PA should not
require RT to run, or we need dedicated system wide PA daemon that can
be made RT :)
Agree in this sense. RT can be seen as 'hardcode' some computing resource
to simulate dedicated hardware, in other words, virtual dsp, should reside
at system scope.
--
Regards,

- cee1
Lennart Poettering
2011-02-14 11:19:10 UTC
Permalink
Post by f***@gmail.com
Hi Lennart,
Thanks for the explanation.
Heya,
Post by f***@gmail.com
a) Desktop distributions disable GROUP_RT in the kernel, then no
rt_bandwidth, all RT-apps can be fully administrated under rtkit.
We actually want to allow people to use the RT limiting features, so I
think this option should be left on.
Post by f***@gmail.com
Or b) cpu cgroup controller should default make sub-cgroups share
rt_bandwidth with their parent.
Internally, kernel uses two separate hierarchies for NORMAL and RT
processes, current logic should be changed as: a new cgroup with a cpu
controller will add a new level in 'NORMAL hierarchy', but not touch
'RThierarchy' unless set sched_rt_runtime_us.
This is more or less what I was asking the kernel folks for. Dhaval, any
news?

Lennart
--
Lennart Poettering - Red Hat, Inc.
f***@gmail.com
2011-03-30 12:19:05 UTC
Permalink
Hi,
Post by Lennart Poettering
Heya,
just wanted to let everybody know that I reverted the logic in
pam_sytemd which automatically creates a cgroup in the 'cpu' hierarchy
for every user/session, since it makes it impossible to use RT
scheduling from user/session processes then. For a longer explanation
see the patch I commited (attachment).
Just mention it, this also enables autogroup for processes in each
user session. The new autogroup feature will not work if a task has
already attached with a CPU controller.

See kernel/sched_autogroup.c, function task_wants_autogroup.

Loading...