Discussion:
systemd config recipes for namespace-isolated webapps
(too old to reply)
Martin Langhoff
2013-07-02 21:18:57 UTC
Permalink
Hi folks!

At OLPC, I got an early chance to use and abuse systemd, and I like it
quite a bit.

We currently have ~500 identical VMs (created from kickstarts, kept
almost in sync via satellite), each hosts apache/mysql daemons, and 2
installs of the same PHP webapp (production, test).

Goal is to reduce the number of VMs radically, as memory and storage
overheads are killing us.

I am now looking at systemd (under F-19, RHEL7 later) and wondering
whether there are any recipes that can guide me a bit through setting
up webapps in CGs with suitable namespaces.

What I _think_ I need is

0 - one target per "customer", which in turn pulls in
1 - apache
2 - mysql
3 - cronjobs
4 - apache/tomcat/java setup {for some customers}
5 - sftp -- namespace-aware?

with 1,2 and 3 set to use the same CG. And stopping the target should
ensure all the CG is down/dead.

If possible, I prefer to avoid containers (and the associated chroot
maintenance).

High on the list of goals is to protect customers from data leakage,
so guidelines towards effective use of namespaces are sought here.

Pointers, hints, anyone else working in a similar direction?

cheers,



martin
ps: I have read all/most of LWN and Lennart's articles, but welcome a
gentle pointer if relevant...
--
***@gmail.com
- ask interesting questions
- don't get distracted with shiny stuff - working code first
~ http://docs.moodle.org/en/User:Martin_Langhoff
Zbigniew Jędrzejewski-Szmek
2013-07-03 04:53:39 UTC
Permalink
Post by Martin Langhoff
Hi folks!
At OLPC, I got an early chance to use and abuse systemd, and I like it
quite a bit.
We currently have ~500 identical VMs (created from kickstarts, kept
almost in sync via satellite), each hosts apache/mysql daemons, and 2
installs of the same PHP webapp (production, test).
Goal is to reduce the number of VMs radically, as memory and storage
overheads are killing us.
I am now looking at systemd (under F-19, RHEL7 later) and wondering
whether there are any recipes that can guide me a bit through setting
up webapps in CGs with suitable namespaces.
What I _think_ I need is
0 - one target per "customer", which in turn pulls in
1 - apache
2 - mysql
3 - cronjobs
4 - apache/tomcat/java setup {for some customers}
5 - sftp -- namespace-aware?
with 1,2 and 3 set to use the same CG. And stopping the target should
ensure all the CG is down/dead.
If possible, I prefer to avoid containers (and the associated chroot
maintenance).
Hi,
I haven't really tried anythng like what you describe, but in general
both container and container-less approaches should work.

with a container: you can have socket activated systemd-nspawn
instance, which boots to a default target containing your services
1-5 + whatever special you want for that "customer". It is currently
not possible to launch a systemd-nspawn container directly from /,
but you can do a bind mount to somewhere else. If by "chroot
maintanance" you mean the need to copy stuff between / and the
container, then it can be avoided this way. Launching systemd-nspawn
containers directly from / is on the list of planned things.
systemd-***@.service already provides part of the installation.

container-less: a bunch of template units with dependencies on one
another should do what you need (instance units can refer to each
other). You can use InaccessibleDirectories= and other settings to
limit what those units can "see".

The version with containers is probably slightly more flexible
and will allow more customizations for each "customer". The other
one has probably lower overhead. But both should work.

HTH,
Zbyszek
Martin Langhoff
2013-07-03 11:40:33 UTC
Permalink
On Wed, Jul 3, 2013 at 12:53 AM, Zbigniew Jędrzejewski-Szmek
Post by Zbigniew Jędrzejewski-Szmek
I haven't really tried anythng like what you describe, but in general
both container and container-less approaches should work.
Thanks for your reply. Yes, I get the sense that "in general, it
should work". As usual, the devil's in the details...

In both container and container-less cases...

- How do I handle cronjobs?
- How do I tell several services to use the "same" cg?

Then... if I setup a single chroot and try to launch many containers
on top of it...

- does the "stateless" service work?
- how can I "key" stateless writable dirs on a per-container instance?

cheers,



m
--
***@gmail.com
- ask interesting questions
- don't get distracted with shiny stuff - working code first
~ http://docs.moodle.org/en/User:Martin_Langhoff
Zbigniew Jędrzejewski-Szmek
2013-07-03 14:49:09 UTC
Permalink
Post by Martin Langhoff
On Wed, Jul 3, 2013 at 12:53 AM, Zbigniew Jędrzejewski-Szmek
Post by Zbigniew Jędrzejewski-Szmek
I haven't really tried anythng like what you describe, but in general
both container and container-less approaches should work.
Thanks for your reply. Yes, I get the sense that "in general, it
should work". As usual, the devil's in the details...
In both container and container-less cases...
- How do I handle cronjobs?
With systemd .timers and systemd .services activated by those timers. If
you mean "real" cronjobs, I don't know.
Post by Martin Langhoff
- How do I tell several services to use the "same" cg?
They can't all use the same cg, because systemd uses groups to group
units. But they can share a slice of resources, by assigning a group
of services to the same systemd .slice. This part is currently in
fast development, but should be usable already.
Post by Martin Langhoff
Then... if I setup a single chroot and try to launch many containers
on top of it...
- does the "stateless" service work?
In general, systemd is happy to only write to /run, which won't be shared,
so going with an read-only root should work.
Post by Martin Langhoff
- how can I "key" stateless writable dirs on a per-container instance?
You can add a template .service which will mount some directory,
let's say /var/lib/container/etc, and make it PartOf the .service
launching the container. IIRC, templated .mount units are not possible
currently, but the same should be achievable with an explicit mount
command.

Zbyszek
Martin Langhoff
2013-07-03 15:47:17 UTC
Permalink
On Wed, Jul 3, 2013 at 10:49 AM, Zbigniew Jędrzejewski-Szmek
Post by Zbigniew Jędrzejewski-Szmek
Post by Martin Langhoff
- How do I tell several services to use the "same" cg?
They can't all use the same cg, because systemd uses groups to group
units. But they can share a slice of resources, by assigning a group
of services to the same systemd .slice. This part is currently in
fast development, but should be usable already.
In F19 / systemd v209, is there any usable way to have slices or
something resembling them?

thanks!



m
--
***@gmail.com
- ask interesting questions
- don't get distracted with shiny stuff - working code first
~ http://docs.moodle.org/en/User:Martin_Langhoff
Zbigniew Jędrzejewski-Szmek
2013-07-04 13:23:18 UTC
Permalink
Post by Martin Langhoff
On Wed, Jul 3, 2013 at 10:49 AM, Zbigniew Jędrzejewski-Szmek
Post by Zbigniew Jędrzejewski-Szmek
Post by Martin Langhoff
- How do I tell several services to use the "same" cg?
They can't all use the same cg, because systemd uses groups to group
units. But they can share a slice of resources, by assigning a group
of services to the same systemd .slice. This part is currently in
fast development, but should be usable already.
In F19 / systemd v209, is there any usable way to have slices or
something resembling them?
I expect systemd-205 only in rawhide. Maybe systemd-206 will be FC19.

Zbyszek
Martin Langhoff
2013-07-04 18:35:43 UTC
Permalink
On Thu, Jul 4, 2013 at 9:23 AM, Zbigniew Jędrzejewski-Szmek
Post by Zbigniew Jędrzejewski-Szmek
Post by Martin Langhoff
On Wed, Jul 3, 2013 at 10:49 AM, Zbigniew Jędrzejewski-Szmek
Post by Zbigniew Jędrzejewski-Szmek
Post by Martin Langhoff
- How do I tell several services to use the "same" cg?
They can't all use the same cg, because systemd uses groups to group
units. But they can share a slice of resources, by assigning a group
of services to the same systemd .slice. This part is currently in
fast development, but should be usable already.
In F19 / systemd v209, is there any usable way to have slices or
something resembling them?
I expect systemd-205 only in rawhide. Maybe systemd-206 will be FC19.
I made a mistake, clearly. I have v204 in my F19 box.

Are you saying slices will land in F19 through an update?

thanks,



m
--
***@gmail.com
- ask interesting questions
- don't get distracted with shiny stuff - working code first
~ http://docs.moodle.org/en/User:Martin_Langhoff
Tomasz Torcz
2013-07-04 18:43:57 UTC
Permalink
Post by Martin Langhoff
On Thu, Jul 4, 2013 at 9:23 AM, Zbigniew Jędrzejewski-Szmek
Post by Zbigniew Jędrzejewski-Szmek
Post by Martin Langhoff
On Wed, Jul 3, 2013 at 10:49 AM, Zbigniew Jędrzejewski-Szmek
Post by Zbigniew Jędrzejewski-Szmek
Post by Martin Langhoff
- How do I tell several services to use the "same" cg?
They can't all use the same cg, because systemd uses groups to group
units. But they can share a slice of resources, by assigning a group
of services to the same systemd .slice. This part is currently in
fast development, but should be usable already.
In F19 / systemd v209, is there any usable way to have slices or
something resembling them?
I expect systemd-205 only in rawhide. Maybe systemd-206 will be FC19.
I made a mistake, clearly. I have v204 in my F19 box.
Are you saying slices will land in F19 through an update?
Sorry for interrupting, but I seriously doubt this. v205 changes are
waaay to big to appear in an update to released version of distribution.
I think you will have to go to rawhide to have slices.
--
Tomasz Torcz To co nierealne -- tutaj jest normalne.
xmpp: ***@chrome.pl Ziomale na życie mają tu patenty specjalne.
Lennart Poettering
2013-07-12 18:27:43 UTC
Permalink
Post by Martin Langhoff
On Wed, Jul 3, 2013 at 10:49 AM, Zbigniew Jędrzejewski-Szmek
Post by Zbigniew Jędrzejewski-Szmek
Post by Martin Langhoff
- How do I tell several services to use the "same" cg?
They can't all use the same cg, because systemd uses groups to group
units. But they can share a slice of resources, by assigning a group
of services to the same systemd .slice. This part is currently in
fast development, but should be usable already.
In F19 / systemd v209, is there any usable way to have slices or
something resembling them?
No, this will only be in F20.

Lennart
--
Lennart Poettering - Red Hat, Inc.
Lennart Poettering
2013-07-12 18:31:55 UTC
Permalink
Post by Martin Langhoff
On Wed, Jul 3, 2013 at 12:53 AM, Zbigniew Jędrzejewski-Szmek
Post by Zbigniew Jędrzejewski-Szmek
I haven't really tried anythng like what you describe, but in general
both container and container-less approaches should work.
Thanks for your reply. Yes, I get the sense that "in general, it
should work". As usual, the devil's in the details...
In both container and container-less cases...
- How do I handle cronjobs?
You could user timer units instead.
Post by Martin Langhoff
- How do I tell several services to use the "same" cg?
In F20 with slices and everything is awesome. F19 you could use the
ControlGroup= settings, but this is really awful and very manual.
Post by Martin Langhoff
Then... if I setup a single chroot and try to launch many containers
on top of it...
- does the "stateless" service work?
You could do this with nspawn. You could use the same /etc, /usr, and
everything else, and use --bind= to mount different /home and /var into
the containers. systemd-nspawn will take care of a private /run for you,
and systemd in the container will take care of a private /tmp.
Post by Martin Langhoff
- how can I "key" stateless writable dirs on a per-container instance?
--bind and --bin-ro.

Lennart
--
Lennart Poettering - Red Hat, Inc.
Michael Scherer
2013-07-05 10:59:13 UTC
Permalink
Post by Martin Langhoff
Hi folks!
Hi,
Post by Martin Langhoff
At OLPC, I got an early chance to use and abuse systemd, and I like it
quite a bit.
We currently have ~500 identical VMs (created from kickstarts, kept
almost in sync via satellite), each hosts apache/mysql daemons, and 2
installs of the same PHP webapp (production, test).
Goal is to reduce the number of VMs radically, as memory and storage
overheads are killing us.
I am now looking at systemd (under F-19, RHEL7 later) and wondering
whether there are any recipes that can guide me a bit through setting
up webapps in CGs with suitable namespaces.
What I _think_ I need is
0 - one target per "customer", which in turn pulls in
1 - apache
2 - mysql
3 - cronjobs
4 - apache/tomcat/java setup {for some customers}
5 - sftp -- namespace-aware?
with 1,2 and 3 set to use the same CG. And stopping the target should
ensure all the CG is down/dead.
If possible, I prefer to avoid containers (and the associated chroot
maintenance).
High on the list of goals is to protect customers from data leakage,
so guidelines towards effective use of namespaces are sought here.
Pointers, hints, anyone else working in a similar direction?
I would take a look at openshift, since that's exactly what the product
is doing. ( http://openshift.github.io/ )

Each user is isolated into a a specific part of the system, separated by
selinux and regular linux namespace. There is quota, support for apache,
mysql, cron and tomcat. And you can access your space with ssh/sftp.

You can also take a look virt-sandbox-service, who can start a service
or a set of service in a isolated minimal container, and no headache on
upgrade due to bind mounts ( ie, everything use the same code ). And
this is using systemd.
See https://fedoraproject.org/wiki/Features/Securecontainers and various
others pages on the web.
--
Michael Scherer
Loading...