[systemd-devel] systemd-nspawn and shared private network

[Igor Bukanov]

Hello,

I am trying to see how to implement with systemd-nspawn a version of
docker's pod when a group of very lightweight containers use a
loopback interface or unix sockets to communicate with each other and
a shared network interface to communicate with the outside world.
Otherwise the containers are isolated and do not share process and
other namespaces.

My impression from the documentation is that I should create a version
of systemd-***@.service that uses JoinsNamespaceOf to join the
namespace of the main service for the pod. That main service should
configures container networking, expose ports to host etc. For that I
plan to use systemd-nspawn --network-veth ...

The problem I do not see how to pass the name of the main service
created with systemd-nspawn to that template. Obviously I can create
own unit for the main service that contains PrivateNetwork=true, but
then I cannot use --network-veth with nspawn as that configures the
namespace that nspawn creates, not the one from the unit.

Any suggestions?

[Lennart Poettering]

Post by Igor Bukanov
Hello,
I am trying to see how to implement with systemd-nspawn a version of
docker's pod when a group of very lightweight containers use a
loopback interface or unix sockets to communicate with each other and
a shared network interface to communicate with the outside world.
Otherwise the containers are isolated and do not share process and
other namespaces.
My impression from the documentation is that I should create a version
namespace of the main service for the pod. That main service should
configures container networking, expose ports to host etc. For that I
plan to use systemd-nspawn --network-veth ...

This won't work. JoinsNamespaceOf= only has an effect on networking if
you set PrivateNetwork= too. But if you use --network-veth (or the
related options) in nspawn then this means that nspawn will open its
own network namespace anyway, independent from the network namespace
it was invoked from. Hence, if you set PrivateNetwork=1 in your
service, you detach nspawn one level from the host's network
namespace, and then with nspawn's --network-veth your detach the stuf
running inside of the container one level further from the host. You
hence end up with three network namespaces in total: one for host, one
for the nspawn process itself, and a thrid one for the stuff running
inside of the container.

Post by Igor Bukanov
The problem I do not see how to pass the name of the main service
created with systemd-nspawn to that template. Obviously I can create
own unit for the main service that contains PrivateNetwork=true, but
then I cannot use --network-veth with nspawn as that configures the
namespace that nspawn creates, not the one from the unit.
Any suggestions?

The only way you can do this right now is by not using any of nspawn's
own network namespacing features, but instead use JoinsNamespaceOf= +
PrivateNetwork=1 to let the service set up things, and then use the
"ip" tool to set up what you need before that. But YMMV...

Long story short: we do not cover this nicely at the moment. And I am
not entirely how we should cover it. One option could be to add
--same-network= or so to nspawn which takes another container name and
then runs the container in the exact same network namespace as that
other container.

Lennart

[Igor Bukanov]

One option could be to add --same-network= or so to nspawn

It seems it would be better to refer to the service unit that executed
nspawn, not the container running in the namespace created with
nspawn. This way I can refer to that unit using a stable name. Another
alternative that is even more useful is to allow for any service that
specifies PrivateNetwork= also specify how that network should be
initialized and connected to the outside world using the same options
that are available with nspawn.