[systemd-devel] Setting up network interfaces for containers with --private-network

[Spencer Baugh]

Hi,

Currently, I can manually set up (or set up with a script) a veth, then
move it in to a systemd-nspawn container with
--network-interface. However, if the container tries to restart (or
exits and needs to be restarted), the network namespace of the container
is destroyed and therefore so is the veth that that namespace
contains. Thus, if the interface isn't recreated by some external force
in the time between stopping and restarting, the next invocation of
systemd-nspawn --network-interface=someif will fail.

To state the problem again more concretely, if I create a veth, assign
one end of the veth to a container started with systemd-nspawn
--network-interface=veth0, then run "reboot" inside the container, the
container will shut down and fail to come back up, as veth0 is
destroyed.

Perhaps, I could hack up some shell script to wrap system-nspawn and
make sure that whenever I run it, an appropriate network interface is
created before actually running systemd-nspawn
--network-interface=someif, but I don't think that's really the best
approach.

I could use --network-bridge on a bridge and get the veth made
automatically, as long as all I wanted to do was add the other end of
the veth to a bridge, and it would be remade whenever the container
restarted. But I think this might be the wrong place for this magic to
live; it's not very configurable and seems rather strange to have
systemd-nspawn doing ad-hoc networking tasks like this.

I'm curious about how this should be done; it seems very important for
serious use of containers. Perhaps networkd could be used to do
something here? What is the best practice?

Thanks,
Spencer Baugh

[Lennart Poettering]

Post by Spencer Baugh
Hi,
Currently, I can manually set up (or set up with a script) a veth, then
move it in to a systemd-nspawn container with
--network-interface. However, if the container tries to restart (or
exits and needs to be restarted), the network namespace of the container
is destroyed and therefore so is the veth that that namespace
contains. Thus, if the interface isn't recreated by some external force
in the time between stopping and restarting, the next invocation of
systemd-nspawn --network-interface=someif will fail.
To state the problem again more concretely, if I create a veth, assign
one end of the veth to a container started with systemd-nspawn
--network-interface=veth0, then run "reboot" inside the container, the
container will shut down and fail to come back up, as veth0 is
destroyed.
Perhaps, I could hack up some shell script to wrap system-nspawn and
make sure that whenever I run it, an appropriate network interface is
created before actually running systemd-nspawn
--network-interface=someif, but I don't think that's really the best
approach.
I could use --network-bridge on a bridge and get the veth made
automatically, as long as all I wanted to do was add the other end of
the veth to a bridge, and it would be remade whenever the container
restarted. But I think this might be the wrong place for this magic to
live; it's not very configurable and seems rather strange to have
systemd-nspawn doing ad-hoc networking tasks like this.
I'm curious about how this should be done; it seems very important for
serious use of containers. Perhaps networkd could be used to do
something here? What is the best practice?

So far I'd recommend running networkd on the host and in the
container. If you run it on the host, then it will automatically
configure the hos side of each of nspawn's veth links with a new IP
range, and be a DHCP server on it, as well as do IP
masquerading. Connectivity will hence "just work", if you use networkd
in most cases.

Of course, if you want to manually set up things, without networkd or
an equivalent service, then a lot of things will be more manual: one
way could be to add some script to ExecStartPre= of the service to set
things up for you each time you start the container.

Another option could be to use write a service that sets up the
interface, uses PrivateNetwork= and then use JoinsNamespaceOf= on the
container service towards that service, and turn off nspawn's own
private networking switch. That way PID1 would already set up the
joint namespace for your container, and ensure it is set up properly
by your setup service. And as long as either the setup service or the
container is running the network namespace will stay referenced.

Lennart

[Spencer Baugh]

Post by Lennart Poettering
So far I'd recommend running networkd on the host and in the
container. If you run it on the host, then it will automatically
configure the hos side of each of nspawn's veth links with a new IP
range, and be a DHCP server on it, as well as do IP
masquerading. Connectivity will hence "just work", if you use networkd
in most cases.

This is in the case where I use --network-bridge, right? Because
otherwise there is no veth to be automatically configured.

Yes, in that case, it is of course very simple, but it is not at all
configurable. I have one thing and one thing only that I want to
configure: The IP address that a given container receives. This seems
like a reasonable thing to want to configure; ultimately there have to
be fixed IP addresses somewhere, and I have a use for containers that
would benefit from having fixed IP addresses.

The way I currently fix the IP address that the container receives is by
fixing the MAC address of the veth; since I am using IPv6 and radvd, the
IP address is deterministically generated from the MAC address. So it
would be helpful if there was a way to do fix the MAC address in
nspawn. Would you accept a patch to add an option to nspawn to specify a
MAC address for the veth? Or is there a better way to go about this?

Post by Lennart Poettering
Of course, if you want to manually set up things, without networkd or
an equivalent service, then a lot of things will be more manual: one
way could be to add some script to ExecStartPre= of the service to set
things up for you each time you start the container.
Another option could be to use write a service that sets up the
interface, uses PrivateNetwork= and then use JoinsNamespaceOf= on the
container service towards that service, and turn off nspawn's own
private networking switch. That way PID1 would already set up the
joint namespace for your container, and ensure it is set up properly
by your setup service. And as long as either the setup service or the
container is running the network namespace will stay referenced.

Hmm, that is an interesting approach... it would be nice to be able to
have networkd be setting up the interface here, though.

I am interested in using networkd to do these things, but at the moment
it doesn't seem to have the required level of power.

[Lennart Poettering]

Post by Spencer Baugh

Post by Lennart Poettering
So far I'd recommend running networkd on the host and in the
container. If you run it on the host, then it will automatically
configure the hos side of each of nspawn's veth links with a new IP
range, and be a DHCP server on it, as well as do IP
masquerading. Connectivity will hence "just work", if you use networkd
in most cases.

This is in the case where I use --network-bridge, right? Because
otherwise there is no veth to be automatically configured.

No, not with --network-bridge, but with --network-veth, i.e. -n or

Post by Spencer Baugh
Yes, in that case, it is of course very simple, but it is not at all
configurable. I have one thing and one thing only that I want to
configure: The IP address that a given container receives. This seems
like a reasonable thing to want to configure; ultimately there have to
be fixed IP addresses somewhere, and I have a use for containers that
would benefit from having fixed IP addresses.
The way I currently fix the IP address that the container receives is by
fixing the MAC address of the veth; since I am using IPv6 and radvd, the
IP address is deterministically generated from the MAC address. So it
would be helpful if there was a way to do fix the MAC address in
nspawn. Would you accept a patch to add an option to nspawn to specify a
MAC address for the veth? Or is there a better way to go about this?

The MAC address is currently generated as hash value from the
container name, it hence should be completely stable already as long
as you keep using the same name for the container?

maybe the ipvlan stuff could work for you?

Post by Spencer Baugh

Post by Lennart Poettering
Another option could be to use write a service that sets up the
interface, uses PrivateNetwork= and then use JoinsNamespaceOf= on the
container service towards that service, and turn off nspawn's own
private networking switch. That way PID1 would already set up the
joint namespace for your container, and ensure it is set up properly
by your setup service. And as long as either the setup service or the
container is running the network namespace will stay referenced.

Hmm, that is an interesting approach... it would be nice to be able to
have networkd be setting up the interface here, though.

Well, it can, but only if you run it inside of the container. I am
pretty sure the networkd of the host should not configure the
interfaces inside of it...

Post by Spencer Baugh
I am interested in using networkd to do these things, but at the moment
it doesn't seem to have the required level of power.

what do you mean precisely with this?

Lennart

[Spencer Baugh]

Post by Lennart Poettering

Post by Spencer Baugh
Yes, in that case, it is of course very simple, but it is not at all
configurable. I have one thing and one thing only that I want to
configure: The IP address that a given container receives. This seems
like a reasonable thing to want to configure; ultimately there have to
be fixed IP addresses somewhere, and I have a use for containers that
would benefit from having fixed IP addresses.
The way I currently fix the IP address that the container receives is by
fixing the MAC address of the veth; since I am using IPv6 and radvd, the
IP address is deterministically generated from the MAC address. So it
would be helpful if there was a way to do fix the MAC address in
nspawn. Would you accept a patch to add an option to nspawn to specify a
MAC address for the veth? Or is there a better way to go about this?

The MAC address is currently generated as hash value from the
container name, it hence should be completely stable already as long
as you keep using the same name for the container?

Well, generally I want to know what MAC/IP address a machine/container
will receive in advance of actually starting it. I could start it once
and immediately stop it to see and record what MAC address is generated
for a given name, or copy the code to generate the MAC address out of
nspawn.c, but neither of those seem like good options.

Post by Lennart Poettering
maybe the ipvlan stuff could work for you?

It's possible, but then I'd be back to the situation where I need to
write a script to keep bringing up the ipvlan devices before starting
the container. Unless ipvlan devices don't disappear when the namespace
disappears?

Post by Lennart Poettering

Post by Spencer Baugh

Post by Lennart Poettering
Another option could be to use write a service that sets up the
interface, uses PrivateNetwork= and then use JoinsNamespaceOf= on the
container service towards that service, and turn off nspawn's own
private networking switch. That way PID1 would already set up the
joint namespace for your container, and ensure it is set up properly
by your setup service. And as long as either the setup service or the
container is running the network namespace will stay referenced.

Hmm, that is an interesting approach... it would be nice to be able to
have networkd be setting up the interface here, though.

Well, it can, but only if you run it inside of the container. I am
pretty sure the networkd of the host should not configure the
interfaces inside of it...

Post by Spencer Baugh
I am interested in using networkd to do these things, but at the moment
it doesn't seem to have the required level of power.

what do you mean precisely with this?

I mean that instead of writing another service (probably a shell script)
to set up the interface on the host, using the PrivateNetwork= and
JoinsNamespaceOf= trick, instead have networkd set up the interface on
the host inside a network namespace and use the same kind of trick.

Or am I misunderstanding the role of networkd? It seems like if I am
writing a service that represents "the network interface and namespace
for this container", I am doing something that networkd should
ultimately do.

"Set up the interface" here just means "create the interface with a
specific MAC address", of course.

[Lennart Poettering]

Post by Spencer Baugh

Post by Lennart Poettering
The MAC address is currently generated as hash value from the
container name, it hence should be completely stable already as long
as you keep using the same name for the container?

Well, generally I want to know what MAC/IP address a machine/container
will receive in advance of actually starting it. I could start it once
and immediately stop it to see and record what MAC address is generated
for a given name, or copy the code to generate the MAC address out of
nspawn.c, but neither of those seem like good options.

Sidenote: if this is about having stable names to refer to containers,
note that nss-mycontainers adds those automatically. If enabled, then
all local container names will be resolvable, automatically. It's
often hence unnecessary to have fixed IP addresses for this at all.

Post by Spencer Baugh

Post by Lennart Poettering
maybe the ipvlan stuff could work for you?

It's possible, but then I'd be back to the situation where I need to
write a script to keep bringing up the ipvlan devices before starting
the container. Unless ipvlan devices don't disappear when the namespace
disappears?

Nope, this is the same for ipvlan, macvlan, veth links.

Post by Spencer Baugh

Post by Lennart Poettering

Post by Spencer Baugh
I am interested in using networkd to do these things, but at the moment
it doesn't seem to have the required level of power.

what do you mean precisely with this?

I mean that instead of writing another service (probably a shell script)
to set up the interface on the host, using the PrivateNetwork= and
JoinsNamespaceOf= trick, instead have networkd set up the interface on
the host inside a network namespace and use the same kind of trick.

Well, I mean how useful would this actually be? THis would only work
for static configuration, everything more complex requires a daemon
watching the interface continously and that's really hard to do for a
set of network interfaces in a different network namespace.

Also, trivial static IP configuration is seldom sufficient, you at
least need to also provide DNS configuration, and if you don't use
DHCP or something similar then you need to configure that inside the
container anyway. But if you do that you might as well configure the
static IP addresses in it too, so what is gained by doing this from a
networkd outside of the cotnainer?

Post by Spencer Baugh
Or am I misunderstanding the role of networkd? It seems like if I am
writing a service that represents "the network interface and namespace
for this container", I am doing something that networkd should
ultimately do.

Sure, absolutely. But our idea so far was that networkd should run
inside the container to configure the container's network, and on the
host to configure the host's network, but not to cross this boundary
and have the host networkd configure the container's network.

Post by Spencer Baugh
"Set up the interface" here just means "create the interface with a
specific MAC address", of course.

Well, of course, we could beef up systemd-nspawn and allow it to take
configurable IP or MAC addresses on the command line, and then it
would become a second networkd, and we already have one of those...

Lennart

[Spencer Baugh]

Post by Lennart Poettering

Post by Spencer Baugh

Post by Lennart Poettering
The MAC address is currently generated as hash value from the
container name, it hence should be completely stable already as long
as you keep using the same name for the container?

Well, generally I want to know what MAC/IP address a machine/container
will receive in advance of actually starting it. I could start it once
and immediately stop it to see and record what MAC address is generated
for a given name, or copy the code to generate the MAC address out of
nspawn.c, but neither of those seem like good options.

Sidenote: if this is about having stable names to refer to containers,
note that nss-mycontainers adds those automatically. If enabled, then
all local container names will be resolvable, automatically. It's
often hence unnecessary to have fixed IP addresses for this at all.

It is about stable names, but I believe those names need to be usable
from off the host.

Post by Lennart Poettering

Post by Spencer Baugh

Post by Lennart Poettering

Post by Spencer Baugh
I am interested in using networkd to do these things, but at the moment
it doesn't seem to have the required level of power.

what do you mean precisely with this?

I mean that instead of writing another service (probably a shell script)
to set up the interface on the host, using the PrivateNetwork= and
JoinsNamespaceOf= trick, instead have networkd set up the interface on
the host inside a network namespace and use the same kind of trick.

Well, I mean how useful would this actually be? THis would only work
for static configuration, everything more complex requires a daemon
watching the interface continously and that's really hard to do for a
set of network interfaces in a different network namespace.

All that I want to do is configuration that can be done at the time of
first creating the interface - like setting the MAC address. That is all
the script that I am using at the moment does, everything else is done
by networkd.

Post by Lennart Poettering
Also, trivial static IP configuration is seldom sufficient, you at
least need to also provide DNS configuration, and if you don't use
DHCP or something similar then you need to configure that inside the
container anyway. But if you do that you might as well configure the
static IP addresses in it too, so what is gained by doing this from a
networkd outside of the cotnainer?

Post by Spencer Baugh
Or am I misunderstanding the role of networkd? It seems like if I am
writing a service that represents "the network interface and namespace
for this container", I am doing something that networkd should
ultimately do.

Sure, absolutely. But our idea so far was that networkd should run
inside the container to configure the container's network, and on the
host to configure the host's network, but not to cross this boundary
and have the host networkd configure the container's network.

Hmm, yes, but I think the problem is the configuration done at
interface-creation-time. It seems to me that that configuration
currently does not fit naturally in either the host networkd or the
container networkd.

Post by Lennart Poettering

Post by Spencer Baugh
"Set up the interface" here just means "create the interface with a
specific MAC address", of course.

Well, of course, we could beef up systemd-nspawn and allow it to take
configurable IP or MAC addresses on the command line, and then it
would become a second networkd, and we already have one of those...

Yes, but what else can configure the interfaces at creation time?

[Lennart Poettering]

Post by Spencer Baugh

Post by Lennart Poettering
Also, trivial static IP configuration is seldom sufficient, you at
least need to also provide DNS configuration, and if you don't use
DHCP or something similar then you need to configure that inside the
container anyway. But if you do that you might as well configure the
static IP addresses in it too, so what is gained by doing this from a
networkd outside of the cotnainer?

Post by Spencer Baugh
Or am I misunderstanding the role of networkd? It seems like if I am
writing a service that represents "the network interface and namespace
for this container", I am doing something that networkd should
ultimately do.

Sure, absolutely. But our idea so far was that networkd should run
inside the container to configure the container's network, and on the
host to configure the host's network, but not to cross this boundary
and have the host networkd configure the container's network.

Hmm, yes, but I think the problem is the configuration done at
interface-creation-time. It seems to me that that configuration
currently does not fit naturally in either the host networkd or the
container networkd.

Well, again, I doubt that configuration exclusivel at
interface-creation-time will be useful for more than the most trivial
cases, already because as mentioned it would not cover DNS server
configuration and thelike.

I am a bit careful with supporting something like this directly in
nspawn if it cannot even cover the most basic cases...

If you really want fixed IP addresses, I think this could work:

We add configurability for the DHCP server address range in networkd,
including taking ranges that contain a single IP address. You could
then assign fixed addresses to your containers simply by dropping a
.network snippet for them, that only contains a single dhcp range IP
address for it. THat should work, no?

Lennart

[Spencer Baugh]

Post by Lennart Poettering

Post by Spencer Baugh

Post by Lennart Poettering
Also, trivial static IP configuration is seldom sufficient, you at
least need to also provide DNS configuration, and if you don't use
DHCP or something similar then you need to configure that inside the
container anyway. But if you do that you might as well configure the
static IP addresses in it too, so what is gained by doing this from a
networkd outside of the cotnainer?

Post by Spencer Baugh
Or am I misunderstanding the role of networkd? It seems like if I am
writing a service that represents "the network interface and namespace
for this container", I am doing something that networkd should
ultimately do.

Sure, absolutely. But our idea so far was that networkd should run
inside the container to configure the container's network, and on the
host to configure the host's network, but not to cross this boundary
and have the host networkd configure the container's network.

Hmm, yes, but I think the problem is the configuration done at
interface-creation-time. It seems to me that that configuration
currently does not fit naturally in either the host networkd or the
container networkd.

Well, again, I doubt that configuration exclusivel at
interface-creation-time will be useful for more than the most trivial
cases, already because as mentioned it would not cover DNS server
configuration and thelike.

Sure, I'm in agreement, but there are things that can only be
"configured" at interface-creation-time. As a trivial example, the type
of the interface. And it looks like for ipvlan you can only choose L2 or
L3 mode at creation-time.

Additionally, it seems to me that the MAC address, at least, is
something that you might want to configure only at creation-time and not
change later.

The earlier two examples are necessary, the third example, the MAC
address, is just nice-to-have. But since there are things that actually
really must be configured at creation-time, it seems that eventually it
will be necessary to figure out a best practice or mechanism for such
configuration. And that method might as well also allow the nice-to-have
of configuring the MAC address at creation time.

Post by Lennart Poettering
We add configurability for the DHCP server address range in networkd,
including taking ranges that contain a single IP address. You could
then assign fixed addresses to your containers simply by dropping a
.network snippet for them, that only contains a single dhcp range IP
address for it. THat should work, no?

This would be a nice feature and would work for many use cases.

My problem with this solution is that I'm using IPv6. With IPv6, I can
set up stateless autoconfiguration to "assign" IPs to interfaces. In
that case the IP is directly determined by the MAC address, so it is not
possible (AFAIK?) to specify a range of IPs of length 1 to force that
single IP to be assigned to a specific interface.

I would somewhat prefer to be using this feature of IPv6, rather than
using DHCPv6; and anyway, networkd doesn't support DHCPv6 right now,
right? So this doesn't necessarily work for me.

The mapping from MAC address to IPv6 address is just a matter of
encoding the MAC address in the last 64 bits of the IPv6 address. So if
I know the MAC address of the container, and maybe listen a little on
the network interface to see what routing prefix is being advertised, I
know the IPv6 address that the container will get. Maybe something
(networkd, networkctl, machined, machinectl?) could reveal this
information? Just an idea.

The stateless autoconfiguration of IPv6 seems pretty similar to
systemd's own aims, it would be nice if they worked well together.

[Lennart Poettering]

Post by Spencer Baugh

Post by Lennart Poettering

Post by Spencer Baugh

Post by Lennart Poettering
Also, trivial static IP configuration is seldom sufficient, you at
least need to also provide DNS configuration, and if you don't use
DHCP or something similar then you need to configure that inside the
container anyway. But if you do that you might as well configure the
static IP addresses in it too, so what is gained by doing this from a
networkd outside of the cotnainer?

Post by Spencer Baugh
Or am I misunderstanding the role of networkd? It seems like if I am
writing a service that represents "the network interface and namespace
for this container", I am doing something that networkd should
ultimately do.

Sure, absolutely. But our idea so far was that networkd should run
inside the container to configure the container's network, and on the
host to configure the host's network, but not to cross this boundary
and have the host networkd configure the container's network.

Hmm, yes, but I think the problem is the configuration done at
interface-creation-time. It seems to me that that configuration
currently does not fit naturally in either the host networkd or the
container networkd.

Well, again, I doubt that configuration exclusivel at
interface-creation-time will be useful for more than the most trivial
cases, already because as mentioned it would not cover DNS server
configuration and thelike.

Sure, I'm in agreement, but there are things that can only be
"configured" at interface-creation-time. As a trivial example, the type
of the interface. And it looks like for ipvlan you can only choose L2 or
L3 mode at creation-time.
Additionally, it seems to me that the MAC address, at least, is
something that you might want to configure only at creation-time and not
change later.

networkd considers the MAC address that can be changed for the network
you connect to, and can hence change dynamically depending on the
network you pick.

Hmm, there's indeed a gap here though: as .link files are applied by
udev, not by networkd, and udev is not available in containers, .link
files are currently not applied at all in containers. It might be an
option to do this within networkd instead of udev when it is run in a
container.

Of course, this will then only cover the props that can be changed
with .link files, not the ones that have to be specified at link
creation time.

For those, things are nasty. I mean, for things like ipvlan which
relate to an interface outside of the container, this means we need to
create them outside of the container, and thus they need to be
configured initially outside of the container.

So far we handled all this inside .netdev files, that are processed by
networkd. We have limited support for creating these devices also with
nspawn (i.e. the --network-veth, --network-macvlan and
--network-ipvlan switches), but I'd be really careful with turning
them into more than just basic switches, I'd really leave more complex
cases with networkd.

Now, I am not really sure how would could hook this up... After all
as you say it will not be sufficient to create the netdevs once before
nspawn uses them, they need to be created before each time before
nspawn uses them.

As soon as networkd gains a bus interface maybe an option could be to
hook up nspawn's --network-interface= with it: if the specified
interface doesn't exist, nspawn could synchronously ask networkd to
create it. With that in place you could then configure .netdev files
outside of the container, and neatly pass them on into the container,
without races. Would that fix your issue?

Post by Spencer Baugh

Post by Lennart Poettering
We add configurability for the DHCP server address range in networkd,
including taking ranges that contain a single IP address. You could
then assign fixed addresses to your containers simply by dropping a
.network snippet for them, that only contains a single dhcp range IP
address for it. THat should work, no?

This would be a nice feature and would work for many use cases.
My problem with this solution is that I'm using IPv6. With IPv6, I can
set up stateless autoconfiguration to "assign" IPs to interfaces. In
that case the IP is directly determined by the MAC address, so it is not
possible (AFAIK?) to specify a range of IPs of length 1 to force that
single IP to be assigned to a specific interface.
I would somewhat prefer to be using this feature of IPv6, rather than
using DHCPv6; and anyway, networkd doesn't support DHCPv6 right now,
right? So this doesn't necessarily work for me.

True. It's certainly our plan to support it eventually.

Lennart

[Spencer Baugh]

Post by Lennart Poettering

Post by Spencer Baugh

Post by Lennart Poettering
Well, again, I doubt that configuration exclusivel at
interface-creation-time will be useful for more than the most trivial
cases, already because as mentioned it would not cover DNS server
configuration and thelike.

Sure, I'm in agreement, but there are things that can only be
"configured" at interface-creation-time. As a trivial example, the type
of the interface. And it looks like for ipvlan you can only choose L2 or
L3 mode at creation-time.
Additionally, it seems to me that the MAC address, at least, is
something that you might want to configure only at creation-time and not
change later.

networkd considers the MAC address that can be changed for the network
you connect to, and can hence change dynamically depending on the
network you pick.
Hmm, there's indeed a gap here though: as .link files are applied by
udev, not by networkd, and udev is not available in containers, .link
files are currently not applied at all in containers. It might be an
option to do this within networkd instead of udev when it is run in a
container.
Of course, this will then only cover the props that can be changed
with .link files, not the ones that have to be specified at link
creation time.
For those, things are nasty. I mean, for things like ipvlan which
relate to an interface outside of the container, this means we need to
create them outside of the container, and thus they need to be
configured initially outside of the container.
So far we handled all this inside .netdev files, that are processed by
networkd. We have limited support for creating these devices also with
nspawn (i.e. the --network-veth, --network-macvlan and
--network-ipvlan switches), but I'd be really careful with turning
them into more than just basic switches, I'd really leave more complex
cases with networkd.
Now, I am not really sure how would could hook this up... After all
as you say it will not be sufficient to create the netdevs once before
nspawn uses them, they need to be created before each time before
nspawn uses them.
As soon as networkd gains a bus interface maybe an option could be to
hook up nspawn's --network-interface= with it: if the specified
interface doesn't exist, nspawn could synchronously ask networkd to
create it. With that in place you could then configure .netdev files
outside of the container, and neatly pass them on into the container,
without races. Would that fix your issue?

Yes, that sounds like it would work. This would destroy and recreate the
interface on reboot, which is fine for my use case.

There might at some point be a desire by someone else to have the
interface not be destroyed on reboot. At that point it would just
require teaching networkd something about network namespaces, which
shouldn't be hard. I don't want that myself, of course.

As a stopgap measure until the feature you described is ready, I'll use
the .service with PrivateNetwork= and JoinsNamespaceOf= suggestion you
made earlier.

Post by Lennart Poettering

Post by Spencer Baugh

Post by Lennart Poettering
We add configurability for the DHCP server address range in networkd,
including taking ranges that contain a single IP address. You could
then assign fixed addresses to your containers simply by dropping a
.network snippet for them, that only contains a single dhcp range IP
address for it. THat should work, no?

This would be a nice feature and would work for many use cases.
My problem with this solution is that I'm using IPv6. With IPv6, I can
set up stateless autoconfiguration to "assign" IPs to interfaces. In
that case the IP is directly determined by the MAC address, so it is not
possible (AFAIK?) to specify a range of IPs of length 1 to force that
single IP to be assigned to a specific interface.
I would somewhat prefer to be using this feature of IPv6, rather than
using DHCPv6; and anyway, networkd doesn't support DHCPv6 right now,
right? So this doesn't necessarily work for me.

True. It's certainly our plan to support it eventually.

That's in reference to just DHCPv6, right? What about stateless
autoconfiguration, out of curiosity?

[Lennart Poettering]

Post by Spencer Baugh

Post by Lennart Poettering
As soon as networkd gains a bus interface maybe an option could be to
hook up nspawn's --network-interface= with it: if the specified
interface doesn't exist, nspawn could synchronously ask networkd to
create it. With that in place you could then configure .netdev files
outside of the container, and neatly pass them on into the container,
without races. Would that fix your issue?

Yes, that sounds like it would work. This would destroy and recreate the
interface on reboot, which is fine for my use case.

I'll add it to the TODO list.

It's not actually nspawn that destroys these interfaces, it's the
kernel.

Post by Spencer Baugh
There might at some point be a desire by someone else to have the
interface not be destroyed on reboot. At that point it would just
require teaching networkd something about network namespaces, which
shouldn't be hard. I don't want that myself, of course.

Hmm, I am not sure that that teaching networkd namespacing is that
easy or desirable.

But anyway, we can discuss this when this comes up.

Post by Spencer Baugh

Post by Lennart Poettering

Post by Spencer Baugh
I would somewhat prefer to be using this feature of IPv6, rather than
using DHCPv6; and anyway, networkd doesn't support DHCPv6 right now,
right? So this doesn't necessarily work for me.

True. It's certainly our plan to support it eventually.

That's in reference to just DHCPv6, right? What about stateless
autoconfiguration, out of curiosity?

THe code for that is already in place.

Lennart