SystemD and Networking

It turns out that network.target does not mean a fully functional network, just that the basic subsystem is up and running. For services that bind to the universal address 0.0.0.0 this isn’t an issue, but if it explicitly binds to an interface or IP it may fail during boot if the interface or IP isn’t available when it tries to start.

I recently had this problem with Samba. The following was in /etc/samba/smb.conf

[global]
interface = eth1
bind interfaces only = yes

Which caused an error on boot

open_sockets_smbd: No sockets available to bind to.

So what we actually need is network-online.target. This target doesn’t complete until after all configured interfaces have come up. This can make your boot slower by making DHCP a blocking step before some services, but in some cases like this one we need that extra time.

So based on my earlier post about overriding SystemD, we’ll create an override and make smb.service also depend on network-online.target.

# cat /etc/systemd/system/smb.service.d/override.conf
[Unit]
After=network-online.target

Now Samba will start up correctly on a reboot!

Further reading:

Advertisements

SystemD unit customization

What is the best, most correct way to customize SystemD unit files that come from packages?

Packages will leave their files in /usr/lib/systemd/system . I used to just copy those files into /etc/systemd/system and edit them as needed. But is this the best way?

Yes! You can create override files! If you create a directory like /etc/systemd/system/some-unit.service.d and create files named *.conf in there they will also be ready by SystemD (after a systemctl daemon-reload)

One trick is that to override some things like ExecStart= you need to first declare an empty one to reset it if the Type= isn’t oneshot.

[Service]
ExecStart=
ExecStart=/usr/sbin/httpd $OPTIONS -DFOREGROUND

You can also use systemctl edit some-unit.service to create and edit the file in one go. This will create override.conf for you, so that’s probably a best bet for naming any override files you might create.

I learned this from this excellent answer on AskUbuntu.

Systemd dependencies

Do you want to make sure two separate services are connected in some way? That one service is up when the other is or even starts first? Thanks to systemd this is now super easy!

In the dark old days of SysV (/etc/init.d) services were started serially during boot and individually on demand. You could adjust the start and stop order by editing a special comment in the init file. For example

# chkconfig: - 85 15

would have the service start at priority 85 and stop at priority 15. Priorities are from 0 to 99 and are done in ascending order. So this would start pretty late in the boot and stop pretty early in the shutdown.

With directives like Requires, Wants, Before, and After you can give systemd more info on what you want. systemd can start things in parallel and will use these hints to build its dependency tree and start or stop everything in the right order.

Packages will install their systemd config files into /usr/lib/systemd/system/ . Don’t edit these files. If you want to make changes, copy them to /etc/systemd/system/ .

I’ll use mariadb and httpd as examples here. It makes sense that we want the database to be up and running before Apache so that we don’t serve broken pages. Lets explore our options and see how they each act under different circumstances.

Requires

This effectively makes one unit cascade the starting of another. If the one is enabled, the other is also effectively enabled. Both units will be started in parallel, but if one of the required units fails to start, the requiring unit will get stopped again.

We’ll add Requires=mariadb.service to the [Unit] section of /etc/systemd/system/httpd.service .

When we issue a start of httpd, mariadb is started in parallel.

When we issue a stop of httpd, mariadb keeps running.

If we break the mariadb config to prevent it from starting, then start httpd, it starts, but shows an error. I think despite what the documentation says, this is just a race condition since both are started in parallel. We’ll get into ordering in a bit.

If when both are running, we stop mariadb, httpd will stop automatically.

Wants

This is just a weaker version of Requires and is the recommended option. If the wanted service fails to start, the wanting service will still start.

When we issue a start of httpd, mariadb is started in parallel.

When we issue a stop of httpd, mariadb keeps running.

If we break the mariadb config to prevent it from starting, then start httpd, it starts with no error.

If when both are running, we stop mariadb, httpd will keep running.

After

Now we can get to some ordering! We’ll tell httpd to start After mariadb.

After=mariadb.service

On its own, it doesn’t enforce the starting of mariadb, just that if both happen to be starting at the same time, mariadb should be started first. And if you are stopping both at the same time, mariadb should be stopped last.

systemctl start httpd mariadb

will start them in order.

Before is just the inverse relationship. You could instead specify in the config for mariadb to start before httpd.

If we combine Requires and After we get what we want. httpd forcing mariadb to start first and fail to start if mariadb fails.

When we issue a start of httpd, mariadb is started first.

When we issue a stop of httpd, mariadb keeps running.

If we break the mariadb config to prevent it from starting, then start httpd, it will fail to start.

If when both are running, we stop mariadb, httpd will stop first.

More info:

Understanding Systemd Units and Unit Files

systemd.unit(5)