Managing services with systemd

Systemd is the init system and service manager on Ubuntu 16.04 and later. It replaced Upstart and handles everything from boot to service management to logging. Love it or hate it, it’s what you’ll be working with.

systemctl basics

Check a service’s status:

sudo systemctl status nginx

This shows whether it’s running, recent log entries, and the process ID.

Start, stop, restart:

sudo systemctl start nginx
sudo systemctl stop nginx
sudo systemctl restart nginx

reload is gentler than restart — it tells the service to reread its config without dropping connections (if the service supports it):

sudo systemctl reload nginx

Enable a service to start at boot:

sudo systemctl enable nginx

Disable it:

sudo systemctl disable nginx

Check if a service is enabled:

systemctl is-enabled nginx

Writing a unit file

Custom services get unit files in /etc/systemd/system/. Here’s a basic one for a Node.js app:

[Unit]
Description=My Node.js Application
After=network.target

[Service]
Type=simple
User=appuser
WorkingDirectory=/opt/myapp
ExecStart=/usr/bin/node server.js
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target

Key fields:

• After=network.target — start after the network is up
• User=appuser — run as this user, not root
• Restart=on-failure — restart if the process exits with an error
• RestartSec=5 — wait 5 seconds before restarting

After creating or editing a unit file:

sudo systemctl daemon-reload
sudo systemctl start myapp
sudo systemctl enable myapp

The daemon-reload step is easy to forget. Systemd caches unit files, so changes don’t take effect until you reload.

Service types

Type=simple is the default. The process started by ExecStart is the main process.

Type=forking is for traditional daemons that fork into the background. You need to specify PIDFile= so systemd can track the main process.

Type=oneshot is for tasks that run and exit. Use RemainAfterExit=yes if the service should be considered “active” even after the process exits.

Viewing logs

Systemd integrates with journald. View logs for a service:

journalctl -u nginx
journalctl -u nginx -f          # follow in real time
journalctl -u nginx --since today
journalctl -u nginx --since "2024-01-15" --until "2024-01-16"

The journal is persistent if /var/log/journal/ exists. Otherwise logs are lost on reboot. To make logs persistent:

sudo mkdir -p /var/log/journal
sudo systemd-tmpfiles --create --prefix /var/log/journal

Targets

Targets are groups of services. They’re the systemd equivalent of runlevels:

systemctl get-default                  # current default target
sudo systemctl set-default multi-user.target  # text mode
sudo systemctl set-default graphical.target   # graphical mode

multi-user.target is what servers use. graphical.target adds a desktop environment.

Dependency management

After= and Before= control ordering. Requires= and Wants= control dependencies:

[Unit]
Description=My App
After=postgresql.service
Requires=postgresql.service

Requires= means the service fails if the dependency fails. Wants= is softer — it tries to start the dependency but continues if it can’t.

Resource limits

Control memory and CPU usage:

[Service]
MemoryLimit=512M
CPUQuota=50%

This keeps a runaway service from consuming all system resources.

Common mistakes

Forgetting daemon-reload. You edit a unit file, restart the service, and nothing changed. Systemd is still using the cached version.

Using ExecStartPre for the main process. ExecStartPre runs before the main process and is for setup commands. The main service goes in ExecStart.

Not checking logs. When a service fails to start, systemctl status shows recent log output. Read it. The answer is usually there.

Remarks

Systemd takes some getting used to, but systemctl and journalctl cover most of what you need. The unit file format is readable, and once you’ve written a few, it becomes straightforward. The biggest improvement over older init systems is the integrated logging — journalctl -u service beats digging through scattered log files.