Increase default file handle limits for daemons

Linux still defaults to a 1024 open file handle limit. This causes
scalability problems for libvirtd / virtlockd / virtlogd on large
hosts which might want > 1024 guest to be running. In fact if each
guest needs > 1 FD, we can't even get to 500 guests. This is not
good enough when we see machines with 100's of physical cores and
TBs of RAM.

In comparison to other memory requirements of libvirtd & related
daemons, the resource usage associated with open file handles
is essentially line noise. It is thus reasonable to increase the
limits unconditionally for all installs.

Signed-off-by: Daniel P. Berrange <berrange@redhat.com>

diff --git a/daemon/libvirtd.service.in b/daemon/libvirtd.service.in
index c72dde564f01c041433a6b98e3c8f8c49d7571ed..899abdf37e2ab9ca472c19a531dc93fcac619e3c 100644 (file)
--- a/daemon/libvirtd.service.in
+++ b/daemon/libvirtd.service.in
@@ -24,8 +24,11 @@ ExecStart=@sbindir@/libvirtd $LIBVIRTD_ARGS
 ExecReload=/bin/kill -HUP $MAINPID
 KillMode=process
 Restart=on-failure
-# Override the maximum number of opened files
-#LimitNOFILE=2048
+# At least 1 FD per guest, often 2 (eg qemu monitor + qemu agent).
+# eg if we want to support 4096 guests, we'll typically need 8192 FDs
+# If changing this, also consider virtlogd.service & virtlockd.service
+# limits which are also related to number of guests
+LimitNOFILE=8192
 
 [Install]
 WantedBy=multi-user.target

diff --git a/src/locking/virtlockd.service.in b/src/locking/virtlockd.service.in
index 69b568f8e4b41e4b4b1d5730323ce82d7501a07a..c369591267c7a97038fe383c33b3f04ff807d784 100644 (file)
--- a/src/locking/virtlockd.service.in
+++ b/src/locking/virtlockd.service.in
@@ -13,6 +13,10 @@ ExecReload=/bin/kill -USR1 $MAINPID
 # cause the machine to be fenced (rebooted), so make
 # sure we discourage OOM killer
 OOMScoreAdjust=-900
+# Needs to allow for max guests * average disks per guest
+# libvirtd.service written to expect 4096 guests, so if we
+# allow for 4 disks per guest, we get:
+LimitNOFILE=16384
 
 [Install]
 Also=virtlockd.socket

diff --git a/src/logging/virtlogd.service.in b/src/logging/virtlogd.service.in
index 09e07401499747f32f96a86444a5c338960f5d40..8b67317cb1eaf59a1e236d150536064a539cf96f 100644 (file)
--- a/src/logging/virtlogd.service.in
+++ b/src/logging/virtlogd.service.in
@@ -13,6 +13,11 @@ ExecReload=/bin/kill -USR1 $MAINPID
 # cause the machine to be fenced (rebooted), so make
 # sure we discourage OOM killer
 OOMScoreAdjust=-900
+# Need to have at least one file open per guest (eg QEMU
+# stdio log), but might be more (eg serial console logs)
+# libvirtd.service written to expect 4096 guests, so if we
+# guess at 2 log files per guest here (stdio + 1 serial):
+LimitNOFILE=8192
 
 [Install]
 Also=virtlogd.socket