作者：牧原
第一個問題
limits.conf的限制在/proc/pid/limits中未生效
# cat /proc/3606/limits
Limit Soft Limit Hard Limit Units
Max processes 31202 31202 processes
Max open files 1024 4096 files在Centos7 & ubuntu 系統中，使用Systemd替代了之前的SysV。/etc/security/limits.conf文件的配置作用域縮小了。
/etc/security/limits.conf的配置，只適用於通過PAM認證登錄用戶的資源限制，它對systemd的service的資源限制不生效。因此登錄用戶的限制，通過/etc/security/limits.conf與/etc/security/limits.d下的文件設置即可。
對於systemd service的資源設置，則需修改全局配置，
全局配置文件放在/etc/systemd/system.conf和/etc/systemd/user.conf，
同時也會加載兩個對應目錄中的所有.conf文件/etc/systemd/system.conf.d/.conf和/etc/systemd/user.conf.d/.conf
system.conf是系統實例使用的，user.conf是用戶實例使用的。
vim /etc/systemd/system.conf
DefaultLimitNOFILE=100000
DefaultLimitNPROC=65535修改並重啟即可
# cat /proc/3613/limits
Limit Soft Limit Hard Limit Units
Max processes 65535 65535 processes
Max open files 100000 100000 files第二個問題
在服務裡面設置LimitNOFILE=infinity為什麼不是無窮大？
在服務裡面設置LimitNOFILE=infinity 後，通過查看pid的limit發現openfile是65536 ，而不是無窮大
查看服務配置
[root@iZwz98aynkjcxvtra0f375Z ~]# cat /etc/systemd/system/multi-user.target.wants/docker.service |grep -vi "^#"|grep -vi "^$"
[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
BindsTo=containerd.service
After=network-online.target firewalld.service containerd.service
Wants=network-online.target
Requires=docker.socket
[Service]
Type=notify
ExecStart=/usr/bin/dockerd -H fd://
ExecStartPost=/usr/sbin/iptables -P FORWARD ACCEPT
ExecReload=/bin/kill -s HUP $MAINPID
TimeoutSec=0
RestartSec=2
Restart=always
StartLimitBurst=3
StartLimitInterval=60s
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
TasksMax=infinity
Delegate=yes
KillMode=process
[Install]
WantedBy=multi-user.target查看配置效果
# cat /proc/11019/limits
Limit Soft Limit Hard Limit Units
Max cpu time unlimited unlimited seconds
Max file size unlimited unlimited bytes
Max data size unlimited unlimited bytes
Max stack size 8388608 unlimited bytes
Max core file size 0 unlimited bytes
Max resident set unlimited unlimited bytes
Max processes 31202 31202 processes
Max open files 65536 65536 files
Max locked memory 65536 65536 bytes
Max address space unlimited unlimited bytes
Max file locks unlimited unlimited locks
Max pending signals 31202 31202 signals
Max msgqueue size 819200 819200 bytes
Max nice priority 0 0
Max realtime priority 0 0
Max realtime timeout unlimited unlimited us這個是systemd的bug，低於240的版本需要手動設置才可以生效
LimitNOFILE=102400
https://github.com/systemd/systemd/issues/6559
第三個問題
為什麼openfile不能設置為unlimited
[root@iZwz98aynkjcxvtra0f375Z ~]# ulimit -n
65535
[root@iZwz98aynkjcxvtra0f375Z ~]# ulimit -n unlimited
-bash: ulimit: open files: cannot modify limit: Operation not permitted原因是 centos7裡 openfile不能大於nr_open
[root@iZwz98aynkjcxvtra0f375Z ~]# cat /proc/sys/fs/nr_open
1048576
[root@iZwz98aynkjcxvtra0f375Z ~]# ulimit -n 1048577
-bash: ulimit: open files: cannot modify limit: Operation not permitted
[root@iZwz98aynkjcxvtra0f375Z ~]# ulimit -n 1048576
[root@iZwz98aynkjcxvtra0f375Z ~]# ulimit -n
1048576第四個問題
使用supervisor管理進程（測試環境ubuntu 1604）啟動進程後，maxfile是1024
需要修改配置文件
#cat /etc/supervisor/supervisord.conf
[supervisord]
logfile=/var/log/supervisor/supervisord.log ; (main log file;default $CWD/supervisord.log)
pidfile=/var/run/supervisord.pid ; (supervisord pidfile;default supervisord.pid)
childlogdir=/var/log/supervisor ; ('AUTO' child log dir, default $TEMP)
下面這兩行
minfds=655350 ; min. avail startup file descriptors; default 1024
minprocs=65535 ; min. avail process descriptors;default 200
# cat /proc/2423/limits
Limit Soft Limit Hard Limit Units
Max cpu time unlimited unlimited seconds
Max file size unlimited unlimited bytes
Max data size unlimited unlimited bytes
Max stack size 8388608 unlimited bytes
Max core file size 0 unlimited bytes
Max resident set unlimited unlimited bytes
Max processes 65535 65535 processes
Max open files 655350 655350 files ------------修改成功
Max locked memory 65536 65536 bytes
Max address space unlimited unlimited bytes
Max file locks unlimited unlimited locks
Max pending signals 61946 61946 signals
Max msgqueue size 819200 819200 bytes
Max nice priority 0 0
Max realtime priority 0 0
Max realtime timeout unlimited unlimited us關於file-max nr_open file_nr的解釋 可參考
https://www.kernel.org/doc/Documentation/sysctl/fs.txt
外部文檔
https://www.cnblogs.com/zengkefu/p/5635153.html
https://blog.csdn.net/google0802/article/details/52304776
http://blog.cloud.360.cn/post/tuning-your-system-for-high-concurrency.html
https://blog.csdn.net/qq_38165374/article/details/104881340