▌1. 安装
安装supervisor
pip install supervisor
运行echo_supervisord_conf程序生成supervisor的初始化配置文件
echo_supervisord_conf > /etc/supervisord.conf
修改/etc/supervisord.conf中的include参数,将/etc/supervisor_conf目录添加到include中
vim /etc/supervisord.conf
前面的分号表示注释,记得去掉
[include]
files = /etc/supervisor_conf/*.ini
在supervisord.conf
中添加下面代码:
[group:pyspider]
program=pyspider-webui,pyspider-scheduler,pyspider-processor,pyspider-result_worker,pyspider-fetcher,pyspider-phantomjs
priority=999
[program:pyspider-webui]
directory=/root/pyspider
command=pyspider -c /root/pyspider/config.json webui
autostart=true
autorestart=true
priority=905
user=root
stdout_logfile_maxbytes = 20MB
stdout_logfile_backups = 20
stdout_logfile = /root/pyspider/logs/webui.log
[program:pyspider-scheduler]
directory=/root/pyspider
command=pyspider -c /root/pyspider/config.json scheduler
autostart=true
autorestart=true
priority=900
user=root
stdout_logfile_maxbytes = 20MB
stdout_logfile_backups = 20
stdout_logfile = /root/pyspider/logs/scheduler.log
[program:pyspider-processor]
command=pyspider -c /root/pyspider/config.json processor
directory=/root/pyspider
autostart=true
autorestart=true
priority=903
user=root
stdout_logfile_maxbytes = 20MB
stdout_logfile_backups = 20
stdout_logfile = /root/pyspider/logs/processor.log
[program:pyspider-result_worker]
command=pyspider -c /root/pyspider/config.json result_worker
directory=/root/pyspider
autostart=true
autorestart=true
priority=904
user=root
stdout_logfile_maxbytes = 20MB
stdout_logfile_backups = 20
stdout_logfile = /root/pyspider/logs/worker.log
[program:pyspider-fetcher]
command=pyspider -c /root/pyspider/config.json --phantomjs-proxy="localhost:25555" fetcher
directory=/root/pyspider
autostart=true
autorestart=true
priority=902
user=root
stdout_logfile_maxbytes = 20MB
stdout_logfile_backups = 20
stdout_logfile = /root/pyspider/logs/fetcher.log
[program:pyspider-phantomjs]
command=pyspider -c /root/pyspider/config.json phantomjs
directory=/root/pyspider
autostart=true
autorestart=true
stopasgroup=true
stopsignal=QUIT
;上面两条针对supervisor停止phantomjs进程
priority=901
user=root
stdout_logfile_maxbytes = 20MB
stdout_logfile_backups = 20
stdout_logfile = /root/pyspider/logs/phantomjs.log
注:
command=pyspider
中的pyspider
如果是用虚拟环境的话,填写虚拟环境的绝对路径,如/root/pyspider/env/bin/pyspider
- 我的json文件在/root/pyspider/,所以直接写上绝对路径
-c /root/pyspider/config.json
/root/pyspider/logs/webui.log
这个路径就是下面logs
的绝对路径
▌2. 新建文件logs
新建pyspider 需要的logs(如果用宝塔可以直接新建文件)
cd /root/pyspider/
mkdir logs
cd logs
vim webui.log
vim scheduler.log
vim processor.log
vim worker.log
vim fetcher.log
vim phantomjs.log
▌3. 启动服务
supervisord -c /etc/supervisord.conf
注意:在执行第一条命令出现以下提示信息时:
Error: Another program is already listening on a port that one of our HTTP servers is configured to use.
Shut this program down first before starting supervisord.
For help, use /usr/bin/supervisord –h
是因为有一个使用supervisor配置的应用程序正在运行,需要在supervisorctl交互终端执行 shutdown命令终止它,或者执行reload 来重新启动配置中的所有程序,或者在xhell命令行直接执行
find / -name supervisor.sock
unlink /tmp/supervisor.sock
使用 supervisorctl 管理进程:
# 停止某一个进程,program_name 为 [program:x] 里的 x:
supervisorctl stop program_name
#启动某个进程:
supervisorctl start program_name
#重启某个进程:
supervisorctl restart program_name
#停止全部进程,注:start、restart、stop 都不会载入最新的配置文件:
supervisorctl stop all
#载入最新的配置文件,停止原有进程并按新的配置启动、管理所有进程:
supervisorctl reload