{"id":1212,"date":"2013-09-10T01:00:00","date_gmt":"2013-09-09T23:00:00","guid":{"rendered":"https:\/\/www.fussylogic.co.uk\/blog\/?p=1212"},"modified":"2013-09-10T15:15:48","modified_gmt":"2013-09-10T14:15:48","slug":"supervisord","status":"publish","type":"post","link":"https:\/\/www.fussylogic.co.uk\/blog\/?p=1212","title":{"rendered":"Supervisord"},"content":{"rendered":"

I\u00e2\u20ac\u2122ve recently written a couple of projects that run as a python-based daemon. I want them to run as long-lived projects on a VPS, with automatic restart should they fail. I thought I was going to have to go through the tedium of writing init.d<\/code> scripts for each.<\/p>\n

Then I stumbled upon supervisord<\/a>, which is already available in Debain. As packaged, it automatically starts as any other python daemon. However, the benefit for us as developers is that it offers a really straight-forward interface for turning our applications into long-lived daemons themselves.<\/p>\n

$ apt-get install supervisor<\/code><\/pre>\n
This will create a configuration (albeit empty) in \/etc\/supervisor\/conf.d<\/code> ready for your individual project configurations, and then will auto-start supervisord<\/code> itself as root<\/code>.<\/p>\n
So, let\u00e2\u20ac\u2122s say this is the command line I run normally:<\/p>\n
root:\/home\/deployed\/myapp$ python main.pyc<\/code><\/pre>\n
I therefore add this to \/etc\/supervisor\/conf.d\/myapp.conf<\/code><\/p>\n
[program:myapp]\ndirectory=\/home\/deployed\/myapp\ncommand=python main.pyc<\/code><\/pre>\n
The defaults<\/a> for supervisord<\/code> are sensible. A program defined like this will auto-start; will auto-restart on unexpected stop.<\/p>\n
Without doing anything else, we now access the supervisord<\/code> terminal.<\/p>\n
$ supervisorctl\nsupervisor> help\n\ndefault commands (type help <topic>):\n=====================================\nadd    clear  fg        open  quit    remove  restart   start   stop  update \navail  exit   maintail  pid   reload  reread  shutdown  status  tail  version<\/code><\/pre>\n
We have it rescan the configuration:<\/p>\n
supervisor> reread\nmyapp: available\nsupervisor> avail\nmyapp              avail     auto      999:999<\/code><\/pre>\n
and our application becomes available. It\u00e2\u20ac\u2122s not actually running yet of course; it\u00e2\u20ac\u2122s simply available to run. Before we do that though, let\u00e2\u20ac\u2122s improve<\/a> our configuration.<\/p>\n
[program:myapp]\ndirectory=\/home\/deployed\/myapp\ncommand=python main.pyc -d\nuser=deployed\nredirect_stderr=true\nstdout_logfile=\/home\/deployed\/myapp\/log\/myapp.log\nstdout_logfile_maxbytes=10MB<\/code><\/pre>\n
The app itself logs to stderr, when we run it as a daemon that means we won\u00e2\u20ac\u2122t get to see the output. supervisord<\/code> conveniently includes redirect and capture facilities so we can turn that stderr output into a file. Even better, it also includes automatic log rotation above a size threshold (which I\u00e2\u20ac\u2122ve set here to 10MB). By default supervisord<\/code> will pick a location (\/var\/log\/supervisor<\/code> by default) and will auto-delete any log file on exit. Setting it by hand causes it to live longer.<\/p>\n
Back to supervisorctl<\/code>: we could now start our daemon with the add<\/code> command (after another reread<\/code>), but better is to rescan and start\/stop whatever needs doing in one simple command:<\/p>\n
supervisor> update\nmyapp: added process group\nsupervisor> status\nmyapp              RUNNING    pid 23386, uptime 0:00:07<\/code><\/pre>\n
supervisord<\/code> will detect if the application keep cyclically starting and stopping and will remove it from its process table. That prevents your badly behaved app from sucking all the cycles from your system. My first use happened to do exactly that; so very soon the status<\/code> command gave:<\/p>\n
supervisor> status\nmyapp              FATAL      Exited too quickly (process log may have details)<\/code><\/pre>\n
In my case, I\u00e2\u20ac\u2122d forgotten to supply a password, easily discovered by reading the log file that supervisord<\/code> created.<\/p>\n
You can check for these runaway processes in \/var\/log\/supervisor\/supervisord.log<\/code>, in case your crashes are happening infrequently and unpredictably. It\u00e2\u20ac\u2122s not of use to me, but it can also manage FastCGI<\/em> applications, sharing a single port,<\/p>\n
All in all, I\u00e2\u20ac\u2122m absolutely delighted. supervisord<\/code> has literally just saved me at least a day of work, if not more. It\u00e2\u20ac\u2122s also made it unnecessary to add a load of extra code to my applications just to make them daemonizable. Very nice.<\/p>\n","protected":false},"excerpt":{"rendered":"
I\u00e2\u20ac\u2122ve recently written a couple of projects that run as a python-based daemon. I want them to run as long-lived projects on a VPS, with automatic restart should they fail. I thought I was going to have to go through the tedium of writing init.d scripts for each. Then I stumbled upon supervisord, which is\u2026 Read More »<\/a><\/span><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[1],"tags":[110,111,6],"_links":{"self":[{"href":"https:\/\/www.fussylogic.co.uk\/blog\/index.php?rest_route=\/wp\/v2\/posts\/1212"}],"collection":[{"href":"https:\/\/www.fussylogic.co.uk\/blog\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.fussylogic.co.uk\/blog\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.fussylogic.co.uk\/blog\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.fussylogic.co.uk\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1212"}],"version-history":[{"count":1,"href":"https:\/\/www.fussylogic.co.uk\/blog\/index.php?rest_route=\/wp\/v2\/posts\/1212\/revisions"}],"predecessor-version":[{"id":1213,"href":"https:\/\/www.fussylogic.co.uk\/blog\/index.php?rest_route=\/wp\/v2\/posts\/1212\/revisions\/1213"}],"wp:attachment":[{"href":"https:\/\/www.fussylogic.co.uk\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1212"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.fussylogic.co.uk\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1212"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.fussylogic.co.uk\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1212"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}