How do you guys deploy Sidekiq.
The big problem is when I deploy Sidekiq through Foreman, then I always end up with hung processes on restart. Somehow, the TERM signal does not work very well.
I have now started using supervisor for deploying Sidekiq, but I would have preferred a Foreman like tool (so that development is also nice and simple)
I just kick off the application with a bash script for both dev and prod and push it through supervisor. (Remember to set `stopasgroup` to true, though, or else you can easily orphan processes and you will be sad.)
I've always just done `bundle exec sidekiqctl stop tmp/blah_worker.pid` at deploy-time, and then let the process monitoring system bring them back up. It works fine. I've never had a problem with orphans in production, but our apps dont rely on sidekiq for their core functionality.
GC Disabled, Memory required to process 10,000 jobs reduced from 1257 MB to 151 MB . So it may mean (unless there's a general overhead) that, 15KB of memory is used vs. 125.7KB previously. assuming the job still itself leaks the same memory it means sidekiq itself was leaking 125.7KB per job previously, am I right?
Leaking is the wrong word. A leak implies that the memory was retained, RAM usage will go up forever and the process will eventually crash as a result. This is memory that sidekiq needs to do it's job, allocating strings, hashes, arrays etc. that will all be cleaned up eventually. Larger, more complex objects take up more memory. I.e. strings take up less memory than a hash. What he has done is eliminate the need to allocate so many large objects. This means less GC is needed, and overall performance is increased.
I know what Sidekiq is, but that doesn't stop me from always thinking these articles are about hacking the old Sidekick phone https://en.wikipedia.org/wiki/Danger_Hiptop and getting weirdly nostalgic. Words are strange.
6 comments
[ 4.9 ms ] story [ 34.2 ms ] threadI have now started using supervisor for deploying Sidekiq, but I would have preferred a Foreman like tool (so that development is also nice and simple)
See http://www.schneems.com/2015/05/11/how-ruby-uses-memory.html for more info.