I like PM2 and it's successfully been keeping my APIs alive quite well, but how does the future look given the popularity of Kubernetes? What is the overlap like and what's not overlapping?
Be that as it may - if you already have Kubernetes up and running, what would be the additional value of having both PM2 running alongside Kubernetes?
Fwiw, last time I asked myself this question my conclusion was that there was no additional value, and we opted to not use PM2. This was a long time ago however, so take it with a grain of salt.
PM2 is great if you have an app composed of multiple services that need to share an environment. You may want to reduce communication overhead between these node processes or avoid involving kubernetes to share resources. Implementation details would drive this.
Also, kubernetes cluster workers can fail. How do you want your workload to behave in that case? When your container is killed, should these services fail together? Do you want a better separation of concerns between your app dev team and your devops/middleware team (think disaster recovery and cluster maintenance)? Where does the app dev end and the devops begin? For other languages such as Java or C++, it would be out of the question to let these details matter to anyone but app devs. Their entire apps are in one docker image. Kubernetes should only manage instances of the app and the obvious dependencies that should be externalized (redis, memcached, postgres, mysql, etc.)
In my experience it's really a question of failure modes in your architecture and squeezing out small gains in performance that matter at scale.
I use PM2 with BrowserBox and it's been great. Reliable, scalable, performant. PM2 has been with us since the early days. We've never looked for an alternative.
the change from docker classic swarm into current docker swarm happened years ago. from what I can tell, usage-wise there are not much difference between the two.
however the sad thing is currently swarm is held by other company (mirantis) after being sold by docker inc. and the new company doesn't seem that invested in improving swarm. you could say its in life support now save for a few community contribution
Writing this at 1:30am, might not be very insightful.
We actually are phasing out the use of pm2 in our org since we are already running within kubernetes. And running pm2 within k8s seemed redundant (and is likely a relict of previous deployment strategies).
It also made certain debugging tasks trickier as we very rarely like to attach debuggers to our staging environment and the cluster/fork behavior of pm2 made it hard to effectively target indivual executions/processes.
Further we had some issues running nextjs after upgrading to the latest version within pm2 when using custom configuration files (we did not end up finding the root cause but decided to drop pm2, least path of resistance etc).
One other thing that also gave us headaches was the fact that pm2 allows specifying different env variable values for different environments but no other configuration (such as cluster count etc.)
We were successful entirely replacing it with docker-compose configs locally that are ultimately closer to how our services are deployed in production.
Overall I think pm2 is suitable for local dev and prototyping small to medium complexity services.
At work, we have kubernetes and pm2 running on a pet server outside of kubernetes. I cannot wait to get rid of pm2 and the pet server. For smaller projects, I'd rather just work directly with systemd or better yet, use the podman-systemd integrations.
I am running a 18 node SOA since 4 years with pm2 and I am very happy. Very reliable. No need for the complexity of Kubernetes or even Mesosphere DCOS back in the day. I do not even consider to change anything.
Actually on 3 hosts, but only the primary is at the edge, one other is a hot-standby at another geographic location and the third is for incremental backup purposes via ZFS. The database sync connection are tunneled via Wireguard. Failover to the hot-standby is done via DNS (ClouDNS).
Also I run the setup in a Linux Container (LXC).
Deployment is done via SCP and Tar. :-D
This setup is very reliable to me and at the other hand cheap!
I don't think systemd has a built-in load balancer. Also, running multiple copies of the same service isn't typical PID-1 stuff, so maybe doing that with systemd involves some friction.
I know about templates, but I don't believe they are designed for fungible copies. You can run a bunch of my-app@a-random-port.service, but it's not the same UX as `pm2 start my-app -i number-of-copies`.
They work very well for "worker" processes. The default ssh package on Debian uses systemd template units for the user process; I've used them extensively for job queue workers, in combination with max execution time and a separate timed job to monitor queue load (essentially automatic scale up/down of queue workers).
Realistically if your prod environment relies on you manually running `pm2... ` or whatever, you're doing something wrong. User Experience should have about 0% impact on how your service daemons run.
Systemd also gives you cpu/memory limiting, file system isolation, locked down capabilities for the process, CPU affinity, can use different Node versions for different apps, etc. It removes the need for Docker for the vast majority of cases.
Systemd won’t do load balancing, but you should have nginx/caddy in front of Node anyway, and either will handle load balancing just fine.
Yep, systemd is an excellent and stable process runner - it can start your processes whenever, with dependencies (eg redis/pg), or on a flexible schedule etc.
systemd can run Node.js apps easily, but unless your applocation is doing something special for multithreading that is single threaded. PM2 makes it trivial to run an application in cluster mode.
I set up my first nodejs backend project on an ec2 instance with pm2 almost 9 years ago. I've also been toying with an instant deploy pipeline that takes advantage of its watch and restart.
PM2 is very good and I really wonder why it's still somewhat unknown.
People complaining that Node.js only use one CPU core, or benchmarks with low scores for Node.js. There are solutions for this! PM2 is one of them and is very good. Use it.
Would you rather scale with more containers? That's heavier.
Properly written node apps are bound by I/O throughput. CPU and memory cost for additional processes is minimal, but latency is significantly decreased with a worker pool.
> Properly written node apps are bound by I/O throughput
Does that mean that node apps shouldn’t do any computation on data, just move bytes between a file and a network socket? If I need my program to think, I wrote it wrong or in the wrong language?
I would rather have shared memory multithreading be a usable feature in my program’s runtime. SharedArrayBuffer exists but 98% of existing code uses objects and arrays so can’t be easily shared between processes without copying, and the cost of copying objects with structuredClone puts a huge optimization barrier in between most code and parallel processing.
> Does that mean that node apps shouldn’t do any computation on data, just move bytes between a file and a network socket? If I need my program to think, I wrote it wrong or in the wrong language?
This is a very uncharitable interpretation of Node's primary use case. I think it would be more accurate to say don't use Node if all of the following apply:
- Your task is CPU bound rather than IO bound (e.g. you're not waiting to insert or retrieve data from the database).
- Your task is highly parallel (some computations are sequential and linear).
You can't use worker threads because:
- You have too much data to bear the cost of message passing via the structured clone algorithm.
- You can't share memory and avoid structured cloning because your data can't be represented using an array buffer.
This is a far cry from the claim that Node can't handle problems that require "thinking". When you're on the JIT compiler's happy path Node can think pretty quickly actually. In practice, many high volume web servers are handling enough requests that there's more than enough jobs for each Node process to have exclusivity over its jobs, and you don't need to share work between processes. Each process or worker thread can be largely independent. Not every task is going to fall within those boundaries, but that's ok, Node doesn't have to fulfill every use case.
How do you feel about using pm2 inside of a container?
At work, most of our app is docker-composed in one VPS (early B2B product with few customers). I would like to have the multiple-process benefits of PM2, but I don’t want to spin up a whole load balancer for it, or split the node service from the other services. Everything is so simple right now and I’d like to keep it that way lol.
Last time I googled this I found conflicting opinions, but if I can just stuff PM2 into my node container, I’d be a happy dev!
We use PM2 in a docker container since we got a memory leak and PM2 has a memory limiter option that allows us to kill and restart a process when it uses too much memory. If we let it grow the response time starts to drop due to GC taking longer and longer.
You have to do so many things just to use more than a single CPU core. There are other language platforms and runtimes where using up the entire available CPU cores is the default option, and taking advantage of multiple cores within the app is easy by design.
I can see why people like pm2, similarly to why I can see why people like Nodejs. But I think pm2 is half-baked, and Nodejs's greatest design feature (single-threaded async execution) is flawed by design.
No, it won't. It's basically just a NodeJS application server. On the plus side, you get a dashboard with statistics like latency, memory usages etc. But from your description it seems that it will be a redundant solution.
PM2 has worked very well in my experience. I have about a dozen services (APIs and full-stack applications) across a number of servers, remote and local. Mostly Node, these days trying with Bun. Some of them have been running for years without a hitch. It's been totally reliable.
I'm sure there are alternatives, lighter-weight and closer to the metal, but I haven't had the need. PM2 does the job.
It may well be that I did not learn to use it properly, but - to provide a counter opinion - pm2 left a sour taste with me. Tried to use it few years back and by default it does not rotate the logs, did not handle machine reboots well, and at times I was confused what is running and what is not. I settled on docker for small projects and kubernetes for larger ones and these things made more sense to me. I admit it could be a matter of habit.
My experience starting out with PM2 was the same. You need a plugin for the log rotation, saving the default environment variables is confusing, and stale processes cause weird behavior.
You may already have your own in-app log rotation before adding PM2 so may want to avoid printing to stdout/stderr. For usage with docker you want pm2-runtime.
I came to the opposite conclusion for the same reasons.
Docker/kubernetes is fine for things that make money, but for small personal projects it is far too much complexity and I dumped it. pm2 gets the job done with less effort overall.
I get to rsync my code up to a server, & pm2 restart it. The server process runs like any other, no container issues to consider, just a json file saying how I'd run it myself if I wanted to
tbf I'd rather replace pm2 with systemd, since I'm using it to run a Rust program, but if I was using node their cluster mode looks neat
Honestly, I hate using docker. It’s a pain to deal with in your dev environment, especially on a Mac. It’s not lightweight for your dev machine. It requires a good internet connection and it uses a lot of data. It’s all very “enterprise”.
If you’re sat in an office, you’re on a work computer, you need reproducible dev environments, probably none of that bothers you.
However, if I’m working by myself, I don’t want to deal with any of that Rube Goldberg shit. I use shell scripts, ssh, rsync and pretend it’s the 00s again, and oh my god I’m so much happier in that universe.
It uses a VM on Macs, it's usually slow and requires too much thinkering to get it to work decently. I gave up development with Linux tools on Macs, very annoying and not worth the hassle. I think it even works better in Windows since WSL.
I'm using runit instead of pm2. Systemd should also do fine. It's over engineered.
Of course it does, but at least it works quite decently. And you're running an actual distribution under it, so no hassles with MacPorts, Homebrew and other third party solutions for installing dependencies. If development on Snow Leopard was quite annoying, I can't imagine what it is now with the newer Apple sandboxes in place.
It’s so much worse on a Mac. Wrong OS, wrong CPU architecture. Never had any issue with it in deployment.
The other problem for me is, let’s say you’re in a hotel with crappy wifi and docker wants to download a large quantity of images, you’re in for a rough time.
There’s a very sharp downwards trajectory of user experience once you leave the happy path.
Again, no idea about osx, but if you have any project with lots of deps, it’s going to take time to download.
A well crafted image may add an extra 500MBs on top of that. By no means minimal, but not really that bad. (Assuming you have no base images like python or Ubuntu already on your system)
And idk what you mean by “happy path” for docker.
Every dockerfile is a snowflake.
It’s not a framework like rails.
If you have a good understanding of the tool you shouldn’t have any trouble.
As others mentioned - logrotate is available as an option, since the system one is more preferable for many. As per reboot handing - it's available via "pm2 startup" script.
I'm with you. I think PM2 is half-baked software. It tries to do too much and none of them well. It has two source-of-truths, the dump file and the ecosystem files. It doesn't have a way to stagger restarts.
For small projects, there is now a way to run docker containers or pods (through podman) via systemd units, so if you are using a systemd enabled distro, then that may be a better alternative than pm2, especially if you already have proficiency in docker and kubernetes.
I used pm2 to run my adhoc Python jobs/bot. Tried with Django and celery and docker, ended up being a pain in the ass since I have to modify on the fly.
Maybe systemd? I just always find the unit files so cumbersome to deal with (actually I feed pm2 to ecosystem files to ChatGPT and let it deal with the conversion)
Does anyone have an alternative to pm2 that's reliable? That's my biggest issue with it: it gets slower and slower to do commands like `pm2 list` over time, and eventually, pm2 itself crashes—taking everything it runs with it.
Also, `pm2 flush` doesn't actually truncate all logs, just the logs of whatever processes pm2 happens to be running at that moment.
I workaround this by killing pm2 entirely and `rm -Rf .pm2` once a month or so.
Dokku itself doesn't run anything in resident (there is a small go binary that listens to the docker event stream and restarts apps as necessary but thats it), so there shouldn't be overhead there.
On the Docker side, there is a _slight_ amount of overhead in cpu/networking, but its so negligible that its sometimes difficult to benchmark. While I can't really make changes to Docker to alleviate issues, I'd be interested in knowing what parts of Docker are heavyweight so we can better surface any such information to Dokku users.
It's a fascinating project, and esp for from before docker+k8s normalized and nodejs was reinventing the universe
Nowadays, docker compose or k8s ecosystems do a lot of this stuff in a more team-friendly & ecosystem-supported manner. I always liked pm2 bc the dynamic API layers... except docker & k8s also have APIs, and you get all sorts of extras with them by being a layer below
I wonder if there is a name for this waller garden phenomena. The pm2 team built a wonderful universe and it keeps improving afaict, but tech ecosystems also have been improving around them and at faster & bigger levels...
I've experienced some strange behavior with pm2 on my projects where a reboot would completely wipe the configs and I'd have to rebuild the processes (even after a pm2 save)
If it happens again I'll likely look into doing it the old fashioned way with systemd.
Skill issue? Most certainly. It still sucks to think I've figured it out (to the point of testing with manual reboots) only to one day have nothing running and have no clue why.
I don't understand why people think k8s is a good replacement for pm2. The resource requirements to run containers over standard processes is tremendously different in practice.
Can you make Kubernetes deployments easier than this?
PM2 fits the job perfectly when you want to start managed processes from the CLI without a config like you would with something SystemD or SupervisorD).
It has some nice interfaces for observability.
Its more lightweight than spinning up Docker containers for each process
Horrible piece of software, it uses a stateful(!) configuration. The daemon updates it's state at runtime and there is absolutely no guarantee it will properly restore that state when it restarts. It's extremely unreliable/flaky, you really don't want that from such a critical service.
Despite a few shortcomings, such as the lack of automatic scaling, it is definitely more stable than, nginx unit, which we had a lot of problems with, due to segfaults and very "tricky" nodejs support.
Is pm2 claiming that the cluster module is a load balancer? Because I don’t really consider round robin to be a full load balancer. Maybe 25 years ago but not today.
The node cluster module needs a way to provide your own balancing algorithm. Then I wouldn’t feel the need to put a reverse proxy in front of it.
I used this to run an entire startup for 4 years (complex data needs, not CRUD). It was a stopgap that was good enough to stay. I always wanted rid of it, but there were always more pressing problems. Used the load balance feature extensively, which worked but starts to falter when you get to big scale. It's not a stable foundation, it's a thing that's been hacked around enough to the point where the problems are mostly gone.
If I did it all over again, I'd pick something proper like systemd or k8s. I would say this is more a surprisingly capable dev tool than a real service manager.
84 comments
[ 0.18 ms ] story [ 133 ms ] threadFwiw, last time I asked myself this question my conclusion was that there was no additional value, and we opted to not use PM2. This was a long time ago however, so take it with a grain of salt.
Also, kubernetes cluster workers can fail. How do you want your workload to behave in that case? When your container is killed, should these services fail together? Do you want a better separation of concerns between your app dev team and your devops/middleware team (think disaster recovery and cluster maintenance)? Where does the app dev end and the devops begin? For other languages such as Java or C++, it would be out of the question to let these details matter to anyone but app devs. Their entire apps are in one docker image. Kubernetes should only manage instances of the app and the obvious dependencies that should be externalized (redis, memcached, postgres, mysql, etc.)
In my experience it's really a question of failure modes in your architecture and squeezing out small gains in performance that matter at scale.
> Docker Swarm mode is built into the Docker Engine. Do not confuse Docker Swarm mode with Docker Classic Swarm which is no longer actively developed.
however the sad thing is currently swarm is held by other company (mirantis) after being sold by docker inc. and the new company doesn't seem that invested in improving swarm. you could say its in life support now save for a few community contribution
more explanation by dockerswarm.rocks website: https://dockerswarm.rocks/swarm-or-kubernetes/
Writing this at 1:30am, might not be very insightful.
We actually are phasing out the use of pm2 in our org since we are already running within kubernetes. And running pm2 within k8s seemed redundant (and is likely a relict of previous deployment strategies).
It also made certain debugging tasks trickier as we very rarely like to attach debuggers to our staging environment and the cluster/fork behavior of pm2 made it hard to effectively target indivual executions/processes.
Further we had some issues running nextjs after upgrading to the latest version within pm2 when using custom configuration files (we did not end up finding the root cause but decided to drop pm2, least path of resistance etc).
One other thing that also gave us headaches was the fact that pm2 allows specifying different env variable values for different environments but no other configuration (such as cluster count etc.)
We were successful entirely replacing it with docker-compose configs locally that are ultimately closer to how our services are deployed in production.
Overall I think pm2 is suitable for local dev and prototyping small to medium complexity services.
I use it to avoid the complexity of those things since my operational needs are usually very modest.
why?
I thought process managers like supervisor and pm and the others had all been superseded by systemd, unless you’re in a non systemd distro of course.
Realistically if your prod environment relies on you manually running `pm2... ` or whatever, you're doing something wrong. User Experience should have about 0% impact on how your service daemons run.
Systemd won’t do load balancing, but you should have nginx/caddy in front of Node anyway, and either will handle load balancing just fine.
People complaining that Node.js only use one CPU core, or benchmarks with low scores for Node.js. There are solutions for this! PM2 is one of them and is very good. Use it.
Properly written node apps are bound by I/O throughput. CPU and memory cost for additional processes is minimal, but latency is significantly decreased with a worker pool.
Does that mean that node apps shouldn’t do any computation on data, just move bytes between a file and a network socket? If I need my program to think, I wrote it wrong or in the wrong language?
I would rather have shared memory multithreading be a usable feature in my program’s runtime. SharedArrayBuffer exists but 98% of existing code uses objects and arrays so can’t be easily shared between processes without copying, and the cost of copying objects with structuredClone puts a huge optimization barrier in between most code and parallel processing.
If you need to compute something complicated, write it in a compiled language and make it a node module. Many popular npm libraries are exactly this.
This is a very uncharitable interpretation of Node's primary use case. I think it would be more accurate to say don't use Node if all of the following apply:
- Your task is CPU bound rather than IO bound (e.g. you're not waiting to insert or retrieve data from the database).
- Your task is highly parallel (some computations are sequential and linear).
You can't use worker threads because:
- You have too much data to bear the cost of message passing via the structured clone algorithm.
- You can't share memory and avoid structured cloning because your data can't be represented using an array buffer.
This is a far cry from the claim that Node can't handle problems that require "thinking". When you're on the JIT compiler's happy path Node can think pretty quickly actually. In practice, many high volume web servers are handling enough requests that there's more than enough jobs for each Node process to have exclusivity over its jobs, and you don't need to share work between processes. Each process or worker thread can be largely independent. Not every task is going to fall within those boundaries, but that's ok, Node doesn't have to fulfill every use case.
At work, most of our app is docker-composed in one VPS (early B2B product with few customers). I would like to have the multiple-process benefits of PM2, but I don’t want to spin up a whole load balancer for it, or split the node service from the other services. Everything is so simple right now and I’d like to keep it that way lol.
Last time I googled this I found conflicting opinions, but if I can just stuff PM2 into my node container, I’d be a happy dev!
I can see why people like pm2, similarly to why I can see why people like Nodejs. But I think pm2 is half-baked, and Nodejs's greatest design feature (single-threaded async execution) is flawed by design.
You write nodemon or npm start- and so it's put at the end of the Dockerfile as such.
I know someone that discovered pm2 because copilot autocompleted their Dockerfile that way.
I'm sure there are alternatives, lighter-weight and closer to the metal, but I haven't had the need. PM2 does the job.
You may already have your own in-app log rotation before adding PM2 so may want to avoid printing to stdout/stderr. For usage with docker you want pm2-runtime.
Docker/kubernetes is fine for things that make money, but for small personal projects it is far too much complexity and I dumped it. pm2 gets the job done with less effort overall.
Why did you prefer pm2?
I get to rsync my code up to a server, & pm2 restart it. The server process runs like any other, no container issues to consider, just a json file saying how I'd run it myself if I wanted to
tbf I'd rather replace pm2 with systemd, since I'm using it to run a Rust program, but if I was using node their cluster mode looks neat
If you’re sat in an office, you’re on a work computer, you need reproducible dev environments, probably none of that bothers you.
However, if I’m working by myself, I don’t want to deal with any of that Rube Goldberg shit. I use shell scripts, ssh, rsync and pretend it’s the 00s again, and oh my god I’m so much happier in that universe.
It saved me from the Rube Goldberg esque pile of bash scripts and puppeteer configs of the 00s.
Funny how perception changes.
I'm using runit instead of pm2. Systemd should also do fine. It's over engineered.
https://learn.microsoft.com/en-us/windows/wsl/compare-versio...
It’s been a long time.
The other problem for me is, let’s say you’re in a hotel with crappy wifi and docker wants to download a large quantity of images, you’re in for a rough time.
There’s a very sharp downwards trajectory of user experience once you leave the happy path.
A well crafted image may add an extra 500MBs on top of that. By no means minimal, but not really that bad. (Assuming you have no base images like python or Ubuntu already on your system)
And idk what you mean by “happy path” for docker. Every dockerfile is a snowflake. It’s not a framework like rails.
If you have a good understanding of the tool you shouldn’t have any trouble.
For small projects, there is now a way to run docker containers or pods (through podman) via systemd units, so if you are using a systemd enabled distro, then that may be a better alternative than pm2, especially if you already have proficiency in docker and kubernetes.
Pm2 is neat, but the monitor solution is pricey.
You can come up with a Unix solution by using bash, piping to logs, daemonizing with &.
And as a bonus, it works for any program, not just with node stuff.
Remember, use the tools available in your foundation before adding more blocks to the pyramid
Also, `pm2 flush` doesn't actually truncate all logs, just the logs of whatever processes pm2 happens to be running at that moment.
I workaround this by killing pm2 entirely and `rm -Rf .pm2` once a month or so.
(This is all on Linux BTW.)
Dokku itself doesn't run anything in resident (there is a small go binary that listens to the docker event stream and restarts apps as necessary but thats it), so there shouldn't be overhead there.
On the Docker side, there is a _slight_ amount of overhead in cpu/networking, but its so negligible that its sometimes difficult to benchmark. While I can't really make changes to Docker to alleviate issues, I'd be interested in knowing what parts of Docker are heavyweight so we can better surface any such information to Dokku users.
Nowadays, docker compose or k8s ecosystems do a lot of this stuff in a more team-friendly & ecosystem-supported manner. I always liked pm2 bc the dynamic API layers... except docker & k8s also have APIs, and you get all sorts of extras with them by being a layer below
I wonder if there is a name for this waller garden phenomena. The pm2 team built a wonderful universe and it keeps improving afaict, but tech ecosystems also have been improving around them and at faster & bigger levels...
If it happens again I'll likely look into doing it the old fashioned way with systemd.
Skill issue? Most certainly. It still sucks to think I've figured it out (to the point of testing with manual reboots) only to one day have nothing running and have no clue why.
Can you make Kubernetes deployments easier than this?
Its more lightweight than spinning up Docker containers for each process
The node cluster module needs a way to provide your own balancing algorithm. Then I wouldn’t feel the need to put a reverse proxy in front of it.
If I did it all over again, I'd pick something proper like systemd or k8s. I would say this is more a surprisingly capable dev tool than a real service manager.
Take a look at the code. E.g. https://github.com/Unitech/pm2/blob/master/lib/API.js
I don't want this thing running my production environment if I can help it.