I got hacked: My Hetzner server started mining Monero (blog.jakesaunders.dev)

[flagged]

Whew, load average of 0 here.

> The Reddit post I’d seen earlier? That guy got completely owned because his container was running as root. The malware could: [...]

Is that the case, though? My understanding was, that even if I run a docker container as root and the container is 100% compromised, there still would need to be a vulnerability in docker for it to “attack” the host, or am I missing something?

> Here’s the test. If /tmp/.XIN-unix/javae exists on my host, I’m fucked. If it doesn’t exist, then what I’m seeing is just Docker’s default behavior of showing container processes in the host’s ps output, but they’re actually isolated.

  /tmp/.XIN-unix/javae &
  rm /tmp/.XIN-unix/javae

This article’s LLM writing style is painful, and it’s full of misinformation (is Puppeteer even involved in the vulnerability?).

This article is very interesting at first but I once again get disappointed after reading clear signs of AI like "Why this matters" and "The moment of truth", and then the whole thing gets tainted with signs all over the place.

I don't use Docker for my containers at home, but I take it by the concern that user namespacing is not the employed by them or something?

Never expose your server IP directly to the internet, vps or baremetal.

> ls -la /tmp/.XIN-unix/javae

Unless ran as root this could return file not found because of missing permissions, and not just because the file doesn't actually exist, right?

> “I don’t use X” doesn’t mean your dependencies don’t use X

That is beyond obvious, and I don't understand how anyone would feel safe from reading about a CVE on a widely used technology when they run dozens of containers on their server. I have docker containers and as soon as I read the article I went and checked because I have no idea what technology most are built with.

> No more Umami. I’m salty. The CVE was disclosed, they patched it, but I’m not running Next.js-based analytics anymore.

Nonsensical reaction.

Recently, those Monero miners were installing themselves everywhere that had a vulnerable React 19. I had exactly the same problem.

tl:dr: He got hacked but the damage was only restricted to one docker container runn ing Umami (that is built on top of NextJS). Thankfully, he was running the docker container as a non privileged non-root user which saved him big time considering the fact that the attack surface was limited only within the container and could not access the entire host/filesystem.

Is there ever a reason someone should run a docker container as root ?

Sure does seem like the primary outcome of cryptocurrencies being released onto the world has been criminals making money.

Is mining via CPU even worthwhile for the hackers? I thought ASICs dominated mining

Was dad notified of the security breach? If not he may want to consider switching hosting providers. Dad deserves a proper LLM-free post mortem.

I wouldn't trust that boot image or storage again, I'd nuke it for peace of mind.

That said, do you have an image of the box or a container image? I'm curious about it.

As an aside, if you're using a Hetzner VPS for Umami you might be over-specced. I just cut my Hetzner bill by $4/mo by moving my Umami box to one of the free Oracle Cloud VPS after someone on here pointed out the option to me. Depends whether this is a hobby thing or something more serious, but that option is there.

Just a note - you can very much limit cpu usage on the docker containers by setting --cpus="0.5" (or cpus:0.5 in docker compose) if you expect it to be a very lightweight container, this isolation can help prevent one roudy container from hitting the rest of the system regardless of whether it's crypto-mining malware, a ddos attempt or a misbehaving service/software.

I still can't believe that there are so many people out here popping boxen and all they do is solve drug sudokus with the hardware. Hacks are so lame now.

Sorry to hear you got hacked.

I know we aren't supposed to rely on containers as a security boundary, but it sure is great hearing stories like this where the hack doesn't escape the container. The more obstacles the better I guess.

You might want to harden that those outbound firewall rules as another step. Did the Umami container need the ability to initiate connections? If not, that would eliminate the ability to do the outbound scans.

Also could prevent something to exfiltrate sensitive data.

No firewall! Wow that's brave. Hetzner will let you configure one that runs outside of the box so you might want to add that too, as part of your defense in depth - that will cover you if you make a mistake with ufw. Personally I keep SSH firewalled only to my home address in this way; if I'm out and about and need access, I can just log into Hetzner's website and change it temporarily.

You're lucky that Hetzner didn't delete your server and terminate your account.

I also run Umami, but patched once the CVE patch was released. Also, I only expose the tracking js endpoint and /api/send via Caddy publically (though, /api/send might be enough to exploit the vul). To actually interact with Umami UI I use Twingate (similar to Tailscale) to tunnel into the VPC locally.

Still confused what I am supposed to do to avoid all this.

What's considered nowadays the best practice (in terms of security) for running selfhosted workloads with containers? Daemon less, unprivileged podman containers?

And maybe updating container images with a mechanism similar to renovate with "minimumReleaseTime=7days" or something similar!?