HN Explorer

Ask HN: What is the cheapest, easiest way to host a cronjob in 2022?

361 points by heywhatupboys ↗ HN
I thought this could start a good debate on the subject. Myself I have had to make a short running web-scraping job that, given a change in the site, sends a notifying email. This running once an hour.

It is 2022, so I had thought it would be tremendously easy and cheap, but it seems no solution is easily implemented.

335 comments

[ 2.6 ms ] story [ 286 ms ] thread
warrenm
Sounds like a task for a cheap VPS
oracle2025
Sometimes you get cron jobs with a cheap webhost, so you would not even need to maintain a cheap VPS
warrenm
Maybe they're out there, but I cannot recall the last time I saw a webhost for less than Vultr's starting $2.50/mo
iforgotpassword
Raspberry Pi at home if you don't have a home server already. Or some low spec cheap vps.
gwn7
Yes yes yes. I came to post the exact same thing then saw this =)
MuffinFlavored
Eh.

I tried to do Chromium/Puppeteer based scraping this way.

Building a Dockerfile took ages due to the low compute. (Rust was a non-starter).

I also had (foolishly) only bought the Pi with 2GB instead of 8GB so RAM was an issue.

Disk was super slow.

I'm not sure how viable this is, especially with how hard it is currently to source a Pi, let alone its computation/memory constraints.

marcosdumay
That's really deviating from the nature of the "cheapest, easiest way to host a cronjob" question. If the OP has that kind of requirement, he won't get good answers.
marginalia_nu
You really don't need docker for this.
Arainach
It's true that you don't, but I can see the advantages.

I have a Raspberry Pi that is natively running a scraper using headless Chromium and cron. It works great, except....

I ended up needing a virtual framebuffer. I got it working on the Raspberry Pi, but I got a new workstation and wanted to edit my script and test it there. I got cryptic errors that I needed to debug to understand they were framebuffer issues, then attempt to recreate the setup that's running on my Pi, then debug that.....

My first mistake was not writing down what I did in my README, but a Docker image would have saved me a ton of time here.

gwn7
Why couldn't you keep editing your script in your Pi?

Isn't proposing Docker as a solution to this going nuclear?

I think there are some many cheaper things you can do to solve this.

For me this is absolutely no justification for Docker use in this scenario.

Arainach
The script takes 60+ seconds to run on an RPi and fewer than 10 on an i9, allowing for significantly faster iteration.
marginalia_nu
Doesn't really answer any of the questions of what docker actually does to help here, other than make everything slower and more complex.
SeriousM
You can use the "browserless" docker service which contains a headless chrome browser in a docker container. It also supports puppeteer and playwright connect api. Works flawless! I use it in combination with n8n. All on a raspberry pi4b (yes, I got one recently)
MuffinFlavored
> You can use the "browserless" docker service which contains a headless chrome browser in a docker container.

A lot of websites can detect the IPs from this and block it, basically almostly like a CAPTCHA.

I had other needs like Postgres, etc.

Scraping data is one thing, actually doing anything with it is another. I quickly hit the limits of a $50 RaspberryPi 4 or whatever they're going for on Amazon these days with the gouging, etc.

TheDong
> Building a Dockerfile took ages due to the low compute

Why compile anything on the raspberry pi? Cross-compile on a machine with more compute (like your laptop, desktop, phone, or ec2 instance) one time, and then transfer the compiled binaries or built docker image over.

> Pi with 2GB instead of 8GB

For headless chrome, that should be enough unless you're doing other stuff with it. Unless you mean for compiling stuff, which as before can be done elsewhere.

gwn7
You're mentioning the worst possible scenario.

First of all the answer recommends to use either a Pi or a VPS. If a Pi doesn't cut it for you, just switch to a VPS with sufficient specs for your requirement. Problem solved, now it's viable.

Besides, a significant part of the web can be scraped without resorting to a heavyweight browser such as Chromium. It should always be the last resort. Even if you have to evaluate Javascript (in case of SPAs), there are much cheaper solutions than Puppeteer (JSDOM being an example) which can get the job done most of the time.

As to Docker, I fail to see why you would need Docker for this kind of job, unless you don't know how to do it without Docker.

When there is no portability requirement, the costs of Docker easily surpass its benefits.

...

So no, it's not that what's being recommended in the answer is not viable. You're doing it wrong. Either you're using a Pi when you need to use a VPS or you are introducing unnecessary layers.

disiplus
i run it on my NAS at home, one of the reasons is that it does not have a PUBLIC IP of one of the providers that is blocked on my scraping target.
dharmab
A use case I've needed cloud services for is web scraping. Sites which IP ban web scrapers will still allow scraping from major cloud provider IP space.
Arainach
How are sites detecting and banning once-an-hour users? Yes, if you want to spider an entire website continuously, you could get IP banned, but if you're making 24 (or even 96) requests a day to visit one site and check some data, your traffic is indistinguishable from baseline.
iforgotpassword
I've seen bigger sites block requests from ip ranges that belong to hosting providers. Don't know about cloud providers, but doing it from home with a residential address has the highest chances of success. You might still end up in the cloudflare verification page though sometimes.
Tempest1981
Although they still seem to be out of stock almost everywhere. (Looking for a Zero W)
Sirened
Take your choice of other random SBCs off Aliexpress for $30-40. They all run Linux well enough for headless, short cronjobs.
malfist
AWS Lambdas could be used for this, you can trigger them on a time schedule using an event bridge.
codegeek
You can checkout https://cronhub.io (I run it). Behind the scenes, it wraps a lambda function with EventBridge (so you can set schedule) and adds other goodies like notifications/alert (slack/email etc), metrics, notifications if job runs slow etc.

Not free but basically you wrap your job in http endpoint and then we take care of the rest.

ThePhysicist
I think OP doesn't want to keep a machine running permanently. If you do that, adding your own crontab entry would be trivial as well.
codegeek
Possibly. We do have paying customers who are using it so that they have better tracking, notifications and guarantee that the job is running (with metrics etc). But I hear you. For some devs, it may not be critical. I do have plans to allow adding your own code directly so that you don't have to come with an http endpoint.
davnicwil
I suppose with this solution you could also run one of those free tier machines that spins down when not in use, or a lambda function, etc
Tombar
Github actions FTW! > https://docs.github.com/en/actions/using-workflows/events-th...

I remember seeing a couple projects shared before, using this technique to scrape sites with GHA

that_guy_iain
I came to suggest the same thing.

I use their cronjob functionality to ensure my docker images are built daily and therefore in theory secure.

rpastuszak
Just note that they’re not guaranteed to be called precisely on time, e.g. my “every 15m” CRON job will be called every 15m _at best_, in practice… twice per hour.

This works perfectly for my case (content syndication for https://potato.horse), and I’m pretty happy with GH actions for this kind of stuff, but if you need something more precise, you might want to look somewhere else.

doubleunplussed
At least for a while before I moved to self-hosted systemd timers, if you were running jobs near 0:00 UTC, the delay was often so long that your job wouldn't run at all. I had weeks of jobs literally not running at all before bailing on GitHub actions for this sort of thing. It was disappointing, but I'm now happier with my current setup.
lofatdairy
> Just note that they’re not guaranteed to be called precisely on time, e.g. my “every 15m” CRON job will be called every 15m _at best_, in practice… twice per hour

Is the spread really that egregious? That's essentially a 50% failure to trigger at all, like I don't think you can call that 15 minutes so much as 15±10 minutes lmfao

IIRC AWS EventBridge is also not guaranteed to execute on the exact minute, but in my experience running a small job every 5 minutes only had about a 30-40 second delay at worst.

rpastuszak
Just checked the logs: it used to be 2 per hour, but now it has improved to 3-5, so closer to the "every 15 m" rule. Again, not a big deal in my case and probably something that's being worked on.
1f60c
AFAIK, scheduled GitHub workflows stop running after a while. But when that happens, GitHub will send you an email with a big green “Continue running workflow” button.
akerl_
I use crons to keep my Docker containers fresh, and have never hit this. But the cron commits to the repo, so I wonder if they’re flagging repos with crons but no commits recently?
tracker1
Definitely... It seems to be in 4-6 week timeframe... I'd thought about making the cron update references in the repo, but hadn't gone through that as of yet.
sauruk
sounds like time to make a “hit the big green button” github action
tracker1
I've got a couple projects like this, that mostly just create bundles of other source projects that I'm not involved with. Creating a windows installer, or docker image for projects that don't have that integrated. It's kind of annoying that they will stop in several weeks when there's no project changes.
moehm
> It's kind of annoying that they will stop in several weeks when there's no project changes.

Can't you configure a GHA which commits nonsense every month?

tracker1
Yeah... I will probably do that, with my daily check action (update a last-check) text file or something.
genericacct
Can you not trigger an action via a local cronjob and the API?
tracker1
But then I need a second system to do that.
[deleted]
(comment deleted)
WirelessGigabit
I wonder if I can make a Github repo with an action that commits to another repo on a cron. Or itself?
cornel_io
If you have a GCP, AWS, or Azure account already and know Javascript, you should be able to whip up a serverless cron task real quick. IMO the simplest setup is probably GCP via Firebase (which makes deployment super simple once you install the CLI): https://firebase.google.com/docs/functions/schedule-function...

You won't spend much at all, it's fractions of a penny per call and I think there's a free tier.

mhitza
Just use a 2.5$/month instance on Vultr for your cronjob, and plain old sendmail for your emails (they will go in your spam folder but at least you don't have to waste time setting up emails).
cube2222
EventBridge + AWS Lambda

It's cheap as in free, thanks to the generous free tier.

ghufran_syed
isn’t the free tier only for one year?
giaour
No, you get some free usage in perpetuity
drpixie
Or until they change the terms & conditions...
Moissanite
Some services have a small "always free" tier too; S3, DynamoDB and Lambda all have enough free usage available for personal projects.
ectospheno
Cheap vps running urlwatch sending alerts with pushover. Turn on unattended upgrades for the server. The one time pushover fee is well worth it.
theNailz
+1 for urlwatch. I run it on my pc with Docker (restart=unless-stopped), and Discord integration on a personal Discord channel with alerts enabled. Takes two minutes to set up and is online whenever I'm online.
m463
I've never known about urlwatch, I always rolled my own in python (and then copy/pasted for the next task)

there it is an apt-get away

Thanks!

newbieuser
apache airflow and a $5 vps might be the cleanest solution
jrib
All free:

* free-tier vps on GCP or Oracle Cloud

* lambda job on AWS

I have a cheap VPS I use for other things and just run my cron jobs there.

Eleison23
I don't know about the others, but AWS is not free and it's not cheap. There are ways to set alerts on budget overruns, but there is no foolproof way to limit monthly spending. Eventually you will configure something that goes out of control, or I was able to conceive of a few scenarios where a malicious third party could rack up charges for, e.g. S3 egress costs. Good luck contesting the charges more than once. I shut everything down and pulled out of AWS. I believe that Azure's free trial featured a hard cap, but still requires a credit card to open your account.
akerl_
The lambda free tier on AWS is forever, and if you’re kicking it via cloud watch vs an api gateway it shouldn’t be possible for it to overrun.
JamesSwift
Lambda on AWS is basically free for personal crons. I ran lots on there until I moved everything to a personal k8s cluster for some more granular control over scheduling.
skelpmargyar
The free tier on Oracle Cloud is nuts. 200GB block storage, 10gb object storage, 24GB RAM 4 core ARM VM, 2 x86 VM's 1GB RAM each. From my experience, it's also really hard to get charged since you have to upgrade your account from a free tier. I haven't been charged one cent in several years.
johnmoberg
You should check out Modal! Does much more than just cronjobs, but super easy and cheap: https://modal.com/docs/guide/cron. They have an example with a simple web scraper as well: https://modal.com/docs/guide/web-scraper

From their pricing example:

> You schedule a job to run once an hour for 15 seconds. This job uses 50% of one CPU core and 256 MiB of memory. This job will cost $0.016/day for the CPU, and $0.001/day for the memory, adding up to $0.017/day, or $0.51/month.

yawnxyz
Ooh and apparently everyone gets $30/mo of credits as a baseline, so this is within the free tier (if you don't want to pay 50 cents per month...)
alexeldeib
This is very cool. Love the pricing model.
dyeje
We use Modal at my startup, it’s fantastic.
AndrewDucker
I have a PowerShell script that collects some RSS and posts it to my journal.

Easiest way I found to do that was Azure Functions. Costs me about 35p per month. Mostly for storage for logs as far as I can tell. I'd sort that, but it's literally not worth the time it would take

Thristle
Any of the managed serverless options with a free tier cloudflare has a free tier but it looks very limited compared to AWS/GCP/Azure
ElevenLathe
If this question came up at work, I would recommend a Lambda function that sends email via SES and is triggered periodically by a CloudWatch event. It would be cheap and require basically no admin overhead once set up.

That said, there are probably different forces at play if this is a personal infrastructure question: while cheap is good, if you aren't fluent in AWS it may be a slog to set up. If you're not, easiest thing is probably a real cron job run on a cheap VPS.

brycewray
https://cron-job.org
donohoe
I came here to add this very same service. Very nice UI. Simple. Solid so far.
c16
For hobbyists/non-critical work, this is the answer. I've been using it for a while now and never had an issue. But as with all free services, it might be here today then gone tomorrow.
AtNightWeCode
I would suggest anything serverless in most cases. Cheapest is not a great measure. Azure and GCP have good options. Cloudflare too if you just want to do something simple.