I've been a cron user forever, I had never heard of systemd timers until this thread.
A quick internet search shows:
- Nice clean config in a file (instead of having to edit crontab)
- Lots of great features that you would have to manually implement and maintain in cron (e.g. missed launches, duplicate runs, random delay, timeouts)
I mean sure, I've got bash script templates that let me solve the problem of preventing duplicate cron job running.
But having seen how easy it is to achieve the same thing with systemd timers whilst not requiring a whole lot of boilerplate code in each shell file ... why wouldn't you switch to systemd timers ? I sure will be.
dcron is very lightweight and has most of those features. Dunno about random delay/timeouts, but worst case those sound like a 10 line shell script away.
> dcron is very lightweight and has most of those features.
Hmm, let's see ...
- Choice 1: Use what comes with my system (two choices, systemd or, if for the anti-systemd people... cron)
- Choice 2: Install some random third-party that has not had any substancial git commits since 2014 (the 2019 commits are just admin metadata).
I don't get why people downvotes you. Clearly consul is not that widespread but if you happen to use it, consul lock is pretty cool, it also permits locks between a cluster of machines
Huh, does python not have a way to flock a file itself?
A simple flock doesn't always work all that well - the devil is in the details. And there is the issue that you miss a run of the script and have to wait another half hour for the next run.
I've used the Highlander, "There can be only one!" code pattern for forever, and it works really well:
That seems a little inefficient to pull in SQLite just so that you can have it lock a file, when you can just lock your own file, don't you think?
glances around nervously is flocking one of those "what's old is new again" concept that people are just now relearning? Surely a blog post on how not to have two concurrent cronjobs running is kind of a weird thing to read when some Linux grey beard figured this out in the 90's on their lunch break.
SQLite comes for free in Python distribution. The only reason why I mentioned it. Of course, just having a nice cross-platform file locking interface would be nice (not the case for Python).
I meant in a complexity/memory way. Exclusive locks themselves aren't actually cross platform - it would vary on your fs. That's why you don't see a nice cross-platform file locking interface. I'm not sure if you can guarantee an exclusive lock in SQLite on something like a nfs?
> SQLite uses POSIX advisory locks to implement locking on Unix. On Windows it uses the LockFile(), LockFileEx(), and UnlockFile() system calls. SQLite assumes that these system calls all work as advertised. If that is not the case, then database corruption can result. One should note that POSIX advisory locking is known to be buggy or even unimplemented on many NFS implementations (including recent versions of Mac OS X) and that there are reports of locking problems for network filesystems under Windows. Your best defense is to not use SQLite for files on a network filesystem.
I guess cross-platform for naive me is POSIX + Windows.
I have 100+ crons & I don't want to write some specific logic in my code for handling this, now what? would you mind sharing your thoughts on the same?
Well, the flock command mentioned in the article will also work, but you might have to deal with having a bit more boilerplate around everything. The moreutils things are pretty small, some are just implemented as small perl scripts even.
If you're writing the program, it's fairly easy to do yourself in it with Flock or Fcntl (you can usually just get an exclusive lock in the absolute path to the running executable/script.
That said, quite a few of those moreutils items are useful in their own right (e.g. ts to prefix timestamps to each line of output is something so simple but so useful for logging output, chronic to silence stdout unless exit error is also very useful for cron, etc).
But doesn’t flock also require installing a utils package? The docker people, recently being forced to choose a life of mass migration, would greatly appreciate tools which do not introduce additional overhead to be carried across the whaling sea to the kubed podlands.
The very small utils are actually perl scripts so there's a few dependencies it may pull in there (especially if there's no Perl in the container already, but this is for CentOS so people might be using different base containers.
But honestly, a package made that excludes the perl portions would be ideal for containers. From what I can tell, there's no other libraries used except glibc, and then you get most of those implemented in ~20k or less each.
If the perl was split into separate packages (CentOS already ships moreutils and moreutils-parallel to split that out), that would be ideal for containers for the portions that are binaries, IMO.
Systemd is much more powerful than cron and manages your exact use case. You will need to deal with process hangs though if you have self dependencies like this.
We used to run crons using a custom wrapper which handled locking, logging, state reporting and enable/disable via puppet. However, systemd timer units basically do all of this for you, so I'd recommend that instead.
Can you please share any resources to understand it more in detail? With flock, I am facing issue with lock files, if cron crashes or system reboots, lock files are a big pain.
IME, flock works fine across SIGKILLs and forced reboots. Are you trying to use flock on a filesystem that runs across multiple kernels (i.e., networked fs), by any chance?
A fairly common pattern would be trying to create a specifically named file with the O_CREAT and O_EXCL flags. That's atomic, so you don't need flock. If it fails, then exit. This pattern is also often extended where you write your pid to the file and subsequent runs do a sanity check that your pid is actually running, and unlink the file if not.
That said systemd-timers, as mentioned in many other comments, is probably the right answer here.
flock locks are released when the file is no longer open. This means that no matter how your command exits the OS will close open files and release the lock for you.
The problem with creating files is that you then have the stale lock file problem to deal with which adds a lot more complication.
I have made a simple "cron with extras" project https://github.com/umputun/cronn handling duplication issue as well as a bunch of other things, like restarting failed jobs, jitter, automatic reload and so on.
47 comments
[ 253 ms ] story [ 178 ms ] threadHow, exactly ?
I've been a cron user forever, I had never heard of systemd timers until this thread.
A quick internet search shows:
I mean sure, I've got bash script templates that let me solve the problem of preventing duplicate cron job running.But having seen how easy it is to achieve the same thing with systemd timers whilst not requiring a whole lot of boilerplate code in each shell file ... why wouldn't you switch to systemd timers ? I sure will be.
I like systemd. It's a good clone of launchd
https://github.com/dubiousjim/dcron
Hmm, let's see ...
https://github.com/systemd-cron/systemd-cron
[0]: https://www.consul.io/commands/lock
A simple flock doesn't always work all that well - the devil is in the details. And there is the issue that you miss a run of the script and have to wait another half hour for the next run.
I've used the Highlander, "There can be only one!" code pattern for forever, and it works really well:
https://www.perlmonks.org/?node_id=14260
You could change things to check on the status of your lock every minute or so for 15 minutes or whatever.
Also in module form,
https://metacpan.org/pod/App::Highlander
And much more updated:
https://medium.com/booking-com-development/highlander-b67172...
No need to attempt to stuff all this in a cronjob IMHO.
glances around nervously is flocking one of those "what's old is new again" concept that people are just now relearning? Surely a blog post on how not to have two concurrent cronjobs running is kind of a weird thing to read when some Linux grey beard figured this out in the 90's on their lunch break.
> SQLite uses POSIX advisory locks to implement locking on Unix. On Windows it uses the LockFile(), LockFileEx(), and UnlockFile() system calls. SQLite assumes that these system calls all work as advertised. If that is not the case, then database corruption can result. One should note that POSIX advisory locking is known to be buggy or even unimplemented on many NFS implementations (including recent versions of Mac OS X) and that there are reports of locking problems for network filesystems under Windows. Your best defense is to not use SQLite for files on a network filesystem.
I guess cross-platform for naive me is POSIX + Windows.
It does. (https://docs.python.org/3/library/fcntl.html#fcntl.flock)
https://git.dpkg.org/cgit/dpkg/dpkg.git/commit/utils/start-s...
https://man.openbsd.org/crontab.5
"-s command
Only a single instance of command will be run concurrently. Additional instances of command will not be scheduled until the earlier one completes."
But I don't think it saves the state somewhere permanently, which isn't a big problem until cron's process dies for some reason.
The approach you're suggesting is what led to the problem you're proposing it as a solution for.
https://joeyh.name/code/moreutils/
If you're writing the program, it's fairly easy to do yourself in it with Flock or Fcntl (you can usually just get an exclusive lock in the absolute path to the running executable/script.
That said, quite a few of those moreutils items are useful in their own right (e.g. ts to prefix timestamps to each line of output is something so simple but so useful for logging output, chronic to silence stdout unless exit error is also very useful for cron, etc).
But honestly, a package made that excludes the perl portions would be ideal for containers. From what I can tell, there's no other libraries used except glibc, and then you get most of those implemented in ~20k or less each.
If the perl was split into separate packages (CentOS already ships moreutils and moreutils-parallel to split that out), that would be ideal for containers for the portions that are binaries, IMO.That said systemd-timers, as mentioned in many other comments, is probably the right answer here.
The problem with creating files is that you then have the stale lock file problem to deal with which adds a lot more complication.
There's also sysv or posix semaphores.