> For parallel processing I'd reach for the nearest standard library at hand on the language of choice. That is a good example of what I mean: The nearest standard library is likely to either buffer output in memory or…
With --results or --joblog and --resume-failed GNU Parallel can do this, too.
I have done that more than once. I often end up with a solution that works on the test set but which breaks after 10 TB just because <" "@example.com> is a valid email address according RFC-822 (Who the f* thought it…
But that approach is potentially dangerous. See: https://mywiki.wooledge.org/BashPitfalls#Non-atomic_writes_w...
> For parallel processing I'd reach for the nearest standard library at hand on the language of choice. That is a good example of what I mean: The nearest standard library is likely to either buffer output in memory or…
With --results or --joblog and --resume-failed GNU Parallel can do this, too.
I have done that more than once. I often end up with a solution that works on the test set but which breaks after 10 TB just because <" "@example.com> is a valid email address according RFC-822 (Who the f* thought it…
But that approach is potentially dangerous. See: https://mywiki.wooledge.org/BashPitfalls#Non-atomic_writes_w...