HN Explorer

12 comments

[ 9.1 ms ] story [ 118 ms ] thread
storyinmemo
My suggestion for learning: re-write this using futures. Here's a quick example of using similar parallelism to search structured json in gzip'd files: https://github.com/autocracy/snippets/blob/master/search_clo...
emj
Basically xargs in python, and I love xargs.

There are some cool examples of places were xargs fails in the docs, this got me very excited I hope it works in 2.7. https://docs.python.org/3/library/concurrent.futures.html

d0mine
There is futures module for Python 2.7 on PyPI.

In simple cases, you could use multiprocessing.Pool():

  results = pool.map(processfile, getpaths())
There is also ThreadPool. There are known issues with multiprocessing especially on Python 2.7 e.g., Ctrl+C handling.

Though you don't need threading to manage concurrent subprocesses or IO (Gevent, Twisted or curio if you use Python 3).

mkesper
And alwyays use if __name__ == '__main__' if you want be able to test your code.
marlynm
I'm learning. Thanks for letting me know. Will use that.
marlynm
Thanks. Will learn and use.
jhoechtl
Where Python warrants a HN post, Golang (Rust, others) calls it natural. Use the right "tool" to do the job?
formula_ninguna
what's the use case of it?
purrcat259
I would guess: checking if machines are online without rolling an overkill monitoring solution. More of a smoke test than a proper solution (ping does not ensure a service is up for example).
marlynm
1) Finding other reachable hosts in a local network. 2) This can be slightly modified to get average latency for each host. This is not a replacement for health check.