Given ACM's Digital Library is currently free to access thru June 30, 2020 what are must read/watch resources you recommend and why?
https://dl.acm.org/
Those are available everywhere. Stuff you can't get from sci-hub would be the recorded videos. The nicest thing about dropping the paywall is you can click through the references.
My pick is Jeff Dean's The Tail At Scale (2013) [1]. If you're interested in performance of web services, it's very, very well-written and describes a counterintuitive (but crucial!) phenomenon: your p99.99 latency can drive the entire user experience because of head-of-line blocking. As microservice architectures have gotten more popular, this has only become more important.
It's a fiendishly difficult problem to get around, but Dean proposes a few mechanisms. I learned a lot and think back to this paper often when designing real software.
Reading it will probably make you a better engineer.
Intuitively, if your request is handled by 1 service you have 1 chance that your request lands at the extreme end of the latency distribution. If your requests require 20 services, that's 20 chances.
In reality maybe someone is able to make each microservice so much more performant and is able to deal with slow or failed requests gracefully in the UX. Some sites do, but it doesn't automatically by any means.
Saying a different way: for fork/join or barrier style parallel requests, stragglers set overall latency, and though the probability of any specific response being a straggler may be low, the probability of at least one response being a terrible straggler gets very high at large scales (or large fan-outs).
A couple weeks ago someone brought up High Frequency Trading and while in theory it didn’t tell me anything I didn’t already know, I’ve been chewing on the thesis of the linked article ever since: that the real trick to doing things quickly is to do them consistently. That variance causes far more kinds of practical problems than does average response time.
This a life changing video by Gil Tene on this topic. Bottom line is because of the number requests to support a single customer "request", the high percentiles are the ones that actually matter.
An argument I’ve had a few times and wish I didn’t:
Yes, this situation only happens a small fraction of the time, but it happens exactly once to each user, and the first time they use it. It doesn’t matter what the stats say. The facts are that every user will see this problem and it will be their first experience with the feature.
In such situations I usually ask "whose percentile are we talking about ? the percentile of the latencies of the service or the percentile of the wait time of the user ?" Then I show that for many scenarios the percentile that the user experiences can be drastically different.
If you are a Ruby programmer or interested in Ruby I maintain a list of Ruby papers, some of which are only now freely available due to this open access, at https://rubybib.org
EDIT 2: I was just downloading, and got my IP address blocked. Nothing crazy or automated, just manually clicking through to all the articles in HOPLs I–III. Hopefully it'll clear up eventually, but just a warning to anyone else trying to build a library in a day.
Most of the really good and informative papers and books have no time limit. I think the papers and resources that made the biggest impact on me would have been easily dismissed by junior-me because of being before 2010 but fortunately, less-junior-me read them anyway and they changed how I work in many ways for the better.
That's a very good recommendation. And if you can't find the preprint on the authors website, I've had good success just emailing the author and asking for a copy. That's successful in 99% of the cases where I couldn't find it officially for free or as a preprint. Most authors seems more than happy to provide you with a copy.
Are authors technically allowed to do that? (Not _should_ they be allowed, but _are_ they)?
If so, where does the line get drawn? Are they allowed to have an auto-responder that responds to any specially-formatted email with a copy of the requested paper? Can they just put the paper up on their website? Can they just allow their website to be aggregated by someplace that collects all such papers?
I'm sure the legal line is drawn somewhere between those two extremes but I have no idea where.
The exact policy depends on the venue, but to take the named orgs above as an example, if I remember correctly both IEEE and ACM allow publishing the preprint version of a paper on the researchers website, but an aggregator could probably not copy them without permission.
Good question. I initially got told by friends in academia that this is all fine and what they all do if they can't access the paper and they would also hand it over if someone emailed them asking for a copy.
In the end, I guess it depends on who the copyright of the published paper belongs to. If it belongs to the authors, they are free to do as they please, if it belongs to the publisher, you should actually email the publisher and ask for a copy (but good luck with that).
Here are some resources with more thoughts from people in academia and outside on the subject:
Speaking from experience here, don't go overboard trying to grab papers to read later. Doubly so if you have a static IP. Academic resources, the ACM included, are incredibly ban-happy, and the line between good behavior and bad is kept secret.
Basically the entire internet is built on TCP. But TCP wasn't always very good. There was a time when congestion nearly killed the early internet. Congestion avoidance and control talks about how they fixed it. It's light hearted, accessible and one of my all time favourite reads:
This is a good opportunity to remind folks that an ACM subscription is very affordable ($99/year; 25% off on first year [1]), and the benefits are more than worth it:
- Print and online "Communications of the ACM" magazine
- Safari Books subscription included - all 40,000 books (!)
- Online courses, webinars, special interest group conferences
- Full ACM Digital Library (only with the all-access membership option - $198)
note: the promo code in the URL came on a Google search, and seems to give a 25% discount for first year (which is higher than on the site directly). I have no affiliation with ACM other than being a happy subscriber
https://queue.acm.org/detail.cfm?id=1814327 followed by http://varnish-cache.org/docs/trunk/phk/notes.html have completely transformed how I view and architect software. I basically started to strive to leave as much memory management as possible to the kernel thus not only simplifying my code but also making it behave better as citizens in the country that the operating system is, if you will.
Communications of ACM Volume 37, Issue 7 July 1994
This was an issue that came around the time I was starting my dive into agents after reading [1] and it showed me my particular interest is a small part of what other people label the area.
46 comments
[ 4.5 ms ] story [ 115 ms ] threadGo to https://dl.acm.org/conferences
Click on the Conference with the subject area that interest you, like "Hypertext and Hypermedia" (https://dl.acm.org/conference/ht)
Scroll down to "Most Popular" and sort by either Downloads or Cited.
Go bananas with reading everything.
My immediate recommendations are the onward! papers, https://www.sigplan.org/Conferences/Onward/
It's a fiendishly difficult problem to get around, but Dean proposes a few mechanisms. I learned a lot and think back to this paper often when designing real software.
Reading it will probably make you a better engineer.
[1] https://dl.acm.org/doi/abs/10.1145/2408776.2408794
In reality maybe someone is able to make each microservice so much more performant and is able to deal with slow or failed requests gracefully in the UX. Some sites do, but it doesn't automatically by any means.
https://www.youtube.com/watch?v=lJ8ydIuPFeU&list=WL&index=22...
Yes, this situation only happens a small fraction of the time, but it happens exactly once to each user, and the first time they use it. It doesn’t matter what the stats say. The facts are that every user will see this problem and it will be their first experience with the feature.
https://dl.acm.org/doi/book/10.1145/3335772
EDIT 1: HOPL was pointed out earlier by spdegabrielle (https://news.ycombinator.com/item?id=22795488).
EDIT 2: I was just downloading, and got my IP address blocked. Nothing crazy or automated, just manually clicking through to all the articles in HOPLs I–III. Hopefully it'll clear up eventually, but just a warning to anyone else trying to build a library in a day.
https://techpack.acm.org/
Hopl II: https://dl.acm.org/doi/proceedings/10.1145/154766
Hopl III: https://dl.acm.org/doi/10.1145/1238844.1411838 (use next button on the left to proceed)
Lots of absolutely fascinating stuff in there.
The easiest way is to search for the paper on Google Scholar and see if there is a link that says "[PDF]" or "All X versions".
And if that finally doesn't work out, there is always sci-hub.si and their alter egos (https://en.wikipedia.org/wiki/Sci-Hub)
If so, where does the line get drawn? Are they allowed to have an auto-responder that responds to any specially-formatted email with a copy of the requested paper? Can they just put the paper up on their website? Can they just allow their website to be aggregated by someplace that collects all such papers?
I'm sure the legal line is drawn somewhere between those two extremes but I have no idea where.
In the end, I guess it depends on who the copyright of the published paper belongs to. If it belongs to the authors, they are free to do as they please, if it belongs to the publisher, you should actually email the publisher and ask for a copy (but good luck with that).
Here are some resources with more thoughts from people in academia and outside on the subject:
- https://academia.stackexchange.com/questions/21440/is-it-rig...
- https://www.quora.com/How-can-I-send-requesting-email-to-res...
- https://www.researchgate.net/post/Can_we_send_published_pape...
1979 reprint: https://dl.acm.org/doi/10.1145/1113634.1113638
- MapReduce: Simplified Data Processing on Large Clusters by Jeff Dean et al.
- The Design Philosophy of the DARPA Internet Protocols by David D. Clark
- Chord: A Scalable Peer-to-peer Lookup Service for Internet Applications by Ion Stoica et al.
https://dl.acm.org/doi/10.1145/52324.52356
- Print and online "Communications of the ACM" magazine
- Safari Books subscription included - all 40,000 books (!)
- Online courses, webinars, special interest group conferences
- Full ACM Digital Library (only with the all-access membership option - $198)
[1] https://services.acm.org/public/qj/keep_inventing/qjprofm_co...
note: the promo code in the URL came on a Google search, and seems to give a 25% discount for first year (which is higher than on the site directly). I have no affiliation with ACM other than being a happy subscriber
This was an issue that came around the time I was starting my dive into agents after reading [1] and it showed me my particular interest is a small part of what other people label the area.
1) https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.123...