HN Explorer

Ask HN: $8k on AWS in August for you, any interesting computations?

124 points by adhambadr ↗ HN
for reasons too long to write here I have around $8k of credit on AWS from my last failed venture that will expire end of this august. Id hate to let those credits go in thin air and I don't have the time to build the stuff I imagined myself building to use them, so I thought Id give it to the HN community if they come up with a really cool 'demanding' computational task to use this kind of server power.

Comment on how you'd like to use it and if you impress us they're yours.

p.s id be super happy if somehow the computational power is used to build/crawl some kind of data that would directly/indirectly help anyone for free. Second to that would be if hackers in the comments come up with an idea and do it together.

93 comments

[ 3.0 ms ] story [ 63.4 ms ] thread
jdimov9
That's very generous.

Not sure if this matches your criteria, but I've been wanting to run some extensive benchmark tests on the most multi-core EC2 instance that I can get for this Elixir project: https://github.com/a115/exmatrix

I probably don't need more than a couple of days of uptime to complete this, and the results will of course be made public, with proper credit for your support. jdimov at a115 co uk

smt88
If someone wants you to do webscraping, even for a good cause, I wouldn't waste my time if I were you. Most AWS IPs have been long-since blacklisted by the most interesting sites (e.g. craigslist).
autotune
How about building an Einstein@Home lab with some of those credits? Help discover some neutron stars.
Debogue
Or Folding at Home for protein folding
[deleted]
(comment deleted)
jcoffland
That would be a waste since this would contribute very little to F@H. I'm the lead dev at F@H.
hellbanner
I thought F@H was processing dependent?
tempestn
Can you explain in more detail why the contribution wouldn't be proportional to the value of the credits?
joshstrange
You could spin up a large number of ArchiveTeam warrior images
pjungwir
Are they a 501(c)(3)? If the OP's marginal combined tax rate is, say 50%, then donating $8k in credits would be like $4k in their pocket.

EDIT: Of course OP should talk to an accountant first. . . .

JoshTriplett
The Internet Archive is indeed a 501(c)3, and this is a great idea. It also fits very well with the request in the PS.
sirsar
The Internet Archive is a different entity than ArchiveTeam. I'm skeptical that running AT instances (which scrape websites into archives which then get hosted by IA) counts as a donation towards the IA.

From archiveteam.org:

    Archive Team is in no way affiliated with the fine folks at ARCHIVE.ORG
JoshTriplett
Granted, but you could contact archive.org and ask how $8k of AWK credit could be donated and used productively. I'm certain they'd have a valuable use for it.
adhambadr
id rather spent my time writing infinitely nested while loops then talk to my accountant about one more topic
lucb1e
If I get this correctly, this would help make a backup of archive.org. It would download a big amount of data and then store it for... one month, until the instances expire? That seems rather wasteful of bandwidth and effort, or am I missing the point?
sp332
That's a separate AT project. The Warrior downloads data from various websites (usually ones that look like they're about to go down) and uploads the data to a central AT server. It's distributed to get around rate-limiting and banning. Data is not stored long-term on the Warrior.
[deleted]
(comment deleted)
DavidSJ
It'll be closer to about $200 maybe. And it's an awful waste of energy (not saying mining bitcoin in general is a waste of energy, but doing it this way is).
ronreiter
ha, lol, that was exactly my estimate.
ronreiter
who said you lose only 4k? 8k general compute power in AWS is roughly 200$ in bitcoin.
[deleted]
(comment deleted)
JoshTriplett
GPUs aren't nearly as effective as they used to be, given all the ASIC miners. And AWS GPU instances don't have the right GPU, either; there's a very specific line of AMD GPUs that can accelerate the blockchain hashing algorithm, and AWS uses nVidia.
simonebrunozzi
With 8,000$ worth of AWS credit, you would probably get a few cents worth of Bitcoin. Mining with computers or GPUs has stopped to be in any way profitable years ago.
eudoxus
You can certainly mine various altcoins that are still GPU mining based and see some return on it for 8K. Although AWS GPUs aren't that good at mining in general.
j0hnb
I donate to http://www.worldcommunitygrid.org/ when possible. Right now I'm donating my monthly $100 MSDN Azure credit. I would do more through AWS but I don't have the budget to do so at the moment.
rajington
I'd like to use Riot's official API (https://developer.riotgames.com/) to scrape a BUNCH of game data, specifically "timeline" data, and see which players do things the best.
airza
Their rate limiting is laughable compared to the amount of data they produce (even on prod apps) so good luck :/
kauffj
If you're a FOI supporter, I'm one of the founders of http://lbry.io a startup launching this month that could absolutely put those to good use. We're creating a fully decentralized market for information, with an emphasis on consumer-friendly names.
api
Ethereum GPU mining?
[deleted]
(comment deleted)
simonebrunozzi
Give it to Watsi, a non-profit that is making extensive use of AWS. It would probably equate to donating the entire sum to them.
philip1209
Bayes Impact is another YC nonprofit that may be interested!
adhambadr
Watsi sounds like an awesome organisation, but would they need such computing power within a month ? (the credit is only usable in August), if then can you connect me with them ?
madchops1
You are so aweosome for doing this. I built GoReturnMe.com single handedly with my own php framework Tatanka. It needs an update but we could use some credit :) I love building bands/musicians/artists websites. And I host them at no charge on AWS, credit would help there :) I also an an engineer for label insight.com :) we could use some credit too.

Bands/Artists: http://www.deathbyicon.com http://www.deannadevore.com http://www.theivorysband.com

GoReturnMe: http://www.goreturnme.com LabelInsight: http://www.labelinsight.com

You're awesome!

metral
how about donating it to some sort of intern/mentorship program? i have a friend attempting to start one with gov't support and is in the trial'ing phase of it now at his company. he may find it useful. I could put you two in contact
firloop
I'm pretty sure what you're trying to do is not allowed under Section 1.3 of the AWS Promo Credit agreement. Personally, I would be wary accepting "free" credit that I may end up being on the hook for.

https://aws.amazon.com/awscredits/

[deleted]
(comment deleted)
nadams
IANAL but I don't see that a violation. As long as he isn't selling or renting it out for a price then he is fine. I have a feeling that provision is there to make sure people don't try to sell credits, screw over the buyer, then the buyer tries to complain to Amazon.

Steam has a similar stipulation - and a lot of people are pissed about it. Basically you can't buy or sell steam accounts - and if you are caught Valve will suspend the account. I personally think it's completely stupid and if actually challenged would not hold up in court (I think the first sale doctrine fits perfectly because if you sell your steam account then the games are no longer accessible to you). But again - I get why, they don't want to put up with scammers or stupid people who try to sell an account and disappear before they send the credentials.

deadmik3
> You may not sell, license, rent, or otherwise transfer Promotional Credit. Promotional Credit may be applied only to your AWS account, and may not be applied to any other account.

Seems pretty clear to me...

wernercd
> You may not sell, license, rent

He's giving it away, no money changing hands.

> or otherwise transfer Promotional credit

Is what he has "promotional" credit? He says he has credit... is there a difference? If there is, then that quote doesn't apply. If they are the same thing, and it's used on his account?

If there is no "sale" or "transfer", then yes... it is clear: He's fine to use it as he wishes.

deadmik3
Yes, if he bought $8,000 in AWS credits then he can use it as he wants, but I doubt this kind soul is trying to just throw out $8k.

This is very very likely promotional credit. They hand it out like candy.

Edit: Okay, I can see that if his approach is to use it on his own account, but for someone else's purposes, then it's probably fine. I don't think they even have any system to transfer AWS credits

Edit2: Then again I would be wary of publicly announcing that I'm actively trying to use my AWS credits for other people even though I'm not technically "transferring" them...

bitJericho
Be careful the AWS police will get you.
adhambadr
-yup its a promotional credit (started at 10k), i didnt really buy it -nope I can't transfer it -yes my approach was simply to give the person/people access to my account -Its my fucking server and I can run whatever script I see fit .. but ok realistically maybe its not the best idea to give it to a commercial startup/project that has a legal entity to be sued. but I don't think its really that big of a deal to run some crawlers
pc86
If he does the computations and simply gives the results as a donation is that transferring?
nadams
I would argue no - what would be the difference between him spinning up VMs and handing off the credentials and hosting a public web service on it? The VMs are still his responsibility - he is just allowing other people to use them.
wc-
If you are into baseball and advanced sabermetric stats, there's a bunch of c3.8XL and g2.8XL instances out there cranking through data to train various neural networks. The results can't be published to the public but I'd be happy to add you to the distribution list of results and future projections.
icelancer
Hmm? First I've heard of it, and I work in professional baseball and have been active in the SABR community. Can you add me? mike.anon@hotmail.com
wc-
Unfortunately it's been caught up in the classic proprietary closed-source vs public source political debates... Pretty much zero chance anything we do gets added to the public domain.
xzlzx
How could I get on that list? I would love to be involved.

Edit: email -> dustin@xdlbx.io

katharas
I am doing my thesis (in the general area of video classification) as an electrical engineer in a Greek University and I am currently running some experiments that try to learn a video representation from local features (kernel codebook encoding) using an SVM variant and stochastic gradient descent.

I could use a couple of c4.8xlarge instances to run the experiments trying out learning parameters. I am currently "competing" for time in the lab's cluster so I end up running the experiments in whatever computer I find available.

Anyway, thanks for the opportunity, it is really generous of you.

plicense
You know it pains me to see you ask for resources. Everyday at work, my team uses a couple r3.4x large and c4.8x large for trivial tasks. My team does not realize how valuable a resource they are accessing(and wasting) everyday. I wish I can donate those to you, but company policy forbids me to.
katharas
Thanks for the thought!

I hope I didn't sound ungrateful... There are resources in the university but they come with overhead and not as much freedom as a VM on which you can do anything.

That said our cluster is about as big as two c4.8xlarge vms.

cheald
FWIW, you can have 2 cc2.8xlarge spot instances for about $0.50/hour. $12/day isn't nothing, but it's pretty affordable. Stick your data on an EBS instance attached to the spot so you keep your data if the spot instance gets nuked and you have a functioning cluster for around 13% of the on-demand price.
mryan
Your thesis sounds interesting. I'm working with a company doing some research in a potentially similar area. They might be able to spare some time on their cluster - drop me an email if interested and I'll see if I can set something up.
the_economist
Amazon was generous enough to give you the credit trying to help your company, and in general is extremely helpful to the startup community as a whole with AWS credits. Maybe you should just give it back to them so they continue to be so kind in the future.

(On the other hand, the main reason AWS gives out these credits is to get startups hooked on AWS, then charge borderline outrageous prices once the credit is done for, profiting billions) :)

SomeCallMeTim
AWS is constantly reducing prices, as their costs go down. I wouldn't accuse them of profiteering, at least any more than any public company (with the requirement of a fiduciary responsibility to shareholders).

They're not a non-profit. They don't always have the best price/performance ratio, but they do offer scaling options that few competitors can match.

Disclaimer: I am a former Amazon employee, though I sold my stock. Regretting it now, but what can you do.

ranman
(former AWS employee) You can't give the credits back or transfer them to another account (at least not in a short period of time). Best to spend them.

There's just no infrastructure to support that scenario.

kretor
Not spending the credits is like giving them back to Amazon
the_economist
If you don't use the credits, they expire in two years. How is that not giving the credits back to Amazon?
yitchelle
If the credits ended up being donated to a non-profit org, and Amazon can't overlook it, then Amazon is just being a prick about it. Hopefully, I am jumping the gun a bit here..
wyldfire
I've been fuzz testing python modules on some whimpy local computers, I'd love to give AWS a try!

I hope to find security problems w/CPython which would likely help the world at large.

sbhavnani
I would use it to find 1,000,000,000 digits of Tau.
geyang
I am working on a real-time collaborative notebook for code and mathematics. We are trying to upgrade to a small ec2 from a micro tier. Some AWS credit would be helpful for us to try this out!

www.escherpad.com

mahouse
I can send you some WPA handshakes for you to decrypt them. ;P
dimino
Is it just me or has this post been made before here on HN?
dchichkov
If you are Bay Area local and would like to support soaring pilots community (Glider, Hang Glider, Paraglider), I'm maintaining local soaring forecast and run relatively computationally expensive NOAA WRF model. It is free, no ads, I've been maintaining it for ~7-8 years and for last couple of years it runs at about $700/year using spot/auction Amazon pricing. Available at http://norcalsoaring.org (my e-mail is HN user id @gmail)

edit: after reading a link describing Watsi (current top comment), I would support donating credits to Watsi as well.

alan_si
For my Master's Project (UC Berkeley), I'm working on sports analytics and furthering in-game sports prediction. We are building off some initial work out of MIT that you can find out more about here: http://www.sloansportsconference.com/?p=6137

To start, the goal is to perform real-time MLB pitch prediction. To facilitate this we will be building a crawler to curate MLB data from http://gd2.mlb.com/components/game/mlb/ as well as testing some Machine Learning algorithms. We'd like to use the credits to facilitate this and potentially put together some crawlers for other sports data (NBA, NHL, etc.) before the end of the month!