Ask HN: Raspberry Pi Server Farm with Load Balancing

Hey HN,

I am a Web developer with only rudimentary knowledge of how a server farm is physically designed and maintained. Nevertheless I would like to build a tiny rack filled with about a dozen Raspberry Pi computers (when suppliers allow buying that many at once.) I would like to use each as its own modest server instance, akin to Webfaction, and use another to manage the load balancing across them.

I know this has been considered before, but I would like to ask HN about advice and resources. I have never had to run an application distributed on more than one server so any materials on horizontal scaling, building and managing a small farm would be appreciated. Also, it will need to account for more storage than just a small SD card in each RPi. What's the best way to mount a harddrive RAID into this setup for shared storage?

4 comments

[ 4.4 ms ] story [ 13.8 ms ] thread

ariejan 14y ago ↗

What should the server do? Server static files? Ruby? Python? I think with the latter you'll get into trouble quickly (at least Ruby) in the memory-usage area.

But, given you can run your stack on a single RasPi. Use one to run HaProxy in front of it.

aespinoza 14y ago ↗

I recommend starting from reading the HAProxy site. http://haproxy.1wt.eu/

It has a ton of information about building a load balanced infrastructure using HAProxy.

This article helped me too: http://horms.net/projects/has/html/node6.html

Read some architecture articles from this site: http://highscalability.com

I am also planning on doing that, so once suppliers allow multiple machines I'll write a blog post about I had to do.

JoachimSchipper 14y ago ↗

Aside from the Raspberry Pi being cool, what would this buy you over AWS, a server running some VMs, or some old spare boxes? Getting some experience with load-balanced systems isn't necessarily a bad idea, but the Pi doesn't seem like the best hardware for the job.

Assuming you do want to do this, consider getting one more capable box, and using it as a firewall, load balancer, NFS server and database server. Your Pis can then talk to this machine and do the webby things. (If you want to avoid a single point of failure, use CARP/VRRP, some sensible load balancer, DRBD and database replication. Reboot all Pis on failover, failing over NFS mounts is very painful.)

lumberjack 14y ago ↗

I thought of this before myself. From a purely economical point of view, you'd be better of if you grab a couple laptops or thin clients off Ebay. This after considering the power consumption too.