HN Explorer

Ask HN: Can you recommend a modern, accessible P2P framework/library?

78 points by rayvy ↗ HN
I'm interested in building p2p applications.

I've checked into IPFS, GNUnet, Freenet, libp2p, and a few more.

My main tools are Rust and Python.

I haven't found anything (I could be reading things wrong of course) that I can directly build GUI applications on top of, with great documentation, and minimal head banging.

Does anyone have experience building p2p apps? Can you recommend/suggest a route to take?

Definitely don't want to have to implement my own thing.

Thanks

36 comments

[ 6.8 ms ] story [ 71.2 ms ] thread
Strom
Peer-to-peer is such a wide term that can mean pretty much anything, e.g. micro loans of money to people in need. It can be viewed just as a way of doing things. So in that sense, all you need is a socket connection to get going.

What are you looking to build more specifically? News sharing? Video chat? File sharing?

If you're looking for file sharing, then a decent solution might be to use BitTorrent via the libtorrent library. https://www.libtorrent.org/

rayvy
> What are you looking to build more specifically?

I think this is my problem. I don't have an idea of anything I want to build specifically, I kinda just want to hack around on something.

My approach to learning new things is to usually read a bit, download it/import it and just start trying things. But that doesn't seem like a practical route when trying to build p2p applications?

lucideer
> I kinda just want to hack around on something.

If this is where you're at, and you don't mind too much learning a new language (js), or porting, the SSB community is very accessible from a social/personal perspective, and there's plenty of small, simple hackable ideas being tossed around discussions on the network for implementing.

fenwick67
Secure Scuttlebutt, DAT and Cabal are working p2p apps that can help expose you to the way things can work. Personally, I wrote frontends for two of those and it's helped me understand them better. These are all written in Nodejs, though.

There's a nice p2p and kappa database primer here: https://github.com/kappa-db/workshop

rolleiflex
When I was building Aether [1] I had to build mine from scratch in Go. It all depends on your needs.

Generally speaking, P2P is not a concept that you can abstract into a library. you have to make your app very deeply integrate with it, because it will present you challenges, limitations (and opportunities) that you have to respond to. There is no such thing as taking a regular app and porting it to P2P.

[1] https://getaether.net

faleidel
Wow, the now version of Aether looks nice! I remember trying your project 3/4 years ago? I'll re-try Aether when I get home.
rolleiflex
Thank you! It’s been a long road. ;)
rklaehn
Not sure what exactly you are looking for, but I find IPFS / libp2p pretty easy to use once you know the basic concepts.

For a simple web based GUI app you can use the js version of IPFS or interact with a locally running go-IPFS node using a pretty simple REST api.

Once you got your feet wet, or if you have a more demanding use case, you can use libp2p directly, from js, go or rust.

Since you mentioned rust: https://github.com/libp2p/rust-libp2p

libp2p could be described as an attempt to implement a common networking stack for the distributed web. IPFS builds on libp2p, as does filecoin.

I work for a company https://www.actyx.io/en that does industrial applications based on IPFS / libp2p, so I use IPFS every day. Obviously it is bleeding edge technology with a lot of rough edges. But I find the current state and the rate of improvement quite promising.

fenwick67
libp2p looks useful but my problem with it (at a glance) has been, "how do I make this into a program?".

When I look at a list of ~10 different modules, and let's say I want to implement a secure chat program, I have no idea where to start. Is there a good primer on something like this?

rayvy
> "how do I make this into a program?".

This! It looks unbelievably promising, but still very much a work in progress, with no deploy-able examples. Can't wait to see what it turns into though

devttyeu
There is a repo with simple examples to help get started - https://github.com/libp2p/go-libp2p-examples

People in IRC channels - #libp2p @ freenode (or #ipfs) - should be able to answer most of your other questions

(I work for PL, on go-ipfs and occasionally libp2p)

jwhitlark
P2P isn't a single thing. But if you know more about what you want to do, you can usually find libraries to do it with.

I recommend Indy's courses for getting the map of the territory: https://www.coursera.org/instructor/indygupta

therein
I remember Indranil Gupta. He was my professor for Distributed Systems in college. He was also my academic advisor but that's just a symbolic thing at my college. Kinda like an "ask this professor stuff about the CS curriculum at this school if you feel like it" arrangement.

He had early lectures but made them fun so I ended up not minding it. If the Coursera follows the same curriculum, I remember it being a pretty comprehensive introduction.

jwhitlark
How cool. I only know of him from these courses, but it was a fantastic intro and overview of distributed systems.
joefourier
If you want real-time data-transmission (for chat, audio or video), what about WebRTC? It's primarily intended to be used in browsers, but there's nothing theoretically preventing you from just using a Python library (e.g. https://github.com/jlaine/aiortc ) that implements the protocols if you wanted.
GFischer
It's not entirely P2P, you need a server for initial connectivity.

But it's probably good enough for most P2P applications (I saw someone building P2P CDNs with it !!!)

nielsbjerg
This guy mad a lot of easy to use libraries https://github.com/mafintosh in nodejs, so don't know if they are useful to you
rapnie
Mafintosh created hypercore and hyperdrive, which is part of datproject.org a nice p2p option to consider too. It is used by Beaker browser among others.
fermuch
He also created peerflix, the magic behind Popcorn Time (now Butter)
notrhodey
There is definitely plenty of things to play around with an experiment in the Dat ecosystem, however their Rust platform is still early stages.

http://datproject.org/ https://github.com/datrs

I'm working on a P2P app built atop Dat right now myself, archiving of police radio using software-defined radio and Dat Hypercores. Maybe the codebase can give you some ideas / inspiration, this is developer-preview level for sure:

https://radiowitness.hashbase.io/

mempko
I built one called Firestr. Check http://firestr.com

I didn't use a framework and built it from scratch. Maybe there is some code there that can help you understand how this stuff works.

The whole app is an environment to make p2p apps in Lua with a built in editor. Not sure there is anything easier than firestr to build p2p apps. Check out the Lua code for some examples here https://github.com/mempko/firestr/tree/master/example_apps

sprucely
Several have been mentioned already, but I'll add one more rust library to the mix: https://github.com/maidsafe/crust
rayvy
I actually emailed one of the contributors from crust and unfortunately the impression I got is that it's still very much a work in progress (but good if you want to actively contribute to the project I guess)
gammateam
Orbit-Db for IPFS

Its just Node.js

You won't be bottlenecked by the language if you are using P2P services. If you want threads, spin up another compute instance, and your provider will do that for you.

zxcmx
Left-field option; embed zerotier SDK. They do a better job selling it than I could: https://www.zerotier.com/blog/zerotier-sdk.shtml

Probably some head-banging required to get it to link/load/run (I see there's an open ticket for Python). Would likely need to do a little bit of c/c++ to get that working properly. Benefit is, you would then never need to touch the p2p layer again and you could just be doing web development if that's your thing.

usgroup
If you’re not familiar with the principals of p2p generally and the kind of problems you get into I’d start with building a simple DHT based solution using something like the Chord protocol:

https://en.m.wikipedia.org/wiki/Chord_(peer-to-peer)

This will teach you all about adhoc networking, routing, the limitations of different architectures and so on. You could also tackle a real use case like distributed key value storage as a learning device.

What you’ll need ultimately will strongly depend on what you intend to build with it.