HN Explorer

Show HN: AOO – C++ library for real-time audio streaming and messaging (aoo.iem.sh)

107 points by spacechild1 ↗ HN
AOO is a lightweight and flexible peer-to-peer audio streaming and messaging solution. It allows to send audio and messages in real-time and on demand between network endpoints - not only in local networks, but also over the public internet!

The C/C++ library can be easily embedded in host applications or plugins. It even runs on embedded devices, such as the ESP32. In addition, the project contains a Pure Data external and a SuperCollider extension.

The following article provides a high level introduction and demonstrates some use cases: https://www.soundingfuture.com/en/article/aoo-low-latency-pe...

AOO has been in development since 2020 and it has been used in several art projects since. Now it has finally reached a stage where I feel comfortable presenting it to a larger public.

This is still a pre-release and I want to get more feedback before making a final release, so tell me what you think!

25 comments

[ 4.2 ms ] story [ 64.1 ms ] thread
taneq
This looks really interesting! Is there anything to stop it being used for video frames as well as audio? Any war stories or interesting projects using it?
spacechild1
Thanks!

> Is there anything to stop it being used for video frames as well as audio?

Generally, the library is aimed at audio applications and follows the typical model of audio plugins: there is a process() function that takes an array of audio buffers and is called by the host application in the audio callback.

That being said, you could abuse the so-called "stream message" feature to embed images resp. video frames in the audio stream, but I'm not sure how practical that would be... Someone should try it :)

> Any war stories or interesting projects using it?

Check out the section "Use cases" in my article: https://www.soundingfuture.com/en/article/aoo-low-latency-pe...

jarmitage
Hi Christof! Really interesting project! I have used your VSTPlugin before (and probably other stuff..).

Have you tried AOO on embedded platforms e.g. Bela, RPi?

Would you consider supporting bindings to other langs, e.g. Python? At iil.is we have an OSC package called iipyper and I'm curious what we could do with AOO in the Python ecosystem https://github.com/Intelligent-Instruments-Lab/iipyper

epcoa
They claim it will work on an ESP32. If it fits and runs decently there it will be nothing an RPi or Bela.
spacechild1
It actually does run on an ESP32 :) And yes, it's really nothing for an RPi or Bela.
spacechild1
Hi, nice to see you here :)

> Have you tried AOO on embedded platforms e.g. Bela, RPi?

Yes, AOO also works on embedded platforms! I managed to run AOO on the Olimex ESP32-ADF board (https://www.olimex.com/Products/IoT/ESP32/ESP32-ADF/open-sou...) to build low-cost wireless speakers that can be played directly from Pd or SC. With two cores @ 240 MHz, the ESP32 is not exactly a powerful chip :) Bela or RPi is no problem at all.

> Would you consider supporting bindings to other langs, e.g. Python?

Actually, that has been on my mind. C# and Java might also be worthwhile, in particular for mobile devices. I don't think I will have the time to do it myself in the near future, but if someone's interested in creating langauge bindings, I'm happy to assist! Since AOO also has a plain C interface, it shouldn't be a big deal.

EDIT: the IIL looks amazing btw!

Zopieux
* How does this compare to Roc, if you've looked at it? https://github.com/roc-streaming/roc-toolkit

* Would it be possible to have pulse/pipewire sink/source examples to quickly get started on desktop?

spacechild1
> How does this compare to Roc, if you've looked at it?

Strangely, I haven't heard of Roc, thanks! Looks neat, albeit a bit minimal. AOO seems to have more features. Most importantly, it has an (optional) connection server to facilitate peer-to-peer streaming over the internet, but there are also many little things like stream metadata, embedded stream messages, event handling, etc. Some of these are very useful or even essential for online jamming applications, for example. (https://sonobus.net/ uses AOO under the hood.)

> Would it be possible to have pulse/pipewire sink/source examples to quickly get started on desktop?

ATM, there are only portaudio examples (https://git.iem.at/aoo/aoo/-/tree/master/examples/cpp), but they should translate easily to pulse/pipewire.

jononor
Cool to see that it works even on ESP32 devices. That will open up many possibilities for affordable speakers/microphones with this integrated.
bsaul
Does jamming over the internet requires special techniques on the musician side ? Whenever i played with my DAW, any kind of hardware latency made it impossible to play correctly. I can’t imagine how that would work with two or more people over the net.
atmanactive
Jamming online in real-time is possible only between participants in the same city or district. It all boils down to internet-induced latency. More than 20ms and no real-time is possible. For long distance (high latency) online jamming, there is Ninjam [1], developed by Cockos, creators of Reaper DAW, where latency is actually expanded to match a musical division (bar), so everyone can jam in non-realtime but still musically correct.

For long distance collaboration and recording (not jamming), I'm currently working on developing a Reaper + Sonobus (which uses AOO) solution which I named ReaConnect [0].

[0] https://github.com/AtmanActive/ReaConnect

[1] https://www.cockos.com/ninjam/

spacechild1
With good hardware and a stable system you should be able to get the harware latency down to 2-5 ms. Then everybody needs a very stable and fast internet connection with low jitter, so you get average ping times of 20 ms or less. On top of that you have a jitter buffer latency of maybe 20 ms or less. (If you're very far apart geographically, you also need to consider the speed of light for the network transmission.) Under ideal circumstances you may end up with a total end-to-end latency of 30 ms. This is the same latency you get when standing 10 meters apart on a large stage. You probably won't be able to play bebop or technical death metal, but pop ballads or stoner rock should be feasable :)
mindcrime
Doom metal might be viable as well. But that's pretty closely related to stoner rock, so maybe that's just saying the same thing.
stargrazer
So given the ESP32 aspect, is there also Android support? Such that it could then be used for an interactive door bell?
spacechild1
AOO works fine on Linux ARM64 devices, so Android shouldn't be a problem. Personally, I haven't tried it yet. I definitely want to add an Android example because mobile devices are a great use case.
pjmlp
NDK doesn't support GNU/Linux userspace APIs other than what is required for ISO C and ISO C++ standard libraries implementation, plus Android NDK specific APIs.

Everything else from traditional GNU/Linux userspace is a matter of luck if it happens to work across devices, or updates, because it is not part of the oficial NDK APIs.

You need to plug into OpenSL ES or AAudio, https://developer.android.com/ndk/guides/audio

spacechild1
Fortunately, AOO has an option for building a stripped down version (`AOO_NET=OFF`) without internal networking, mainly for embedded devices. Then it's just pure standard C++. The downside is that you can't use the connection server.

I definitely need to test on Android and maybe I need to add some extra hooks. Thanks!

pier25
This is very cool.

Is there like a GUI server and client?

spacechild1
If you count Pd and SC, then yes :-D

In the future, I want to add a client plugin and standalone, probably with JUCE.

I'm not sure how useful a GUI would be for the server, though, because it typically runs as a service/daemon.

pier25
I was thinking about a free replacement for the Listento service.

https://audiomovers.com/listento/

I do sound design work for media composers and very often I demo the work in realtime during a video call. The problem is that Google Meet, Skype, Zoom, etc don't have good enough audio quality for this. So in parallel with the call I stream high quality audio from the DAW to a web UI the service offers.

spacechild1
I think you could use https://sonobus.net/ for this purpose. It is based on AOO and has a very nice and easy-to-use UI. You could run the SonoBus plugin in your DAW and your clients could listen with the standalone. Just like AOO, SonoBus supports both Opus and raw PCM. Opus at 128 kbps/channel is pretty much transparent. With PCM you might run into packet loss issues.

(AOO has an option for resending lost packets which can be used to create very stable PCM streams. However, I'm not sure if SonoBus actually supports this option...)

That being said, Listeto looks very neat and the pricing seems reasonable. Is there a particular reason why you want to move away?

pier25
Thanks I will check it out!

Listento is a fantastic service but I don't use it that often to justify a subscription and it's overkill for what I need.

fenesiistvan
VoIP reinvented?
spacechild1
If your question is why AOO does not use any of the (many) existing audio network protocols:

We wanted a protocol that

1. is simple to understand and implement

2. supports all the required features (as outlined in https://aoo.iem.sh/overview/)

If you know of any existing protocol that could achieve that I would be honestly curious to know!