HN Explorer

Ask HN: How to get started with audio programming?

358 points by Flex247A ↗ HN
I recently picked up 'The Audio Programming Book' [0] and so far, I am really liking it. But I am not sure which resource to pick after reading that. Can you recommend me some audio programming resources which are beginner friendly?

[0] https://mitpress.mit.edu/books/audio-programming-book

Edit: My goal is to make a mini-synth which takes input from the computer keyboard.

122 comments

[ 4.3 ms ] story [ 174 ms ] thread
Flex247A
Reading 'The Audio Programming Book' also clarified the concept of OOP for me. There was a small section where function pointers in C structs was discussed. Just after that, there was a section on struct classes in C++ which made it clear why data members should be grouped with their related functions.
smlss_sftwr
Haven't read that book so I'm not familiar with the contents, but if you're interested in creating VST plugins you might want to check out some frameworks out there like JUCE: https://juce.com/

I don't know if you're a musician yourself but I'd highly recommend downloading a DAW and playing around with the plugins to get a sense of common audio processing modules, you can perhaps make it a near-term goal to create an EQ or filter plugin that you can install and use in the DAW. IIRC I think Ableton Lite should have a free demo, Reaper is another well-regarded free DAW from what I've heard

gerlofs
It depends on what you're trying to achieve. I'd say pick a project you want to work on and go from there, resources on JUCE/C++ aren't always going to be helpful if you don't want to write plugins.
rmnclmnt
Two books I loved during my digital signal processing studies (especially in the audio field):

- "Introduction to Computer Music"[0] by Nick Collins: very focused on audio programming for musical applications but you can easily extrapolate for other domains. Lots of pseudo-code algorithms are provided.

- "The Scientist and Engineer's Guide to Digital Signal Processing"[1] by Steven W. Smith: the bread and butter for everything concept related to DSP. Everything is explained in a crystal-clear fashion, lots of sample code to get you started with digital filters and analysis (although written in BASIC but easily translatable to C/C++ or even Python).

Also there is this one article "Real-time audio programming 101: time waits for nothing"[2] by Ross Bencina (creator of AudioMulch, Portaudio, etc.) about the basics concepts of real-time audio programming

[0] https://www.wiley.com/en-gb/Introduction+to+Computer+Music-p...

[1] https://www.dspguide.com/

[2] http://www.rossbencina.com/code/real-time-audio-programming-...

clauswitt
I wrote a CLI version of a sequencer (multitrack sequencing with plugins and automation where a simple xml format controls the sequencer events) a couple of years ago. I had a quite clear vision of what I wanted to achieve, and then picked C++ and JUCE and just started figuring things out.

I generally learn best by having a real thing to build; and then finding the resources I need when I get stuck.

I buy books aspirationally (I have bought the Audio Programming Book as well come to think of it) - so I often end up with many books on subjects that interests me, without learning much from them.

lxdesk
Implement a MIDI 1.0 sequencer and get it to play back SMF files with a sine wave synth - you can have it output a WAV file, or learn an API to do realtime rendering. It's not a large spec, there are lots of old documents on how the protocol functions in practice, and then once you start getting it working you'll get results instantly(lots of SMF files around to test with) but will want more features, better synthesis; complications start to arise and you will then pick up a lot of knowledge by doing.
Grustaf
Not answering your question, but if you're interested in audio programming you might find Pure Data interesting, it's sort of visual audio programming. It might seem like a toy but it's actually incredibly powerful: https://puredata.info
yummypaint
Second for puredata. It's pretty different from other options but very powerful. It's also quite visual and uses the same kind of design concepts involved in setting up a traditional modular synth.
Nekorosu
There is a more up to date fork of it called Purr Data https://www.purrdata.net/
spacechild1
Purr Data is not really more “up to date“, it is rather Pd vanilla with batteries included + a nicer UI. Both are actively developed. Note, however, that they are not 100% compatible: both have features that the other one is (still) missing.
tomduncalf
This is a topic close to my heart as I've always been very into music production and programming, and have dabbled with a few things over the years, but have only started working professionally with audio software in the last few years.

What is your background, and which parts of audio programming would you like to focus on? e.g. are you interested in writing your own low-level DSP code, or more interested in hooking together higher-level blocks?

If you're from a web programming background, you can have a lot of fun with the WebAudio API [1]. It provides high level building blocks like oscillators, filters, effects, etc., which you can hook together. It has some rough edges but it's a great starting point.

Tone.js [2] is a great way to get started with WebAudio, it provides some useful abstractions that let you build fun things quickly, and there are some great examples of WebAudio creativity out there e.g. Blokdust [3]. If you want to go lower level, all major browsers now support the AudioWorklet API [4], which lets you write your own custom DSP algorithms and have them run efficiently within WebAudio.

However, WebAudio has limitations in terms of performance, and you can't use a WebAudio synth as a plugin in your music production software of choice (well, you might be able to get a browser which runs as a plugin, but it's not going to be ideal!).

If you want to write professional audio code, you can't really avoid working with C++ - it's the industry standard approach. This is largely because C++ allows programmers to very carefully design the performance characteristics of their code - specifically, you can avoid doing anything which might take an unknown amount of time (e.g. allocating new memory, locking) while processing on your audio thread - audio is realtime and very low latency, so you need to be sure your code is going to run in the amount of time you think it will.

Personally, I think JUCE [5] is a great starting point. It abstracts away some of the complexities of both C++ and audio APIs, allowing you to build cross platform audio apps and plugins without having to worry about platform-specific quirks. They have a pretty good selection of tutorials too [6]. An alternative you could look into is iPlug2 [7]. There are a couple of books by Will Pirkle on the topic of programming audio effects and synths with C++ - the author uses his own framework, but the same principles apply to other frameworks. Make sure you get the new editions if you do get these.

You'll probably want to take time to learn some C++ basics first if you don't already know it, it's a hard language to master! I'm actually working on an integration right now to allow you to build web UIs for JUCE apps, which takes away the complexity of also building the UI in C++ - I'll hopefully have something to share in the next few weeks.

If you want something a bit more straightforward and you're happy just supporting MacOS and iOS, AudioKit [8] is worth a look. It's kind of in between WebAudio and JUCE in terms of complexity I'd say - you write your code in Swift and it provides a load of high level building blocks, but you can drop down to a lower level if you need to write your own stuff in C++. Also the CoreAudio/CoreMIDI APIs on Apple platforms are very powerful, though I've struggled to find great documentation.

One other option you could look at are specialist audio languages like Supercollider [9] and Faust [10], or visual programming tools like Max [11] or Pure Data [12]. I'm not very well versed in any of these, but I've heard good things about Supercollider especially.

Phew, hope that helps! As you can probably tell, I'm really interested in this stuff so please give me a shout with any questions!

Edit: one more resource I remembered, there's a Youtube channel dedicated to learning audio programming, mostly using JUCE, called The Audio Programmer [13]. I haven't actually watched much content so can't say ...

Flex247A
Thank you so much for your detailed comment! I am mostly interested in writing professional audio code, so I am learning C++ at the moment. Also, do you think Rust is a good alternative to C++ for writing high performance code? I am thinking of learning Rust too, but I am not sure whether Rust audio programming is popular or not.
tomduncalf
My pleasure!

My understanding of Rust audio programming is that it is possible, but still fairly early days... something like JUCE provides a huge amount out of the box that you might not initially appreciate, e.g. it abstracts away the details of all the different plugin formats and all the different platform's audio APIs, and it has been around for long enough that you can be confident it does it well.

I'm really curious about Rust as it feels like a nicer language to work with than C++, but if you are serious about building professional audio software I personally don't think you can avoid C++ so you might as well embrace it.

I have a feeling that with Rust, you'd spend half your time figuring out how to do stuff that JUCE does out of the box, which is fine if you are interested in the figuring out part, but ultimately it might not help you focus on the actual audio programming.

Also there are a tonne of resources on C++ and especially JUCE out there, never underestimate the power of being able to Google for your answers!

Edit: on the C++ side, I found Bjarne Stroustrup's "C++ Programming Language" book quite a good way to understand the fundamentals. To be honest, I usually learn new languages just by doing and Googling, but with C++ it quickly became apparent after the 100th mystery crash that I'd actually have to understand what was going on under the hood with memory etc!

Flex247A
Thank you! Looks like I might have to embrace C++ after all ;)
tomduncalf
Haha yeah sorry about that ;) To be honest I actually found it really interesting to be exposed to lower level concerns like thinking about memory allocation, runtime complexity, locking etc. after working in higher level languages throughout my career, but there’s no doubt that it can be a frustrating language at times.

One thing to watch out for is that the language has evolved a lot recently, but a lot of code samples are still written in an old style. You want to make sure you are using “modern” C++ (e.g. post C++11) features where possible e.g. smart pointers. It can be a bit confusing as some JUCE classes have been superseded by the C++ standard library, they date back to the days before “modern C++”... but I wouldn’t stress too much, using the JUCE versions is usually equally fine.

ajxs
Depending on what kind of audio programming you intend to do, it should be possible to use Rust or other alternative programming languages. Your mileage may vary, depending on the dependencies of what you intend to build. If you want to interface with your operating system's low-level audio libraries you should be able to do that from any language that can interface with C. I was able to use Ada together with ALSA on Linux in this way. In this case the code responsible for generating the sound was in Ada, and the code for interfacing with the audio hardware (via ALSA) was in C. If you intend to write audio code for bare metal, you should be able to use any programming language your target hardware's toolchain supports. I've heard of people using alternative languages to write VSTs. I'm not an expert in this area, however theoretically you should be able to do something similar to the above as long as you have a good way to interface with C/C++ code.
diimdeep
> JUCE

I would add to that VCV Rack [1] is another great starting point.

Very low barrier to entry, see developer manual [2], hundreds open source modules to learn from. [3]

[1] https://vcvrack.com

[2] https://vcvrack.com/manual/PluginDevelopmentTutorial

[3] https://github.com/search?o=desc&q=vcv&s=&type=Repositories

tomduncalf
Nice, I hadn’t looked into building VCV Rack modules but it’s a super fun piece of software - there’s also an iOS port called MiRack, personally I love making music on the iPad so I use that more.

I saw this the other day that lets reuse gen~ code from Max as a VCV module: https://github.com/isabelgk/gen-rack

varjag
> If you want to write professional audio code, you can't really avoid working with C++ - it's the industry standard approach.

No, you can do with C just fine.

tomduncalf
True, although most of the frameworks for building plugins etc. are in C++ as far as I know, so using C might be making life hard for yourself if that’s your goal.
Bridgeburner4
Thanks for this answer!

I don't know much about audio programming, but here's a question.

When emulating sounds from physical instruments, can you get close by using only code without using audio samples? If yes, is it harder/easier?

tomduncalf
It's not something I know a lot about (I'm more interested in synthesised sounds!) but I believe you can create pretty impressive emulations of physical instruments using techniques like physical modelling. Not sure if it's comparable to a well recorded sample though.

I believe Audio Modelling are one of the leaders in this field: https://audiomodeling.com/

In terms of harder/easier... I'd guess it's a lot harder to code, but easier in terms of not having to sample thousands of different articulations for each instrument! Judging by the huge selection of sampled physical instruments and the small amount of synthesised ones, I guess overall harder. Would be interesting to hear from someone who knows more though!

cjaybo
> I'm actually working on an integration right now to allow you to build web UIs for JUCE apps, which takes away the complexity of also building the UI in C++

I assume you've seen the Blueprint project for building React UIs on top of JUCE? Very interested to hear more details about what you've been working on whenever you have something ready to share!

tomduncalf
I have indeed, it's great.

My primary motivations for doing something different was that Blueprint currently doesn't work on mobile (and I'm not sure it could ship on the iOS app store currently, with it using Duktape), and I wanted to get away from writing any C++ at all for the UI – and also that I love playing with this kind of stuff, heh. The app that I work on is built using React Native and JUCE (and Unity), so it's an area I have a lot of thoughts about!

TBH the web stuff started out as an experiment, but I'm actually pretty impressed by how practical web is for building UIs – I've got some nice integration going on to make it pretty natural to write plugin UIs. I'll post it on here when it's ready, but if you have a Twitter/email/whatever you'd like to share or can contact me via mine in my profile, I'd be happy to share with you in advance – I'd love to get feedback from people interested in this area.

imvetri
You can try to build an visual audio editor and learn everything else from there, step by step.
antfarm
If you are on macOS, AudioKit is a nice simplified layer on top of CoreAudio and CoreMidi: https://github.com/AudioKit/AudioKit

It ccomes with many Xcode playgrounds for interactive exploration of the SDK.

Mizza
It depends what you're trying to accomplish, but I found VCV Rack to be a great environment to implement synth ideas without having to worry about reinventing the wheel. Trying making a Rack module!

https://vcvrack.com/

Archit3ch
If you're interested in synths, there is a new version of the Pirkle synth book coming out soon.

Also, I second the suggestion to join the Audio Programmer community.

gregsadetsky
I’ll repost a comment I made a few years ago. Hopefully the (informative, and authoritative) reference will be useful!

——

The founder, and sole developer of Mutable Instruments (a well known modular audio/synth maker) wrote about her recommendations on how to get started in audio programming here [0]. Much recommended for the cited books, and general/business advice on making a hardware audio product.

[0] https://modwiggler.com/forum/viewtopic.php?p=2858814#p285881...

——

My own post about this was from a mini thread on HN about programming sound that may also interest you:

https://news.ycombinator.com/item?id=20491766

——

Last note! Another synth maker personally recommended the same DSP book as noted in the comment above (on Modwiggler), so it’s probably worth checking out!

Udo Zolzer's "DAFX"

capableweb
Just today there was a Audio Operating System featured on HN that looks like a great starting point for audio programming and it comes with everything you need. Unfortunately it looks like everything is out of stock, but looks interesting none the less and you can probably run it in a VM until they get the development kits in stock again.

- https://news.ycombinator.com/item?id=27269589 - Elk OS – Audio Operating System

- https://elk.audio/audio-os/

adriancooney
Shameless plug for my own audio programming experiment: https://noise.sh

It's not as powerful as something like PureData but it does give you a simple and intuitive introduction to DSP and audio synthesis.