(Thanks skrebbel, I appreciate it. I'll checkout talkjs as well)
Just to add, I have 8 recorders now in Limburg. Pretty much any large explosion we can localize to likely within around a 40m diameter location within a 7km stretch. Anyone else wanting to join this network and run some sound localizing listening nodes in the Netherlands please reach out to me.
Well, it's not automatic yet. So sure, it can :) I did record all of the explosions from New Years eve and I'm hoping to have a shot at using it for ML training data this year.
That will be an interesting task to deal with solving the location of multiple events that may overlap.
Over 7km, you will have delays of 10s or so which makes it quite possible that multiple explosions within that window will need disambiguation in the sound record. There should be a parsimony principle that would apply.
Very cool project and fun things to work on related to it.
That part is going to be challenging. But not too challenging I think because the number of nodes that will be relevant will be limited and finite. Even brute force solving would work I think. Many nodes will be instantly ruled out because the time differences will be too great to be viable. That's going to be a lot of fun. But I'm sure that when that part arrives where will be plenty of smart people willing to offer directions.
Till that time, I have a collaborator in America helping with gathering gunshot sounds and I have intentions to make a matcher that also determines the start time of the event accurately enough to be used for the next stage, to feed into the solver that does the localization.
I expect I'll be able to model this to some extent on the way that birdnet from Cornell University works, except that's just identifying a bird and not trying to find with high accuracy the start time of the event. But ultimately that part comes down to generating spectragrams and doing object detection on the spectragram image. Even YoloV6, which should be more accurate than resnet as used in birdnet, can run a nano version of this object detect quite quickly on a Raspberry Pi.
It should be possible to make this part within a year.
Doing localizations by hand I once heard one that localized to the middle of a car park over 3km away. I drove there and could still smell the sulpher. Also, only in that carpark, not in an adjacent carpark. This suggests that accuracy could well have been within a 40-50m diameter. Amazing!
I also tested this with an ultrasonic mic. I could see signatures of small critters that were likely mice as we had chickens at the time. I would need a bunch of ultrasonic mics and an rtk GNSS system to do testing but I expect it might accurate enough in a very small area to determine where rodents are calling from.
I'd like to develop this solution with a mind to extending it's use more generically so that it could be used for species surveys in bioacoustic research.
Very nice to have this topic in this position. So I'll add some more info.
I have a formed a small group to localize large explosions we here just to gain insights into the effectiveness of the localization over distance. The real goal would be to make something like a real time gunshot localization system. Something I'm working on this year. Again, testing on the random fireworks explosion we hear a lot is a great way to test how well it works.
This system records sounds to a raspberry pi that has it's time synchronized with chrony to within less than a microsecond of error. It's recording off a jackd real time audio network so that one can use multiple clients. One client is the recorder but another client can be a real time audio ML pipeline for detecting and localizing gunshots, explosions. Or for localizing bird calls in real time for example. A lot of potential value here in bioacoustics.
I've made this install and configure with one command on all of the Raspberry Pi's and it creates new partitions so that it can run resiliently with an in-memory overlayFS architecture to avoid SD card corruption. Very important to make for a practical product.
The people here might appreciate the effort to shrink and repartition the file system in order to facilitate that. It was an interesting challenge. I had to interrupt the system boot at a very early stage, create a minimal rootfs, pivot the root, shrink the filesystem, create the partitions, pivot back and continue the boot. As far as I know this is the only Raspberry Pi project that does this. If I'm wrong, please let me know, I'm curious and quite proud of this part :)
Very cool project! It is fascinating to see that sensor nodes can be time synchronized so well, using standard technology. That is a key precondition here. Localization is still quite tricky over long distances though, especially due to multi-path propagation, reflections, non-homogeneous propagation conditions from wind, inversion etc.
We did some work on noise monitoring of shooting ranges / explosive training facilities, using machine learning to track sound events.
This was focused on a simpler problem formulation than full localization; "did the impulsive sound event come from A) known site or B) elsewhere?". But as a precondition one also needs to solve "was there an impulsive sound event consistent with gunshot/explosion". Which in the 24/7 monitoring scenario is non-trivial, keeping the false positive rate down.
I link the papers here, in case they are of interest.
https://arxiv.org/abs/2107.11453https://arxiv.org/abs/2108.11758
I don't really work on this niche anymore, but I would be happy to discuss a bit this usecase, maybe it can be helpful in getting the first efforts to automated / semi-automated analysis going. My email is in the papers, and can be found on GitHub/website etc.
7 comments
[ 2.7 ms ] story [ 30.5 ms ] threadJust to add, I have 8 recorders now in Limburg. Pretty much any large explosion we can localize to likely within around a 40m diameter location within a 7km stretch. Anyone else wanting to join this network and run some sound localizing listening nodes in the Netherlands please reach out to me.
Can it handle New Year's Eve?
Over 7km, you will have delays of 10s or so which makes it quite possible that multiple explosions within that window will need disambiguation in the sound record. There should be a parsimony principle that would apply.
Very cool project and fun things to work on related to it.
That part is going to be challenging. But not too challenging I think because the number of nodes that will be relevant will be limited and finite. Even brute force solving would work I think. Many nodes will be instantly ruled out because the time differences will be too great to be viable. That's going to be a lot of fun. But I'm sure that when that part arrives where will be plenty of smart people willing to offer directions.
Till that time, I have a collaborator in America helping with gathering gunshot sounds and I have intentions to make a matcher that also determines the start time of the event accurately enough to be used for the next stage, to feed into the solver that does the localization.
I expect I'll be able to model this to some extent on the way that birdnet from Cornell University works, except that's just identifying a bird and not trying to find with high accuracy the start time of the event. But ultimately that part comes down to generating spectragrams and doing object detection on the spectragram image. Even YoloV6, which should be more accurate than resnet as used in birdnet, can run a nano version of this object detect quite quickly on a Raspberry Pi.
It should be possible to make this part within a year.
Doing localizations by hand I once heard one that localized to the middle of a car park over 3km away. I drove there and could still smell the sulpher. Also, only in that carpark, not in an adjacent carpark. This suggests that accuracy could well have been within a 40-50m diameter. Amazing!
I also tested this with an ultrasonic mic. I could see signatures of small critters that were likely mice as we had chickens at the time. I would need a bunch of ultrasonic mics and an rtk GNSS system to do testing but I expect it might accurate enough in a very small area to determine where rodents are calling from.
I'd like to develop this solution with a mind to extending it's use more generically so that it could be used for species surveys in bioacoustic research.
I have a formed a small group to localize large explosions we here just to gain insights into the effectiveness of the localization over distance. The real goal would be to make something like a real time gunshot localization system. Something I'm working on this year. Again, testing on the random fireworks explosion we hear a lot is a great way to test how well it works.
This system records sounds to a raspberry pi that has it's time synchronized with chrony to within less than a microsecond of error. It's recording off a jackd real time audio network so that one can use multiple clients. One client is the recorder but another client can be a real time audio ML pipeline for detecting and localizing gunshots, explosions. Or for localizing bird calls in real time for example. A lot of potential value here in bioacoustics.
I've made this install and configure with one command on all of the Raspberry Pi's and it creates new partitions so that it can run resiliently with an in-memory overlayFS architecture to avoid SD card corruption. Very important to make for a practical product.
The people here might appreciate the effort to shrink and repartition the file system in order to facilitate that. It was an interesting challenge. I had to interrupt the system boot at a very early stage, create a minimal rootfs, pivot the root, shrink the filesystem, create the partitions, pivot back and continue the boot. As far as I know this is the only Raspberry Pi project that does this. If I'm wrong, please let me know, I'm curious and quite proud of this part :)
Oh and here are the other relevant direct links:
https://github.com/hcfman/sbts-aru
https://medium.com/@kim_94237/tdoa-sound-localization-with-t...
We did some work on noise monitoring of shooting ranges / explosive training facilities, using machine learning to track sound events. This was focused on a simpler problem formulation than full localization; "did the impulsive sound event come from A) known site or B) elsewhere?". But as a precondition one also needs to solve "was there an impulsive sound event consistent with gunshot/explosion". Which in the 24/7 monitoring scenario is non-trivial, keeping the false positive rate down. I link the papers here, in case they are of interest. https://arxiv.org/abs/2107.11453 https://arxiv.org/abs/2108.11758 I don't really work on this niche anymore, but I would be happy to discuss a bit this usecase, maybe it can be helpful in getting the first efforts to automated / semi-automated analysis going. My email is in the papers, and can be found on GitHub/website etc.