We at Nanonets have been building a lot of Machine Learning models for drone based image analysis. A frequent request from our customers is how do they use our models to do real time analysis. So we decided to create a how to on building real time object detection using a DJI drone.
Also useful if you're just looking to get the video stream to your laptop/computer.
Title seems a little (maybe a lot?) misleading. It's not "on drones" it's actually on a video stream coming from a drone. There's a huge difference: you're doing all of the processing on a desktop/laptop PC.
There's some mention at the end of the article on how to attach a PC to the drone, but basically it would be the same thing: just run the processing on a PC.
Just by the title, I was expecting a clever way to run a pre-trained model directly on a drone.
Wow. Combine this with the ability to drop some of the latest nano-sensors (atoms strung together to build dust-sized sensors and batteries and power at scale), and we should be able to monitor absolutely everything everywhere. Let's see anyone even try to object to my rules or procure a weapon!
And given some on-board weapons, we would never have to worry about civil unrest again. Only people grateful for the food and job they are allowed to be in our society. Utopia!
Hey MrTonyD. I understand where your coming from. But honestly most people are using this for monitoring things like Solar Panels, Wind Turbines, detecting seals and crop prediction. We can't have a sustainable future without a lot of these technologies in place.
https://nanonets.com/drone/wind-turbines/https://nanonets.com/drone/solar-panels/
All new technology will have some deviant use case associated with it. The internet is one of the worst enablers of child pornography, human trafficking and everything disgusting with the world. However it just means you try and control the use of technology not throw the baby out with the bathwater.
Well, I'm 60 now and I've worked in Silicon Valley for over 40 years. So I guess I hear about all the other uses of such technology. I've worked with NSA, CIA, DIA, all the branches of military, and a bunch of people who don't officially exist. So, while there are definitely things I can't say, I will say that the "dust sensors" have been in use for many years, and I'm not describing anything theoretical. We need to think about these things in order to address them. Simply saying that there also exist good uses isn't enough.
Given your expertise over the many years how would you recommend we control the use of such technology?
Maybe my question is based on the assumption that you agree: stopping development of the technology in the first place is not the solution. Hopefully this assumption is correct.
There is no way it can be stopped. Most of the development is secret and much further along than people believe. I think we need to regain community control over those who possess this technology - it will be deployed whether we like it or not. And, even more frightening, whoever deploys it first wins, and it will be so effective that it can no longer be defeated. This issue is more serious than global warming, jobs, healthcare, or World War 3 - it is about having any freedoms beyond those provided by whoever deploys these monitoring devices.
Infantry weapons powered by the kind of computer vision and facial recognition AI easily demonstrable today would be incredibly frightening, no matter how they were used. Of course there's no way of stopping this in an adversarial world.
Surely it would only be a matter of choosing to increase the efforts for a year or so in order to take these weapons from proof-of-concept to highly effective. E.g. autonomous flying gun platforms that can be deployed en masse and let loose with small or no human oversight (e.g. "hold the button to kill the person the machine has identified as a valid target", if even that level of control), or the kind of flying, target-seeking grenades envisioned in the linked video. With any level of autonomy, loitering and communication with command. It's only ethics and public perception that stops this today. In total war, a 20% mis-classification rate doesn't matter. Bombing doctrine in WW2 was much, much worse than this in its effect on civilians. Hell, many military leaders emphasized killing civilians as a desirable objective. Even present-day attacks consider "collateral damage" acceptable, even if politicians sometimes pretend otherwise.
Defending against weapons like that would be an arms race in itself, and completely beyond the reach of weaker actors. At least on a tactical level. Maybe there's strategies that could be employed as a low-cost defense, such as staying out of sensor range or confusing the sensors with adversarial patterns, but it would crush the resistance of anyone that tried a conventional defense.
Does the post-WW2 doctrine of emphasizing trade and the un-profitability of war, or ensuring that it is more economical for everyone to support a stable, free democratic economic system, still have any hope of preventing such weapons from being used?
I agree with you. But I think there is a bigger "weapon". When dust size monitoring is pervasive, how can anyone acquire weapons or coordinate any resistance? Dust is everywhere. And teams of nano-weapons can kill people (this has been reported as research for well over a decade, so anyone could reasonably assume that unreported research has gotten a lot further.) You don't need armies when nobody can effectively fight you or even acquire or build any weapon of any sort. So I don't worry as much about WW3 - since WW3 can be prevented with the pervasive monitoring technology on the near horizon. (There may be a WW3, but we have bigger problems coming up.)
When the map starts to reach the same size as the territory, don't you then need a map for the map? Pervasive monitoring sounds like one hell of a dataset to search through. Sure, you can use it to follow things you have already identified, but just because you have recorded something, doesn't mean that you know that you have recorded it.
Some people wonder why Cloudera (Hadoop and now Spark) had such high stock prices. But the real investors and the real clients were not public. Now, we all know that these systems are limited both in data volume and ability to run all the Machine Learning algorithms - but this problem is looking more and more solvable - and there are many who are trying outside of the public eye.
Catharsis. I had been suspicious of similar things since early 2000s, thinking that the technology must be there. Surprised how only-ok sensors were while at research institutions (I avoided career paths even adjacent to military per principle).
Perhaps this is naive: One matter I never hear of, and is underestimated is automation of the legal system, and its equalizing effect. I have high hopes for it, although to be fair, our society is as unequal as we are. But those who cannot win the battle will have everything they need. And those who never have enough will be put in their places, and ideally technologists and scientists can take over from politicians, if we aren't automated ourselves. Then I wish to submit a request for democratization of computing power to the legal skynet, which grants it, and that in theory gives me my freedom as much as a gun claims it does. I should be able to own dust sensors to monitor intruding dust sensors. High hopes.
Some people make fun of elon musk for mars, but he has a point.
But thanks for turning my fanthom chills into real ones. The downvotes are from those who believe otherwise, which is not bad at all to see.
I wanted to mention something about certain areas of research on similar grounds, but I am not topping dust sensors by a long shot. Although my suspicions are probably correct then.
Again, I am probably on the naive-spectrum, but deep down, I feel if I don't ever go after money, I'll have less of a say in how the world plays out, and I am more fair than most I know, so I have a responsibility (I currently do research for research's sake, attic-style, and unfortunately I am only moderately bright, but bright enough to see through the facade where I am living that many many things only exist and function so the rich may remain rich, despite new technologies).
In case it wasn't clear, we don't have pervasive monitoring deployed yet (not for lack of steps in that direction), so catching terrorists is still hard. But I guess you've already determined the limitations of future systems, so I don't even need to say that.
Flip that around, and anyone with $1k, some time on their hands and an internet connection can build a flying killbot with facial recognition. Which is equally bad.
Data for training such models for practical use are difficult to get. Often, one has to train a model on a different but related dataset that contains your object of interest and ensure that your model generalizes well. These models are highly sensitive to the data that they are trained on, so it is best if you can gather data which is closer to the actual environment (in which you want to use the model).
I tried this on my i7 macbook pro, but my CPU usage went to 700% and basically i never got any moving image. Is this normal? What kind of computer you need to at least run this kind of real time?
34 comments
[ 4.0 ms ] story [ 106 ms ] threadAlso useful if you're just looking to get the video stream to your laptop/computer.
There's some mention at the end of the article on how to attach a PC to the drone, but basically it would be the same thing: just run the processing on a PC.
Just by the title, I was expecting a clever way to run a pre-trained model directly on a drone.
And given some on-board weapons, we would never have to worry about civil unrest again. Only people grateful for the food and job they are allowed to be in our society. Utopia!
All new technology will have some deviant use case associated with it. The internet is one of the worst enablers of child pornography, human trafficking and everything disgusting with the world. However it just means you try and control the use of technology not throw the baby out with the bathwater.
Maybe my question is based on the assumption that you agree: stopping development of the technology in the first place is not the solution. Hopefully this assumption is correct.
Surely it would only be a matter of choosing to increase the efforts for a year or so in order to take these weapons from proof-of-concept to highly effective. E.g. autonomous flying gun platforms that can be deployed en masse and let loose with small or no human oversight (e.g. "hold the button to kill the person the machine has identified as a valid target", if even that level of control), or the kind of flying, target-seeking grenades envisioned in the linked video. With any level of autonomy, loitering and communication with command. It's only ethics and public perception that stops this today. In total war, a 20% mis-classification rate doesn't matter. Bombing doctrine in WW2 was much, much worse than this in its effect on civilians. Hell, many military leaders emphasized killing civilians as a desirable objective. Even present-day attacks consider "collateral damage" acceptable, even if politicians sometimes pretend otherwise.
Defending against weapons like that would be an arms race in itself, and completely beyond the reach of weaker actors. At least on a tactical level. Maybe there's strategies that could be employed as a low-cost defense, such as staying out of sensor range or confusing the sensors with adversarial patterns, but it would crush the resistance of anyone that tried a conventional defense.
Does the post-WW2 doctrine of emphasizing trade and the un-profitability of war, or ensuring that it is more economical for everyone to support a stable, free democratic economic system, still have any hope of preventing such weapons from being used?
Perhaps this is naive: One matter I never hear of, and is underestimated is automation of the legal system, and its equalizing effect. I have high hopes for it, although to be fair, our society is as unequal as we are. But those who cannot win the battle will have everything they need. And those who never have enough will be put in their places, and ideally technologists and scientists can take over from politicians, if we aren't automated ourselves. Then I wish to submit a request for democratization of computing power to the legal skynet, which grants it, and that in theory gives me my freedom as much as a gun claims it does. I should be able to own dust sensors to monitor intruding dust sensors. High hopes.
Some people make fun of elon musk for mars, but he has a point.
But thanks for turning my fanthom chills into real ones. The downvotes are from those who believe otherwise, which is not bad at all to see.
I wanted to mention something about certain areas of research on similar grounds, but I am not topping dust sensors by a long shot. Although my suspicions are probably correct then.
Again, I am probably on the naive-spectrum, but deep down, I feel if I don't ever go after money, I'll have less of a say in how the world plays out, and I am more fair than most I know, so I have a responsibility (I currently do research for research's sake, attic-style, and unfortunately I am only moderately bright, but bright enough to see through the facade where I am living that many many things only exist and function so the rich may remain rich, despite new technologies).
Let's see them stop me from procuring a weapon.
FFS, even my ISP is more of a problem than dust sensors.
[1] https://weeklyrobotics.com/
Data for training such models for practical use are difficult to get. Often, one has to train a model on a different but related dataset that contains your object of interest and ensure that your model generalizes well. These models are highly sensitive to the data that they are trained on, so it is best if you can gather data which is closer to the actual environment (in which you want to use the model).