Launch HN: Sterblue (YC S18) – Software for drones to inspect power lines
I am Vincent of Sterblue (https://www.sterblue.com). We build software for drones to inspect power lines and wind turbines automatically.
We are three drone enthusiasts. I met Nicolas in 2006 during our Aerospace Engineering studies at ISAE, in France. Small drones were still in their infancy and we eagerly enrolled in the brand-new student micro-drone club. Nicolas later met Geoffrey when working in India for a large aerospace company, while I was working on my PhD in Computer Science.
In 2015, we shared a feeling of disappointment that the vision people had for drones back in 2006 had not fully happened yet. For example, utility companies need to inspect millions of power line pylons. They still mostly perform inspections using helicopters or rope access technicians. They sometimes use drones for short distance inspections, but they could not scale it up to inspect all of their grid.
We decided to try to make drone inspections happen in the renewable energy industry.
In the beginning, we were super optimistic: we started by designing and patenting actual drone hardware configurations, building and testing drone prototypes. Then we realised that hardware is really hard! and worse, that new drones are not what this industry needs right now. Even with super high-performance drones, you still need a top-gun-level pilot to capture precise photos 8 hours a day from a drone flying high in the wind. And then you need to spend ages trying to find small anomalies on thousands of captured images.
So our approach became to automate most of the inspection process, using off-the-shelf DJI drones. We built two core technologies in the last 2 years to try to get there.
The first one is a mission planning and execution engine based on ideas developed during my PhD. It allows drones to fly on complex 3D trajectories that wrap tightly around structures, flying the drone sometimes less than 3m away from the objects to inspect. Most other drones companies fly high above the objects to inspect, so their apps show trajectories on 2D maps. Our app shows trajectories on 3D views because our trajectories are intricated closely around objects. As an example, we had to take into account the fact that wind turbine blades bend under their own weight. Here is an image of power grid inspection with our software: https://imgur.com/KRE1Oce
The second is a deep learning framework built on top of Tensorflow, PyTorch and Caffe2. It allows to detect 130 classes of defects on images: corrosion, broken equipment, and other safety hazards. The typical use case for drones today is photogrammetry, in order to create 3D models from captured images. This is not super useful for wind turbine and power line inspection, so we focus on our thing: finding defects directly on images, just like human operators do now. For several classes of common defects, our software has 98% recall and 80% precision. We run up to 18 different neural networks on some images in order to find defects on all kinds of equipment. Here is an image of our algorithm detecting an anomaly on a wind turbine: https://imgur.com/iGIxtAZ
These two technologies are linked to a GraphQL API and cloud platform. Our customers get to visualize inspections results on PDF reports and 3D interfaces online. Our UIs are made using React and React Native. Here is an image of an HTML report for distribution grid inspection: https://imgur.com/dkznc6o
In the last 6 months we had successful pilot projects with utilities in 7 different countries, and some have even started to use our software in production mode now. This is exciting for us. But we still have a lot to learn, and we’d like to hear your feedback and ideas in this space!
45 comments
[ 3.1 ms ] story [ 104 ms ] threadWe also did some amount of training on our own GPU machines on premises.
But we recently started to work with our YC batchmates of https://snark.ai , which enables us to speed up training by A LOT
Good luck, there's a lot of cash up for the taking if you're able to automate the process.
We capture images of all structures exhaustively, which means that the analysis is done offline, not in a rush. It also allow to scroll back in time to see if a newly found anomaly was already present during a previous inspection.
Are you doing anything special for 3D trajectory estimation (any visual tracking / SLAM)? Or is it just GPS or whatever DJI puts in?
Also, would it help with defect detection and classification if you had LiDAR data as well?
Some of our customers provide us with LIDAR data, which enhances both navigation and accuracy of report generation.
Question: do you have any plans to offer self-hosted solutions? I imagine that while many companies will be happy to let you guys do pilot projects on some lines, once you start talking about "let's put the whole national grid in" then national security concerns start popping up. Power distribution grids are highly critical infrastructure. Probably wise to pre-empt any troubles on that front; I'd guess a wise move would be partnering with a trusted entity in each country where you want to operate, letting them run it as a "local, trusted SaaS".
- We host data on AWS on data centers in appropriate regions for each customer (read: in their own country, or a nearby one). AWS compliance programs also help a lot in building credibility: https://aws.amazon.com/fr/compliance/programs/
- We use solutions such as https://www.sqreen.io/ to add additional layers of security on our infrastructure
- Having data on our side allows to retrain neural networks more effectively, which gives them great value that would be more difficult to get with on-premise deployments. We are looking into edge computing and training however.
- So far we perform visual inspection of publicly visible assets. We do not host any data that would not be visible to anyone looking up to overhead lines in the countryside. This might also change in the future, and the amount of data we gather is also a concern, even if it is public.
In the end, we take this topic very seriously, and if you have any additional info on easy ways to deploy with trusted entities in various geographical areas, I would be happy to hear your experience about that!
For Europe, you probably want to follow closely the new “EU DSO Entity” which is to be formed following the new Electricity Regulation, it's an entity intended to enhance cooperation between Distribution System Operators. One doesn't know if it will be just a paper tiger, though.
The electric utility world can be a challenging industry due to the reluctance of companies to take risks on new technologies.
Another challenge can be the somewhat antiquated data delivery policies. When I was working in the space around 2012 our vegetation violation deliverables were required to be sent via Excel spreadsheets, then followed up using PDFs, and GIS Shapefile formats.
I'm glad somebody is trying to push a more modern workflow into the industry to make the process more efficient.
Good luck on your journey.
But we noticed that utilities are now getting more and more challenged on different fronts, so many of them are opening to new technologies as a way to improve their operational efficiency.
The younger employees were always excited for us to work with them to make their work easier. Many advocates within the companies had excellent suggestions which were unfortunately sidelined by their managers.
I think I would not apply at a company where there's no english version available. I don't think i'd be welcome there as a non-Frenchman.
In general, why is there still such a poor internationalization in everything in france? From universities, to jobs in startups (!) etc. I recently tried searching for a master in france and it was super difficult.
It's not that hard. lol. Half of advanced math was done in France anyway...Laplace, logarithmes, Cauchy, Riemann...
"analyser les résultats des algorithmes de manière quantitative" => analyze the results of algorithms in a quantitative manner
I certainly wouldn't hire you, after reading that.
Looks like you're insecure and looking to be offended.
I study near the french-border and, in my experience, relying on english in france is hard and even the universities are often not really easy to navigate without speaking french.
I can't learn the language immediatly and so I think starting at the startup will be hard or impossible for me if everything they'r doing is in french. That's what I mean with "I don't feel welcome".
Essentially, you're implying the French are chauvinists that don't care about "non-frenchmen", because they write in French. You are also implicitly accusing these specific French people of being chauvinists because they wrote the jobs section in their native language.
This is completely unwarranted. As they've commented they didn't just have the time to translate the section.
Furthermore, I would add, that seemed obvious from the start to me and it's the charitable, or generous, interpretation anyway.
Therefore, you strike me as someone who looks for reasons to be aggravated. So I wouldn't hire you.
I just want to stress that didn't want to accuse anyone of anything. I just stated the impression the job-section gave to me and think it could help them assuming my experience is not unique.
seems like we have a mutual understanding of not wanting to hire each other
- We tune the algorithms to give very high recall (up to 98%), at the cost of lower precision (down to 70% in certain cases). This means that we miss very few defects, but with sometimes 30% false positives.
- We capture pictures of each point of the structure to inspect from at least 2 or 3 viewpoints. This means that the 98% recall for each image compounds: if we missed a defect on a picture, we have chances of finding it on other pictures from other angles.
- All our software allows humans in the loop, from data capture to data analysis. This means human experts can validate or correct results from the AI, human drones operators can take back manual control of the drone, and so on. Our automation is here to help humans perform their job, not fully replace them.
- Our customers worry about that a lot, and most of them aknowledge that our method is more effective than traditional methods
Can hi-res cameras mitigate the need for close-proximity fly-bys?
Are there any companies doing home security drones? Like circling the property routinely, auto docking to recharge, reporting human thermals around the perimeter etc. Maybe even neighbourhood-watch, crime-fighting drone fleets in the near future
So much potential, can’t wait for it to unleash.
There are lots of companies in the drone security landscape. This is not our market as we are not too much sold to the idea of drones tracking people and potentially using weapons... We are more than happy with our mission of helping the development of clean energies and a more energy-efficient future.
However I know a stealth company that uses drones to assist firefighters, their solution is impressive and saves lives!
- Factory plant inspections is on our roadmap. We have technical proof of concept on this, especially on generating flight plans for arbitrary complex 3D structures, and detecting defects on images. It will be more of a commercial effort than a technical effort for us.
- Pipeline inspection is an interesting application, a bit less in our scope right now because there is not so much of a "3D" aspect to the flight, it is mostly linear assets, which can be dealt with existing mission planning software and drones. But we see where we could add value to this, and will do this if we have more time and resources one day!
If a customer had a structure without a preexisting 3d model how would you create the flight plan to operate close enough for your defect detector to work? Do you have issues with moving structures/obstacles (e.g. a wind turbine rotor offset from its orientation in the model)?
Full Disclosure: I work on the SDK Platform for Skydio R1 (a fully autonomous drone doing real-time visual SLAM and motion planning). If you're interested in chatting more I'm at roshan [at] skydio.com
We have been looking at Skydio for some time, your product is amazing! We'll get in touch
As a deaf person, the way the auto-generated captions are shown is so much harder to read than just your normal sync'd captions.
That said, when you take care, there are so many ways that this solution is better than lines people in trucks inspecting the infrastructure!
However we never had a crash when inspecting power lines! Here is a video of power line inspection using only satellite imagery as input, without giant electric arc!
https://www.linkedin.com/feed/update/urn:li:activity:6441158...
Looking forward to see how your project evolves!