I will be very curious to see how this makes it into production applications. It demos extremely well, but that doesn’t necessarily translate into something that’s production ready.
I must have missed it, but Lobe is owned by Microsoft. The product looks clean and well suited for CV 101 applications. Looks like a no-code meets AI solution. Anyone using it beyond research / personal project implementations?
I've been wanting for quite some time to build a device with a camera that could recognize my cat on the counter and turn on a servo that would release a jet of compressed air. It looks like I could actually use this for that.
Exactly. Minus the job and kids, I'd love to have the time to learn all the tools I need for all the dozens of projects in my head. But if this makes it easier, it greatly increases the chance of the project actually happening. (And heck, I still have to work out how I'll actually send the signal to the servo...)
omg please make this project haha I love this - we'll make it a project highlight on our twitter @lobe_ai! And be a hit on our subreddit https://reddit.com/r/lobe
I’ve been debating doing this too. I’m not too worried about the cat detection but haven’t got a clue how to programmatically release compressed air? Wouldn’t it be as easy just to play the sound of compressed air through a speaker? My dumb cat wouldn’t be able to tell the difference.
Use high quality relays purchased from Digikey or another distributor (should be $10-30 in single quantities for AC mains) and put thermal fuses on the mains side of the relay (I generally put several, each going to different parts that are at risk of shorting). Slather everything but the heatsinks (add some if necessary) with fire retardant epoxy, carefully pushing the thermal fuse into the epoxy until it makes contact and pushes out most of the epoxy under the fuse.
Burning down the house isn’t even an issue for me. USB relay throws 24VDC from a UL-approved power supply that opens the valve. The noise from the air compressor OTOH....
Check out the oss project [autofocus](https://github.com/uptake/autofocus) used by the Chicago zoo to operate camera traps. Could be useful for you too!
Add a random delay 0-60 sec delay for the jet air. This would create a fear and whenever she would sit there will think about jet of air coming any moment of time and now you can remove the machine after 1 month when your cat is trained.
Yup. I also seem to remember studies showing that if the reward/punishment is given randomly, it strengthens the Pavlovian response.
I plan (if I ever do this) to program a decay over time, starting at 100% chance/zero seconds, and moving to lower chance and higher random time interval.
Is there any recommendations for any other robust plant/insect/microbe identification ML solutions? I usually post in respective subreddits for identification, would think of a ML solution but never acted on it.
What's that old joke? Something like in the 1980s a Media Lab teacher gives the class a computer vision assignment where they're supposed to be able to tell whether or not an image contains a bird, and 40 years later they're still working on it? Lobe.ai reminds me of trying to identify plants with Google Goggles 10(ish?) years ago. It didn't work very well then, and then Google killed Goggles. Side note: none of the "click on the leaf feature" web-based plant identifiers gave a satisfactory answer either.
There are a number of new apps that take a crack at it. I've been using PlantSnap, and it seems to do OK. It doesn't always get it, and I wouldn't stake my life on it, but it's been good for assuaging idle curiosity.
Seek (the iOS) app works great -- I use it all the time on walks to identify plants (and even bugs and birds, if I can get close enough and they'll sit still). I think it's based on iNaturalist, which has a large community of professional botanists etc. collecting and classifying image data.
Google Lens (on the stock Android camera app) works pretty great for identifying plants to me. I'd say it's right about 95% of the time (in New England) if I can see a flower, and about 75% of the time for a leaf.
I've used Google Lens a little bit, to see if it could be used for object identification.
It always seemes assume that I'm trying to buy something and tries to find products me that are somehow visually related to what it's looking at, or else to the content of some lettering that it's able to detect.
Unsure. That maybe sounds like the Google Lens standalone app from a couple years ago. The Google Lens mode with the stock camera app never shows me anything to buy. ....actually, I just pointed it my rollerball pen, and it identified the pen correctly and then showed me a link to the pen on Alibaba. But I think that was just the image it was matching. It never does that for natural objects.
That explains it; it is the Google Lens app I used. Or does it?
However, if you go to the Google Store page for it, it does tout this feature:
"IDENTIFY PLANTS & ANIMALS Find out what that plant is in your friend's apartment, or what kind of dog you saw in the park."
I have the current version, which was updated on August 13, 2020.
I will give it another spin, and also look for the lens mode in the camera app. (My phone is from Google, so the camera is the Google one; if any Android camera app has a Google Lens mode, it should be that one.)
I was going to say, Lens works incredibly well. Working with a farm box of random vegetables that we couldn't identify, Lens had a 95% accuracy and instantly, too.
"Working" can be interpreted differently. 75% accuracy (?) is not that great for predicting a specific class (depending on distribution on course).
If you need high accuracy, let's say for e.g. estimating eco system performance based on specific plant distribution, 75% is very low (especially if you want to feed it into another predictor) compared to a professional field biologist.
We're really excited to bring machine learning to more people and make it more accessible. What we are most excited about is empowering people to create custom machine learning models for specific and personal use-cases and solve problems in new and unique ways
I think your comment is behind the state of the art. I bought a house with a beautiful garden in the backyard. I didn't know what any of this stuff was or how to take care of it. The app my wife got (not sure what it is) got all of them correct except 1, with a single photo. That's for at least 50 different plant species.
I recall that in the 18th century people tried to build a computer, so... What's your point? Computer vision didn't work back than, and it does work now, but not for everything. If you don't understand it you should go read about it, not posting pointless replies
Hey Markus from Lobe here :) all images and labels stay private to your computer, we don't ever see any of it. We only collect some generic app usage data for telemetry if you opt-in to sharing analytics after installing Lobe.
You can read our privacy policy, but more than that, you could use any sort of network traffic visualizer to see that we are not lying we are not in the business of selling data, we are in the business of making machine learning accessible to everyone.
Where is the app-specific privacy policy? the link the footer links to some general "Microsoft privacy policy", which covers all kinds of things and if it has anything specific to this app, it's impossible to find.
We do ping online to check if an app update is available! It checks the current app version and our hosted app version to see if we should show a notification that an update is available and a link to our website download.
We do not send any app analytics when it is turned off.
Seems to be marketed as 'machine learning', but upon closer look is only for machine learning on images. Anyone know of something similar for analysis of other kinds of data; particularly interested to analyze records (like spreadsheet data)?
Under project templates - coming soon they say it will work with tabular data.
I expect, as they allude to, it will begin with simple classification tasks in order to stick with the clean user experience they've built. But I'm super eager to see what they propose in this area.
I'm the founder, but you can use sysrev.com to review pdfs, json, text, etc. and assign labels / do annotations. You can see https://blog.sysrev.com/simple-ner/ for how to build something like a gene named entity recognizer in text. We have mechanical turk like compensation tools too, but you'll need to ping me (tom@insilica.co) for access.
There are other options for this too, I think spacy.io has an annotation app.
Yep we are starting with image classification for this initial beta launch, but plan to expand to more data types and problem types in future releases! The vision is to make a tool usable by anyone to build custom machine learning
We are working on adding more project templates in the future, and Lobe is designed upon the idea that machine learning should be made easy—no matter the problem type you are facing.
I really like how the website is done. Visually and content-wise. It transports the message pretty well into my brain.
Concise, not overloaded, good font sizes and looks good on mobile and desktop.
Maybe this is just on Firefox Mobile, but for me there is an image of some berries and no text shown for 0.5 seconds, which then gets replaced by a black rectangle, and then text appears that is borderline impossible to read over the black rectangle. The first thing I see is basically broken to the point of uselessness.
Edit: yep, on Chrome Mobile I actually see an animation and stuff seems to work. On Firefox it's borked.
Thanks for the feedback! We will be adding "Object Detection" next, which is identifying objects within an image and adding a bounding box around it. We are also exploring multi-label classification to classify an image as "dog" and "cat".
For Image Classification, that is a good approach is predicting if an image has both a cat and dog is important.
Thank you very much Aldipower! I am one of the designers from the Lobe team here speaking. This is what we strived for when we designed the website, and to be honest, when we designed the product, too, so I am really happy it came across this way.
In fact, our process for every feature we work on is the same—we start thinking about it with the way users are going to learn about it in mind, that allows us to simplify the way we talk about it and massage the messaging as much as possible, so when we have to talk about it externally, it's so tested that it just comes natural to us, and hopefully to the world.
We believe there are several advantages of Lobe over tools like Google AutoML :) Lobe is making the entire process of creating custom machine learning accessible, from creating your dataset to training and playing with your model, to integrating it into apps:
* Easy to use - no coding, cloud configuration or machine learning experience required.
* Free & private - train for free on your own computer without uploading your data to the cloud. No accounts required.
* Ship anywhere - available for both Mac and Windows. Export your model and ship it on any platform you choose.
AutoML requires paid accounts with high friction setup and is focused on just training a model on your data. You would have to pay and retrain your model manually every time you want to make an iteration. Lobe gives fluidity with iterating and providing feedback to your model through Play.
Thanks for the details! It's good to know this is an option for projects that require a custom image recognition solution. I have a feeling my company's clients will appreciate being able to train their data privately on their own computer.
Wondering this too. It seems to compete with some Azure AI services so my feeling is that either there's a gotcha, or it won't last long. Hopefully I'm wrong.
No gotcha! We're trying to fill a sweet spot for customers looking for a simple and quick way to get started with machine learning using their PCs or Macs without requiring any need for the cloud.
No catch! We are first and foremost trying to make this technology accessible to as many people as possible, and we want to grow an ecosystem around it. Business models around that can come later.
I wonder if they could do something with Proton to make a Linux desktop version. Since Valve made Proton for games I imagine that graphics cards support for things like CUDA might work just fine still.
All the Cuda stuff runs fine in Linux already, no wine required. (But possibly a weekend of tricksy driver setup... For which I blame nvidia, not anyone in the Linux side.)
One thing I thought of when I saw the demo video, that is probably on the team's radar:
There would be a lot of cool ways to improve the model by giving feedback, either showing training images where the model is uncertain, or some more advanced explanations for classifications flagged as incorrect, in order to guide the user to gather the training data that can improve it.
And possibly providing a summary of where it knows it works well.
There are a lot of benefits there, both for improving models people are building but also to help users understand why their model is performing as it does.
Thanks for your suggestions here. We are always looking at ways to improve Lobe, and the feedback loop of how to improve your model is one of the most important ones for us.
I'm on the Label page of the app and it's asking for five images, but I don't have any... could you please give a few example sets of images, e.g. drinking/not drinking, holding up # of fingers, etc, so I don't have to create the images myself?
Hey there! I'd recommend using your webcam to quickly add images of things on your desk or in your house :) to get a feel for the magic of machine learning. Press and hold on the shutter button to take a burst of images and move the object around to get variety. Then use the Play tab to try it out and see live results from the model!
Not yet! We are starting in beta with image classification. We are working on adding more project templates in the future, and Lobe is designed upon the idea that machine learning should be made easy — no matter the problem type you are facing.
i am really looking for an ai service which is able to detect signatures and threads out of email messages and extract the „real new message“ part - does anyone know some tool?
I don't think AI is needed here -- I wrote an Outlook plugin about 15 years ago that used fuzzy diff between messages in the thread that extracted only the new information added by each person and presented it as a message digest or as a labeled people digraph with msg-bubbles. For the life of me I don't understand why this isn't built in to all email clients -- the way gmail especially quotes replied emails, I need to spend time hunting up and down the message trying to recreate the timeline -- even getting the most recent bit seems harder than it should be.
Do you use Outlook? -- If there is interest, I can try and resurrect it. Although it's not as necessary as it once was -- not as many "Re: re: FW: re: fw: hello!" messages now that people use Slack and Teams, etc.
yes i am using outlook, would be awesome to see some approach here.
I thought of AI as there are so many different ways of mail thread formartings these days.
I played around with a machine learning demo and used a banana, apple and an orange for learning via webcam, and used speech synthesis to make it speak out load. After the accuracy was good I point the cam on my wife and it said: - 100% certainty a banana
hah! keep in mind the model will always make a prediction with one of the labels it is trained with for any image it is shown. You can add a "none" label and add images of things that are not banana, orange, apple, to learn the important features of a picture that make it a banana. if you are using your webcam, you can collect images of you, your office, backgrounds, etc
Do you ever think that it's a fundamental limitation of these systems that they aren't good at knowing what they don't know? Like they always give an answer, and their failure modes are so different to ours that that it can be hard for non-experts to interpret the outputs.
In some of the less harmless applications of computer vision and machine learning, sometimes it will have very severe consequences for real people that a computer says yes or no when it really doesn't have the information to say either or. Some people are afraid of what will happen to society when these systems become as accurate as humans - I am honestly more worried about what will happen if they don't.
How is the failure mode different from a human’s? The human mind comes with an answer no matter the situation it’s presented with, and that may be a stupid answer.
There is thankfully no (known) input that makes the human mind fail. There are known inputs for some animals though (like chickens).
Pretty inconvenient, with chickens apparently this sometimes happens by accident. They don't get up, ever. They lie there until they get attacked or just die.
Fascinating... That must be a "drop dead" self-defense mechanism triggered by what looks to it like its beak drawing a line in the dirt due to it being dragged away by a predator.
Though humans do often confabulate, they can also say, "I don't know", "what is that", "wait a minute", "there's something fishy about this", "huh?". Sometimes they can invent a new label or phrase on the spot that captures component attributes.
That would be a response, wouldn't it? If you give an AI "a way out", which is essentially the same, it will take a way out when it thinks it the smartest thing to do.
The trick is to lower the punishment for taking the way out. It's not free, but saying a car is a dog gets you -1, where as seeing a car and saying I don't know, only gets you a -.1 punishment (or even a .1 reward, vs a 1 reward for a correct answer).
Who's going around designing a system that will have severe consequences without mitigating the problem of misclassification? There are techniques available, such as autoencoders, sensor fusion, ensembles, using multiple images, training on "neither" examples, asking for human confirmation, etc. It might never be perfect but neither are humans. We see monsters in the dark, the virgin Mary on toast, a face on Mars, optical illusions, get our attention distracted (magic tricks), act maliciously, fall asleep, etc.
The weird failure modes thing already happened with lossy image compression. Characters in non-OCRed text go replaced with different ones by photocopiers, and people saw spaceships in space probe photos of the sun. We'll get used to the odd banana riding a motorbike and realize what's up.
The app is beautifully done. I'm really impressed by how well it works given the knobs available.
However I tried to train it to recognize some images of characters from an anime (so a little different than facial recognition), and I managed to break the model: achieving 64% error with significant number of examples per class. I think one downside is Lobe doesn't expose how potentially overconfident the model is. I would love the ability to take the existing model and test it on a new image that I can import into the app.
EDIT: I would love to see the following in a future version:
1. What are the percentages associated with each image per class. I see that an image was misclassified, but did it at least include my desired class in its top 5 predicted classes?
2. Test the model on unlabeled inputs directly in the app to see how well the model might generalize. I would like to see a "Test" tab on the left once training is complete.
3. View other metrics of model goodness like F-1 score and training details like CV partitions in the app somehow.
Hey there! Thank you so much for the feedback, we are planning improvements like this.
Here's a few tips for now:
1. You can view by "Test Images" on the Train tab (view options). So you can see how well your model is performing on your test images (a random 20% split from all of your images).
2. You can test your model on the Play tab, by dragging in new images your model has not seen, to see how well it is performing. You can also tell Lobe if it was correct or not and iteratively improve your model.
Hmm seems the title of this post was changed and I cannot edit it anymore, can someone change it to 'for training' or 'to train'. In its current form it's incorrect and sounds like some sort of locomotive AI.
This begs the question: why not ship it as an app itself targeted to normal users and let them custom fit it to their needs (unless this is already the case and I am missing something thinking it's targeted to engineers to build their apps with)?
this link here promises a react based web-app sample, but then it says "You need to get you setup so you can build, launch, and play with your app. These instructions are written for macOS, the only system you can develop iOS apps on." It then proceeds to provide instructions only for mac os. How come? Why does one need mac os to run a web application?
This seems pretty cool -- but one issue to me is that (similar to the chasm that exists in low-code app building once the magic doesn't suit you) that if I already have the skills to create a mobile app that integrates tensorflow, I probably also have the skills to train my models. It would be cool if feature-extraction (image pre-processing and first network layer(s)) could run on the front-end, and the rest of the network/search on the back-end, similar to how distributed speech recognition works. Then I could use a canned lib on the device that integrates w/the camera, and get my results via a websocket. (Of course, I could still run everything on the client still as well.)
159 comments
[ 3.4 ms ] story [ 231 ms ] threadHowever, they do have opensourced bootstrap apps here: https://github.com/lobe
https://gist.github.com/YashasSamaga/e2b19a6807a13046e399f4b... (download links for yolov4.weights is at https://github.com/AlexeyAB/darknet)
Using this, you will be able to detect if a cat is present in your image.
The cat invades the same counter space where we cook and do, well, everything.
There are probably a few kinks to work out that will come up in practice!
https://www.youtube.com/watch?v=QPgqfnKG_T4
I plan (if I ever do this) to program a decay over time, starting at 100% chance/zero seconds, and moving to lower chance and higher random time interval.
It always seemes assume that I'm trying to buy something and tries to find products me that are somehow visually related to what it's looking at, or else to the content of some lettering that it's able to detect.
So I gave up on it.
Are there some settings I can tweak?
However, if you go to the Google Store page for it, it does tout this feature:
"IDENTIFY PLANTS & ANIMALS Find out what that plant is in your friend's apartment, or what kind of dog you saw in the park."
I have the current version, which was updated on August 13, 2020.
I will give it another spin, and also look for the lens mode in the camera app. (My phone is from Google, so the camera is the Google one; if any Android camera app has a Google Lens mode, it should be that one.)
If you need high accuracy, let's say for e.g. estimating eco system performance based on specific plant distribution, 75% is very low (especially if you want to feed it into another predictor) compared to a professional field biologist.
https://dspace.mit.edu/handle/1721.1/6125
That's awesome they have the assignment for download.
I was wrong, Lobe has done a little bit of communication online. It was the lobe backend process I checked previously.
Downloaded 20.42kb and upload 5.78kb.
(Screenshot https://ibb.co/VLbSHQv)
I turned off sending crash info, analytics... in settings.
We do not send any app analytics when it is turned off.
Love the info site design.
I expect, as they allude to, it will begin with simple classification tasks in order to stick with the clean user experience they've built. But I'm super eager to see what they propose in this area.
There are other options for this too, I think spacy.io has an annotation app.
We specialize in tabular data and are building a pipeline-based approach for creating and serving models.
Edit: yep, on Chrome Mobile I actually see an animation and stuff seems to work. On Firefox it's borked.
In pictures where there are both dogs and cats, Lobe would say Dog or Cat instead of "Dog, cat."
I had to create a separate label called "dog and cat." I hope you're working to remove this extra step in future.
For Image Classification, that is a good approach is predicting if an image has both a cat and dog is important.
In fact, our process for every feature we work on is the same—we start thinking about it with the way users are going to learn about it in mind, that allows us to simplify the way we talk about it and massage the messaging as much as possible, so when we have to talk about it externally, it's so tested that it just comes natural to us, and hopefully to the world.
More info on AutoML: https://cloud.google.com/automl
* Easy to use - no coding, cloud configuration or machine learning experience required.
* Free & private - train for free on your own computer without uploading your data to the cloud. No accounts required.
* Ship anywhere - available for both Mac and Windows. Export your model and ship it on any platform you choose.
AutoML requires paid accounts with high friction setup and is focused on just training a model on your data. You would have to pay and retrain your model manually every time you want to make an iteration. Lobe gives fluidity with iterating and providing feedback to your model through Play.
There would be a lot of cool ways to improve the model by giving feedback, either showing training images where the model is uncertain, or some more advanced explanations for classifications flagged as incorrect, in order to guide the user to gather the training data that can improve it.
And possibly providing a summary of where it knows it works well.
There are a lot of benefits there, both for improving models people are building but also to help users understand why their model is performing as it does.
Do you use Outlook? -- If there is interest, I can try and resurrect it. Although it's not as necessary as it once was -- not as many "Re: re: FW: re: fw: hello!" messages now that people use Slack and Teams, etc.
In some of the less harmless applications of computer vision and machine learning, sometimes it will have very severe consequences for real people that a computer says yes or no when it really doesn't have the information to say either or. Some people are afraid of what will happen to society when these systems become as accurate as humans - I am honestly more worried about what will happen if they don't.
There is thankfully no (known) input that makes the human mind fail. There are known inputs for some animals though (like chickens).
https://www.youtube.com/watch?v=8Yo2UkL-n_Q
Pretty inconvenient, with chickens apparently this sometimes happens by accident. They don't get up, ever. They lie there until they get attacked or just die.
The trick is to lower the punishment for taking the way out. It's not free, but saying a car is a dog gets you -1, where as seeing a car and saying I don't know, only gets you a -.1 punishment (or even a .1 reward, vs a 1 reward for a correct answer).
>Optimist: AI has achieved human-level performance!
>Realist: “AI” is a collection of brittle hacks that, under very specific circumstances, mimic the surface appearance of intelligence.
>Pessimist: AI has achieved human-level performance.
The weird failure modes thing already happened with lossy image compression. Characters in non-OCRed text go replaced with different ones by photocopiers, and people saw spaceships in space probe photos of the sun. We'll get used to the odd banana riding a motorbike and realize what's up.
However I tried to train it to recognize some images of characters from an anime (so a little different than facial recognition), and I managed to break the model: achieving 64% error with significant number of examples per class. I think one downside is Lobe doesn't expose how potentially overconfident the model is. I would love the ability to take the existing model and test it on a new image that I can import into the app.
EDIT: I would love to see the following in a future version:
1. What are the percentages associated with each image per class. I see that an image was misclassified, but did it at least include my desired class in its top 5 predicted classes?
2. Test the model on unlabeled inputs directly in the app to see how well the model might generalize. I would like to see a "Test" tab on the left once training is complete.
3. View other metrics of model goodness like F-1 score and training details like CV partitions in the app somehow.
Again, this is a really cool idea :)
Check settings -> export -> local Api
Here's a few tips for now: 1. You can view by "Test Images" on the Train tab (view options). So you can see how well your model is performing on your test images (a random 20% split from all of your images). 2. You can test your model on the Play tab, by dragging in new images your model has not seen, to see how well it is performing. You can also tell Lobe if it was correct or not and iteratively improve your model.
Boo hoo, I'm running Linux on my desktop...
https://github.com/lobe/web-bootstrap