Great question! Let me rephrase so you can confirm my understanding: I have some labeling functions (LFs) that are far more accurate on a majority subset of the data than on one or more minority subsets or "slices" of…
Agreed! As noted in other answer, Snorkel certainly does not work for everything :) And indeed, in many cases it may be easier to express what you know extensionally (label examples) vs intensionally (write functions).…
More advanced structured prediction tasks are still definitely on the cutting edge- mainly IMO down to defining the semantics of the programmatic user input like labeling functions for these kinds of tasks. Some recent…
Yup! The LFs can also actually be correlated- just not too correlated (roughly, think of needing at least three mostly uncorrelated cliques, to be precise). Part of the high level description, though, is that a lot of…
Yeah definitely- and would love to chat sometime, as this is a space I've at least had less direct hands-on interaction with. There's a line of work in the ML literature on "Positive unlabeled (PU) learning"--basically,…
You can always do multi-label as a multi-task learning model (or just a set of binary models), which is something we (and many others) have explored before! A lot of the adjustments for mainline Snorkel have to do with…
Lurking for a few more min... great question! Dealing with both class imbalance and issues of pernicious biases in both underlying data distributions and training labels is an extremely important topic. Our underlying…
Single label has been the applicable one for most of the applications we've tackled to date, but agreed that multi-label is also very important! More coming here soon...
First: Snorkel Flow absolutely does not generalize to every ML problem :). IMO defining where different systems and approaches do and don't work best is one of the most important and most challenging problems in ML…
Thanks for this kind note! Would love to chat sometime and hear about your findings working with Snorkel- pros, cons, feature requests, etc. Multi-label is one on our list and under development (see comments below and…
Hi all, this is Alex from the Snorkel team- thanks for all the great comments! Excited to respond to a few questions directly, but first highlighting some up here: - Where to find more about the core Snorkel concepts:…
The grouping of theoretical, 'QED' style math with music and football is interesting to think about. I think football and math go together and can be seen as inverse processes wrt to music. Both football and theoretical…
To be annoying by tacking a long, overly thought-out comment onto yours (sorry): Agree with this root sentiment, for many other things that used to be done casually and amicably in real life and are now replaced by…
Sounds really cool. I like services that offer solutions that are easier along the dimension of horrible-awkward-interactions. I.e. I think that maybe the biggest appeal of this is not actual the time saved finding the…
Haha. Someone should do a web scrape of all parkour club directories, and then look at the intersection of that set with Boston Dynamics' team... could explain some things
I should caveat subsequent comments by saying that I would have given my childhood for this toy (and therein I would have learnt the meaning of irony too, but I digress). Anyway I am wondering what this could really be…
The patents are assigned to NASA (see for example: http://assignments.uspto.gov/assignments/q?db=pat&pat=75...). If NASA had been funding an outside contractor they would have had a non-exclusive non-transferable…
Cool article! I think a broader question raised is one I have thought about a lot: what is the optimum trade-off between narrow clarity of function/purpose and broad ambition/expansiveness/spread of features? On the one…
What I don't quite understand about this piece is the motivation for saying that the biggest value-add would be to 'kindly inform the person that the correct term is 'co-founder'" (I am purposely trying to keep this…
Great question! Let me rephrase so you can confirm my understanding: I have some labeling functions (LFs) that are far more accurate on a majority subset of the data than on one or more minority subsets or "slices" of…
Agreed! As noted in other answer, Snorkel certainly does not work for everything :) And indeed, in many cases it may be easier to express what you know extensionally (label examples) vs intensionally (write functions).…
More advanced structured prediction tasks are still definitely on the cutting edge- mainly IMO down to defining the semantics of the programmatic user input like labeling functions for these kinds of tasks. Some recent…
Yup! The LFs can also actually be correlated- just not too correlated (roughly, think of needing at least three mostly uncorrelated cliques, to be precise). Part of the high level description, though, is that a lot of…
Yeah definitely- and would love to chat sometime, as this is a space I've at least had less direct hands-on interaction with. There's a line of work in the ML literature on "Positive unlabeled (PU) learning"--basically,…
You can always do multi-label as a multi-task learning model (or just a set of binary models), which is something we (and many others) have explored before! A lot of the adjustments for mainline Snorkel have to do with…
Lurking for a few more min... great question! Dealing with both class imbalance and issues of pernicious biases in both underlying data distributions and training labels is an extremely important topic. Our underlying…
Single label has been the applicable one for most of the applications we've tackled to date, but agreed that multi-label is also very important! More coming here soon...
First: Snorkel Flow absolutely does not generalize to every ML problem :). IMO defining where different systems and approaches do and don't work best is one of the most important and most challenging problems in ML…
Thanks for this kind note! Would love to chat sometime and hear about your findings working with Snorkel- pros, cons, feature requests, etc. Multi-label is one on our list and under development (see comments below and…
Hi all, this is Alex from the Snorkel team- thanks for all the great comments! Excited to respond to a few questions directly, but first highlighting some up here: - Where to find more about the core Snorkel concepts:…
The grouping of theoretical, 'QED' style math with music and football is interesting to think about. I think football and math go together and can be seen as inverse processes wrt to music. Both football and theoretical…
To be annoying by tacking a long, overly thought-out comment onto yours (sorry): Agree with this root sentiment, for many other things that used to be done casually and amicably in real life and are now replaced by…
Sounds really cool. I like services that offer solutions that are easier along the dimension of horrible-awkward-interactions. I.e. I think that maybe the biggest appeal of this is not actual the time saved finding the…
Haha. Someone should do a web scrape of all parkour club directories, and then look at the intersection of that set with Boston Dynamics' team... could explain some things
I should caveat subsequent comments by saying that I would have given my childhood for this toy (and therein I would have learnt the meaning of irony too, but I digress). Anyway I am wondering what this could really be…
The patents are assigned to NASA (see for example: http://assignments.uspto.gov/assignments/q?db=pat&pat=75...). If NASA had been funding an outside contractor they would have had a non-exclusive non-transferable…
Cool article! I think a broader question raised is one I have thought about a lot: what is the optimum trade-off between narrow clarity of function/purpose and broad ambition/expansiveness/spread of features? On the one…
What I don't quite understand about this piece is the motivation for saying that the biggest value-add would be to 'kindly inform the person that the correct term is 'co-founder'" (I am purposely trying to keep this…