1) Is a Non-Compete Agreement required and what are the terms (time and geographic coverage; compensation during the non-compete time frame)?
2) Is the company going to assert ownership over IP created before employment began?
3) Is the company going to assert ownership over IP created that is unrelated to current responsibilities (or IP in a area unrelated to the company's current business), especially IP created on the employee's own time, not using any company resources?
These are great questions to ask any company, not just AI/ML companies. Also, if you're planning to do it:
4) What is the companies policy on employees moonlighting / doing side contract work for other companies not related to it's business?
Inquire about this before you get your offer. Ask me how I know...
EDIT: Another one, which is kind of risky but I personally think is important and would ask any small company:
5) How transparent/open are you about the company's financial health? Are the books open or semi-open?
Keeping in mind that some companies will not share this information and may even find the question offensive. One time I asked the owner/founder this, and he would not share--I found out later through a side channel that he was very offended that I asked for this information (but hired me anyway because he needed someone like me).
"Why does anyone need this? Like all advice, this sounds deceptively simple. But make sure you get a very compelling answer here."
Right now I'm at the Web Summit, in Lisbon. I saw a few ML/AI startups here, but I was surprised there weren't a lot more. There was some imbalance. There were many startups trying to create new online social networks for niche groups -- business professionals, women, entrepreneurs, sales professionals, etc. I do think there is some innovation that can happen in social networks, but generally I regard it as an over-crowded space. For the most part, the benefits offered by the Internet seemed to have been mostly absorbed by the 1994-2008 cycle, and what's left is fairly minor compared to what happened previously.
One of the few areas where I still see the possibility of significant traction (the creation of large companies) is with information gathering and a combination of ML and NLP. We've already seen a wave of startups which were no more than pretty GUIs over existing technology. At the risk of being unfair, BigML.com is just a nice interface over existing ML tools, and API.ai is just a nice interface over some NLP tools. AmenityAnalytics goes a bit further combining their web scraping and NLP scripts with a nice interface that customers can use to filter the incoming data. But there is still a wide space for companies to go much further in this field.
All the same, I agree this is a good question: "Why does anyone need this?" You should really ask it whenever you are joining any startup.
Shouldn't the interviewing engineer know the answers to these business questions beforehand during their research of the company? (particularly if the company is upfront about their ML/AI-usage)
In my experience the answers I find on the company home page and the answer I get when asking the question in an interview only rarely sync up perfectly and the difference can be very illuminating.
Agreed. There are many flavors of moats. I originally wrote something more complex but it quickly turned into an amateur HBS study about Google. I hope the point is interpreted more metaphorically.
I've heard the line that Google succeeds because of network effects so many times on HN that I feel like that's the only business concept commenters know about.
I would suggest that in many cases the "data network effect" advantage of ML startups is overappreciated relative to the branding advantage they have. For example, DDG and others can, for the most part, match Google in quality of search, but almost no one will use them because of entrenched branding, this is especially true IMO in smaller ML vertical industries ideal for startups. e.g., if there is only room for one ML company that analyzes farm pipe leaks through camera, then branding and market dominance helps considerably there.
I may be wrong, I'm (mostly) a DL hardware guy, this is just my two cents :)
Google will I'm sure vehemently deny that it is unbeatable but I think with 60k+ searches every second, Google will keep getting better. I think Google is unbeatable in search as we know it.
I'm not counting out google, but there are only two searches that actually count from a business perspective. product and services. All the rest are cost of doing business.
If you do your research on google and then buy from amazon or somewhere else and don't use the adword link google's business strategy fails.
And to add to that - I don't think when Google was launched, anybody thought about AdWords. This statement is technically true:
> How do you make money? Be on the lookout for what I call “multistage rockets”: “Today, we’re doing X. But our grand plan is to do Y, which will be really profitable”. These usually fail.
But, it is also completely non-informative. All startups usually fail. All (completely) new endeavors, even in established companies, usually fail. That does NOT mean though that it's not worth a try. You cannot generally anticipate apriori what will work and what not - and you CERTAINLY cannot anticipate it from the business plan (what was the business plan of Facebook? Instagram?). If you stand any chance to anticipate "what will work" - it's from the team, not the business plan.
One of the primary factors that enabled Google to stay great was their ability to crawl and index the ever growing web at a rate no other company could.
Should the engineers ask the same question when a high profile VC, like YC have already invested on the company? The questions look more like what a VC should (have) asked before funding.
Many ML/AI companies especially if funded and therefore highly visible may get acquired as the market is very hot. Isn't that good for the engineers?
> Many ML/AI companies especially if funded and therefore highly visible may get acquired as the market is very hot. Isn't that good for the engineers?
Only if the potential acquirers are happy with the answers to the same questions. As a percentage of net worth, individual engineers have a lot more at stake than VC's or potential acquirers.
Most of the big companies(apple/google/microsoft/fb/amazon) at current stage are acquiring for talent/IP, not necessarily market fit.. they do have enough money to do the R&D without worrying about finding a market fit soon
If you're a friend of the VC and he suggests you work for a portfolio company, then maybe you can rely on the VC's judgment without asking questions.
General it's always legitimate to ask these questions. Why? Because they cut straight to how the company will be valued, by a customer or by a potential acquirer.
Or, you can't figure out how AI is helping solve the problem they claim to solve, and you realize they are just throwing every buzzword out there in hopes you'll be impressed.
Holy shit. That is amazing. I watched two minutes of video and I now know less than I did when I started. It took me a disturbingly long period of time to decide whether or not it was a parody.
I decoded it from the first 60 seconds. It's a courseware platform for internal use in companies, presented as white label, the company can pass it as their own product.
Generally, their advantage is in the way they use data to solve a vertical specific problem. They more "bolt on" AI.
AI isn't really the focus of most of these companies. This is just what investors want to hear about.
The article here isn't calling them out specifically, but AI infra startups are what are being discussed here as the "hammer looking for a nail" startup.
YC funded a few of them including Skymind (my company) and deepgram (mainly audio). There are a lot more of these startups such as vicarious that mainly publish papers but raise tons of funding hoping for a deepmind style exit (they didn't really have a business model).
With that context out of the way, I'll also maybe just state some learnings from the other side of the table.
Skymind has typically made money trying to reduce dependencies on cloud providers by decoupling the AI infra from google cloud and co from the cloud provider itself.
A horizontal play like this is by its nature very hard. We maintain the whole stack including our own framework. We also started in 2014 and have a decent foothold in enterprise. I wouldn't recommend trying to do this in 2017.
That being said, one other semi similar horizontal AI startup that is being started are the chip companies. Those are significantly harder to run than even what we're doing.
The most common "failed" type of startup that I think of when we think "horizontal" plays are Machine Learning as a service, which is hugely a loss leader for the various cloud providers (some companies are building on top of these providers though).
This is where you see Metamind, Alchemy API, Scaled Inference (founded in 2015), even Nervana before they sold were trying to a "nervana asic cloud" among others.
Maybe a lot of investors from SV view this as a waste of time dissecting, but I would personally love to see a bit more content on acknowledging some of these trends in the market.
The allure of "AI as a service" and the horizontal dev tools infra play is that you can try to build the next AWS similar to what the container and database companies are trying to do. Execution is definitely key for this to work though. Research also can't be the primary focus.
The fastest way to make progress on these business questions is often to build hacky MVPs that look like they're doing something smart, but behind the scenes are powered by humans or dead-simple algorithms, and get them in front of customers ASAP.
I recently joined a seed-stage startup solving a business problem via audio analysis in the manner I described above. I'm not spending much time doing ML yet, but I'm banking on my belief that we're solving a valuable problem (customers want to buy our hacky MVP) and that ML can and will be needed to scale our solution. By deeply understanding the customer as a first step, I think the ML systems we build will be business critical and enduring. Time will tell
The most successful models I've ever built have been logistic regression models. If you can rephrase your problem in a way that's amenable to run-of-the-mill statistical techniques, you can frequently achieve much better results than you can with 'deep learning'.
I had a pretty good regression model but it was not taken seriously. So I wrote it using "a neural network in TensorFlow" and the next thing you know the whole company is asking me how it works and what it does.
This feels so dirty, but these kinds of tricks work. Your stakeholders get to participate in the titillating fiction that they are on the bleeding edge of technology, and you get to deploy a scalable, explainable, and (hopefully) high-performing solution. That's 95% of a win.
Amen.. I like to use exotic techniques just out of intellectual curiosity and to put my education to use, but ultimately a basic linear regression is all people want! Ease of interpretation is paramount.
Question 6. How often do people talk about ARPUs with potential hires? I am sure people will tell you that there are 150 million potential users but not their revenue models. It is easy to under shoot or overshoot ARPU.
This is really solid advice when looking at it backwards too--if you, as a founder, cannot produce reasonable and compelling answers to these questions, you don't know what you're doing, where you're going, or how to actually get there.
Unfortunately, sometimes the hype is so strong, the cart comes before the horse and gets acquired for quite a lot - which is why these companies exist.
The best use of these questions is to get an idea whether your equity might be worth anything. Ask how many customers they have, how fast they're growing, how long does it take to onboard customers, etc. What do they see as an exit (is it a 10M, 100M, 500M a unicorn?). When do they see it happening?
If, let's say, you have 0.01% of a Series A that exits at a billion in five years. You'd expect to be diluted by half and there is around a 20% chance of exit that high so roughly ((0.01%billion)/2).20 /5 or about 2k a year in 'expected value'(or why, numerically at least, it's likely not worth it to take a paycut to work at a startup unless you're seeing substantial equity).
Edit: The reason anyone would wax poetically about everything else is because he or she realizes that they are not being paid enough. It is like day trading - the moment it goes against a bad trader he or she calls it "investment"
Edit: Thanks for the down votes! It makes me feel warm and fuzzy inside to know that it is still possible to swindle people!
Nah. This is a good list of questions. Partly because startups will include equity, and you need to know these things (and more) to make a guess at valuing the equity. And partly because a company without strong answers is likely to be out of business soon; changing jobs unexpectedly can be expensive, especially if you get stiffed on wages.
But mostly because it really sucks to spend a couple of years building stuff that never gets used. There are a few people who seem happy to get paid even if nothing is achieved. But most engineers I know do the work because the like building things that get used, things that are valuable.
Unless you are employee #1-5, the correct approach to valuing equity is to not consider it to be worth anything. For employees 1-5, divide the numbers the CEO gives you by at least 10 to account for risk, and then demand a competitive base pay.
Yes and no. I agree should expect zero to come out of it, as it's a pretty low probability you'll see anything. But you still have to do the work of valuing it.
First off, the people you're negotiating with see it as valuable, so it's a very bad negotiating move to say, "Keep your dumb equity and just give me cash." You'd be slightly better off just pointing to their family photo and saying, "Wow, your kids are ugly." Second, going through the work of valuing equity is necessary to see how the investors (that is, they people actually paying your salary) look at the company. Third, on the off chance you are successful, you're going to really wish you took the equity part of the negotiation seriously.
People you’re negotiating with want you to buy their line that their equity is worth something so they could pay you less. What would you get as eg employee #10? Half a percent or less. Most startups exit under $100M, and your 0.5% turns into 0.25% due to dilution. It’s also vested over 4 years. We’re talking $250k max over 4 years, with a 5% chance you’ll get anything at all. You’d have to be seriously arithmetically challenged to take that seriously.
People are arithmetically challenged. People also like to boast about doing good etc. Typically they realize how stupidly naive they were just around the time they no longer have ability to say "That's nice to know. So how much are you going to pay me?"
The down votes on calling spade a spade demonstrate it quite well
Sure, but the 'correct' answer to that question depends on a lot of other factors related to the job. If you want me to work long hours, doing dull, morally dubious, work in a high stress environment you're going to have to pay me a lot more.
You are looking at the other factors to justify the smaller pay than you know you are get elsewhere. Should you be telling people about how much money you make you would be using a word "but" followed by the list of these factors.
You are looking at the other factors to justify the smaller pay than you know you are get elsewhere
Absolutely. Money is just one of many factors I consider when choosing a job. I see no 'shame' in that. I honestly have/earn enough money to cover all my expenses, so getting a bit more isn't that super important to me. As long as you meet my floor I'm willing to negotiate away money for other perks. I'd rather have time to spend my money rather than just more money.
Typical of SV to focus solely on the business viability of a firm to the detriment of everything else. I’d like to work somewhere knowing that I have a clean conscience. Why not add:
Where do you get your data from?
If your data is sourced from users of your product, do you tell them what you’re collecting?
Exactly. Informing the user does nothing except maybe clear your conscience. Expecting the user to extrapolate to how you will use that data, what will be inferred from it, what risks come with centralizing and storing it, all of that is not the users job. It’s the job of the collector to be as conservative as possible. Any company that treats users privacy as a resource to be extracted is not somewhere I want to work.
Huh? There are dozens, if not hundreds, of things one should consider when contemplating taking a job. That an article focuses on things on one particular category (business viability) does not imply that the author thinks that things from other categories (such as ethical considerations) are less important.
After working for a few companies who (tried to) make ML products, I think one of the most important questions to ask is "who owns the data you are building models on?". It is way harder for a company to build good models off data they don't have complete access to and full knowledge about. The worst (and unfortunately common) scenario for companies trying to do AI is that data scientists don't have full access to all priors necessary to build good models, and the owners of the data don't really know much about it either. Spells death of company
Well said. It is a good tester of knowing how clear the people understand the problem themselves. Many people (particularly non-science background ones, no offence) don't know which kind of data they should access to build a product and, even worse, whether they have access to the data.
Not knowing the answer of this question = they do not know they are travelling on Titanic at best, if not a boat full of holes.
And here I was thinking these would be questions about the moral / ethical / social implications of the ways in which many companies use these technologies..
114 comments
[ 3.7 ms ] story [ 186 ms ] thread1) Is a Non-Compete Agreement required and what are the terms (time and geographic coverage; compensation during the non-compete time frame)?
2) Is the company going to assert ownership over IP created before employment began?
3) Is the company going to assert ownership over IP created that is unrelated to current responsibilities (or IP in a area unrelated to the company's current business), especially IP created on the employee's own time, not using any company resources?
4) What is the companies policy on employees moonlighting / doing side contract work for other companies not related to it's business?
Inquire about this before you get your offer. Ask me how I know...
EDIT: Another one, which is kind of risky but I personally think is important and would ask any small company:
5) How transparent/open are you about the company's financial health? Are the books open or semi-open?
Keeping in mind that some companies will not share this information and may even find the question offensive. One time I asked the owner/founder this, and he would not share--I found out later through a side channel that he was very offended that I asked for this information (but hired me anyway because he needed someone like me).
Right now I'm at the Web Summit, in Lisbon. I saw a few ML/AI startups here, but I was surprised there weren't a lot more. There was some imbalance. There were many startups trying to create new online social networks for niche groups -- business professionals, women, entrepreneurs, sales professionals, etc. I do think there is some innovation that can happen in social networks, but generally I regard it as an over-crowded space. For the most part, the benefits offered by the Internet seemed to have been mostly absorbed by the 1994-2008 cycle, and what's left is fairly minor compared to what happened previously.
One of the few areas where I still see the possibility of significant traction (the creation of large companies) is with information gathering and a combination of ML and NLP. We've already seen a wave of startups which were no more than pretty GUIs over existing technology. At the risk of being unfair, BigML.com is just a nice interface over existing ML tools, and API.ai is just a nice interface over some NLP tools. AmenityAnalytics goes a bit further combining their web scraping and NLP scripts with a nice interface that customers can use to filter the incoming data. But there is still a wide space for companies to go much further in this field.
All the same, I agree this is a good question: "Why does anyone need this?" You should really ask it whenever you are joining any startup.
Groan
I think this overstates network effects and under-appreciates branding and simply reinvesting significant capital into continued R&D.
I may be wrong, I'm (mostly) a DL hardware guy, this is just my two cents :)
Even for the long tail? I'm pretty skeptical of this.
If you do your research on google and then buy from amazon or somewhere else and don't use the adword link google's business strategy fails.
Google got great because of PageRank, but it stayed great due to AdWords.
Google got great because of PageRank, but it stayed viable due to AdWords.
Fixed that for ya
> How do you make money? Be on the lookout for what I call “multistage rockets”: “Today, we’re doing X. But our grand plan is to do Y, which will be really profitable”. These usually fail.
But, it is also completely non-informative. All startups usually fail. All (completely) new endeavors, even in established companies, usually fail. That does NOT mean though that it's not worth a try. You cannot generally anticipate apriori what will work and what not - and you CERTAINLY cannot anticipate it from the business plan (what was the business plan of Facebook? Instagram?). If you stand any chance to anticipate "what will work" - it's from the team, not the business plan.
Many ML/AI companies especially if funded and therefore highly visible may get acquired as the market is very hot. Isn't that good for the engineers?
Only if the potential acquirers are happy with the answers to the same questions. As a percentage of net worth, individual engineers have a lot more at stake than VC's or potential acquirers.
General it's always legitimate to ask these questions. Why? Because they cut straight to how the company will be valued, by a customer or by a potential acquirer.
https://www.youtube.com/watch?v=kzT3yfe2o-I
https://www.edcast.com
This product is certified buzzword compliant.
Generally, their advantage is in the way they use data to solve a vertical specific problem. They more "bolt on" AI. AI isn't really the focus of most of these companies. This is just what investors want to hear about.
The article here isn't calling them out specifically, but AI infra startups are what are being discussed here as the "hammer looking for a nail" startup.
YC funded a few of them including Skymind (my company) and deepgram (mainly audio). There are a lot more of these startups such as vicarious that mainly publish papers but raise tons of funding hoping for a deepmind style exit (they didn't really have a business model).
With that context out of the way, I'll also maybe just state some learnings from the other side of the table.
Skymind has typically made money trying to reduce dependencies on cloud providers by decoupling the AI infra from google cloud and co from the cloud provider itself.
A horizontal play like this is by its nature very hard. We maintain the whole stack including our own framework. We also started in 2014 and have a decent foothold in enterprise. I wouldn't recommend trying to do this in 2017.
That being said, one other semi similar horizontal AI startup that is being started are the chip companies. Those are significantly harder to run than even what we're doing.
The most common "failed" type of startup that I think of when we think "horizontal" plays are Machine Learning as a service, which is hugely a loss leader for the various cloud providers (some companies are building on top of these providers though).
This is where you see Metamind, Alchemy API, Scaled Inference (founded in 2015), even Nervana before they sold were trying to a "nervana asic cloud" among others.
Maybe a lot of investors from SV view this as a waste of time dissecting, but I would personally love to see a bit more content on acknowledging some of these trends in the market.
The allure of "AI as a service" and the horizontal dev tools infra play is that you can try to build the next AWS similar to what the container and database companies are trying to do. Execution is definitely key for this to work though. Research also can't be the primary focus.
I won't comment on what will or won't work there
I recently joined a seed-stage startup solving a business problem via audio analysis in the manner I described above. I'm not spending much time doing ML yet, but I'm banking on my belief that we're solving a valuable problem (customers want to buy our hacky MVP) and that ML can and will be needed to scale our solution. By deeply understanding the customer as a first step, I think the ML systems we build will be business critical and enduring. Time will tell
The most successful models I've ever built have been logistic regression models. If you can rephrase your problem in a way that's amenable to run-of-the-mill statistical techniques, you can frequently achieve much better results than you can with 'deep learning'.
I had a pretty good regression model but it was not taken seriously. So I wrote it using "a neural network in TensorFlow" and the next thing you know the whole company is asking me how it works and what it does.
Great read.
Unfortunately, sometimes the hype is so strong, the cart comes before the horse and gets acquired for quite a lot - which is why these companies exist.
If, let's say, you have 0.01% of a Series A that exits at a billion in five years. You'd expect to be diluted by half and there is around a 20% chance of exit that high so roughly ((0.01%billion)/2).20 /5 or about 2k a year in 'expected value'(or why, numerically at least, it's likely not worth it to take a paycut to work at a startup unless you're seeing substantial equity).
That figure probably oughta be at least one order of magnitude less.
There's only one question they need to ask:
"How much are you going to pay me?"
Edit: The reason anyone would wax poetically about everything else is because he or she realizes that they are not being paid enough. It is like day trading - the moment it goes against a bad trader he or she calls it "investment"
Edit: Thanks for the down votes! It makes me feel warm and fuzzy inside to know that it is still possible to swindle people!
But mostly because it really sucks to spend a couple of years building stuff that never gets used. There are a few people who seem happy to get paid even if nothing is achieved. But most engineers I know do the work because the like building things that get used, things that are valuable.
First off, the people you're negotiating with see it as valuable, so it's a very bad negotiating move to say, "Keep your dumb equity and just give me cash." You'd be slightly better off just pointing to their family photo and saying, "Wow, your kids are ugly." Second, going through the work of valuing equity is necessary to see how the investors (that is, they people actually paying your salary) look at the company. Third, on the off chance you are successful, you're going to really wish you took the equity part of the negotiation seriously.
The down votes on calling spade a spade demonstrate it quite well
Sure, but the 'correct' answer to that question depends on a lot of other factors related to the job. If you want me to work long hours, doing dull, morally dubious, work in a high stress environment you're going to have to pay me a lot more.
Sleep on that.
Absolutely. Money is just one of many factors I consider when choosing a job. I see no 'shame' in that. I honestly have/earn enough money to cover all my expenses, so getting a bit more isn't that super important to me. As long as you meet my floor I'm willing to negotiate away money for other perks. I'd rather have time to spend my money rather than just more money.
Where do you get your data from?
If your data is sourced from users of your product, do you tell them what you’re collecting?
Not knowing the answer of this question = they do not know they are travelling on Titanic at best, if not a boat full of holes.