Was interesting to read all the sort of "boring" back end pieces, then see that he went with half-server and half-client rendered React for the front end. Was expecting Angular or similar, given the pattern of the other picks.
This was really informative and delight to read. It's great to see developers making a go at the bizdev side and actually being really successful at it. Light on the heavy technical, light on the buzzwords, just the right amount of information to say, "here, if you even use half my stack you're well on the way to a viable product."
TBH, this is more inspiring than I thought. I thought this would be some BS article about promoting your company but turns out, it was a fun and inspiring read.
My dream is to find a good idea and work solo. It is a little harder as a mobile engineer but you're right as you mention in the blog. There are technology nowadays that help me with backend/dev tools.
key missing info though: does it make money? I checked the site and it looks like a mix of ads (that are blocked of course) and a patreon with 5 supporters.
I'm going to go out on a limb here and confidently say he's doing fine. As he mentioned, he rented an expensive one person WeWork office in SF (when he doesn't even have to).
I think the assumption is that ListenNotes is his only source of income and that he isn't burning personal savings or the capital he has invested on non-mandatory expenses.
Granted those assumptions, you've got to get pretty far up Maslow's Hierarchy of Needs to hit "rent a WeWork"
I respectfully disagree. HN is very much about the -business- of tech. Telling us how you made your mousetrap is all well and good... Having us ask how many mice it has captured is absolutely relevant. Germane even.
What are the pros and cons of charging a fee to transcribe someone else's audio content? Does ListenNotes need a lawyer to deal with contracts or problems related to original content creators?
Looks like they're taking an approach to transcribe first and then deal with it later. The transcription link has an FAQ asking them to contact them if they think they dont want others to transcribe their episode.
When you started working on this full time was there enough profit to pay yourself anything? How did you come to make the decision to work full time on it?
And does it make enough money to cover the time when it wasn't making money? As in: was the time spent bootstrapping worthwhile? The concept of a one-person company is extremely appealing in its simplicity and apparent lack of risk; however, the devil is in the details.
That's awesome and it is a tech stack that I try to mirror and am confident running myself as well!
It's incredible the amount of knowledge required for a single person tho when you think about it eh? It's the full frontend (which I would have more trouble with) + databases + caches + search engine + metrics + deployment + source control + sysadmin all baked in a single person who is also trying to make it a business!
Kudos for the effort and making it happen, one day I might be joining the same journey with the same stack! Just gotta figure out what actually motivates me to build a business on top of =)
as someone who has only worked on small teams, i thought this was normal. My current position is leading a small team, but we have interactions with a number of teams inside of FANG companies and it's amazing the limited amount of knowledge and access each position/person has. Most of my engineers can run circles around our partners.
I always thought every engineer should know how to deploy a server, install deps, understand caching, etc and setup an app... turns out, that is apparently not even remotely expected at most companies. I guess the bigger the company the more narrow the skillset required.
Yeah, I worked in start ups and that's where I got the fast pace and I think that allowed me to rise to high ranks very fast at bigger companies as I have a solid understanding of all the stack.
In bigger companies traditionally they had a person / department more specialized per part of the stack, I think is a lot about the "devops" culture.
And the thing is, generalists do better work. They do less work overall because there are fewer of them, but compare a typical enterprise 50 person team to the output of a team of 5 generalists, hands down the generalists will make a better product every time because there's no buck passing or shoulder shrugging about problems.
I think specialists make more robust systems, but generalists get more done. When I work at large companies, it makes me sad how long it takes to get anything done. But once the work is complete, it is really solid. It scales well, it handles edge cases gracefully, it deploys cleanly, all because a large team of specialists made each part of the puzzle work really well.
The slow progress was dictated by the extra communication to coordinate the specialists, but at the end of the day it ends up being worth it.
(All of this of course assumes you work in an org with lots of really good specialists).
i agree on this point. having highly specialized SMEs certainly helps with scale, for sure.
iteration is slower at bigger companies but some of those teams are inventing wheels as they go. which requires slower movement than general web/mobile dev.
“Every time” until they are asked to build a product that requires a deep specialized skill set.
I am a generalist full stack developer who has built a SaaS business very similar to the way OP has. No team of me’s could’ve produced, say, TensorFlow.
Great teams and companies have generalists and specialists.
IMO, the reduced output of a team of specialists vs a team of generalists has less to do with generalists doing better work, and more to do with it being impractical to hire specialists at lower levels of scale - and higher levels of scale are less efficient.
A 3 person engineering team doesn't want to hire a specialist DB admin who knows Postgres back to front but can't code. A company of 1000+ engineers might, because they might have problems that require that expertise.
But any engineer working in a company of 1000+ engineers is also subject to friction resulting from resource allocation inefficiencies, communication overhead, regulatory compliance, etc: the reduced output from those factors has little to do with the specialist engineers doing worse work, and more to do with them working at the size of company that can afford to hire specialists.
(You could make the case that buck passing is an example of resource allocation inefficiencies, as it is often accidentally or purposely enabled by company management, but nonetheless, that's not a generalist vs specialist tradeoff.)
Completely agree. IMO the optimal efficiency is easily a team of either 2 or 3, with one FE and one BE, and optionally someone doing dev-ops-y stuff for both of them. After that point your 'output per man hours' can only really go down.
In economic terms, sounds like the marginal utility of adding an engineer goes down significantly as companies get larger and past the start-up phase ;)
You're right. That's been my experience, as well. But... the crème de la crème from FANG are different beasts altogether.
I must also point out that what FANG employees do get in abundance is the ability to learn from highly scalable systems that others build, cutting edge tech others work on, experiment on company's dime, pursue an eng/mgmt tract more in line with their personal interests, and job stability.
It also affords them an opportunity to be part of a team that one day builds a groundbreaking tech and pushes the envelope forward not just for the company but the entire industry.
Having been the CTO of a startup and then a SWE at a FAANG company for 8 years, I disagree. Being at a startup does indeed require a broad technical skillset, but being effective inside a large company requires a lot of skills (both technical and not) that early startup employees may not have or need. I've grown a lot from both environments.
Are you me? I am constantly being gutchecked that a startup mentality does not fly when you have several hundred million in the bank. That said, my enterprise and upper management butt kissing experience from the 00s don’t seem to matter either. I can stay heads down in code and I’m not in fear over being downsized. The hard part is teaching these young kids how to scope.
It might take Alice 10 hours to do the sysadmin, 5 for the database, and 15 for the devops.
Bob however can do the devops in 2 hours, sysadmin in 3 hours and database in 4 hours.
Bob is of course more productive, but if you let Alice do all the DBA, then Bob will be even more productive, and the total output of your team will increase.
I finally created an account because your comment resonated with me. I have created 95% of my platform by myself which itself was the manifestation of several business ideas. I started out buying servers and starting a dedicated, then vps, then shared hosting farm that requires all of the frontends you mention. I took a different approach, I went all out open-source and spent time creating glue and flashy bootstrap frontends to orchestrate everything reliably.
I currently run the remains of my companies as a lab that is spread out to a few datacenters and provides a UX where anyone can request a VM, launch a container, or drop a php/java war/RoR/django/etc onto a custom app server of varying security restrictions. You can request a service/vm/container by API, by chat, or any other host of events through my half-baked event controller and change mgmt database. In a lot of cases, changes are a two-way street. You can modify e.g. a bind zone file and that will reflect upwards in a CMDB or vice-versa and watch the zone file update automagically. The original idea was to allow mixing sysadmin strengths and still maintain a reliable complex system.
So now I have a platform that spans multiple datacenters, uses infra as code as you would expect (supporting another cloud provider is simply adding glue to their apis), has loadbalancing and SSO, and it's just literally sitting on the sidelines exhausting the remaining budget until I finally get tired and liquidate it all. The motivation of building a business on it is so tiny after years of failed attempts and seeing the shared cloud model completely destroy ROI on holding hardware. I can and have built e.g. fleet tracking services. I have gobs of storage, so I run an object store for giggles. But have no clue how to generate revenue from these ideas when the market is already saturated. My last ditch idea is to create a learning ground for the public. Training on how to build apps that scale, manage systems at scale, and give a real world environment to folks who may otherwise not work at an organization with more than 100 servers. shrug . until then I chop-chop away at my dayjob :)
There are people who create consumer products, and there are people who create the products that power those products. It's no different to when id Software sold licences of their engines to other game developers. Perhaps you can turn your infrastructure into a business of its own.
My thought exactly. I was expecting good ol'LAMP or something along these lines. I think here "boring" means "has been released for more than a couple of years".
As someone still in school, this type of article -- though inspiring-- kind of scares me, to be honest. I don't think I could do any one of the things mentioned in this article.
Thanks Alan(?). I haven’t gone too far with Django. I can see what you’re saying about being seasoned. Flask clicked for me quickly but everything just made sense, ORM, Jinja, modules etc.
I couldn't do all these things when I was in school :) I worked in companies for a few years and learned some engineering practices. Then I had basic skills to prototype my own side projects. Then after working on many silly side projects, I started Listen Notes.
And initially, Listen Notes was running on 3 tiny DigitalOcean servers ($5/month each?). I logged in each server to git pull to "deploy to production". Then I added things little by little, day by day. It's a process. The key is to get started. People say that showing up is 80% (or whatever percentage) of success. I think this is very true. Just get started and you'll figure out things along the way.
Exactly what I'd like to tell younger devs and students. You build smaller projects on the side for a bit to familiarize yourself with things until you're ready to start something or something comes along by way of the side projects!
Thanks for the info. It's an interesting set of tools today, but it's helpful to hear that it has evolved to that point in terms of automation and infrastructure.
The first version of a web service can be as simple as flask app which you run in a screen session somewhere. Better to start somewhere than get overwhelmed and never do the thing at all.
As a senior dev: it will take years of experience and constant learning to get to this point, but it is attainable. You don't have to be a master of each part of the stack and each tech within it - you just need to be able to do enough, and solve the problems you face. You can build a professional SaaS product with a simpler stack as well.
You take a project like this one step at a time. Some bits of it are relatively easy - setting up a few postgres servers doesn't take much knowledge. ElasticSearch is a little more obscure, but for the most part, things like this are running a few commands, and setting up a few config files with the help of docs and google. Same for Redis, nginx... etc. Which isn't to diminish devops - you can dive deep into each of these configurations and develop pretty complex setups, but by the time you actually need to you hope to be making enough money to pay someone else to do it.
You won't get everything perfect all the time. You'll have to revisit parts of the stack and tweak them. But you can take it a day at a time and do what you need to do.
Yep, learning how to run your own business is the only way to convert 10x development aptitude to a 5-10x pay increase. Otherwise you get 1x pay, 10x the expectations, upset colleagues that feel inferior in comparison, and very little promotion chance because the company needs you right where you are.
Picking your stack/architecture based on team size is something most engineers miss.
Microservices, great for companies with many teams. Not so much when it's three people scrambling to create something meaningful. Monolith all the way.
It's funny because microservices are explicitly targeted at solving problems for development with many teams, and lots of single team companies cargo cult them. Did they miss the first paragraph when they were reading up on what microservices are?
- different SLA and scaling requirements per component/endpoint
- different security domains per component/endpoint
- different rollout strategies/policies per component/endpoint (no need to restart your long-lived client TCP sessions in parallel to rolling out a business logic fix)
- ...
I've been happily developing and deploying microservices for small customers, in teams of 1 to 10 people. But I also don't work with customers who just need simple CRUD apps.
Everything depends on context, but I don't think microservices are the best solution. I think in nearly every case there is an easier way to address those problems.
Who says only multiple teams need microservices, and who put them in charge of what people find helpful?
If the 'app' has enough different 'bits' I'd favour a SOA no matter the team size.
Somebody now wants to say that isolation doesn't necessitate distinct services; they're probably right, but the alternative is great discipline - why not make it easy?
It's a form of defensive programming isn't it? - if widget factory shouldn't be using a function from auth helpers, make it impossible!
There are better ways to solve that problem. Different packages, visibility modifiers, using interfaces between modules, etc. Taking a little time to iron out a disciplined process for your team is so much better than using tcp/ip as your process and all the problems that brings.
If you can't trust your team to follow a process correctly, you're fucked in a way microservices can't save you from.
To all the people thinking “I couldn’t possibly do this myself” just look at the mention of UpWork towards the bottom. Looks like the author has brought in contractor help at some point.
First, that's a great stack and very well written/presented.
One comment - he dismisses serverless as being overengineering. I think the correct POV, moreso for the single-man company, is that running a server to perform a task is the overengineered option.
One can see from the snapshot the servers are indeed severely overprovisioned and underutilized. Building an api with api-gateway + lambda is less work than running django in uwsgi behind self-managed nginx, and is guaranteed to be more cost-effective for unpredicted load.
Same logic applies to the db servers - why not hosted?
And last - the inf is a good reminder that prefixing your api routes with /v1, /v2 is always a good habit.
Yes, and no. If you already have a lot of experience building apps that run on servers, there's a learning curve to switching to serverless. Is it huge? Not really. But there are certainly pitfalls and best practices to learn about. The costs can be harder to predict, especially when starting out. And the tooling is different (and much less mature). So now you have a bunch of stuff to learn about or consider, or you can just go do the same thing you already know how to do with minimal friction. It's possible that the cost savings of not overprovisioning servers is worth it, but I don't think it's that straightforward of an answer, and if your server costs aren't massive, you might be better of spending your time building a great product than learning a new way to build.
How? His boxes are running at 100%.
He might be able save some money by switching to hosted db and maybe run his webservers serverless (and creating some keep-alive triggers).
Nontrivial cost reduction would be switching to a different host instead of aws.
Not to mention that he might not stick with AWS if he runs the numbers- he might hop ship to a different cloud provider if he gets a better deal. I don't think it's appropriate for such a fledgling enterprise to lock in to AWS-isms so they can keep the change provider cost as low as possible.
Also, people seem to be dismissing the value of a rock-solid local end-to-end dev stack which with the latest iteration of technology I feel has become very complicated and expensive. The OP can run the same Ansible for his dev stack as well as his prod and his dev stack doesn't turn into an AWS bill. Most of the AWS managed tech & services is only reliably testable on AWS at a functional/integration level, which is a cost that a company this small shouldn't have to absorb.
Not sure I agree with the anti-lock in sentiment. It's a nice to have, but if what you know well is AWS or GCP or whatever, definitely lock yourself in.
Someone who has been doing this for years before serverless existed can set up and manage this infrastructure trivially, using almost no time compared to developing the application and the business. It also provides you with security: you know the pitfalls and the problems with hosting these, serverless is another new technology which can go wrong.
As team of one, you have to be very judicious with your time. You often have to have a philosophy of "if it ain't broke, don't fix it". Yeah they could burn a week or two switching over to lambda and whatnot but is it going to have a higher ROI than all the other things they could be working on?
As me how I know. When I was a solo dev doing my own thing, I'd spend way too much time working on things that really wouldn't affect the business but were "good engineering things" to do. If I spent more time working on things that would grow the business instead of wasting weeks writing fancy deployment scripts, maybe I'd still be doing my own thing now!
> As team of one, you have to be very judicious with your time. You often have to have a philosophy of "if it ain't broke, don't fix it". Yeah they could burn a week or two switching over to lambda and whatnot but is it going to have a higher ROI than all the other things they could be working on?
Timeframe has to be considered. If it only takes a week, I'd strongly suggest biting the bullet. A week's budget can be quickly spent troubleshooting issues, scaling servers up and down, installing software updates, hardening systems and whatnot.
It will most likely be much cheaper to run too. Which, in a single-man operation, may be worthwhile.
Think of it the other way round - if your workload is not realtime, but "within minutes/hours", as in processing feeds here - scaling the worker nodes with an ansible scripts needs to be done on maybe a weekly (if not monthly or less) basis and you get 4/3 instead of 3/3 - for a few minutes of work. This just does not seem to be the workload for autoscaling and adding complexity. I've ran a similar setup at a past job - we planned for whatever but in the end it was one person clicking a few buttons to provision a new instance once in a while.
Also some workloads are really bad for lambda, because you can (total napkin math) run 4 cores at 100% load for 24h a day to do your processing, all at the cost of "1 instance".
> Building an api with api-gateway + lambda (...) is guaranteed to be more cost-effective for unpredicted load.
Depends on the use case. I run a cron monitoring service on a similar nginx/uwsgi/django/postgres stack [1]. My service needs to handle lots of really small and simple HTTP requests, and almost every request needs to do a (small and quick) database write. I did napkin math – at the current usage levels, Lambda per-request fees alone would use up significant chunk of my current hosting budget.
Good point! For people who have tons of experience with serverless, serverless is probably a better choice than running servers for some use cases.
As a small business owner, there are two types of cost that I need to consider:
Time: the time I use to do A is the time I can't use to do B. Unfortunately I haven't used serverless so far in my professional career -- in this sense, I'm not full-stack enough :) It takes time for me to learn it, understand it, operate it, and experience various outage scenarios to gain the true learnings. It's more costly for me (probably not for others) to use serverless than the things that I already understand. I'd rather spend more time on other non-engineering things nowadays -- believe it or not, I spend 1/3 of my working hours replying emails :)
Money: the money I spend on A is the money I can't spend on B. I decided not to use api-gateway + lambda & hosted db servers, primarily because of $$$. I actually did the pricing calculation a few times last year. In addition, api-gateway + lambda also require some time for me to learn, which I should use to talk to users, marketing, building new product feature, thinking (yep, thinking also uses some time budget :)...
I used a bit of serverless Javascript (Firebase functions) in a previous startup, and it actually worked really nicely for small self-contained tasks - saving files to S3, triggering customer emails, triggering notifications or webhooks in other services (and ofc in Firebase, actually doing things with the database, but it's not a fit for everything).
Just wanted to add another reason not to use serverless, or at least AWS. For small projects I would rather experience downtime than big invoice. I had a project running on AWS, but it turns out you can't limit the budget, the best you can do is make alerts when costs exceed some limits. I have now moved the project to old-skool server and sleep much better since.
Thank you for this excellent write-up. Monetizing APIs is always a great topic; did you consider or will you consider using a 3rd party API management service such as RapidAPI or Apigee to keep track and charge for API usage?
Completely agree. Cloud functions are in many cases a better option than maintaining a server, specially for those background tasks that fire occasionally.
The only issue I've found about using serverless is the database. In most cases (Firebase, Fauna, Cosmos, DynamoDB) you have to couple your stack to the DBaaS provider which is not a great idea. AWS recently announced Amazon Aurora PostgreSQL Serverless but while it allows you to use regular Postgres tools/queries you are again tied to AWS.
One hopes that GraphQL abstractions will enable the decoupling. So if you opt for a database that offers GraphQL natively, it might address the issue. You do have to learn the GraphQL ecosystem (new tools/language etc.)
> lambda is less work than running django in uwsgi behind self-managed nginx
If you discount building an AWS-specific deployment process that includes "pip install" from an AWS linux machine image, zipping the project, and putting it in S3.
If serverless were easier, more people would be using it. But it's not simple or straightforward. You have to learn new systems and conventions, it has a bunch of weird considerations depending on the use case, and most people just use it when they don't want to figure out what instance to run some periodic, one-off job on.
It's a niche, just like all solutions that aren't a single Unix process on a single Unix box. Even CGI scripts are a niche. You pick the niche that you know.
I think the correct POV, moreso for the single-man company, is that running a server to perform a task is the overengineered option.
I was all gung-ho about serverless for a while. I wanted to release a demo for my product and thought I'd cut through all the hassles of managing my own server.
I found it bewildering. It was a whole new skillset with new benefits, but also new considerations and headaches. When push came to shove and the clock to release my demo started ticking down, I just went back to a linux server.
I use the same linux distro at home and on the server, and there are about 3 technologies I need installed. On retrospect I think I made the right decision, but happy to have my mind changed.
I've yet to find someone explain a serverless API based setup that isn't more complicated than good old fashioned LAMP or equivalent. Serverless seems to always come out more expensive as well.
I've been exploring serverless for a project at work, and this is my conclusion too. It's great that we don't need to manage any servers and can easily deploy tasks with one command - but we already have infrastructure to do all that for that for our existing apps.
For someone starting out you don't need to worry about learning Linux and how to configure Apache to serve SSL certificates in the right way, but that is important to know, unless you are happy to always rely on (and pay for) someone else to do that.
To me serverless is similar to Heroku, it's great for starting out but as you start to grow it's going to quickly become a lot cheaper to maintain you own systems. Except with serverless it's not so easy to self-host because you end up relying on all the tooling the vendor provides.
I take (very minor) issue with your first line and the point "well written" (I do agree with well presented).
Since the author is reading/commenting here, and there was a large amount of space in the original article outlining tools/services he uses, can I humbly suggest they use a tool like "Grammarly" or similar to help with the word-choice(s)?
Some distracting use of plurals for terms - e.g. "traffics", "stuffs", etc - may have been avoided and other spelling and grammar aspects could have helped make this easier to read. That all being said - the 'essence' of the article is to be commended.
I don't know if overprovisioning servers to counter traffic spikes is over engineering, the mental model is pretty simple. May not be as cost effective or infinitely scalable, but it's simpler to wrap your head around.
I'm a one-person company too ( geocode.xyz ). My tech stack is even more boring than that. (Nginx, MariaDB, Perl on AWS Ec2 Linux instances. I don't have an office either.
I really like the idea of monolithic repos but can see some downsides when there is more than one person working on a project. It would be cool if there was a simplified way to have an entire business operate under source control.
This interests me. Can you elaborate on what downsides you see? Even in a small team we're often working within the same repo at the same time without any issues.
The glaring issue would be permissions ie who can see what. Pull requests could get muddied up but that probably wouldn't be too much of a problem to overcome. Some foundational git protocols would have to be adjusted, like .gitignore.
I think it's a great way for start up to manage the business but as more people get hired the organizational complexity might be too much to bear.
> It would be cool if there was a simplified way to have an entire business operate under source control.
A lot of projects can and do apply CI tooling to achieve this. Every commit to a branch triggers a set of declarative deployment pipelines, simple. IIRC buzzword is "GitOps" if you want to find out more.
461 comments
[ 4.1 ms ] story [ 313 ms ] thread/s
#HNPropaganda
My dream is to find a good idea and work solo. It is a little harder as a mobile engineer but you're right as you mention in the blog. There are technology nowadays that help me with backend/dev tools.
Cheers.
Granted those assumptions, you've got to get pretty far up Maslow's Hierarchy of Needs to hit "rent a WeWork"
https://news.ycombinator.com/hackernews.html
Yes, Listen Notes is making some money - not a lot, but enough to cover all the cost and bring in a bit profit as of today.
The basic idea is that Listen Notes should be free to 99% of users, while making some money from 1% of super users.
We run ads (obviously) on the website and we provide API: https://www.listennotes.com/api/pricing/ And I've been experimenting some paid features that are needed for PR/marketing/journalists to do their job, e.g., https://www.listennotes.com/datasets/
And today is special, because on Sep 16, 2017 (Exactly 2 years ago today), I started to work on Listen Notes full-time!
Just curious, is your company still one-person?
Here is the pricing page: https://www.listennotes.com/api/pricing/
Looks like it's an ad-driven revenue model, and it looks like it gets ~ 1m views/month: https://www.similarweb.com/website/listennotes.com
Assuming $1 per 1,000 impressions, we get $1,000 / month.
That's purely the website. I can't speculate on the API side of his business though.
It's incredible the amount of knowledge required for a single person tho when you think about it eh? It's the full frontend (which I would have more trouble with) + databases + caches + search engine + metrics + deployment + source control + sysadmin all baked in a single person who is also trying to make it a business!
Kudos for the effort and making it happen, one day I might be joining the same journey with the same stack! Just gotta figure out what actually motivates me to build a business on top of =)
I always thought every engineer should know how to deploy a server, install deps, understand caching, etc and setup an app... turns out, that is apparently not even remotely expected at most companies. I guess the bigger the company the more narrow the skillset required.
In bigger companies traditionally they had a person / department more specialized per part of the stack, I think is a lot about the "devops" culture.
The slow progress was dictated by the extra communication to coordinate the specialists, but at the end of the day it ends up being worth it.
(All of this of course assumes you work in an org with lots of really good specialists).
iteration is slower at bigger companies but some of those teams are inventing wheels as they go. which requires slower movement than general web/mobile dev.
I am a generalist full stack developer who has built a SaaS business very similar to the way OP has. No team of me’s could’ve produced, say, TensorFlow.
Great teams and companies have generalists and specialists.
A 3 person engineering team doesn't want to hire a specialist DB admin who knows Postgres back to front but can't code. A company of 1000+ engineers might, because they might have problems that require that expertise.
But any engineer working in a company of 1000+ engineers is also subject to friction resulting from resource allocation inefficiencies, communication overhead, regulatory compliance, etc: the reduced output from those factors has little to do with the specialist engineers doing worse work, and more to do with them working at the size of company that can afford to hire specialists.
(You could make the case that buck passing is an example of resource allocation inefficiencies, as it is often accidentally or purposely enabled by company management, but nonetheless, that's not a generalist vs specialist tradeoff.)
I must also point out that what FANG employees do get in abundance is the ability to learn from highly scalable systems that others build, cutting edge tech others work on, experiment on company's dime, pursue an eng/mgmt tract more in line with their personal interests, and job stability.
It also affords them an opportunity to be part of a team that one day builds a groundbreaking tech and pushes the envelope forward not just for the company but the entire industry.
I currently run the remains of my companies as a lab that is spread out to a few datacenters and provides a UX where anyone can request a VM, launch a container, or drop a php/java war/RoR/django/etc onto a custom app server of varying security restrictions. You can request a service/vm/container by API, by chat, or any other host of events through my half-baked event controller and change mgmt database. In a lot of cases, changes are a two-way street. You can modify e.g. a bind zone file and that will reflect upwards in a CMDB or vice-versa and watch the zone file update automagically. The original idea was to allow mixing sysadmin strengths and still maintain a reliable complex system.
So now I have a platform that spans multiple datacenters, uses infra as code as you would expect (supporting another cloud provider is simply adding glue to their apis), has loadbalancing and SSO, and it's just literally sitting on the sidelines exhausting the remaining budget until I finally get tired and liquidate it all. The motivation of building a business on it is so tiny after years of failed attempts and seeing the shared cloud model completely destroy ROI on holding hardware. I can and have built e.g. fleet tracking services. I have gobs of storage, so I run an object store for giggles. But have no clue how to generate revenue from these ideas when the market is already saturated. My last ditch idea is to create a learning ground for the public. Training on how to build apps that scale, manage systems at scale, and give a real world environment to folks who may otherwise not work at an organization with more than 100 servers. shrug . until then I chop-chop away at my dayjob :)
It mostly worked really well though, kudos to whoever figured it out. And you could mock a dev setup right in PyCharm.
Do you understand what the web, api, and DB servers are doing?
If your interested in Python and want to start small I can recommend Flask. Flask is smaller and could be more user friendly than Django.
Here’s a great tutorial. You’ll build a blog with Bootstrap, Python, Flask and SQLAlchemy.[1]
[1]https://www.youtube.com/playlist?list=PL-osiE80TeTs4UjLw5MM6...
https://wakatime.com/blog/14-pirates-use-flask-the-navy-uses...
(I'm the author of this blog post)
I couldn't do all these things when I was in school :) I worked in companies for a few years and learned some engineering practices. Then I had basic skills to prototype my own side projects. Then after working on many silly side projects, I started Listen Notes.
And initially, Listen Notes was running on 3 tiny DigitalOcean servers ($5/month each?). I logged in each server to git pull to "deploy to production". Then I added things little by little, day by day. It's a process. The key is to get started. People say that showing up is 80% (or whatever percentage) of success. I think this is very true. Just get started and you'll figure out things along the way.
The first version of a web service can be as simple as flask app which you run in a screen session somewhere. Better to start somewhere than get overwhelmed and never do the thing at all.
You take a project like this one step at a time. Some bits of it are relatively easy - setting up a few postgres servers doesn't take much knowledge. ElasticSearch is a little more obscure, but for the most part, things like this are running a few commands, and setting up a few config files with the help of docs and google. Same for Redis, nginx... etc. Which isn't to diminish devops - you can dive deep into each of these configurations and develop pretty complex setups, but by the time you actually need to you hope to be making enough money to pay someone else to do it.
You won't get everything perfect all the time. You'll have to revisit parts of the stack and tweak them. But you can take it a day at a time and do what you need to do.
One question: what all do you use contractors for? How has your experience been in managing them?
Thanks
1. Built some reusable ReactJs components. 2. Design / illustrations 3. Proofread website copy / blog posts 4. Built experimental app like this one https://itunes.apple.com/us/app/just-listen-simple-podcast-a...
(probably there were other random things... I got to look at the billing history on my upwork :)
Microservices, great for companies with many teams. Not so much when it's three people scrambling to create something meaningful. Monolith all the way.
- different SLA and scaling requirements per component/endpoint
- different security domains per component/endpoint
- different rollout strategies/policies per component/endpoint (no need to restart your long-lived client TCP sessions in parallel to rolling out a business logic fix)
- ...
I've been happily developing and deploying microservices for small customers, in teams of 1 to 10 people. But I also don't work with customers who just need simple CRUD apps.
If the 'app' has enough different 'bits' I'd favour a SOA no matter the team size.
Somebody now wants to say that isolation doesn't necessitate distinct services; they're probably right, but the alternative is great discipline - why not make it easy?
It's a form of defensive programming isn't it? - if widget factory shouldn't be using a function from auth helpers, make it impossible!
If you can't trust your team to follow a process correctly, you're fucked in a way microservices can't save you from.
To all the people thinking “I couldn’t possibly do this myself” just look at the mention of UpWork towards the bottom. Looks like the author has brought in contractor help at some point.
One comment - he dismisses serverless as being overengineering. I think the correct POV, moreso for the single-man company, is that running a server to perform a task is the overengineered option.
One can see from the snapshot the servers are indeed severely overprovisioned and underutilized. Building an api with api-gateway + lambda is less work than running django in uwsgi behind self-managed nginx, and is guaranteed to be more cost-effective for unpredicted load.
Same logic applies to the db servers - why not hosted?
And last - the inf is a good reminder that prefixing your api routes with /v1, /v2 is always a good habit.
Of course, the overarching rule is always "if it works, don't touch it".
Nontrivial cost reduction would be switching to a different host instead of aws.
As a solo entrepreneur I can say time risk is a crucial thing to be mindful of. I'll take 10h +/-1h vs 5h +/- 15h any day.
Also, people seem to be dismissing the value of a rock-solid local end-to-end dev stack which with the latest iteration of technology I feel has become very complicated and expensive. The OP can run the same Ansible for his dev stack as well as his prod and his dev stack doesn't turn into an AWS bill. Most of the AWS managed tech & services is only reliably testable on AWS at a functional/integration level, which is a cost that a company this small shouldn't have to absorb.
As me how I know. When I was a solo dev doing my own thing, I'd spend way too much time working on things that really wouldn't affect the business but were "good engineering things" to do. If I spent more time working on things that would grow the business instead of wasting weeks writing fancy deployment scripts, maybe I'd still be doing my own thing now!
Timeframe has to be considered. If it only takes a week, I'd strongly suggest biting the bullet. A week's budget can be quickly spent troubleshooting issues, scaling servers up and down, installing software updates, hardening systems and whatnot.
It will most likely be much cheaper to run too. Which, in a single-man operation, may be worthwhile.
Also some workloads are really bad for lambda, because you can (total napkin math) run 4 cores at 100% load for 24h a day to do your processing, all at the cost of "1 instance".
Depends on the use case. I run a cron monitoring service on a similar nginx/uwsgi/django/postgres stack [1]. My service needs to handle lots of really small and simple HTTP requests, and almost every request needs to do a (small and quick) database write. I did napkin math – at the current usage levels, Lambda per-request fees alone would use up significant chunk of my current hosting budget.
[1] https://blog.healthchecks.io/2019/08/a-look-at-healthchecks-...
As a small business owner, there are two types of cost that I need to consider:
Time: the time I use to do A is the time I can't use to do B. Unfortunately I haven't used serverless so far in my professional career -- in this sense, I'm not full-stack enough :) It takes time for me to learn it, understand it, operate it, and experience various outage scenarios to gain the true learnings. It's more costly for me (probably not for others) to use serverless than the things that I already understand. I'd rather spend more time on other non-engineering things nowadays -- believe it or not, I spend 1/3 of my working hours replying emails :)
Money: the money I spend on A is the money I can't spend on B. I decided not to use api-gateway + lambda & hosted db servers, primarily because of $$$. I actually did the pricing calculation a few times last year. In addition, api-gateway + lambda also require some time for me to learn, which I should use to talk to users, marketing, building new product feature, thinking (yep, thinking also uses some time budget :)...
Thank you for this excellent write-up. Monetizing APIs is always a great topic; did you consider or will you consider using a 3rd party API management service such as RapidAPI or Apigee to keep track and charge for API usage?
The api was launched in Dec 2017 right here on HN: https://news.ycombinator.com/item?id=15825900
The only issue I've found about using serverless is the database. In most cases (Firebase, Fauna, Cosmos, DynamoDB) you have to couple your stack to the DBaaS provider which is not a great idea. AWS recently announced Amazon Aurora PostgreSQL Serverless but while it allows you to use regular Postgres tools/queries you are again tied to AWS.
You'd also see that 7 of his 8 worker boxes are almost at 100%.
If you discount building an AWS-specific deployment process that includes "pip install" from an AWS linux machine image, zipping the project, and putting it in S3.
It's a niche, just like all solutions that aren't a single Unix process on a single Unix box. Even CGI scripts are a niche. You pick the niche that you know.
I was all gung-ho about serverless for a while. I wanted to release a demo for my product and thought I'd cut through all the hassles of managing my own server.
I found it bewildering. It was a whole new skillset with new benefits, but also new considerations and headaches. When push came to shove and the clock to release my demo started ticking down, I just went back to a linux server.
I use the same linux distro at home and on the server, and there are about 3 technologies I need installed. On retrospect I think I made the right decision, but happy to have my mind changed.
For someone starting out you don't need to worry about learning Linux and how to configure Apache to serve SSL certificates in the right way, but that is important to know, unless you are happy to always rely on (and pay for) someone else to do that.
To me serverless is similar to Heroku, it's great for starting out but as you start to grow it's going to quickly become a lot cheaper to maintain you own systems. Except with serverless it's not so easy to self-host because you end up relying on all the tooling the vendor provides.
Since the author is reading/commenting here, and there was a large amount of space in the original article outlining tools/services he uses, can I humbly suggest they use a tool like "Grammarly" or similar to help with the word-choice(s)?
Some distracting use of plurals for terms - e.g. "traffics", "stuffs", etc - may have been avoided and other spelling and grammar aspects could have helped make this easier to read. That all being said - the 'essence' of the article is to be commended.
Also curious if you have a monitoring/recovery strategy for when you go camping or on honeymoon and need to be offline.
I think it's a great way for start up to manage the business but as more people get hired the organizational complexity might be too much to bear.
A lot of projects can and do apply CI tooling to achieve this. Every commit to a branch triggers a set of declarative deployment pipelines, simple. IIRC buzzword is "GitOps" if you want to find out more.