I’ve a small side project I’m working on. I am writing the backend in Clojure and then deploy with Jenkins using the Groovy DSL — it’s useful that everything runs on the JVM. The Jenkins code calls Terraform to set up the servers, drawing on optimized AMIs I set up with Packer. I avoid Docker for all the reasons I’ve written about elsewhere (you can Google if you’re interested). Packer does everything that people would otherwise use Docker for. For now the frontend is plain HTML and jQuery. I haven’t done the mobile app yet so no need for Angular/Cordova or React.
For my recent projects, I've used Go + DynamoDB for a GraphQL backend, and Next.JS with SSR on the frontend. Deploying both to lambda with Apex Up. Stripe for billing, SendGrid for transactional email.
"Next.JS with SSR on the frontend" I call that backend, and node is pretty good for graphql too (graphql-js), full JS make a lot of sense (add flow/TS to get more type-strength)
Any tips on the go/dynamodb side of things? I’ve looked at it and experimented a little and it seems quite easy to get data into dynamodb and quite convoluted to get it out.
I'm using https://github.com/guregu/dynamo for the data reading/writing. In my experience, the trick with Dynamo is the schema design. I haven't had problems with the libraries/queries as long as the schema made sense for how I'd be accessing the data.
My side project is using Rails, React, Postgres, and deployed to heroku. There’s nothing flashy but I can prototype quickly and it’s easy for 2 people to manage.
That's my stack as well. But sure you mean SNS, not SQS?
Though I have been thinking if I should replace elastic beanstalk with Kubernetes, since it seems to become the non-proprietary standard for deploying containers?
A 20$ admin theme from themeforest, based on bootstrap3. Nice enough that it does not warrant massive re-styling right now. If the app becomes bit of a success, we'll outsource this part to more competent people ( me and partner are both backend-ish people)
Is there a guide on the internet that would help me figure out how to use Vue with Django? I tried a couple last month but they were poorly structured or didn't explain enough about what they were doing, and since I'm not great at JavaScript I gave up..
I've completely separated it out. Except for some authentication pages, it's all JSON APIs. Django project does not have any frontend assets of Vue/vuex
Prioritize. If your goal is to get something out quickly and make money, use what you know. If your goal is to learn a new stack, learn the new stack. If you learn enough to comfortably release to customers, do it; otherwise, either get comfortable or rebuild.
I had to make this decision recently and my conclusion was if your goal is to build something that makes money, do it with what you know. Why? Because you can spend 100% of your time building instead of 50/50 learning/building (or worse). Projects with the goal of being a business should iterate and fail fast. If you win the app lottery and actually build something people will pay for, they won't want to wait for you to learn while you build it out. Someone else will do it with some "old" tech like ruby on rails and take your customers and revenue. Spend your time working on the business, not the tech. Developers think code is important but the reality is that it almost doesn't matter. You can build something successful with the worst tech decisions imaginable. I know it's hard to accept but it's true. I've personally witnessed an absolute frankenstein's monster app, made of a mish-mash of what are now considered the worst technologies, get sold for $30+ million and the founders retired to the Caribbean where the build a mansion and an upscale restaurant just for fun. I pity the people devs who have to inherit that, but the value wasn't in the tech and it almost never is.
If you're spending 100% of your time building you're doing it wrong anyway.
I feel like this is just missing so much conte xt.
If you are working full time and can hack in your after hours to learn a new tech stack first and then start your project there might be an aggregate benefit over just starting the project right away using what you know.
Context matters to a huge degree in these kinds of discussions. I feel like people just like to give blanket statement advice.
I've also heard for side projects when you want to learn a new technology to experiment with one piece of it, so you're not totally overwhelmed, and so you can see how it works with stuff you know. For instance, use the rails backend deployed to AWS that you know, but swap out angular for react. I think this works for your question pretty well - if the side project becomes a money-making project, at least there's only one part of it that will be a technical risk, and often if you need to do something quickly, you can work around and do it in the parts that you know.
Before you write a line of code, make a clear decision as to whether it's a learning experience or a business venture. Plan your project based on the goal you've chosen. If it's a learning experience, consciously avoid doing anything that doesn't push you to learn something new. If it's a business venture, consciously avoid doing anything unless you believe it has the highest RoI.
Some purely recreational projects do turn into viable businesses, but far more projects have failed due to indecision and yak shaving. If you're building a business, you don't gain anything by experimenting with a new stack. If you're learning React, you don't gain anything by writing a bunch of CSS and marketing copy. Don't be afraid to step away from the keyboard and ask yourself "Is this a good use of my time?".
So much this. If you want to build a business then there will be a lot of non-technical things to learn along the way. Keep the tech familiar so you can focus on those.
The intention at the outset. There's a low chance of success whether you are trying to make money or not. If you make money accidentally in a new tech stack... who cares because you can afford to support it.
Also build using what you know isn't always the best advice. Figure out what you want to optimize for and optimize for it. That may include doing things in a different stack.
Python for the code, Postgres for the backend, Markdown for the frontend. Because I am too lazy to reinvent the wheel, and most of that is good enough.
I would have much preferred to use Perl, R, dotnet, or just about anything else, but there was an already made library in Python that was good enough for 90% of my needs, and 100% with some elbow grease, so my lazyness made me learn Python instead!
But wait- it gets dirtier than this! The code is done is Vi, with no care in the world about version control (rsync does most of what I need: backups), and deployed to servers by manual SCP, with no automatic tests, no care about CD or automatization - yet: as I need more and more servers, I realize I have to spend time learning say ansible to provision them faster than a copy paste of bash (I have low standards but I won't do curl|bash, just... no) but I am delaying that as much as possible to concentate on implementation.
The servers are on Debian VMs on Azure, just because they gave me free credits and do not perform as bad as AWS. When the free credits run out, I will move to DO. Then I will see what else I can use, and that will be the right time to care about ansible or whatever.
It is as ugly as I can get away with to get my MVP out, using tech from 1996 lol.
I am not convinced that git is easy... I believe it's very powerful if you put the time into learning it, but anything with 50,000 separate guides on the internet must be somewhat unintuitive, right?
It’s exactly like merging but you have to resolve any conflicts commit-by-commit instead of all at once in a merge commit. Useful for if you’ve been working on a feature and want to pull in changes your colleagues have made on the develop or master branch without muddying up your history with merge commits. Having clean history makes diffing easier for code reviews. I’ll admit even after years of rebasing, I still screw it up from time to time.
Definitely are way more intricacies to the cli than I'll ever learn, but 99.7% (estimated) of my git usage is running the same 4-5 commands all day, which I have aliased to convenient acronyms. Don't even have to think about it.
I’ve still been unable to find something that has the right balance of intuitiveness as well as surfacing advanced features and workflows besides SourceTree. The only downside is that SourceTree is slow and buggy at times.
I agree that managing complex workflows, or really any situation involving a lot of contributors/branches/activity may send you running to your nearest search engine with regularity.
That being said, for GP's use, they can work solely from master, requiring only a few commands with little to no chance of difficulty.
There are 50,000+ ways to learn anything, and most won't work for most people. Sometimes I have to read 20 different takes on something before it finally clicks.
The fact that there are so many people trying to explain it in their own way is a good sign.
You can write off something as being hard if lots of people want to help you learn it and share their tips and advice, but then you're just ruling out learning anything worthwhile, really.
I agree with you 95%. If you're running something basic with less than 5 people this is reasonable with one minor caveat:
I'd still use git.
Hell even on my own I frequently use git for single file shell script. Commit a working copy. Tweak it to add a feature. Decide it's unmaintainable. Revert to before I fucked it up.
Also, theres a decent half-step between ssh and Ansible: mssh. You can run ssh across a set of hosts quite trivially. It working on the order of a few dozen machines just fine.
True, that works too. I'm good with bash but I find many aren't. I actually work with bigger setups (think thousand machines) that have range set up which is a big value add with mssh but not in the kinds of small setups we're discussing.
I meant all this is super old school and super dirty, except markdown that is somehow recent (2004)
Yet markdown is not essentially different from running sed to render something into html, which I would have totally done if I hadn't also needed some other features like basic math that would have been too tedious in sed
For my hobby stuff I am similar. Perl and either SQLite or Text files. Sometimes Postgres, but I rarely find it is needed, even for fairly popular anonymous boards.
I use vi. I do use git locally, but I don't trust it, so I also use rsnapshot, probably similar to your rsync. You might want to try rsnapshot some time if you have not. It uses less disk space by hard linking dupes using a perl script.
People saying they wouldn't use git don't know git. I admit it does take investment to learn but once you are over the hump it is insanely easy and useful (yes even easier than rsync and 10x more useful). Especially when you consider that using rsync precludes your from collaborating even with one other person which is crazy easy if you use github (for example). People not using git: get over the hump, it is worth it, even on little side projects.
>> ...with no care in the world about version control (rsync does most of what I need: backups), and deployed to servers by manual SCP...
> People saying they wouldn't use git don't know git.
A few things:
* to the grandparent, rsync is awesome, but comparing it to version control is comparing apples to orangutans.
* to the parent, version control != git, and there are easy arguments to be made against git
* distributed version control is one of the greatest things to emerge in the last 20 years. If git isn’t your thing, check out (haha) fossil or mercurial
Recently I've seen a friend collaborate with teammate by sending each other zips. Using Skype! And as far as I can tell they're pretty productive. Dedication and hard work generally tends to produce results regardless.
When you use git you still need to use it right. How many times have you cloned a repo just to find out it won't build/run/compile without help? Personally, with small projects - more often than not. And I have a habit of asking if they do know for a fact it's going to build without assistance. It doesn't help.
I like git as a glorified change-log, slack commit messages especially. I try to keep everything tidy and build-able as much as anybody. But I'll still give people an archive (with .git included). And ironically this keeps git tidier as you don't obsess over whether or not to commit certain things.
When I first started learning how to program, I didn't know git existed. When I eventually learned a year later, I was already well acquainted with sending myself zips by email, and using email as "history". Git was quite scary to me at the time, (merges and reset/revert as I recall) so for a good year or two more I just kept using emails. It certainly wasn't pretty, but it required almost 0 mental cycles, worked without question, and actually gave some really nice features comparatively. (searching for a "commit" using complex queries)
We were eventually forced to use git/hg/svn for some courses over that time, so I became comfortable out of necessity and now vc for anything serious; had to unfortunately even use TFS for a while. (guilty secret, for many toy projects, I absolutely just have a folder on my fileserver and rely on its backup/replication, zipping the folder if I want to "snapshot".)
That's pretty easy. All you have to do is extract the first zip, go into the directory, run `git init; git add -a; git commit -m first` then unzip the second one and run... Hey wait a second.
Please don't try to sell the world on working this way. It's just terrible and lazy. You can learn enough git to use it productively in 5 commands. It's not even difficult.
I think you've entirely misread my post. I was giving a "cute" story concurring with the author that you can manage to be productive with duct tape and spit, and that it can be a learning period when understanding proper tooling, especially if one is coming in without any real oversight or mentorship at first; if you're reading this thread on HN you're probably far more exposed to CS than I was at that point. There's no "selling anyone" on anything.
I would however frankly say that I think the inability to recognize how intimidating git can be to an uninitiated was the _reason_ it took so long for me and many of my classmates; (this was over a decade ago, and I've been using vc professionally the entire interim) certainly far more damaging than anything you accuse me of propagandizing; It was extremely frustrating and dismissive to constantly be told by more experienced programmers that I should feel far more confident than I was about a tool that in hindsight, I don't look askew at myself for finding some aspects of unintuitive, and I'm a constant user now.
And, while I'm on this tirade, if you're addressing "working this way" as a negative to put unimportant toy projects on a heavily replicated and backed up server, I think you and I have different priorities in how we use our time.
Git's UX is horrible for people who don't know git, i.e. git beginners. All the documentation and error messages are written as if the user is already intricately familiar with the inner workings of git. As a result, it's very hard to learn because you don't know what you're about to do without hours of googling cryptic error messages.
Perfect example that happened to me today:
$ git push
fatal: The current branch insert_branchname_here has no upstream branch.
To push the current branch and set the remote as upstream, use
git push --set-upstream origin insert_branchname_here
What does it mean to "set the remote as upstream"? That phrase makes no sense to me. Is that going to overwrite master on origin? I have no idea, but I'll be damned if I am going to hit enter and risk overwriting my upstream branch, so I Ctrl+C out of there and had to read a bunch of awful reference docs posing as a tutorial.
Git is just plain terrible in almost every way. It just happens to be less terrible for certain workflows than CVS/SVN/mercurial/whatever.
I don't know. I've gotten away with using pretty much the commands in this guide for almost three years now (professionally): https://rogerdudler.github.io/git-guide/ - I even structure my orgs internal git documentation around these commands for newbies (with sources of course).
There are times when you need to troubleshoot a few things but most of the time you can just delete your local repo and reclone it - easy peazy. One of the other great things about git is that even if you do somehow fuck something up upstream it is really easy to get it back. Heck I think git is pretty awesome.
The command failed, with a clear error message, and a direct instruction on exactly how to redolve it. That is good UX.
Your example boils down to not knowing the word "upstream". I would argue that is table stakes domain knowledge for decentralized version control, and is trivially easy to search for. That does not make bad UX.
Good UX does not eliminate the need for domain knowledge, it eliminates incidental complexity from a user's task, and one could argue makes a tool pleasant to use.
I don't disagree that there are rough edges to git (the debate over merge vs rebase is a glaring example), but "just plain terrible in every way" is gross hyperbole.
Sorry but if the error gives you the right command to run first time then it should just run it first time by default. Sane defaults for new users is good UX. Not an error telling you exactly what command to run instead. It already knows what you intended, for Christ sakes, because it's telling you what you should copy and paste instead!
Well considering how many people who now know git and understand it, it doesn't make sense to do that necessarily. Rather they should make it a configurable option which do now see push.default (https://git-scm.com/docs/git-config)
But it doesn't know what you intended. It's guessing. The consequences of guessing incorrectly could be very, very bad, as you are pushing your code to a remote host.
Again, the whole point of git is that it is decentralized; there is no requirement that you have one single central server that you are pushing to; there could easily be multiple places that you might want to push something to.
Using something as default that makes sense for a centralized VCS when git is not centralized is, imho, worse than bad UX, as it teaches incorrect assumptions.
Again, this is table-stakes knowledge for using decentralized version control.
Yes, git’s UX is absolutely terrible. It is an abomination. It is the worst command line tool I’ve extensively used in that regard. For a long time I stubbornly used mercurial, which is vastly superior.
But, it has become a de facto standard. This is a tragedy. But it is a fact.
So it is best just to spend time learning it, find a subset of commands and make a cheat sheet you can refer to, and accept that the world is imperfect. Hopefully at some point in the future git will die and we will transition to the next big thing, and they will get it right that time.
There are many such things in the world of tech, and in most cases it is best to say “yes, it’s awful, but no, I can’t change it, and rather than expending energy on hating it (which is totally justifiable) I will expend the energy on minimising its damaging impact to my productivity and focus on accelerating my work elsewhere.”
I say this because I have spent so many hours ranting about git and every time it gets in my way (git submodules anyone?) I used to get so stressed by its design that it would hurt my productivity.
Be the Zen Dev: acknowledge it, isolate it, keep it at arms length.
Tips: Make a cheat sheet, use a minimal feature set, don’t try to do anything clever with it, and always have a way out (eg backups! Take copies of your directory before dealing with lesser know commands! Try things out on dummy repo’s before breaking your own, etc.).
I don't think it's a tragedy. UX can be fixed (even if difficult because of backwards compatibility and social concerns). Changing the underlying technology is much harder.
git has become a de facto standard because it gets everything else right.
Have you tried the official GitHub client? I've worked at a few places where non developers started to use it because the ability to collaborate through pull requests has so much benefit outside of development. I'm a huge fan of IntelliJ and use the inbuilt git client most of the time, it abstracts most of the complexity. Sourcetree is another good product.
Even if you only use Git for save points, it's game changing. I honestly can't imagine working without it. I'd be too afraid to change anything, in case I break it.
You can do some Googling for how to use them and contrary to popular opinion it’s really not much to it. Mercurial users are moaning about it, but Git has a better track record of maintaining backwards compatibility.
The advantage over SVN is in working with local or personal repositories. And forks of other projects in your local / personal repos, which happen all the time; heck, you will want to work with forks of your own repos for experiments; post-Git of course, b/c doing that with SVN is a pain in the ass.
Pretty much the same except that I use `git push` to deploy instead of `scp`. If I ever need to move beyond a single server that'll be an issue but for now it's fine.
Currently been working on a web app for about 8 months.
Python for the backend under the argument of "build using what you know". ReactJS for frontend development because Python for frontend work I find very antiquated.
Architecture wise I'm hosted in AWS. Data is stored in MySQL and Redis. All self hosted because I'm to cheap to pay per request pricing and the overhead is very minor when done right when rolling my own
My current indie web app is still in development so final numbers for prod are TBD. However I'm using it for
* Storing time counters on user actions (timestamp of last time a user posted/edited X, meant to be a throttling mechanism against abuse)
* Site content caching, around 20-50kb per page, each page being user generated content
Sizes will obviously vary on traction so not sure about the final numbers.
----
My last 9-to-5 employer was a very well known and my largest caching tier in Redis there was 512GB in a cluster configuration. I'm using the same server configuration and sharding logic for the indie thing, just on a smaller scale
React/Redux frontend and .Net Core for backend API. The site takes advantage of Server-Side Rendering and Lazy Loading based on Routes/components. Authentication is built into the App with JWT and Authorize attributes on controllers.
Python and Cython, flask, gunicorn, various ML frameworks, Postgres for web services.
Python with Cython because this allows extremely nice flexibility between concise, expressive code and low-level targeted performance optimization.
Flask with gunicorn has scaled extremely well for us, but there are many good alternatives. Postgres because flexibility with customizations and data types in the database has been the most important thing for us.
Go and, honestly, sqlite with backups written to S3. It's the absolute cheapest. I can run multiple apps on a t2.nano (t2.micro if I am feeling fancy). My apps cost something like $1.50 to run a month, and they can easily handle medium-sized traffic, plus Go is just so dead simple to deploy. scp a build binary, and boom I am done. Once the app grows and more collaboration/developers/requirements call for it, I will add more infrastructure.
net/http and REST api all the way down, I for the most part try to stick to the stdlib as much as possible. The few exceptions are logging (I like zerolog) the sql driver, and auth. I do keep the front/backend separation because doing site hosting in S3 with a cloudfront cache is pennies on the dollar (.63 cents a month) and I tend to lean towards using vue.js more these days. I have been known to just serve the html statically directly from go as well, and use go templates in the past, and use minimal jquery where it's needed (I still like jquery :-/, not everything needs to be full-blown SPA-mode)
I've recently decided to drop javascript frontends for a hobby project of mine and just go with bare go templates. Someone hopefully will laugh at it, but it has been a great, productive decision. It's been easy and quick again to slap ugly lists or tables of stuff together.
The entire headache of choosing a web framework, generating a seed of magnificient complexity, wrestling with the javascript build ecosystem, figuring out how to balance state between JS ui components, JS state management, backend state management and databases. Not necessary. Do an SQL query, put that into a neat datastructure and run html templating, followed by <go build>. Maybe add bootstrap for some neat CSS.
Sure, I don't have anything formal written up, but I think this might make a good topic for my first blog post. It really isn't complicated though in all honestly. I primarily stick to the stdlib, use a go sql driver for sqlite, I used to just run s3 sync on a cron but I am experimenting doing it in code now with the go aws lib, and just a couple bash scripts to "deploy" (I mean, really, it's go build, scp, and a webpack build + s3 upload for the frontend when that's needed. Go and sqlite really does most of the work here in terms of keeping things light and small.
My most complicated build required somewhat of an SOA approach, but rather than go heavy-duty microservices and use grpc where you need a bunch of load balancing and service discovery involved, I went with something simple called nats, that ended up costing me 5 dollars a month, unfortunately.
Thanks for sharing your setup here. Go and SQLite are great! I have a few questions :-)
Is the Go service directly accessible from Internet, or is it hosted behind a reverse proxy?
When you deploy a new binary, is there a small downtime between stopping the old binary and starting the new one?
How do you supervise the Go process? You use something like systemd?
When SQLite backup is ongoing, does it block writes to the database?
When you backup to S3, if an attacker gets control of your EC2 instance, is he able to erase your S3 backups, or is it configured in some kind of append-only mode?
Where do you store your logs and do you "read" them?
I will answer my own questions, in the context of the apps I develop and maintain:
- The Go service is hosted behind a reverse proxy (nginx or haproxy) to enable zero downtime deployments, by 1) starting the new process, 2) directing new requests to the new process, and 3) gracefully stopping the old process.
- Since we've started to use Docker, we let the Docker daemon supervise and restart our services. Before Docker, we used systemd. Before systemd was available on our system, we used supervisord.
- We thought about using SQLite for some apps. But SQLite can only have a single writer at a time, which goes against the zero downtime deployment described above (two processes can be processing requests at the same time). Thus we use PostgreSQL (and MySQL for legacy reasons) which provides online backups. Must be noted that online backups are possible with SQLite, provided the application implement it using SQLite Online Backup API [1]. Another solution, which doesn't require application cooperation, is to snapshot your disk, if your system supports this.
- We backup to rsync.net, which provides an append-only mode, through their snapshot feature [2]. An attacked cannot override or erase the snapshots of your previous backups. I think it's possible to do something similar with S3, albeit in a bit more cumbersome way, using S3 versioning and MFA deletion.
- About logs, we're still not satisfied by what we use currently.
I use multiplatform Kotlin to compile shared business logic to JVM and Javascript. The frontend then uses Typescript+React/Redux while the backend is a Kotlin Spring app with Postgres.
Not the OP but I started with cookie based auth and Spring Security and then moved to OAuth 2 - it’s extremely painless to setup (100 lines of code at most) but it did take a while to understand it all.
The Baeldung Spring Security course helped a lot.
If in a hurry though you can always use an external service like Auth0. I had that setup in an afternoon whereas understanding OAuth 2 and Spring Security took a week.
For getting web apps up fast, React, Firebase, Git, and Heroku. It's a well trodden path so the tooling is simple and the components are reliable.
For my VR stuff... Unity is really the only choice that's fast to get off the ground, so Unity it is.
For hacking together backend heavy stuff, Python. Flask if I need to expose an API endpoint. Postgres if I need a real database, sqlite if I don't. Deployed to my evergreen playpen virtual instance via Docker containers because it's, once again, a well trodden, simple, and undramatic path.
I'm curious, I've started doing development in unity but I find myself unaware of good patterns to use, any pointers to advice on unity design patterns?
Unity is pretty squarely in the space of event based programming, so whatever design patterns apply there are generally useful. Because of stuff like animations making timelines unpredictable, you should embrace the event based approach as much as possible.
Other than that, try your best to decouple game state from the actual objects that are flying around. You will never do it 100%, but the more egregious Unity code I've seen all shares the trait that they just jammed variables into scripts for objects without really thinking about how they'd tie together a distributed network of objects that have no easy way of addressing each other. The answer is that you don't: you still need a central game state.
Do you use firebase auth? It looks really good. I just had a heck of a time implementing user management and a couple oauth2 flows and it looks like it could have saved me a bit of time. My only concern is their plans are a little weird, it just jumps from free tier to jumbo size, wish there was something a little in between for less money.
Yup, I do use it. From a security and convenience standpoint it's just so much easier than whatever I could have come up with myself. But yes, the pricing sucks. I don't have a good answer there unfortunately.
Wow, lots of Vue mentioned! I was expecting mostly react, but surprised to see there was actually some pushback against it.
> I've heard React described as "10 times as much work for a 20% better user experience", and I think that's about right (maybe not 10x, but probably 2x)
I haven't done an official count (comments plus the site) but a very strong showing for full stack Rails or Django, and when people do stick in a front end it is usually React or Vue.
For the backend I start with Node, then get tired of the runtime errors and switch to Go. Go is cool but soon I get tired of writing “if err == nil” so I switch to Haskell. Haskell is nice but soon I get tired of not finding decent libraries for some not very popular things that I use, so I say fuck it and go to rails. Ruby is nice and Rails feels well thought out and solid, but then I can’t help be annoyed that it’s not fast, so fuck it, I switch to Rust. I get tired of lifetimes and wrapping everything in RefCells so I switch to Erlang. Erlang is cool and all, but no fucking meta programming. Sucks, let’s try Lisp...
I am not making fast progress, I think the problem is my keyboard, I should try buying that Ducky one I have been eyeing for quite a while.
Erlang with Ruby-like syntax plus macros..something like that. It compiles to the same VM as Erlang's and has almost all of the same semantics as Erlang.
You are right! I should stick with Scala. It will also give me the opportunity to learn a new language!
Play looks good, but maybe I should go with Scalatra instead? Should I be using Slick? It's quite different than any ORM I used so far, maybe I should bake my own Scala ORM. OK then, here we go. One more decision, REST or GraphQL? GraphQL seems to be the future. Let's go with that. Man, types are nice and Scala's type system is powerful, but sometimes I wish I had the flexibility of a dynamic language. It would make prototyping much faster.
May I recommend creating an AI to automatically rewrite your codebase into the flavor of the month, thus saving yourself significant amounts of labor which you could use to create an AI to improve the AI which is rewriting your codebase?
> types are nice and Scala's type system is powerful, but sometimes I wish I had the flexibility of a dynamic language. It would make prototyping much faster.
You can use both scala and clojure seamlessly.
I'm writing my first gui app in clojure for prototyping swing, but if I choose to rewrite parts of it in scala the rest of the clojure code would continue to work.
On a single project, I agree. But each time I start a new pet/side project I like to use a different stack. It's a good motivation to learn new technologies, and you learn more about them by using them "in anger" than you do in a purely theoretical environment.
Looking at nuxt.js, express and mongodb and then deploying customer pages as static pages on s3 rather than running these dynamically from the app. Picked up 100 customers in a single day so now I have to move quick.
Honest question. Is metaprogramming really that beneficial for any application development? I can certainly see the appeal, but how much will it improve productivity or quality compared to shipping features without it?
Isn't the actual wrong turn focusing on a metaproblem rather than the problem you set out to solve?
Metaprogramming is just macros: code that generates code. One use is to get rid of repetitive code patterns. Another is to move certain computations to compile time. Your statement that metaprogramming is used to solve a metaproblem is just wrong. Those two words are only related in that they share a prefix.
Third is to introduce new concepts into your language as if they were part of that language from the get-go. Which can entail both eliminating repetitive code patterns and doing computation at compile time.
I get what and why of metaprogramming. I was specifically trying to quantify working without it. My reference to metaproblem wasn't what metaprogramming you want to do but rather solving the problem of a lack of it by switching languages mid-development.
How often has a commercial project been rewritten to gain metaprogramming mid-dev? I added commercial because personal projects have different objectives.
Flask (hosted by pythonanywhere) and Neo4j from Graphene.
Chose both because I wanted consistency between the abstraction of my business logic, and the spec of my implementation.
I get some chippy guff about using graphs, but by taking the time to define my business logic as a grammar, expressing that grammar as a graph, then implementing it directly into the model, I get a lot of "aha," moments from potential customers.
Graphs can be analogous to functional languages in that if you are using them, there is a higher likelyhood you've reasoned something through before implementing it.
I actually wound up building most of my current project in Rust, on top of actix-web.
Partly because I'm apparently a masochist, but also because... I mean c'mon, modern architecture is CRUD and job queues. Rust can do that fine if you don't get distracted by the bells and whistles.
203 comments
[ 4.0 ms ] story [ 266 ms ] threadIt was a great--thanks.
Vue+vuex on the front end.
Gitlab CI/CD for deployment
Extremely fast dev/deployment turnaround time with Django and Beanstalk.
Vue/vuex is simple enough for me to understand and build SPA
Though I have been thinking if I should replace elastic beanstalk with Kubernetes, since it seems to become the non-proprietary standard for deploying containers?
SNS is "good enough" for my use cases and I don't have to run a beanstalk worker (like for sqs).
I chose beanstalk because of how dead simple it is. It is non-standard in a way, but was the fastest way to get the app out.
Again, I know the app may outgrow it's needs at some point. At that point I'll invest time to move it out to the flavor of the day:).
How do I separate what might be a side project to learn a new stack from a side project that might eventually make money?
I feel like this is just missing so much conte xt.
If you are working full time and can hack in your after hours to learn a new tech stack first and then start your project there might be an aggregate benefit over just starting the project right away using what you know.
Context matters to a huge degree in these kinds of discussions. I feel like people just like to give blanket statement advice.
Before you write a line of code, make a clear decision as to whether it's a learning experience or a business venture. Plan your project based on the goal you've chosen. If it's a learning experience, consciously avoid doing anything that doesn't push you to learn something new. If it's a business venture, consciously avoid doing anything unless you believe it has the highest RoI.
Some purely recreational projects do turn into viable businesses, but far more projects have failed due to indecision and yak shaving. If you're building a business, you don't gain anything by experimenting with a new stack. If you're learning React, you don't gain anything by writing a bunch of CSS and marketing copy. Don't be afraid to step away from the keyboard and ask yourself "Is this a good use of my time?".
Also build using what you know isn't always the best advice. Figure out what you want to optimize for and optimize for it. That may include doing things in a different stack.
I would have much preferred to use Perl, R, dotnet, or just about anything else, but there was an already made library in Python that was good enough for 90% of my needs, and 100% with some elbow grease, so my lazyness made me learn Python instead!
But wait- it gets dirtier than this! The code is done is Vi, with no care in the world about version control (rsync does most of what I need: backups), and deployed to servers by manual SCP, with no automatic tests, no care about CD or automatization - yet: as I need more and more servers, I realize I have to spend time learning say ansible to provision them faster than a copy paste of bash (I have low standards but I won't do curl|bash, just... no) but I am delaying that as much as possible to concentate on implementation.
The servers are on Debian VMs on Azure, just because they gave me free credits and do not perform as bad as AWS. When the free credits run out, I will move to DO. Then I will see what else I can use, and that will be the right time to care about ansible or whatever.
It is as ugly as I can get away with to get my MVP out, using tech from 1996 lol.
I get the simplicity...but using git/hg/svn is just so easy. And you can easily rewind, diff, view changelogs for a file, etc.
rsync is run by cron.
The other stuff is there if you need it though.
That being said, for GP's use, they can work solely from master, requiring only a few commands with little to no chance of difficulty.
The fact that there are so many people trying to explain it in their own way is a good sign.
When you're setting up for the first time:
When you've made a change you want to keep: When you did something wrong and want to go back to when you last ran git commit: Everything else builds on top of that workflow.Also, theres a decent half-step between ssh and Ansible: mssh. You can run ssh across a set of hosts quite trivially. It working on the order of a few dozen machines just fine.
Yet markdown is not essentially different from running sed to render something into html, which I would have totally done if I hadn't also needed some other features like basic math that would have been too tedious in sed
Speaking of HTML, I do my HTML sites all static, with no external scripts or resources! It is so 1994.. or so 2018??
I use vi. I do use git locally, but I don't trust it, so I also use rsnapshot, probably similar to your rsync. You might want to try rsnapshot some time if you have not. It uses less disk space by hard linking dupes using a perl script.
> People saying they wouldn't use git don't know git.
A few things:
* to the grandparent, rsync is awesome, but comparing it to version control is comparing apples to orangutans.
* to the parent, version control != git, and there are easy arguments to be made against git
* distributed version control is one of the greatest things to emerge in the last 20 years. If git isn’t your thing, check out (haha) fossil or mercurial
Edit: some reference reading. If it’s not already integral to your workflow, get familiar with distributed source control - you’re unlikely to regret it: https://en.wikipedia.org/wiki/Distributed_version_control
When you use git you still need to use it right. How many times have you cloned a repo just to find out it won't build/run/compile without help? Personally, with small projects - more often than not. And I have a habit of asking if they do know for a fact it's going to build without assistance. It doesn't help.
I like git as a glorified change-log, slack commit messages especially. I try to keep everything tidy and build-able as much as anybody. But I'll still give people an archive (with .git included). And ironically this keeps git tidier as you don't obsess over whether or not to commit certain things.
We were eventually forced to use git/hg/svn for some courses over that time, so I became comfortable out of necessity and now vc for anything serious; had to unfortunately even use TFS for a while. (guilty secret, for many toy projects, I absolutely just have a folder on my fileserver and rely on its backup/replication, zipping the folder if I want to "snapshot".)
(Of course, proper VCS wins any day.)
I would however frankly say that I think the inability to recognize how intimidating git can be to an uninitiated was the _reason_ it took so long for me and many of my classmates; (this was over a decade ago, and I've been using vc professionally the entire interim) certainly far more damaging than anything you accuse me of propagandizing; It was extremely frustrating and dismissive to constantly be told by more experienced programmers that I should feel far more confident than I was about a tool that in hindsight, I don't look askew at myself for finding some aspects of unintuitive, and I'm a constant user now.
And, while I'm on this tirade, if you're addressing "working this way" as a negative to put unimportant toy projects on a heavily replicated and backed up server, I think you and I have different priorities in how we use our time.
Perfect example that happened to me today:
What does it mean to "set the remote as upstream"? That phrase makes no sense to me. Is that going to overwrite master on origin? I have no idea, but I'll be damned if I am going to hit enter and risk overwriting my upstream branch, so I Ctrl+C out of there and had to read a bunch of awful reference docs posing as a tutorial.Git is just plain terrible in almost every way. It just happens to be less terrible for certain workflows than CVS/SVN/mercurial/whatever.
There are times when you need to troubleshoot a few things but most of the time you can just delete your local repo and reclone it - easy peazy. One of the other great things about git is that even if you do somehow fuck something up upstream it is really easy to get it back. Heck I think git is pretty awesome.
Your example boils down to not knowing the word "upstream". I would argue that is table stakes domain knowledge for decentralized version control, and is trivially easy to search for. That does not make bad UX.
Good UX does not eliminate the need for domain knowledge, it eliminates incidental complexity from a user's task, and one could argue makes a tool pleasant to use.
I don't disagree that there are rough edges to git (the debate over merge vs rebase is a glaring example), but "just plain terrible in every way" is gross hyperbole.
I'm arguing against the idea that showing an error message with the exact command to use is good UX - see the original comment. It is not.
Again, the whole point of git is that it is decentralized; there is no requirement that you have one single central server that you are pushing to; there could easily be multiple places that you might want to push something to.
Using something as default that makes sense for a centralized VCS when git is not centralized is, imho, worse than bad UX, as it teaches incorrect assumptions.
Again, this is table-stakes knowledge for using decentralized version control.
Yes, git’s UX is absolutely terrible. It is an abomination. It is the worst command line tool I’ve extensively used in that regard. For a long time I stubbornly used mercurial, which is vastly superior.
But, it has become a de facto standard. This is a tragedy. But it is a fact.
So it is best just to spend time learning it, find a subset of commands and make a cheat sheet you can refer to, and accept that the world is imperfect. Hopefully at some point in the future git will die and we will transition to the next big thing, and they will get it right that time.
There are many such things in the world of tech, and in most cases it is best to say “yes, it’s awful, but no, I can’t change it, and rather than expending energy on hating it (which is totally justifiable) I will expend the energy on minimising its damaging impact to my productivity and focus on accelerating my work elsewhere.”
I say this because I have spent so many hours ranting about git and every time it gets in my way (git submodules anyone?) I used to get so stressed by its design that it would hurt my productivity.
Be the Zen Dev: acknowledge it, isolate it, keep it at arms length.
Tips: Make a cheat sheet, use a minimal feature set, don’t try to do anything clever with it, and always have a way out (eg backups! Take copies of your directory before dealing with lesser know commands! Try things out on dummy repo’s before breaking your own, etc.).
I don't think it's a tragedy. UX can be fixed (even if difficult because of backwards compatibility and social concerns). Changing the underlying technology is much harder.
git has become a de facto standard because it gets everything else right.
I feel this sentiment so often in this field. It’s really distressing.
Learn some Git and stick to it, it’s a one day job. And most of the time you’ll use these commands
You can do some Googling for how to use them and contrary to popular opinion it’s really not much to it. Mercurial users are moaning about it, but Git has a better track record of maintaining backwards compatibility.The advantage over SVN is in working with local or personal repositories. And forks of other projects in your local / personal repos, which happen all the time; heck, you will want to work with forks of your own repos for experiments; post-Git of course, b/c doing that with SVN is a pain in the ass.
I'm not in web apps.
Python for the backend under the argument of "build using what you know". ReactJS for frontend development because Python for frontend work I find very antiquated.
Architecture wise I'm hosted in AWS. Data is stored in MySQL and Redis. All self hosted because I'm to cheap to pay per request pricing and the overhead is very minor when done right when rolling my own
* Storing time counters on user actions (timestamp of last time a user posted/edited X, meant to be a throttling mechanism against abuse)
* Site content caching, around 20-50kb per page, each page being user generated content
Sizes will obviously vary on traction so not sure about the final numbers.
----
My last 9-to-5 employer was a very well known and my largest caching tier in Redis there was 512GB in a cluster configuration. I'm using the same server configuration and sharding logic for the indie thing, just on a smaller scale
Python with Cython because this allows extremely nice flexibility between concise, expressive code and low-level targeted performance optimization.
Flask with gunicorn has scaled extremely well for us, but there are many good alternatives. Postgres because flexibility with customizations and data types in the database has been the most important thing for us.
The entire headache of choosing a web framework, generating a seed of magnificient complexity, wrestling with the javascript build ecosystem, figuring out how to balance state between JS ui components, JS state management, backend state management and databases. Not necessary. Do an SQL query, put that into a neat datastructure and run html templating, followed by <go build>. Maybe add bootstrap for some neat CSS.
My most complicated build required somewhat of an SOA approach, but rather than go heavy-duty microservices and use grpc where you need a bunch of load balancing and service discovery involved, I went with something simple called nats, that ended up costing me 5 dollars a month, unfortunately.
https://blog.sgmansfield.com/2016/06/how-to-block-forever-in...
Is the Go service directly accessible from Internet, or is it hosted behind a reverse proxy?
When you deploy a new binary, is there a small downtime between stopping the old binary and starting the new one?
How do you supervise the Go process? You use something like systemd?
When SQLite backup is ongoing, does it block writes to the database?
When you backup to S3, if an attacker gets control of your EC2 instance, is he able to erase your S3 backups, or is it configured in some kind of append-only mode?
Where do you store your logs and do you "read" them?
- The Go service is hosted behind a reverse proxy (nginx or haproxy) to enable zero downtime deployments, by 1) starting the new process, 2) directing new requests to the new process, and 3) gracefully stopping the old process.
- Since we've started to use Docker, we let the Docker daemon supervise and restart our services. Before Docker, we used systemd. Before systemd was available on our system, we used supervisord.
- We thought about using SQLite for some apps. But SQLite can only have a single writer at a time, which goes against the zero downtime deployment described above (two processes can be processing requests at the same time). Thus we use PostgreSQL (and MySQL for legacy reasons) which provides online backups. Must be noted that online backups are possible with SQLite, provided the application implement it using SQLite Online Backup API [1]. Another solution, which doesn't require application cooperation, is to snapshot your disk, if your system supports this.
- We backup to rsync.net, which provides an append-only mode, through their snapshot feature [2]. An attacked cannot override or erase the snapshots of your previous backups. I think it's possible to do something similar with S3, albeit in a bit more cumbersome way, using S3 versioning and MFA deletion.
- About logs, we're still not satisfied by what we use currently.
I'd be curious to read about what others do :-)
[1] https://www.sqlite.org/backup.html
[2] https://www.rsync.net/resources/howto/snapshots.html
The Baeldung Spring Security course helped a lot.
If in a hurry though you can always use an external service like Auth0. I had that setup in an afternoon whereas understanding OAuth 2 and Spring Security took a week.
For my VR stuff... Unity is really the only choice that's fast to get off the ground, so Unity it is.
For hacking together backend heavy stuff, Python. Flask if I need to expose an API endpoint. Postgres if I need a real database, sqlite if I don't. Deployed to my evergreen playpen virtual instance via Docker containers because it's, once again, a well trodden, simple, and undramatic path.
Other than that, try your best to decouple game state from the actual objects that are flying around. You will never do it 100%, but the more egregious Unity code I've seen all shares the trait that they just jammed variables into scripts for objects without really thinking about how they'd tie together a distributed network of objects that have no easy way of addressing each other. The answer is that you don't: you still need a central game state.
> I've heard React described as "10 times as much work for a 20% better user experience", and I think that's about right (maybe not 10x, but probably 2x)
I thought this was a gem.
Tempted to learn a 2nd language well, thinking Go but curious about the experience of Kotlin + IntelliJ
Apps: Flutter and if I need it switch to native. But flutter is really good for my simple needs!
I am not making fast progress, I think the problem is my keyboard, I should try buying that Ducky one I have been eyeing for quite a while.
Play looks good, but maybe I should go with Scalatra instead? Should I be using Slick? It's quite different than any ORM I used so far, maybe I should bake my own Scala ORM. OK then, here we go. One more decision, REST or GraphQL? GraphQL seems to be the future. Let's go with that. Man, types are nice and Scala's type system is powerful, but sometimes I wish I had the flexibility of a dynamic language. It would make prototyping much faster.
Fuck it, "git checkout -b node_v2"
You can use both scala and clojure seamlessly. I'm writing my first gui app in clojure for prototyping swing, but if I choose to rewrite parts of it in scala the rest of the clojure code would continue to work.
There are several ways. using clojure in a scala project: https://github.com/Geal/sbt-clojure using scala in a clojure project: https://github.com/technomancy/lein-scalac jvm polyglot with maven: https://stackoverflow.com/a/9676895
example: https://blog.michielborkent.nl/blog/2016/07/26/clojure-from-...
See now this is where you went wrong. Should have switched to Nim at that point. Can't beat it's familiarity, speed and metaprogramming.
Isn't the actual wrong turn focusing on a metaproblem rather than the problem you set out to solve?
How often has a commercial project been rewritten to gain metaprogramming mid-dev? I added commercial because personal projects have different objectives.
I use PHP for prototypes and Go for serious stuff.
Chose both because I wanted consistency between the abstraction of my business logic, and the spec of my implementation.
I get some chippy guff about using graphs, but by taking the time to define my business logic as a grammar, expressing that grammar as a graph, then implementing it directly into the model, I get a lot of "aha," moments from potential customers.
Graphs can be analogous to functional languages in that if you are using them, there is a higher likelyhood you've reasoned something through before implementing it.
https://github.com/HugoDaniel/gridgenerator
But the video producer/converter is also in TS (in that same repo). It produces your making-of videos like these:
https://gridgenerator.com/static/37.mp4
Partly because I'm apparently a masochist, but also because... I mean c'mon, modern architecture is CRUD and job queues. Rust can do that fine if you don't get distracted by the bells and whistles.
I ripped out and open sourced my user-module stuff for authentication handling, if anyone's interested: https://github.com/ryanmcgrath/jelly