Exactly! Many of the things mentioned in the post aren't really helpful in building a web app from scratch. In fact, Go simply presents way too much overhead for building a webapp from scratch.
I see Go as an optimization tool when concurrency is needed, the same way I would consider using it to break some part of the app when it gets way too big. It works very well for those purposes. But I would never consider Go for building a web app from scratch.
Is djangos default authentication and authorization [1] not enough (since you mentioned django in your post)? If so you could take a look at django-allauth it seems [2] or maybe at Stormpath [3] (external service) or ABAC [3] (which is probably total overkill, too enterprisy and is only used for authorization as far as I know).
Even more people are making good money developing in the language/tooling they know/favor, while it is (comparatively) poorly suited to the problem at hand.
You forgot jumping on the hype bandwagon and getting a piece of the hype-rush.
Seriously though, Go is a really good language. Its a simple yet solid & powerful language, but why would people feel the need to present it as the silver bullet for everything?
My web development experience, apart from an experience with TCL that teached me never to rely on scripting languages for serious web applications, is on the JVM and .NET stacks.
As someone, who is right now debugging an .ear, my preferences right now would have JVM or NET stack at the end, behind all scripting languages imaginable.
By day I make good money using C# where I get all of the above at what seems to be a much more web appropriate level than Go.
One of the things that I'm really liking about the ASP.Net Core rewrite is that a framework that has been around for ages is taking an opportunity to look at what it does well, what it doesn't and what other languages / frameworks have innovated on more recently and stealing liberally.
C# has been through several cycles of how to write concurrent code now and offers both Task and async/await depending on your needs. Not only is web sockets support there but SignalR is there as a useful API for common cases. If you want even better static typing you can switch to F#.
Not everyone was doing web development using single threaded dynamically typed languages before Go came along.
My comment was more about Go posts on HN being upvoted like crazy whenever they appear rather than any specific issue with your post. It's like a magic recipe for upvotes around here.
It feels like the author made the jump straight from PHP to Go without spending much time actually using Node or Ruby and he's subsequently shocked at how much better modern languages are when contrasted to PHP.
Many of these "solved problems" have also been solved with the other languages, albeit not always at the surface level. For Node specifically:
WebSockets -> SockJS, Socket/Engine.io
Concurrency -> Generator Functions
Testing -> import assert from 'assert' and npm test
Syntax -> Some great improvements with ES6 and solid styleguides (airbnb, standard) + eslint as an autoformatter
The major win here for Go is that as a new platform it was designed with simplicity in mind. These features are probably easier to pick up in Go than any of the alternatives, which is great. However, don't make the mistake of thinking that Go is the first language to solve these problems.
Author here, I have used both Node and Ruby fairly extensibly. I think I touched upon Ruby's performance issues in the post. That's why I don't consider Ruby a solid solution. Also, as far as I'm aware, Ruby doesn't have built in concurrency like Go does. Yes, Nodejs is close to being concurrent, but it really doesn't hold a candle to Go. Yes Node has socket.io, something I've used a lot and love. But Go has all of those features natively.
Not yet, I've got as far as reading the Elixir docs start to back, I really enjoyed what I saw, I'm yet to have a play around with it though. I'm hoping to this weekend :)
I looked at it a while back. Even played around with Elixir, which I liked as a language quite a lot. But the deployment and configuration for Elixir/Phoenix/Ecto was pretty horrendous. Lots of dependencies, lots of tweaking, I think even NPM was required too?!
I do have fun bashing Go, but I also use it. It's too practical not to. And the Go team, above all other of the latest wave of language that are ready for production, got that 100% right. I just can't overstate how simple Go makes setup, deployment, and updates. And how complex Elixir is in comparison.
I wish Elixir et al. could have a zero dependency, zero external configuration, and single/xcopy file deployment too.
Using exrm its a very simple setup. You get a deployable tar file which gives you the ability to migrate up and down as well. The only issue i have is wrt to running migrations on deploy server.
Go is very good for services. But i didn't find any mature libraries like Phoenix.
Exrm looks good for updating an already established application. Doesn't look like it does anything for the initial setup and configuration of a new system though.
For me, the ultimate test is can I copy my application to a flash drive, walk it to a new clean system, and "paste" it to that new system. Go really does let me do that.
> But i didn't find any mature libraries like Phoenix.
Yup. But then I tend not to need larger frameworks with my Go apps. Like you allude to, Go apps tend to be less monolithic.
> For me, the ultimate test is can I copy my application to a flash drive, walk it to a new clean system, and "paste" it to that new system. Go really does let me do that.
That used to be the case for erlang deployments that looked pretty similar, and Exrm looks like it does the same thing:
There are five stages to creating and deploying a first release. The first is just generating it, the next three are for moving the release to a new place and the third is just decompressing it.
Clearly I'm not as up to speed as I might be. How do I avoid installing and configuring Erlang, Elixir, Phoenix, etc. on each machine I want to "paste" to? How do I bundle all these into one extract and go package?
Then definitely more interesting than my cursory look. I don't see any mention of bundling the runtime but I'll take your word and dig around a bit more. Thanks for the info.
If its the same architecture then you can copy the tar file to another system and run it. Its the same for Go apps except you get a binary.
I do not want to recreate all the structure( I basically do web apps). Phoenix provides good defaults. If it was few lines of code then Go is fine, for larger project i prefer Phoenix( prefer might not be the right word since i don't have elixir prod apps while i do have Go apps)
I think that would be about 20 lines in Java. You've got to make a `HttpUrlConnection`, then call `setDoInput(true)` (or is that `setDoOutput(true)`), wrap it in a `try`, etc. etc. etc.
> I think that would be about 20 lines in Java. You've got to make a `HttpUrlConnection`, then call `setDoInput(true)` (or is that `setDoOutput(true)`), wrap it in a `try`, etc. etc. etc.
Nothing prevents you to wrap those 20 lines in a generic method.
Also there are other options in Clojure, Scala, Kotlin, Groovy besides pure JDK API calls. Here Scala style, easy to mimic in Java 8
val result = http.postData("http://example.comupload").header("Content-Type", "image/jpeg").responseCode
> Go's standard library is amazing.
So amazing that is doesn't have proper database drivers for the SQL databases that matter.
I recently rewrote one of our services in Go and there was a SQL interface in the standard library with backends for all SQL databases that matter. I don't remember if the backend we used (for PG) was from the community or the standard library, but that's rather inconsequential.
Also this is non blocking, while the go code will bock, by default. there is even akka-http now which could be wrapped really great which is streamed by default, even better.
Nothing prevents you from writing an entirely new Java library.
The point is Go's library already has all the nice convenience methods. With Java I would be endlessly writing them myself, and even then they wouldn't be as good as Go's because they wouldn't be standard (no help on StackOverflow for example).
Actually it is easier to write generic code in Java than in Go.
Also many Java bashers in the Go community seem to be unaware of what is delivered in the JDK.
When I used to daily read gonuts, many weren't even aware that since Java 6, there is a similar infrastructure for HTTP(S), no need for third party servers.
Either you make a method of it and then you can call it in 1 line, or you can use some libraries like OkHttp from square which will let you do that with only 5 indented lines, not 20+.
Request request = new Request.Builder()
.url("http://example.com/upload")
.post(RequestBody.create(IMAGE_JPEG, file)
.build();
Response response = new OkHttpClient().newCall(request).execute();
But yes Go's standard library is cool, but Java's ecosystem is too, and larger.
.net might be good I have not used that. But as Java developer I do find this quote apt.
'I had a problem so I start using Java now I have 2 problems.'
In my experience when a project makes the decision to switch to jRuby it's time to think about breaking parts of it into another language, you end up having to write Java in Ruby and subjecting yourself to the worst of both.
> Ruby doesn't have built in concurrency like Go does
Seriously. Pretty much all languages have support for concurrency via threads. Ruby even have the equivalent of channels since a decade: http://ruby-doc.org/core-2.2.0/Queue.html
Now if you meant to talk about the GIL, well it prevent parallelism only for purely CPU bound tasks. And in the context of a web app, if I have a CPU bound tasks that I can just fire and forget, I don't want to execute it on my frontend server anyway.
Sorry, your definition of "concurrent" is all over the place.
Ruby doesn't have "built in concurrency"? Ruby has fully native threads; it just has the GIL that blocks for IO (but you could use jRuby). It gives you several options for message passing. You also have the extremely under-rated dRuby system. Not to mention the whole EventMachine world.
Node is "close" to concurrent? It's god damn single-threaded! At the very least it's possible to spawn multiple processes and communicate over a Unix socket, good luck doing something like that with PHP. What does socket.io have to do with anything?
Sorry man, I'm hearing a lot of cargo cult opinions and precious little data to back it up.
This was one of the more accessible Go articles I've found through hn but it would really help me if these types of examples included well-written implementations in other languages and frameworks (including third-party library implementations, if required). Does anyone have a link to an article or tutorial like that?
"Or you can use channels, if you need a little more control." that means the guy doesn't even understand the `share memory by communicating` model of go
Author here, as I mentioned in the post I'm fairly new to Go still, so some of the edicts are unfamiliar to me. So apologies for any mistakes. This post was very much a 'why I really like Go over PHP these days'.
I haven't used Go, but have spent a lot of time in Scala. Any statically typed language means increased pain with JSON. Static typing really has made me appreciate a well thought out JSON data model where each key in a document can have one and only one type as its value.
That said, as someone who spends a lot of time in Python as well, I can't express how much I love the deep autocompletion powers and dumb-mistake suppressing abilities of static typing.
It is a bit of a pain, but I've found it to be a worthy trade-off for static typing. Finding documentation on the json struct tags was a lot more difficult than it should've been.
Not total pain, but certainly loses in convenience to python. Deserializing large and arbitrary JSON inputs is trivial with python's `json.loads()`. Element access rules have no dark corners (as long as you don't have pickled data in there).
In contrast, Go and its `json.Unmarshal()` aren't that well suited for parsing arbitrary and/or complex inputs. However, if the expected input is well formed and can be directly mapped to a predefined datastructure, the convenience factor is pretty high.
So, from my limited experience, I would say that JSON parsing in golang is more geared towards deserialising well defined transport formats than ingesting arbitrary inputs.
> In contrast, Go and its `json.Unmarshal()` aren't that well suited for parsing arbitrary and/or complex inputs. However, if the expected input is well formed and can be directly mapped to a predefined datastructure, the convenience factor is pretty high.
I think go revolves more around well-defined structures, and JSON is anything but. For inter-process pipelines, protobuf (polyglot) and gob (go-only) work better in my experience.
Yes, it's pain! Unless you have the struct ready for unmarshalling json. I use gojson [https://github.com/ChimeraCoder/gojson], it auto generates struct from json with a simple command.
I have spent a lot of time working with PHP and have also recently started using Go. The comparison between the two in this article seems shallow and kinda comes across as simply jumping on the PHP-bashing bandwagon.
tl;dr Someone who doesn't understand his tools chooses PHP, and then he tries Go and likes it. But he still doesn't understand his tools. For examples, he "likes the idea of" typing, but never explains why (or why not) others should feel the same way.
Exactly. I really think so many people happy with go would be way happier with C#. I see one of the reasons people don't adopt C# is "because Microsoft", which is sad, because it's such a great language.
more and more clients specifically required Linux at the start of conversation, this ruled out C# despite our 10+ years of experiences on MS products in our industry. (also ruled out Mono, we decided to transform and adapt and there are far better choices)
i guess it depends on what one means by headaches. mono on linux has never been a significant interest for xamarin, and core-clr is nowhere near production ready.
maybe if jetbrains' project rider gains traction and core-clr and/or mono-on-linux matures it will be a broadly appealing option.
I've spent years coding C# apps and really like the core language. But .Net app dependencies and deployment are a huge pain, especially if you decide Linux might be a cheaper way to go. However, the biggest problem with .NET is the huge overhead of having to learn some new framework every time you want to try something new. Months and maybe even years of wasted time learning MVC, MVVM, Sliverlight, WCF, Remoting, etc. etc. I cringe every time Microsoft announces some new framework as the next standard for .NET apps.
Also, bringing a developer without .NET experience into a project means 6 months before they can do most anything and a good year to be truly up to speed. The ramp-up time for both C# and Java applications are now about as long as C++.
I had high hopes for .NET core. I saw it as a way to start over creating a Go like situation on top of the C# language. But it's still about two years from being production ready and its already gotten complex and bloaty. Just to install the development tools and environment means multiple downloads, dependencies, and install scripts. Download a zipped file, extract to the directory of your choice, and set a few environment variables. That's the way to ensure reducing complexity.
Actually currently C# isn't easy to use on Linux and Mac, however this will change soom (tm), especially when Project Rider is ready. I'm really happy if I can try out the EAP and work a bit with C# .NET core.
I'm still not get the go rant. I mean it's not bad, but it's not great either.
Programming in Java8 could be way easier than with go and has a better Concurrency model (CompletionStage), it sometimes could be overly verbose however that's not always a problem.
One thing that always seemed pretty hard to do in Go was security.
Most companies will stop at SQL Injections but I consider defense against XSS, and CSRF attacks just as important. And that is something Go does not excel at because I have not found any framework that makes adding these features as painless as possible. (I have only toyed with Go for the web so I might be wrong).
Django (a python web-framework), on the other hand, is pretty good at these things[1] because they require little to no extra work, so that junior developers like me will get it right. That is the reason I would always go to Python for web development instead of Go, although, I like Go just as much and it is so much quicker.
I'd be interested to hear how the big projects migrate these kind of attacks in Go.
I may be mistaken, but I'm reasonably sure with respect to XSS (in HTML templates anyway), the built-in html/template autoescapes provided parameters by default, much as Django templates do.
You have to mark "trusted" or "pre-escaped" parameters explicitly by wrapping them in e.g. `template.HTML`.
nosurf [1] is a pretty popular CSRF protection package for Go. You need to realize that Go's built in "web framework" is extremely minimal unlike Django. It's more comparable to something like Flask, which also doesn't have CSRF protection built-in.
As someone who is fairly enthusiastic about Go and has implemented a few livelihood-providing production systems in it I am baffled by the number of people who keep trying to use it for application (and especially web application) development.
I mean it does feel like superpower when applied to tasks you'd previously have to use C for. But have you tried any of the modern web development stacks? They are like a zillion times more productive. Hint: PHP is not one of those.
I guess some people just can't get by without the iron hand of the compiler :)
I'd say PHP is definitely one of those. Modern PHP web development is pretty productive and definitely high quality.
EDIT after reading the article: I do have to agree that concurrency in PHP is a bit of a pain, but those 3d party tools are pretty easy to use and maintain.
> Without significant forethought to several levels of caching; such as in memory caching, op caching ...
This is so trivial to setup it's not even worth mentioning.
> Sure, PHP has PSR standards etc, but they're fairly new and developers have been slow to adopt them.
Everyone I know and every library I use uses those standards. PHP is pretty standardized now, especially compared for example to js.
PHP frameworks work well out of the box and contain solutions for most use-cases.
Go is nice but when I tried it I really missed something like composer. "go get" and commit your dependencies to the repository? Seriously? That's wrong on so many levels. Meanwhile I think composer is one of the best package managers around.
(Actually, I do that from the start of the project, so that adding something as pedestrian as concurrency doesn't involve a complete port to another language.)
"It is, but the overwhelming majority of tasks in a web app don't happen concurrently, so it doesn't matter."
But is that cause, or is that effect? The dominant web programming languages for the last 20 years have all been single-threaded, so of course the web stack looks single threaded. Personally, I find there's a ton of things that ought to be concurrent but aren't. I mean, most other network apps need to do things concurrently, why not web?
Once you start looking at it this way, the staggering number of hacks employed to pry concurrency out of fundamentally single-threaded languages is astonishing.
Go is not the only solution to this, there are many. (Go is one of the more Algol-ish language choices that doesn't involve learning a new paradigm, though, and can make it a relatively easy sell where "Let's all learn Lisp!" or "Let's all learn immutable functional programming!", where "all" may be dozens+ people, may not happen. [1]) But I think in five or ten years, "the overwhelming majority of tasks in a web app don't happen concurrently" is going to sound very 200x-ish.
[1]: Incidentally, for those who are still baffled by Go's success, notice how all the alternatives involve trying to teach everyone a brand new paradigm. While that has its advantages, when it comes to language popularity, not requiring that has its advantages too. I'm a polyglot myself, so I get how Haskell is nice and Clojure is nice and how they open your mind to new paradigms etc. etc., no sarcasm, fully mean it, if you only know C-style languages I fully recommend you go fix that soon, but when you're trying to get a team of people on the clock to switch languages for a nicer concurrency story, cutting the requisite training time by 90%+ raises the bar very high for those other languages, in relative terms. (Even moreso than it sounds; not only is training costs less, but the risk of training failure is also that much less. It compounds.) I do not mean this as "advocacy", I mean it as explanation.
Personally, I find there's a ton of things that ought to be concurrent but aren't.
What sort of things? I've been writing web software for ~20 years, and the vast majority of things that I've made boil down to "request comes in > validate input data > transaction to database > check transaction worked > write a response". Concurrency has never really been a problem, or even something I've wanted.
Sure, there are plenty of CRUD apps out there. But an awful lot of apps end up getting jammed into that pattern just because it's the standard paradigm, and not because it's the right one.
Just this week I met with a friend who wanted an app for community meetings that would let people express their preferences in real time, changing their expressed opinions as discussion happens. That way everybody could see how the consensus was shifting in response to discussion without people having to say it out loud.
I quickly whipped together a prototype, and I intentionally didn't use a database because I wanted this to be dynamic. Whenever one person changed something, everybody else needed to see it immediately. That's very easy in a context where you have all your data hot in RAM in one process with easy concurrent-ish access to it: you have a bunch of observer objects that get notifications each time you change something. In database-land, it's a pain.
And it works for me at size as well. Years ago I led a team where we were building a new web product. We were big on test-driven development, and databases are a notorious pain point. We started out without one, figuring that we'd pull one in when we needed it. We ended up never needing one. Our system had incredibly low rendering times (<5ms, even on 2005 hardware), and its 1800 unit tests ran in a couple of seconds. Starting with concurrency saved us so much development pain.
Another common example of where concurrency is valuable is in notifications. Quora and Facebook both have excellent notification systems: dynamic and instantaneous. Twitter's are much rougher, and it's clear they've put a lot less thought into concurrency.
Games, of course. Imagine trying to build agar.io without concurrency. Or look at Google Docs: concurrency is at the heart of what makes them valuable in an office setting.
There's nothing wrong with CRUD apps. There's nothing wrong with all the mainframe batch-processing apps that preceded them, either. But there's more that we can do.
I still don't like it but the landscape definetely changed, and it keeps getting better.
> Meanwhile I think composer is one of the best package managers around.
Apart from the fact that it uses gigabytes of ram on real projects. At work we ended up using it outside virtual machines, to later copy back the dependencies.
Are you committing the composer.lock file in your projects? Likewise you should try and use as close to absolute version numbers as possible in your composer.json `requires` sections.
I used did a quick test (having run `composer clearcache` each time) on one of my projects:
Without composer.lock:
Memory usage: 75.96MB (peak: 87.5MB), time: 38.54s
With composer.lock:
Memory usage: 7.47MB (peak: 8.99MB), time: 28.6s
One of not negligible size. They usually have many dependencies. I can see on our composer.json around ~60 dependencies, and the number will only grow (apart some lucky circumstances).
Ditto. I used it for something that I would have written in C (and had prior). A glorious language. Gives safety to places where pointer arithmetic and function pointers adds danger while keeping structs and arrays of primitives. Plus great libraries and memory management.
And then I thought "why not use it for something web-like". My experience was dreadful. After coming from Python+Django, it felt like writing Java 1.4 (not a good thing). Especially for returning JSON objects with lots of mixed types.
So I switched to Clojure and never looked back (Python would have been an equally valid choice).
Go's a great language, but it's being applied in some very odd (and potentially unsuitable) places.
I had the same experience as you. Django (and RoR, I guess?) blows everything else I've tried out of the water in terms of ecosystem, and Python is a joy to program in. Go is fantastic when I want either a small service or a script that I want to deploy as a single executable and have it be reasonably fast, but I find Python much easier to write in. However, that may be because I'm more familiar with Python.
How did you like Clojure? Does the ecosystem it compare well against Django? I tried it once during an on-site interview but was put off by the multi-minute startup times.
It's unfair to compare a framework like Django to a language like Clojure.
So, comparing the languages, I tried Clojure because a colleague was using it and to me it feels like the best language I have ever used professionally (comparing to C#, C, Obj-C, Python, Java, JavaScript, Ruby, Go) and I'm very happy using it. Great functional programming features, great libraries, I really like the LISP syntax and the stress I'd been having with Python kind of melted away (things like state in objects and the type system (which kind of doesn't really exist most of the time with Clojure)).
In terms of doing web type things, the Liberator and Ring libraries feel a lot like a big chunk of Django's routing and middleware functionality. Selmer does a good job of templating.
For most user interface work I'm using React.js (which I also really like) so my Clojure code is only serving up JSON on an HTTP API. But for simple templated HTML, Selmer works well.
One question is "Django is an entire framework. How does it compare to lots of little libraries?". I have to say that Django is probably the most well constructed pieces of software I have worked with and it hangs together very well. Clojure with a few libraries is very good, but beating Django is a very high bar to reach for any framework or group of libraries.
Startup time is an over-done argument. Once you're running a REPL you don't need to restart it, so development isn't slowed down. Once you're running a server you don't need to restart it, so production isn't slowed down.
I see, thanks. The startup time problem was because of tests, they had to re-run every time and the suite took multiple minutes just to start running.
Since you work with React, how do you like JS? I'd imagine that, coming from Clojure, you'd be frustrated with its inconsistencies, as I am, coming from Python.
I'm running Python + Django with uWSGI. Building the app is a pleasure. With type hints in Python 3.5, the code becomes almost as maintainable as statically typed languages too.
One problem I have is that the application is extremely CPU intensive. I can't get past 35-40 requests per second with 500 concurrent users (at 4 CPU cores, 14 GB RAM), which seemed too expensive economically. (I cached as much data as possible, both at the Nginx tier and with Redis) and tuned the number of uWSGI worker processes.
Do I have to try other languages or do you think I have more room for optimization with Python?
Presuming you've actually profiled it to know its CPU intensive, or you're in a field where this 'goes without saying' (e.g. you're doing a bunch of math calculations within Python)....
Then I'd suggest you use Cython for a speed up or try using PyPy which can be 200% faster without any changes.
When I used Python and worked with financial data, every live algorithm would be recoded as a Cython extension.
Without a doubt you can serve more traffic than that with python, more than likely, the bottleneck isn't your language. There's a good chance if all you did was port your codebase to another language, you'd have the same basic usage profile, maybe +/-50%.
Having done web development for over a decade. My experience is that algorithm is far more important than language. After that data structures trumps language. After that doing work you don't have to do trumps language.
First look for really ugly nested loops in your code. I can't tell you the number of times this type of things ends up in your codebase, especially since dev environments often have a small subset of data. Even an O(n!) algorithm is fast for small values of n.
Next look to see what kinds of datastructures you're using. For instance, dicts are a seductive way to store data, they have a theoretical O(1) lookup, but they have a drawback of randomizing memory lookups, which means that for subsequent looks for a large dict, you'll end up with lots of cache misses, can make each lookup 1000 times slower. Meanwhile, a tuple has O(n) lookups, but if you're finding that you need to iterate over the data, you're going to benefit from memory locality. So know your tradeoffs. Also code that had one usage profile at launch, can often have a very different usage profile a year later, so it doesn't hurt to revisit this occasionally.
Last look for code that isn't doing anything. Are you calling the database twice in a row, asking for the same data, and not using it? It sounds obvious, but over time these kinds of things have a way of showing up in a codebase, and they can really add up.
I also should mention caching at this point. It can be incredibly hard to get caching right, but the performance savings are pretty big. It goes under the bucket of not doing things that you don't need to. Do whatever you can to do page caching. Putting an nginx cache in front of the webserver serving python for the majority of your pages could easily get you to hundreds of requests per second.
The point is, it is broad. You gained by using it in a traditional case where earlier you would have used C.
But I've used it in the web app scenario. Replacing Java (jetty/tomcat servlets with pure Java code) with Golang. My own discovery was it could replace the jetty/tomcat with servlets part seamlessly, just with a bunch of http url handlers. I know it could be possible still more easily in other languages/stacks which are "zillion" times more productive.
Now, we all know that nothing comes free. For e.g. C++ templates cause compilation to take more time; have heard people complain about Ruby/ROR (including one on this page); I myself hated the Java configuration of the app servers, even in minimalist use of just a Servlet (the least needed for a web app thing), not to forget the SOAP-Servlets (Axis et al) before JSON ( good common sensical structure IMHO from the XML schemas/dtds (any one remember them?!)) became fashionable.
So the bloat/overhead can be in terms of useless jargon (e.g. the latter part of above para) which are too verbose and mean something very simple. Slowness e.g. Ruby/Rails. Or simply memory bloat (Which means more servers for the same amount of app processing capability).
So please do understand that there could be reasons other than "just can't get by without the iron hand of the compiler", where people use Go. :-)
People who have come to realize they wasted a decade chasing promises whether they were called OOP, EJBs, SOAP, XML, App Server, etc. Only to have been left with mysterious crashes because of "out of memory exceptions" (heck, jetty/tomcat are open source, but does any app developer should be debugging them. No, right? )
So there are also people disillusioned by the promises they fell for earlier. Who now prefer the minimalist approach. And actually very much appreciate when the wise people developing Golang, are hesitant to add a thing like Generics, despite the pressure from many quarters.
To summarize, why I use it:
Don't try to make it too easy for me, and pinch me at wrong times (when I start to get more users for e.g). Just Give me speed and give me transparency baby. I know how to make it work.
I'd love to hear more about replacing tomcat. Can you lead me to some info material on that topic. In our company we rely heavily on application servers, faster alternatives are always worth checking out.
Sure. Let me try to share some details. But before that, just want to make a point (which you surely know) that its also a major decision of a language change from Java to Go.
In the Tomcat/Jetty/<any other Java app server> world. If we want to have a web app (HTTP) we need to code servlets and add them into web.xml config, matching urls to Servlet Names. And within the Servlet we have coded, we need to call other Java code (the logic).
The container/app server takes care of the TCP/HTTP layers. Thread pools (which you can configure etc.).
Now in the Golang world. We just use the http package[1] for all that.
To listen to url:
http.Handle("/doSearch", &MySearch{"MySearch"})
Where MySearch is a struct which just needs to implement the interface:
func ServeHTTP( w http.ResponseWriter,r * http.Request)
Within this function you can put all the code which you will put in a java Servlet.
And to make this a service in your main package/program you will also do:
So these 3 pieces are the ones needed to have your own micro service or any web service running.
In our case, the similar functionality when moved resulted in lower memory usage and much higher stability (very less crashes, in the rare event of the service needing memory, which OS doesn't have. So the OS will kill that service anyhow - tech independent. We check using dmesg | grep <program name>)
Another benefit of this move, from our experience is getting rid of that webapp folder in any Java Servlet container. You will often end up plugging along multiple smaller apps, alongside your main one. And all reside in the same process.
That's not ample segregation, in our use. Its better you have different processes. So even if one dies, the others are independent.
This is also easier to manage. Deploy new ones - restarting just the ones changed.
Now we all know hot loading (without restart) is offered in several techs including some Java containers. But in practice, there always is some silly reason for you to restart. And sometimes the reason is psychological :-), we are not convinced that there is some baggage until we restart.
So hope my brief experience share is of some use. All the best.
Edit: Just want to add You must review this package page, if you are considering it
Hint, PHP is 100% one of those. The bad rap is either because of hipsterism or for issues that have been long solved.
The rest of its warts are no worse than the kind of BS any language has -- in fact before JS got in fashion the same things was said all the time for it too (for its bizarro coercion rules, only fp arithmetic, bs scope rules etc).
Modern PHP really isn't all that bad, I still wouldn't recommend people go into PHP development out of fear that they might stumble into one of the pre-modern hellholes that I only barely escaped in the early days of my career.
It's almost an irrational fear, but I remember working on plugins for applications where plugin hooks were non-existent, and the canonical approach was to patch code. I remember applications where there was no frontend server, but rather a sprawl of PHP files each doing one thing and hooking into the 'main' system via a 'require settings.php'.
Apart from that, working within PHP still obliges you to do a lot of hairy things---like working with Drupal. Now Drupal is certainly better architected than say Mambo/Joomla ever was, but it's still a better CMS than it is a web framework and for some odd reason lots of companies insist on building things into Drupal even today.
Now ... all of my recent knowledge is second hand, I've been out of PHP and into Python for at least 5 years now
It may be, but the above poster also left out one of PHP's strengths - availability. It's already running and ready to go on whatever cheap shared host you've been on for the last 20yrs. Ruby/Rails likely isn't.
The "fast PHP7 engine" part not. Raw Ruby was slower (but comparable) to raw PHP, so with this new engine which consistently runs real life workloads 2x, PHP7 should be like twice as fast.
And PHP still has more vendors, support in all kinds of hosts, even in third world facilities, and more companies offering commercial support. Plus there's stuff in PHP that don't exist with the same level of maturity/adoption/community in Ruby/Rails, e.g. Wordpress.
I am pretty fluent in ruby, which I learned after PHP specifically because I wanted to migrate, and I really enjoy using it. However, compared to PHP ruby is slow, it took ages before it had a half-decent unicode solution, and once you step out of the rails rah-rah it's a bit desolate on quality libraries... there's 10 half-built versions of everything, and no decent go-to implementation. On the plus side, ruby is nominally better for reflective metaprogramming, but that's really a rare use case.
Things I like about PHP: it's not perl, which it replaced for webdev. It's very optimized, and people with fat budgets pay for that (eg. Facebook). It connects to everything. Most of its (useful to webdev type situation) libraries are more mature than the equivalents in other languages, because more people have used them for longer. It had working unicode from the early days. It runs everywhere. Loads of people know enough of it to be dangerous / collaborate, and they tend to be numerous in the young/cheap hire pool. It's pretty much web-oriented, but can do scripting fine as well. It has a relatively stable configuration. Honestly, I think it's a very Unix-like solution to webdev problems, because it lets you roll your own solution, stays out of your way (unless requested), and encourages you to code in higher level languages. Yes, a lot of the code out there is horrible. Yes, most people code on frameworks which replace half the language with ugly hanging half-implemented abstractions. Yes, the function name thing is a pain. But mostly, really always almost mostly, it just works ... quite fast indeed, and everywhere. Yay PHP!
I originally wrote a long comment about how I've written CPAN modules and worked professionally in perl but will never do it again, explaining why technical betamax superiority was irrelevant because perl's users left before it got its act together, but I think in hindsight a simpler summary would be this: grep 'man perldsc' ~/.bash_history|wc -l
I'm not snarking; I honestly don't know. I tried using PHP years ago, threw up my hands in horror, and have never touched it since. I felt a little bad about that, but a few years back Eevee's "PHP: A Fractal of Bad Design" [1] persuaded me I wasn't missing much, and that my "learn a new language" time was better spent on Scala and then Python.
So what's changed in the last few years that makes it 100% as good as Ruby or Python for web development?
>I'm not snarking; I honestly don't know. I tried using PHP years ago, threw up my hands in horror, and have never touched it since.
Without exactly knowing what exactly bothered you and what you use instead, we can't answer.
As for "PHP: A Fractal of Bad Design", I find the article quite shallow. One could say the same things comparing something like Python to Smalltalk, and yet Python is fine as it is.
> So what's changed in the last few years that makes it 100% as good as Ruby or Python for web development?
That seems like it should be pretty straightforward.
> One could say the same things comparing something like Python to Smalltalk, and yet Python is fine as it is.
If you've already dropped the standard from "100% as good as" to "fine as it is", then I think you've answered my question. Nobody's denying that PHP is a perfectly cromulent language; things get built in it. The servers hum, pages render, revenue is made. For its audience, it's fine as it is.
fine
But as far as language quality, developer experience, or aesthetic quality goes, I think PHP's kind of a mess. I don't mind that people don't care about the things that Eevee raises. Different strokes for different folks, after all. But I care about them.
Personally, having spent the last year writing Python, I agree that it's fine, that it's adequate. But I think it's often no better than adequate. The object orientation is bolted on and a bit clumsy, there are 70-or-so global functions that mostly should go objects, the inclusion stuff is kinda broken, there's a bunch of stuff that looks like Java envy, etc.
Given that's how I feel about Python, which is honestly fine, you could imagine why I think PHP looks like a novice in the 90s ate a half-dozen random O'Reily books and then threw up in a compiler. For people who do almost all their work in that language, have high detail memory, and are on the naturalist end of the naturalist/theorist spectrum, I think that's ok; they'll get by happily. I'm just none of those things.
>If you've already dropped the standard from "100% as good as" to "fine as it is",
Actually there's no logical inconsistency being "100% as good as" to "fine as it is". Obviously something "fine as it is" could be "100% as good as".
>But as far as language quality, developer experience, or aesthetic quality goes, I think PHP's kind of a mess.
If we're comparing it to something like Scheme or Haskell or Smalltalk, yes. So, compared to the "aesthetic quality" of what? Javascript? C++? Perl? Go? Java?
As for the developer experience, can we get any actual qualified statements? Because the average developer experience of getting lost in the JS framework-du-jour land, or the Java developer churning BS enterprise factories upon factories is probably worse. So what exactly is bad with the developer experience? Certainly not the tools, from Composer to IDEs catering to the language. And I'd say not the fact that "it just works" without elaborate setup (or do people love configuring stuff in other environments?).
>Personally, having spent the last year writing Python, I agree that it's fine, that it's adequate. But I think it's often no better than adequate. The object orientation is bolted on and a bit clumsy, there are 70-or-so global functions that mostly should go objects, the inclusion stuff is kinda broken, there's a bunch of stuff that looks like Java envy, etc.
All of points being to it being a pragmatic language that developed from early versions.
>* For people who do almost all their work in that language, have high detail memory, and are on the naturalist end of the naturalist/theorist spectrum, I think that's ok; they'll get by happily. I'm just none of those things.*
Well, I don't do most of my work on the language (I mostly work with JS these days. I've also been using Python professionally since before 2.0, with early Zope et al). But I still find it perfectly fine.
If one has an issue with "70-or-so global functions", for example, I can't imagine how he would feel with JS "global by default" scoping -- which unlike PHPs is a core language issue.
> there's no logical inconsistency being "100% as good as" to "fine as it is". Obviously something "fine as it is" could be "100% as good as".
One phrase I take meaning "adequate", the other "good". Of course, if the thing you're comparing it with is merely serviceable, then yes. If all you're saying is that PHP is no worse than Javascript as a language, then sure, fine. I also don't think Javascript is a particularly good language.
I thought you were saying that PHP's issues "have been long solved", so I thought perhaps I was missing something since my last encounter. But given the tone of your responses here and your repeated evasion of the question "So what's changed in the last few years that makes it 100% as good as Ruby or Python for web development?" I'm just going to keep running with my theory that I'm not missing much.
Yep. When you're talking about "web development" and your first argument is about "concurrency", your odds of doing it wrong are likely very high. Learn to profile, learn to cache appropriately, learn to set up data pipelines based on access patterns. Many of us who have learned the dangers of threading the hard way consider it a last resort, and just because you have wonderful primitives like goroutines and channels doesn't mean you shouldn't think first. I'm inclined to link to JRuby's first rule of writing concurrent code: https://github.com/jruby/jruby/wiki/Concurrency-in-jruby#con...
Also, it's relative newness combined with the fact that it is on the lower level of things means that basic tasks that have been iterated to precision on other platforms are still very raw around the edges and require quite a bit more boilerplate, which is not at all suited to almost all "web development".
I know, I know. "But it's faster!" I'd take a clean, cohesive system that can scale horizontally even if it takes me 5 times the servers of my entire-system locking clusterfuck that makes me guess about data.
There is probably an interesting metaphor to be made that is corollary to the adage about the hammer, something about how if you give that man a screwdriver he starts looking for nails to pound with it?
You likely see this because web applications are one of the most common things developers build, and it is easier to learn a language when you start by building something you are familiar with.
When I learned Ruby I came from a Java background, and being able to search for things like "java interfaces in ruby" was incredibly helpful. It is a very succinct way of defining what you are trying to do in the new language, and often leads to a useful post explaining how to do what you want to do (even if that means doing it a completely different way in the new language).
I suspect this is also why you see frameworks like Revel gaining so much popularity in Go. Developers coming from Rails, etc all want something they are familiar with, and Revel looks familiar. It may not be the best way to build web apps in Go (I don't know), but limiting what you have to learn has its benefits.
I personally view this as a positive thing. It makes it easier for developers to learn Go and find out what it is useful for so that when they can benefit from it they realize it.
I also found myself relying on a lot of third party vendors in order to do some more complex tasks, such as queuing, web sockets etc. I found myself using Pusher, RabbitMQ, Beanstalkd etc, etc.
This felt kind of wrong.
And yet it isn't wrong. Standing on the shoulders of giants who built, tested, and optimised applications that do complex things far better than anything an individual could ever do on their own is how you build scalable applications that don't fall over.
Although I don't deny that tanker trucks are pretty amazing feats of engineering, sometimes I just want a glass of water. Instead of figuring out how to purchase one, drive it, and park in my back yard, it's pretty nice if I can just turn on a tap, fill a glass, and turn the tap off again. The tanker truck isn't "wrong", but it sure is a lot to deal with.
Go is as great for concurrency as Node.js: it forces one particular type of concurrent programming on the programmer, which IMHO does not qualify as "great".
Many languages give the programmer a range of options when it comes to concurrency, including the options Go and Node.js offer. Some languages even allow these options to be implemented as libraries (so they do not have to be built-into the language; though can become part of the standard lib).
I've done real-life web dev using PHP and using Go, and have dabbled with Node. Thoughts:
1. Go's strict, statically typed, thou-shalt-check-thy-errors approach is a breath of fresh air. The compiler does a whole load of checking which can (and therefore should) be automated, which in many other languages is left to the developer. Go code tends to have the property that if it compiles, it probably works as designed. I love this.
2. IMHO it makes more sense to think of a web app as a server application one of whose functions is to listen for and respond to HTTP requests. So, the question I'd ask is "is this a good language for server programming, and how easily can I make it speak HTTP?" In Go's case I've found the answers to be "hell yes" and "very".
3. The tooling is really nice. go fmt takes a whole load of pain away from everyone (no more effort required to format code properly, the tab/space and where-to-put-the-{ debates are solved decisively, reading other people's code is made easier), the package management is lovely, statically-linked executables are easy to move around and go doc encourages proper commenting.
4. Really good performance.
5. The biggest drawback I've seen with Go is verbosity - although a lot of the time it turns out that my verbose Go code is checking errors properly and my compact PHP code wasn't.
> The compiler does a whole load of checking which can (and therefore should) be automated, which in many other languages is left to the developer. Go code tends to have the property that if it compiles, it probably works as designed.
As someone who works with Go daily, these statements are both very untrue. Claiming "if it compiles it works" applies to Go code makes my eye twitchy.
For instance, type assertions have no exhaustivity checker. Interface{} and liberal usage of it throws type safety out the window.
> the package management is lovely,
Go doesn't have package management per SE, just vendoring.
I did fudge a little - "tends" and "probably". Of course compilation is not a guarantee of correctness, I just find that it gets me most of the way there. JavaScript, by comparison, feels like it's trying to give you rope to hang yourself.
Maybe I'm abusing the term "package management". I'm talking about go get, the way imports function, the cute private/public lowercase/uppercase mechanism, the recently-added vendoring support etc.
The real pain for me to share variables from different handlers in the chain (middlewares). I know some frameworks and libraries provide "contexts", but it's something that should have been part of the standard library.
What about DB (sql) access ? The 90% user case for any webservice today is :
- parse request (ok)
- issue a serie of DB requests <-- GO PAIN POINT
- serialize result into either HTML (ok) or Json <-- PAIN POINT
I'd say 2 pain points out of 3 is pretty bad IMHO.
I would recommend go only for very specialized and technical web services, such as a websocket server. But really not for the average web developer task.
Where are the pain points in DB requests or serialization? I've written several web apps with Go, and lots of ETL, and have yet to feel those as pain points.
Golang has nothing close to sqlalchemy, linq, activerecord or hibernate when it comes de querying a database. Last time i used go i had to write every single query and bind them manually to records in both ways.
For writing simple api endpoints, this is really not up to today's standards.
164 comments
[ 3.2 ms ] story [ 202 ms ] threadSeems to be upvoted because #Go.
I see Go as an optimization tool when concurrency is needed, the same way I would consider using it to break some part of the app when it gets way too big. It works very well for those purposes. But I would never consider Go for building a web app from scratch.
Rendering HTML templates / JSON
Asset compilation / Cache busting
Caching
Sending Email
Previewing/Testing Email
Background Job queuing(with failure/retry/backoff)
Reliable and easy integration/feature testing
Connecting to database and modeling data
Authentication / Account signup / Confirmation / Forgot Password
Security
Payment handling
PDF / Invoice rendering
Deployment
Making pretty good money over here and how many ROFL scale GO routines I can run never really came up.
¯\_(ツ)_/¯
+1
And Authorized data access.
[1] https://docs.djangoproject.com/en/1.9/topics/auth/
[2] http://www.intenct.nl/projects/django-allauth/
[3] https://docs.stormpath.com/python/
[4] https://github.com/cedadev/ndg_xacml
¯\_(ツ)_/¯
Seriously though, Go is a really good language. Its a simple yet solid & powerful language, but why would people feel the need to present it as the silver bullet for everything?
* Static typing
* Web sockets
* Concurrency
* Unit testing
* Speed
* Syntax
* Formatting tools
Both eco-systems fill all those bullet points.
Luckily I only had to use them on Websphere based deployments.
By day I make good money using C# where I get all of the above at what seems to be a much more web appropriate level than Go.
One of the things that I'm really liking about the ASP.Net Core rewrite is that a framework that has been around for ages is taking an opportunity to look at what it does well, what it doesn't and what other languages / frameworks have innovated on more recently and stealing liberally.
C# has been through several cycles of how to write concurrent code now and offers both Task and async/await depending on your needs. Not only is web sockets support there but SignalR is there as a useful API for common cases. If you want even better static typing you can switch to F#.
Not everyone was doing web development using single threaded dynamically typed languages before Go came along.
Many of these "solved problems" have also been solved with the other languages, albeit not always at the surface level. For Node specifically:
WebSockets -> SockJS, Socket/Engine.io
Concurrency -> Generator Functions
Testing -> import assert from 'assert' and npm test
Syntax -> Some great improvements with ES6 and solid styleguides (airbnb, standard) + eslint as an autoformatter
The major win here for Go is that as a new platform it was designed with simplicity in mind. These features are probably easier to pick up in Go than any of the alternatives, which is great. However, don't make the mistake of thinking that Go is the first language to solve these problems.
yeah no, please compare node/ruby whatever to modern PHP and not some ancient versions you used some time ago. Modern PHP is very much up there.
At a glance his previous articles seem to be about Go and PHP (with a little bit of shell scripting) as well, so GP's point seems to stand.
[1] https://klibert.pl/posts/weekend_with_elixir.html
I do have fun bashing Go, but I also use it. It's too practical not to. And the Go team, above all other of the latest wave of language that are ready for production, got that 100% right. I just can't overstate how simple Go makes setup, deployment, and updates. And how complex Elixir is in comparison.
I wish Elixir et al. could have a zero dependency, zero external configuration, and single/xcopy file deployment too.
For me, the ultimate test is can I copy my application to a flash drive, walk it to a new clean system, and "paste" it to that new system. Go really does let me do that.
> But i didn't find any mature libraries like Phoenix.
Yup. But then I tend not to need larger frameworks with my Go apps. Like you allude to, Go apps tend to be less monolithic.
That used to be the case for erlang deployments that looked pretty similar, and Exrm looks like it does the same thing:
https://hexdocs.pm/exrm/extra-deployment.html
There are five stages to creating and deploying a first release. The first is just generating it, the next three are for moving the release to a new place and the third is just decompressing it.
Live upgrades/downgrades seem pretty similar: https://hexdocs.pm/exrm/extra-upgrades-and-downgrades.html
We are also web developers on those stacks.
I mean, tell me the equivalent Java code to this:
I think that would be about 20 lines in Java. You've got to make a `HttpUrlConnection`, then call `setDoInput(true)` (or is that `setDoOutput(true)`), wrap it in a `try`, etc. etc. etc.Go's standard library is amazing.
Nothing prevents you to wrap those 20 lines in a generic method.
Also there are other options in Clojure, Scala, Kotlin, Groovy besides pure JDK API calls. Here Scala style, easy to mimic in Java 8
> Go's standard library is amazing.So amazing that is doesn't have proper database drivers for the SQL databases that matter.
Not community projects that kind of work.
The point is Go's library already has all the nice convenience methods. With Java I would be endlessly writing them myself, and even then they wouldn't be as good as Go's because they wouldn't be standard (no help on StackOverflow for example).
Also many Java bashers in the Go community seem to be unaware of what is delivered in the JDK.
When I used to daily read gonuts, many weren't even aware that since Java 6, there is a similar infrastructure for HTTP(S), no need for third party servers.
Seriously. Pretty much all languages have support for concurrency via threads. Ruby even have the equivalent of channels since a decade: http://ruby-doc.org/core-2.2.0/Queue.html
Now if you meant to talk about the GIL, well it prevent parallelism only for purely CPU bound tasks. And in the context of a web app, if I have a CPU bound tasks that I can just fire and forget, I don't want to execute it on my frontend server anyway.
> Seriously. Pretty much all languages have support for concurrency via threads.
You missed the most important point: like Go does
Ruby doesn't have "built in concurrency"? Ruby has fully native threads; it just has the GIL that blocks for IO (but you could use jRuby). It gives you several options for message passing. You also have the extremely under-rated dRuby system. Not to mention the whole EventMachine world.
Node is "close" to concurrent? It's god damn single-threaded! At the very least it's possible to spawn multiple processes and communicate over a Unix socket, good luck doing something like that with PHP. What does socket.io have to do with anything?
Sorry man, I'm hearing a lot of cargo cult opinions and precious little data to back it up.
Thanks.
That said, as someone who spends a lot of time in Python as well, I can't express how much I love the deep autocompletion powers and dumb-mistake suppressing abilities of static typing.
This article seems to address a lot of the pain points I originally had: https://eager.io/blog/go-and-json/
Go type system is flat so it's often hard to express something without being tempted to "opt-out" from its rigid type system.
The question is, is Go type system a good compromise between simplicity and expressiveness ?
I guess language design is hard, because the competition (Nim,Rust,...) also have its share of issues.
In contrast, Go and its `json.Unmarshal()` aren't that well suited for parsing arbitrary and/or complex inputs. However, if the expected input is well formed and can be directly mapped to a predefined datastructure, the convenience factor is pretty high.
So, from my limited experience, I would say that JSON parsing in golang is more geared towards deserialising well defined transport formats than ingesting arbitrary inputs.
I think go revolves more around well-defined structures, and JSON is anything but. For inter-process pipelines, protobuf (polyglot) and gob (go-only) work better in my experience.
curl -s https://api.github.com/repos/chimeracoder/gojson | gojson -name=Repository
Good on him for learning a new language, I guess.
maybe if jetbrains' project rider gains traction and core-clr and/or mono-on-linux matures it will be a broadly appealing option.
Also, bringing a developer without .NET experience into a project means 6 months before they can do most anything and a good year to be truly up to speed. The ramp-up time for both C# and Java applications are now about as long as C++.
I had high hopes for .NET core. I saw it as a way to start over creating a Go like situation on top of the C# language. But it's still about two years from being production ready and its already gotten complex and bloaty. Just to install the development tools and environment means multiple downloads, dependencies, and install scripts. Download a zipped file, extract to the directory of your choice, and set a few environment variables. That's the way to ensure reducing complexity.
Go is just good for some things, not for all.
Most companies will stop at SQL Injections but I consider defense against XSS, and CSRF attacks just as important. And that is something Go does not excel at because I have not found any framework that makes adding these features as painless as possible. (I have only toyed with Go for the web so I might be wrong).
Django (a python web-framework), on the other hand, is pretty good at these things[1] because they require little to no extra work, so that junior developers like me will get it right. That is the reason I would always go to Python for web development instead of Go, although, I like Go just as much and it is so much quicker.
I'd be interested to hear how the big projects migrate these kind of attacks in Go.
[1] https://docs.djangoproject.com/en/1.9/topics/security/
You have to mark "trusted" or "pre-escaped" parameters explicitly by wrapping them in e.g. `template.HTML`.
[1] https://github.com/justinas/nosurf
I mean it does feel like superpower when applied to tasks you'd previously have to use C for. But have you tried any of the modern web development stacks? They are like a zillion times more productive. Hint: PHP is not one of those.
I guess some people just can't get by without the iron hand of the compiler :)
EDIT after reading the article: I do have to agree that concurrency in PHP is a bit of a pain, but those 3d party tools are pretty easy to use and maintain.
> Without significant forethought to several levels of caching; such as in memory caching, op caching ...
This is so trivial to setup it's not even worth mentioning.
> Sure, PHP has PSR standards etc, but they're fairly new and developers have been slow to adopt them.
Everyone I know and every library I use uses those standards. PHP is pretty standardized now, especially compared for example to js.
PHP frameworks work well out of the box and contain solutions for most use-cases.
Go is nice but when I tried it I really missed something like composer. "go get" and commit your dependencies to the repository? Seriously? That's wrong on so many levels. Meanwhile I think composer is one of the best package managers around.
It is, but the overwhelming majority of tasks in a web app don't happen concurrently, so it doesn't matter.
When it does matter just use something else.
That's what I do!
(Actually, I do that from the start of the project, so that adding something as pedestrian as concurrency doesn't involve a complete port to another language.)
But is that cause, or is that effect? The dominant web programming languages for the last 20 years have all been single-threaded, so of course the web stack looks single threaded. Personally, I find there's a ton of things that ought to be concurrent but aren't. I mean, most other network apps need to do things concurrently, why not web?
Once you start looking at it this way, the staggering number of hacks employed to pry concurrency out of fundamentally single-threaded languages is astonishing.
Go is not the only solution to this, there are many. (Go is one of the more Algol-ish language choices that doesn't involve learning a new paradigm, though, and can make it a relatively easy sell where "Let's all learn Lisp!" or "Let's all learn immutable functional programming!", where "all" may be dozens+ people, may not happen. [1]) But I think in five or ten years, "the overwhelming majority of tasks in a web app don't happen concurrently" is going to sound very 200x-ish.
[1]: Incidentally, for those who are still baffled by Go's success, notice how all the alternatives involve trying to teach everyone a brand new paradigm. While that has its advantages, when it comes to language popularity, not requiring that has its advantages too. I'm a polyglot myself, so I get how Haskell is nice and Clojure is nice and how they open your mind to new paradigms etc. etc., no sarcasm, fully mean it, if you only know C-style languages I fully recommend you go fix that soon, but when you're trying to get a team of people on the clock to switch languages for a nicer concurrency story, cutting the requisite training time by 90%+ raises the bar very high for those other languages, in relative terms. (Even moreso than it sounds; not only is training costs less, but the risk of training failure is also that much less. It compounds.) I do not mean this as "advocacy", I mean it as explanation.
What sort of things? I've been writing web software for ~20 years, and the vast majority of things that I've made boil down to "request comes in > validate input data > transaction to database > check transaction worked > write a response". Concurrency has never really been a problem, or even something I've wanted.
Just this week I met with a friend who wanted an app for community meetings that would let people express their preferences in real time, changing their expressed opinions as discussion happens. That way everybody could see how the consensus was shifting in response to discussion without people having to say it out loud.
I quickly whipped together a prototype, and I intentionally didn't use a database because I wanted this to be dynamic. Whenever one person changed something, everybody else needed to see it immediately. That's very easy in a context where you have all your data hot in RAM in one process with easy concurrent-ish access to it: you have a bunch of observer objects that get notifications each time you change something. In database-land, it's a pain.
And it works for me at size as well. Years ago I led a team where we were building a new web product. We were big on test-driven development, and databases are a notorious pain point. We started out without one, figuring that we'd pull one in when we needed it. We ended up never needing one. Our system had incredibly low rendering times (<5ms, even on 2005 hardware), and its 1800 unit tests ran in a couple of seconds. Starting with concurrency saved us so much development pain.
Another common example of where concurrency is valuable is in notifications. Quora and Facebook both have excellent notification systems: dynamic and instantaneous. Twitter's are much rougher, and it's clear they've put a lot less thought into concurrency.
Games, of course. Imagine trying to build agar.io without concurrency. Or look at Google Docs: concurrency is at the heart of what makes them valuable in an office setting.
There's nothing wrong with CRUD apps. There's nothing wrong with all the mainframe batch-processing apps that preceded them, either. But there's more that we can do.
I still don't like it but the landscape definetely changed, and it keeps getting better.
> Meanwhile I think composer is one of the best package managers around.
Apart from the fact that it uses gigabytes of ram on real projects. At work we ended up using it outside virtual machines, to later copy back the dependencies.
I used did a quick test (having run `composer clearcache` each time) on one of my projects:
Without composer.lock: Memory usage: 75.96MB (peak: 87.5MB), time: 38.54s
With composer.lock: Memory usage: 7.47MB (peak: 8.99MB), time: 28.6s
https://github.com/composer/composer/issues/1898
I use composer every day, and have 0 problems when doing a `composer install` on production.
I also recommend setting up Toran Proxy, which will allow you to ensure you're not boned when Github goes down.
One of not negligible size. They usually have many dependencies. I can see on our composer.json around ~60 dependencies, and the number will only grow (apart some lucky circumstances).
https://github.com/composer/composer/issues/1898
> I also recommend setting up Toran Proxy, which will allow you to ensure you're not boned when Github goes down.
Yes, that's a good idea
Why is this a pb? Makes your app is more resilient to external outages and no need to git clone all the repos with historical data.
And then I thought "why not use it for something web-like". My experience was dreadful. After coming from Python+Django, it felt like writing Java 1.4 (not a good thing). Especially for returning JSON objects with lots of mixed types.
So I switched to Clojure and never looked back (Python would have been an equally valid choice).
Go's a great language, but it's being applied in some very odd (and potentially unsuitable) places.
How did you like Clojure? Does the ecosystem it compare well against Django? I tried it once during an on-site interview but was put off by the multi-minute startup times.
So, comparing the languages, I tried Clojure because a colleague was using it and to me it feels like the best language I have ever used professionally (comparing to C#, C, Obj-C, Python, Java, JavaScript, Ruby, Go) and I'm very happy using it. Great functional programming features, great libraries, I really like the LISP syntax and the stress I'd been having with Python kind of melted away (things like state in objects and the type system (which kind of doesn't really exist most of the time with Clojure)).
In terms of doing web type things, the Liberator and Ring libraries feel a lot like a big chunk of Django's routing and middleware functionality. Selmer does a good job of templating.
For most user interface work I'm using React.js (which I also really like) so my Clojure code is only serving up JSON on an HTTP API. But for simple templated HTML, Selmer works well.
One question is "Django is an entire framework. How does it compare to lots of little libraries?". I have to say that Django is probably the most well constructed pieces of software I have worked with and it hangs together very well. Clojure with a few libraries is very good, but beating Django is a very high bar to reach for any framework or group of libraries.
Startup time is an over-done argument. Once you're running a REPL you don't need to restart it, so development isn't slowed down. Once you're running a server you don't need to restart it, so production isn't slowed down.
Since you work with React, how do you like JS? I'd imagine that, coming from Clojure, you'd be frustrated with its inconsistencies, as I am, coming from Python.
I've tried ClojureScript (without React) and it didn't quite click -- some of JavaScript's weird behaviour still shines through.
One problem I have is that the application is extremely CPU intensive. I can't get past 35-40 requests per second with 500 concurrent users (at 4 CPU cores, 14 GB RAM), which seemed too expensive economically. (I cached as much data as possible, both at the Nginx tier and with Redis) and tuned the number of uWSGI worker processes.
Do I have to try other languages or do you think I have more room for optimization with Python?
Then I'd suggest you use Cython for a speed up or try using PyPy which can be 200% faster without any changes.
When I used Python and worked with financial data, every live algorithm would be recoded as a Cython extension.
Having done web development for over a decade. My experience is that algorithm is far more important than language. After that data structures trumps language. After that doing work you don't have to do trumps language.
First look for really ugly nested loops in your code. I can't tell you the number of times this type of things ends up in your codebase, especially since dev environments often have a small subset of data. Even an O(n!) algorithm is fast for small values of n.
Next look to see what kinds of datastructures you're using. For instance, dicts are a seductive way to store data, they have a theoretical O(1) lookup, but they have a drawback of randomizing memory lookups, which means that for subsequent looks for a large dict, you'll end up with lots of cache misses, can make each lookup 1000 times slower. Meanwhile, a tuple has O(n) lookups, but if you're finding that you need to iterate over the data, you're going to benefit from memory locality. So know your tradeoffs. Also code that had one usage profile at launch, can often have a very different usage profile a year later, so it doesn't hurt to revisit this occasionally.
Last look for code that isn't doing anything. Are you calling the database twice in a row, asking for the same data, and not using it? It sounds obvious, but over time these kinds of things have a way of showing up in a codebase, and they can really add up.
I also should mention caching at this point. It can be incredibly hard to get caching right, but the performance savings are pretty big. It goes under the bucket of not doing things that you don't need to. Do whatever you can to do page caching. Putting an nginx cache in front of the webserver serving python for the majority of your pages could easily get you to hundreds of requests per second.
Hey, Haskell and Scala exist!
What stacks are you referring to exactly?
[1] https://www.djangoproject.com/start/overview/
But I've used it in the web app scenario. Replacing Java (jetty/tomcat servlets with pure Java code) with Golang. My own discovery was it could replace the jetty/tomcat with servlets part seamlessly, just with a bunch of http url handlers. I know it could be possible still more easily in other languages/stacks which are "zillion" times more productive.
Now, we all know that nothing comes free. For e.g. C++ templates cause compilation to take more time; have heard people complain about Ruby/ROR (including one on this page); I myself hated the Java configuration of the app servers, even in minimalist use of just a Servlet (the least needed for a web app thing), not to forget the SOAP-Servlets (Axis et al) before JSON ( good common sensical structure IMHO from the XML schemas/dtds (any one remember them?!)) became fashionable.
So the bloat/overhead can be in terms of useless jargon (e.g. the latter part of above para) which are too verbose and mean something very simple. Slowness e.g. Ruby/Rails. Or simply memory bloat (Which means more servers for the same amount of app processing capability).
So please do understand that there could be reasons other than "just can't get by without the iron hand of the compiler", where people use Go. :-)
People who have come to realize they wasted a decade chasing promises whether they were called OOP, EJBs, SOAP, XML, App Server, etc. Only to have been left with mysterious crashes because of "out of memory exceptions" (heck, jetty/tomcat are open source, but does any app developer should be debugging them. No, right? )
So there are also people disillusioned by the promises they fell for earlier. Who now prefer the minimalist approach. And actually very much appreciate when the wise people developing Golang, are hesitant to add a thing like Generics, despite the pressure from many quarters.
To summarize, why I use it:
Don't try to make it too easy for me, and pinch me at wrong times (when I start to get more users for e.g). Just Give me speed and give me transparency baby. I know how to make it work.
Edit: Typos and minor rephrase
In the Tomcat/Jetty/<any other Java app server> world. If we want to have a web app (HTTP) we need to code servlets and add them into web.xml config, matching urls to Servlet Names. And within the Servlet we have coded, we need to call other Java code (the logic).
The container/app server takes care of the TCP/HTTP layers. Thread pools (which you can configure etc.).
Now in the Golang world. We just use the http package[1] for all that.
To listen to url:
Where MySearch is a struct which just needs to implement the interface: Within this function you can put all the code which you will put in a java Servlet.And to make this a service in your main package/program you will also do:
So these 3 pieces are the ones needed to have your own micro service or any web service running.In our case, the similar functionality when moved resulted in lower memory usage and much higher stability (very less crashes, in the rare event of the service needing memory, which OS doesn't have. So the OS will kill that service anyhow - tech independent. We check using dmesg | grep <program name>)
Another benefit of this move, from our experience is getting rid of that webapp folder in any Java Servlet container. You will often end up plugging along multiple smaller apps, alongside your main one. And all reside in the same process.
That's not ample segregation, in our use. Its better you have different processes. So even if one dies, the others are independent.
This is also easier to manage. Deploy new ones - restarting just the ones changed.
Now we all know hot loading (without restart) is offered in several techs including some Java containers. But in practice, there always is some silly reason for you to restart. And sometimes the reason is psychological :-), we are not convinced that there is some baggage until we restart.
So hope my brief experience share is of some use. All the best.
Edit: Just want to add You must review this package page, if you are considering it
[1] https://golang.org/pkg/net/http/
Hint, PHP is 100% one of those. The bad rap is either because of hipsterism or for issues that have been long solved.
The rest of its warts are no worse than the kind of BS any language has -- in fact before JS got in fashion the same things was said all the time for it too (for its bizarro coercion rules, only fp arithmetic, bs scope rules etc).
http://www.brightball.com/golang/go-from-a-php-perspective
TLDR: Go is good at the things PHP is bad at. PHP is good at the things Go is bad at. They actually compliment each other very well.
It's almost an irrational fear, but I remember working on plugins for applications where plugin hooks were non-existent, and the canonical approach was to patch code. I remember applications where there was no frontend server, but rather a sprawl of PHP files each doing one thing and hooking into the 'main' system via a 'require settings.php'.
Apart from that, working within PHP still obliges you to do a lot of hairy things---like working with Drupal. Now Drupal is certainly better architected than say Mambo/Joomla ever was, but it's still a better CMS than it is a web framework and for some odd reason lots of companies insist on building things into Drupal even today.
Now ... all of my recent knowledge is second hand, I've been out of PHP and into Python for at least 5 years now
And PHP still has more vendors, support in all kinds of hosts, even in third world facilities, and more companies offering commercial support. Plus there's stuff in PHP that don't exist with the same level of maturity/adoption/community in Ruby/Rails, e.g. Wordpress.
And even a path to static typing, through Hack.
I'm not snarking; I honestly don't know. I tried using PHP years ago, threw up my hands in horror, and have never touched it since. I felt a little bad about that, but a few years back Eevee's "PHP: A Fractal of Bad Design" [1] persuaded me I wasn't missing much, and that my "learn a new language" time was better spent on Scala and then Python.
So what's changed in the last few years that makes it 100% as good as Ruby or Python for web development?
[1] https://eev.ee/blog/2012/04/09/php-a-fractal-of-bad-design/
Without exactly knowing what exactly bothered you and what you use instead, we can't answer.
As for "PHP: A Fractal of Bad Design", I find the article quite shallow. One could say the same things comparing something like Python to Smalltalk, and yet Python is fine as it is.
> So what's changed in the last few years that makes it 100% as good as Ruby or Python for web development?
That seems like it should be pretty straightforward.
> One could say the same things comparing something like Python to Smalltalk, and yet Python is fine as it is.
If you've already dropped the standard from "100% as good as" to "fine as it is", then I think you've answered my question. Nobody's denying that PHP is a perfectly cromulent language; things get built in it. The servers hum, pages render, revenue is made. For its audience, it's fine as it is. fine But as far as language quality, developer experience, or aesthetic quality goes, I think PHP's kind of a mess. I don't mind that people don't care about the things that Eevee raises. Different strokes for different folks, after all. But I care about them.
Personally, having spent the last year writing Python, I agree that it's fine, that it's adequate. But I think it's often no better than adequate. The object orientation is bolted on and a bit clumsy, there are 70-or-so global functions that mostly should go objects, the inclusion stuff is kinda broken, there's a bunch of stuff that looks like Java envy, etc.
Given that's how I feel about Python, which is honestly fine, you could imagine why I think PHP looks like a novice in the 90s ate a half-dozen random O'Reily books and then threw up in a compiler. For people who do almost all their work in that language, have high detail memory, and are on the naturalist end of the naturalist/theorist spectrum, I think that's ok; they'll get by happily. I'm just none of those things.
Actually there's no logical inconsistency being "100% as good as" to "fine as it is". Obviously something "fine as it is" could be "100% as good as".
>But as far as language quality, developer experience, or aesthetic quality goes, I think PHP's kind of a mess.
If we're comparing it to something like Scheme or Haskell or Smalltalk, yes. So, compared to the "aesthetic quality" of what? Javascript? C++? Perl? Go? Java?
As for the developer experience, can we get any actual qualified statements? Because the average developer experience of getting lost in the JS framework-du-jour land, or the Java developer churning BS enterprise factories upon factories is probably worse. So what exactly is bad with the developer experience? Certainly not the tools, from Composer to IDEs catering to the language. And I'd say not the fact that "it just works" without elaborate setup (or do people love configuring stuff in other environments?).
>Personally, having spent the last year writing Python, I agree that it's fine, that it's adequate. But I think it's often no better than adequate. The object orientation is bolted on and a bit clumsy, there are 70-or-so global functions that mostly should go objects, the inclusion stuff is kinda broken, there's a bunch of stuff that looks like Java envy, etc.
All of points being to it being a pragmatic language that developed from early versions.
>* For people who do almost all their work in that language, have high detail memory, and are on the naturalist end of the naturalist/theorist spectrum, I think that's ok; they'll get by happily. I'm just none of those things.*
Well, I don't do most of my work on the language (I mostly work with JS these days. I've also been using Python professionally since before 2.0, with early Zope et al). But I still find it perfectly fine.
If one has an issue with "70-or-so global functions", for example, I can't imagine how he would feel with JS "global by default" scoping -- which unlike PHPs is a core language issue.
> there's no logical inconsistency being "100% as good as" to "fine as it is". Obviously something "fine as it is" could be "100% as good as".
One phrase I take meaning "adequate", the other "good". Of course, if the thing you're comparing it with is merely serviceable, then yes. If all you're saying is that PHP is no worse than Javascript as a language, then sure, fine. I also don't think Javascript is a particularly good language.
I thought you were saying that PHP's issues "have been long solved", so I thought perhaps I was missing something since my last encounter. But given the tone of your responses here and your repeated evasion of the question "So what's changed in the last few years that makes it 100% as good as Ruby or Python for web development?" I'm just going to keep running with my theory that I'm not missing much.
Also, it's relative newness combined with the fact that it is on the lower level of things means that basic tasks that have been iterated to precision on other platforms are still very raw around the edges and require quite a bit more boilerplate, which is not at all suited to almost all "web development".
I know, I know. "But it's faster!" I'd take a clean, cohesive system that can scale horizontally even if it takes me 5 times the servers of my entire-system locking clusterfuck that makes me guess about data.
There is probably an interesting metaphor to be made that is corollary to the adage about the hammer, something about how if you give that man a screwdriver he starts looking for nails to pound with it?
When I learned Ruby I came from a Java background, and being able to search for things like "java interfaces in ruby" was incredibly helpful. It is a very succinct way of defining what you are trying to do in the new language, and often leads to a useful post explaining how to do what you want to do (even if that means doing it a completely different way in the new language).
I suspect this is also why you see frameworks like Revel gaining so much popularity in Go. Developers coming from Rails, etc all want something they are familiar with, and Revel looks familiar. It may not be the best way to build web apps in Go (I don't know), but limiting what you have to learn has its benefits.
I personally view this as a positive thing. It makes it easier for developers to learn Go and find out what it is useful for so that when they can benefit from it they realize it.
This felt kind of wrong.
And yet it isn't wrong. Standing on the shoulders of giants who built, tested, and optimised applications that do complex things far better than anything an individual could ever do on their own is how you build scalable applications that don't fall over.
That's not a thread, that's a child process. Very different model of concurrency...
Many languages give the programmer a range of options when it comes to concurrency, including the options Go and Node.js offer. Some languages even allow these options to be implemented as libraries (so they do not have to be built-into the language; though can become part of the standard lib).
1. Go's strict, statically typed, thou-shalt-check-thy-errors approach is a breath of fresh air. The compiler does a whole load of checking which can (and therefore should) be automated, which in many other languages is left to the developer. Go code tends to have the property that if it compiles, it probably works as designed. I love this.
2. IMHO it makes more sense to think of a web app as a server application one of whose functions is to listen for and respond to HTTP requests. So, the question I'd ask is "is this a good language for server programming, and how easily can I make it speak HTTP?" In Go's case I've found the answers to be "hell yes" and "very".
3. The tooling is really nice. go fmt takes a whole load of pain away from everyone (no more effort required to format code properly, the tab/space and where-to-put-the-{ debates are solved decisively, reading other people's code is made easier), the package management is lovely, statically-linked executables are easy to move around and go doc encourages proper commenting.
4. Really good performance.
5. The biggest drawback I've seen with Go is verbosity - although a lot of the time it turns out that my verbose Go code is checking errors properly and my compact PHP code wasn't.
As someone who works with Go daily, these statements are both very untrue. Claiming "if it compiles it works" applies to Go code makes my eye twitchy.
For instance, type assertions have no exhaustivity checker. Interface{} and liberal usage of it throws type safety out the window.
> the package management is lovely,
Go doesn't have package management per SE, just vendoring.
Maybe I'm abusing the term "package management". I'm talking about go get, the way imports function, the cute private/public lowercase/uppercase mechanism, the recently-added vendoring support etc.
- parse request (ok)
- issue a serie of DB requests <-- GO PAIN POINT
- serialize result into either HTML (ok) or Json <-- PAIN POINT
I'd say 2 pain points out of 3 is pretty bad IMHO.
I would recommend go only for very specialized and technical web services, such as a websocket server. But really not for the average web developer task.
For writing simple api endpoints, this is really not up to today's standards.
For me, it's a non-issue, as it doesn't feel right with Go.
Seriously, why is this even a thing? Why gratuitously prevent Firefox + NoScript from working?