144 comments

[ 2.5 ms ] story [ 214 ms ] thread
> I like having a single binary. I usually run code on EC2 machines to give my scripts closer network proximity to S3 and to our database. With Python, I have to ensure all the packages I need are installed on the remote machine, and that one of my coworkers hasn’t installed anything conflicting.

pipenv

I've written in Python for ~13 years and I'm really enjoying Go. It's fast and its libraries (the few there are) seem to make a lot of sense. I don't think I would use it for daily scripting like the author, but for a large new project I would definitely pick Go over Python now.
Try an ML-family language (e.g. OCaml, F#, or Rust) if you haven't already. You get the best of all worlds: expressiveness, safety and speed.
I like the expressiveness of Rust in theory, but for the programs I write I don't feel like I get sufficient payoff for fighting the borrow checker, as compared to writing in Go. Would I be correct in saying OCaml is pretty much Rust without the borrow checker? I have no prior experience with the language.
> Would I be correct in saying OCaml is pretty much Rust without the borrow checker?

Yeah, it's very much on those lines, though the syntax is a bit more ML-style (Rust pushes a bit towards a C-like syntax).

(I actually mostly use Scala myself these days, but I know the JVM is a dealbreaker for some people, and the design compromises for the sake of Java compatibility mean Scala is probably not the best choice for a first ML)

If you want something closer to Rust, consider looking into Reason. It’s essentially OCaml with braces and semicolons.
Dlang is also very nice if you like Algol style languages. You can get productive in it pretty fast, just like in Go, but it's expressive, it's garbage collected by default and you don't waste time fighting the compiler. Also the code compiles as fast as Go with DMD and LDC. I havent tried GDC.

https://tour.dlang.org/

Plus you get templates and CTFE, which are both awesome. Though if you're doing web stuff I've found D compile times to not be so great anymore, especially when diet templates get into the mix.
Rust was originally described to me as "OCaml + C", so there's some truth to that. The language has strayed a bit farther from OCaml in many ways.

I really like OCaml, and you should learn it, but you should also know that concurrency is not a strength of it. It's being worked on...

Single-process concurrency is great in OCaml. It’s flexible and composable with libraries like Lwt and Async. On the other hand true multicore backend is where the most work is moment. For the latest update on that see: https://discuss.ocaml.org/t/ocaml-multicore-report-on-a-june...

(Tl;dr: It will be officially merged in 2019.)

Yes, I should have said parallelism, my bad.

I hope that it does get merged soon, people have been waiting quite a while, in my understanding. It’s a tough problem.

Why can't he just be happy using Go?
If he(?) wouldn't use it for daily scripting then it isn't meeting all his needs. IME being able to use a single language for both scripting and "real" code is an underrated advantage, since it lets you reuse all your helper functions etc. in your scripts.
What you are describing (one language to rule them all) is a non-goal to 98% of HN readers.
Only because they assume it's impossible.
Or, because like most professionals, we understand certain tools are better than others at specific tasks, even when there's specific tools that do both things equally well. There can be an option that's preferred even when it's "only" good at one of the things you need.
XAR lets you package many files into a single self-contained executable file. This makes it easy to distribute and install.

https://github.com/facebookincubator/xar/

Then just build your XAR executable with:

$ python setup.py bdist_xar

First let me state that a language is just a brush with which to paint your logic. I really don't subscribe to the X is better than Y. I might concede that x is better than y for certain tasks, but not a sweeping generalisation.

The first point about inconsistent spelling mistakes, I 100% sympathise. I suffer from this terribly. Pylint and syntastic are a silver bullet for me. I have been tempted by a more fancy IDE, but I'm so used to vim, I'm reluctant/scared to change.

Parallelism: I do look over the wall at Go and wish that python 3.x had spent more time on removing the GIL, but that is old news now.

However I enjoy threading with python(not its limitations), because its so trivially easy. The Queues primitive saves 90% of the headache. as for ctrl-c not doing anything set all your threads to daemon (<threadObj>.setDaemon(True)) When the main thread exits, so does everything else. Go shines by having built in true parallelism

If I need horsepower, then I have to run multiple processes (not multiprocessing, that has drawbacks.) which has the annoyance of serializing and message passing. (but that does mean scaling horizontally less painful further on). Again, Go really shines here.

Statically compiling thing: YES. This means that I can 90% of the faff with docker (xyz version not working any more, the build fails) and use a real scheduling system that runs on real steel(or AWS) and not the omnifaff that is kubernetes.

I think the pypy people were able to get a trivial program running with the gil unlocked. I'd watch them for further updates.
I think when you really need performance it's a great example of a time to start looking at alternative Python implementations, CPython just isn't the best choice for a lot of tasks that actually have processing speed requirements.
For starters, just adding Cython and/or Numba to the code should introduce great improvements. https://rushter.com/blog/numba-cython-python-optimization/
The word "just" is doing a lot of work. Cython might as well be a different language.
I generally don’t agree with the aversion to the word “just”, butbin this case you’re absolutely correct. Cython is a different language.
After working in the Python speed optimization space as a consultant for a while I have come to see Cython as not really being the right answer for a lot of situations. The main reason is that CPython is slow but Cython really is quite heavily constraining you to the CPython runtime. Cython is significantly different enough to not be Python but it isn't the language I'd want to choose (C/C++/Rust) for going the whole way with foreign function interfacing (I might blog about this again sometime). Sure it's easy to get started with it but I feel like the maintenance costs are higher than people think.

Numba on the other hand has a few nice use cases (where "nice" means close to Pareto-efficient improvement), but I tend to just be using PyPy if I'm going with JIT approach. If for some reason I had to use CPython but had a few bottleneck functions that could be sped up with Numba I'd strongly consider it.

> I have been tempted by a more fancy IDE, but I'm so used to vim, I'm reluctant/scared to change.

I'm happy to report that JetBrains' VIM plugin is almost flawless (Block-based editing, window splitting, save/close, it's all where it's supposed to be).

I used to be in the same camp. But if you add a linter and parser and full blown backend to make lookups for code completion to your VIM configuration you might as well use an IDE. I've switched to IntelliJ for Java.

I did see that it had vim support, but its always difficult to know which _bits_ of vim it supports. Thanks for this, I'll investigate.
Regarding multiprocessing and serializing between processes in python, and linux specifically, have you tried SharredArray?

It dumps numpy arrays in /dev/shm which can loaded by other processes by the filepath, muuuuuch nicer than passing things around

Ooo, I have not. This sounds interesting.
> I really don't subscribe to the X is better than Y. I might concede that x is better than y for certain tasks, but not a sweeping generalisation.

Not even if X is better than Y for every specific task?

_if_ it was, then I'd be using that brush instead of the other. but to be honest, most languages are pretty much the same
Some issues:

- I make stupid mistakes in Python constantly. I misname a variable or a function or I pass in the wrong arguments: Try mypy

- If the task is IO bound: Threads are the wrong the answer to that. You're going async by using Golang, why not in python?

- With Python, I have to ensure all the packages I need are installed on the remote machine: pipenv Like slezyr mentions

- Consistent Styling: Try autopep8 or black

- Intellisense: I would say "use Pycharm" but vscode is getting better I heard

I had a very good experience using PyInstaller for a few of my Python programs, it made "binaries" out of them really easily.
Pylint is a bit hard to configure, use Prospector

Prospector does a better job and it prevents you from wasting time with what essentially are stylistic choices and BS like a hard 80 chars limit per line (remember, A Foolish Consistency is the Hobgoblin of Little Minds)

And yes, yes, automated stilling tools are better

Intelisense is needed but even the standard vim autocompletion plugin does an ok job

> You're going async by using Golang, why not in python?

In Golang, you're still using a thread-based abstraction.

In Python, "going async" involves switching to a slightly different style of programming and using different I/O libraries. While that has advantages, using threads lets you stick with the standard style.

“Python is much more flexible.”
Re: gofmt - python has black now. It's excellent.

Pylint solves the undefined variable issue, but it is annoying as hell and always requires configuration. I wish there was a linter for python that didn't care about 80 character limit and was careful not to drown important errors with unimportant stylistic tics.

Black is excellent ideed, and solves the problem of having code compatible with PEP8 once and for all.

flake8 is a linter that doesn't need much config, and you can config it to give you only "serious" errors and not stylistic issues.

I gave a talk on these subjects last year at the Paris Open Source Summit: https://speakerdeck.com/sfermigier/python-quality-engineerin... (NB: Black didn't exist yet at the time).

flake8, like black, shouldn't really require any config on most projects yet annoyingly it always requires at least a little. I really would switch to another tool in a heartbeat if they could just fix that.

Ideally instead of needing to be told whether to only show "serious" errors, for instance, it should just give you a project "score" (sum of the errors x their severity) and show the top 20 or so errors in order of importance and by default it should never exit with error code 1 with just stylistic errors unless explicitly configured to do so.

Have you tried pyflakes?
No, and I think you just made my day. Thanks!

Or rather, I'd vaguely heard the name but I erroneously thought it was just another flake8 type thing.

Black? Why the undifferentiated name? Another OSS naming fail.
The real benefit of gofmt is that it's zero-config and used by 100%±1 Go developers. Virtually all Go code is properly formatted and in the same style. It's priceless.
Does someone know a ressource where I can easily learn how to switch from Bash to Go?

To give some context: When it comes to serious programs I use Go already, but often I just want to code a little script to watch a website for changes for example:

  #!/usr/bin/env bash
  url="https://example.com"
  mem="memory.txt"
  oldsum="$(cat "$mem")"
  newsum="$(curl -sL "$url" | sha512sum)"

  if [ "$oldsum" != "$newsum" ]; then
    xmpp_notify "$url changed"
    echo "$newsum" > "$mem"
  fi
I haven't tried it yet, but in my mind it feels like writing such a program would be more time consuming/complex in Go than writing it in bash. Neverthelss, I feel like Go is overall a much better language than Bash (with all those quotes), so I would like to transition to writing more short programs in Go and for that I would like to learn how to do things like reading/writing/grepping files as easily in Go as I do it in Bash. Any idea if someone wrote some hints together already?
The benefit of bash is that it has system binaries as first class citizens; you can leverage the hundreds of thousands of lines of GNU projects with a single keyword. It's hard to beat that expressiveness.

Saying "Go is overall a much better language" is like saying a wrench is a better tool than a screwdriver. You should keep both in your toolbelt and switch out when the need arises.

For your question, start with the regexp and os package, learn how to open a file, read in the contents, and then while iterating over the contents, apply a regex to the text. That should get you to the point that you can search plaintext for character strings. Good luck.

Go is the wrong tool for that. I'm no fan of bash, but it's often the the quick, clear (enough) solution for small tasks. I'm not a python dev - maybe that would be a suitable compromise?
i'd say bash is the perfect tool for the problem that you demonstrated. go may be a better language but it's just another tool.
Depends how much you care about robustness. If the answer is "more than zero" than Bash is a terrible tool.
As a professional gopher, don't.

Just a few months ago I replaced an almost 100-line Go "script" with two lines in the Makefile. The fact that Go is great for big projects doesn't automatically make it good for everything. Go is for programs, not scripts.

Thanks for your warning :-)

My problems come from the other end: My Bash scripts tend to become larger. A few weeks ago I wrote a 'cache' utility function (very useful to cache curl requests for example). A few days ago I wanted to use it in a second script was just about to write a package manager for bash modules... I think that is the kind of situation you are getting into when the line count of a bash script increases and you are trying to do proper software development ;-)

So I am not going to build every script in Go from today on, but I would like to learn a bit more on when Go is a viable alternative to Bash.

Writing a program like that would not be too difficult in Go. It's something like this:

    func main() {
      url := "https://example.com"
      mem := "memory.txt"
      oldsum, err := ioutil.ReadFile(mem)
      if err != nil {
        log.Fatal(err)
      }
      
      // Download URL
      resp, err := http.Get(url)
      if err != nil {
        log.Fatal(err)
      }
      defer resp.Body.Close()
      body, err := ioutil.ReadAll(resp.Body)
      if err != nil {
        log.Fatal(err)
      }
      
      // Hash it
      newsum := sha512.Sum512(body)
      
      if oldsum != newsum {
        err = exec.Command("xmpp_notify", url + " changed").Run()
        if err != nil {
          log.Warn(err)
        }
        err = ioutil.Write(mem, newsum)
        if err != nil {
          log.Error(err)
        }
      }
    }
Don't listen to the naysayers. Go is fantastic for scripts like this - it's really good at network stuff, has a ton of built in functions, etc. And you get actual robustness and error checking that isn't a complete joke. Look how many ways your Bash script could fail! That's fine if you're literally watching the output with your eyes. If not it's going to cause issues when `curl` fails for example.

The only downside really is that you have to compile the Go code, which is fast but not as fast as just interpreting a Bash script (the first time).

Another possible issue is executable size - if you have a lot of Go scripts they may take up a fair bit of space.

I wish Go had an interpreter / JIT mode for scripts.

Edit: Before someone says "but that's 3 times longer than the bash code!" please write the equivalent Bash code that has proper error checking.

Just add this as a second line?

  trap "exit 1" ERR
I think you mean:

  set -euo pipefail
That is so repetitive to handle errors. Doesn't go have a cleaner way of handling errors than making the code look far less readable without unified error handling like exceptions?
It is a little repetitive. The best solution is Rust's Result system. But I'd say Ho's error handling system is second to that.

You might be thinking "but... exceptions!". They are a pretty bad solution. The problems are:

* Error handling is hidden - it's impossible in most implementations to know which exceptions can be thrown by a function * Flow control context is lost. If you wrap lots of lines with try {} you don't know which one caused the error and can't add context information. You end up with shit error messages like "file not found" rather than "error opening config file foo.cfg: file not found". The solution is to wrap every line in a separate try/catch but that is just Go-style error handling but shitter and even more verbose.

Go is a good balance. If you want proper error handling you have to actually write it. Yes it is a little tedious but that's what it takes to do it properly.

If you don't want to do it properly then fine, use Bash and set -e. But accept that it will be shit.

Thanks for giving an equivalent example in Go :-)

I think for this small size of an example program the error checking in Go is pure overhead, but as soon as the program starts to have a few more functions it will be very useful to have more options than just to abort on fail (Bash trap style / set -e). So I don't have any problem with 'polluting' my source with `if err != nil` checks ;-)

What I am more concerned about are the different ways I have to call functions from different libraries like

  resp, err := http.Get(url)
  ...
  defer resp.Body.Close()
  body, err := ioutil.ReadAll(resp.Body)
That is a lot more 'special' than the other function calls and the bash version and probably requires more documentation reading than '$ curl --help'. But hey, thanks to your example I have a starting point and probably are going to experiment with it in the new future.

For all the others who wrote that Bash is the right tool for the job: I read your warnings, opinions and suggestions and will keep them in mind while exploring the possibilities of using Go for script like jobs.

I love Go and use it for allllmost everything, but I think bash is still better for something like this. The ease with which it can run programs and capture output is the killer feature.

I'd love to see some future bash like language that's strongly typed, has a consistent and simple syntax, but retains these advantages.

for me, I use bash for anything less than 10 lines long, python for anything between 10 and 500 lines, and then I switch to a heavier programming language for anything longer.
Being such a concise language, Haskell could be a good option here. You can do quick scripting while still having a powerful type system (especially compared to Go's) backing you up. I somewhat quickly typed this code up and haven't tested it, but it should replicate that bash script pretty closely:

  #!/usr/bin/env stack
  -- stack script --resolver=lts-12.2
  
  import System.Process (system)
  
  import Control.Lens
  import Crypto.Hash.SHA256
  import qualified Data.Bytestring.Lazy as B
  import Network.Wreq
  
  url = "https://example.com"
  mem = "memory.txt"
  
  main = do
      oldsum <- B.readFile mem
      newsum <- (hashLazy . view responseBody) <$> get url
      when (oldsum /== newsum) $ do
          exitCode <- system $ "xmpp_notify " ++ url ++ " changed"
          B.writeFile mem newsum
I didn't make use of it, but there's also the Turtle library which aims to provide a fairly solid shell scripting experience, providing a lot of coreutils as simple Haskell functions. You also get a REPL in the form of GHCI, which comes default in Haskell installations.
Does the unreachable web page use Go or Python?
This is like saying "Goodbye car, hello airports!" If your needs change, then of course your favorite solution may change.
Goodbye plenty, matured, well tested libraries and frameworks and big community. Hello go.

(Nothing against golang, just wanted to point out it's not that easy for every usecase).

(comment deleted)
So much this. No one is writing large production applications without some sort of imported library. Go simply has not been around long enough or had enough users to mature. For all of the pros Go has (not even mentioning the cons) they seem miniscule compared to what we give up switching to it. It has always sounded like the hot new fad and not a stable thing. Especially with daily articles like coaxing people into using it.
Exactly! It will be many more years before big mature projects like Kubernetes, Docker or CockroachDB would consider switching to Go.
This are all sort of very close to system frameworks itself. I would never write something like that in Python. This is a good example for golang.
> I’m not a fan of full blown IDEs that require their own configuration.

Hmm, but those IDEs actually do solve most of the problems author has had with Python. If one regularly code in Python, spending a few hours configuring the IDE is justifiable. I for example never write scripts outside the IDE, because why?

It seems the author is one of those employees who think if he works hard he does a good job.
No they don't. No IDE can detect all typos in Python and do accurate code completion like they can for statically typed languages because the problem is unsolvable in general.

They can use heuristics to get fairly okish. Pycharm is not bad and VSCode's new Python extension seems to work fairly well. But it's never going to be as rock solid as Java code completion for example.

Note: I like Python, so this is me playing devil's advocate here.

If you need an IDE to solve common pain points of a language, isn't it fair to say that another language which lacks those pain points is a better fit for you? Using an IDE to aid you, to smooth things out and allow for more rapid development and better workflow is one thing. Being forced to use one because the language is lacking is a bad look on the language!

You could also say that this attitude holds us back. What if Smalltalk had only been accessible from vi back in the day, would Smalltalk have been as appreciated? I think not.
> I make stupid mistakes in Python constantly. I misname a variable or a function or I pass in the wrong arguments.

> Unit tests would catch most of these, but it’s hard to get 100% code coverage, and I don’t want to spend time writing unit tests for a one-off script.

Sorry, I actually do like golang but this is telling of a bigger issue. Someone else is going to have to delete/fix it

One of the biggest benefits of go vs python is that you only manage your dependencies during build. You end up with a single executable that is easy to deploy and mostly immune to the deployment of other things. In python, you have to also worry about deployment. To be safe, you need to carry the vm and other dependencies with you.

This means you can't just give any python program to someone and have it just work, unless it's pretty trivial.

Depends on your usecase, really. Static linking has been there for a long time and it has its disadvantages. I have more than a dozen programs on my system that share the same Python runtime and libraries. Compare that to every Go program shipping its own copy of the same things.
>every Go program shipping its own copy of the same things.

Is that really an issue? It takes up a bit more space, but nothing that's really any concern on a modern system. People often resort to using Docker to manage dependencies in other programming languages and end up shipping an entire copy of an operating system.

I honestly don't see many downside to shipping a static binary. At least in that case you can chroot the entire thing, something I really wish Python (and most other languages) would make a whole lot easier.

Security fixes require a recompilation, that perhaps the largest downside, but if you use Docker, then you need to rebuild that image as well.

Also, deploying to production is as easy as writing a small shell script to simply copy the new binary and restart your service definition. (or using ansible/puppet/chef/saltstack et al)
Yes, if you can keep things simple. If you've fixed a ux bug and your binary depends on a certain new version of an image/icon etc - it may or may not be that easy.
You can bundle assets, like icon, in your binary. In that case changing an image results in a new build and a new binary. Then deployments are still just one copy via Ansible, or similar.
Can? Sure. Should? Sometimes.
Go also carries all its dependencies with it; this is why it produces large single static binaries.

Python lacks a tool to painlessly produce the same, that is, a self-contained binary that includes the VM and all the dependencies. Determining Python dependencies may be slightly tricky, because `import` is an executable statement, and dependencies may be native-code libraries.

There are several tools for that, but all have their downsides, and none is in widespread use. Polishing such a tool until it's as simple and fool-proof as Go's build tool could be quite beneficial for Python community.

Pyinstaller solves the problem for me, but it can (in some particular cases) be a real bitch to configure properly with your dependencies at first.
Right, its not magic to build such a tool. But until it exists, this is a benefit for Go. This is also why I keep using Tcl, with tclkit you can build single-file executables for you tcl programs.
Why does it have to be a self-contained binary? When you pack everything into an virtualenv you get more or less the same result, don't you? If you want to have a single file you can zip it :)
Usability-wise, there's a huge gap between even low-effort and no-effort actions.

Another upside of a single binary is that it's not easy to modify; you can even sign it. Quite an upside if you want to audit what you deploy to your production that handles millions of dollars.

Can't you tar the python venv and sign it as well? Might be more difficult to get reproducible builds tgisbway, but it's sort of the same thing isn't it?
You can deploy an executable on a server just by typing

    GOOS=linux GOARCH=386 go build foo && scp foo user@server:
That's a lot of time saved, especially when you do it on a daily basis, and not something you can do as easily with virtualenv and such.
Yea I just do not buy it. How much time is saved? Is it writing an additional SSH command to extract the file or hitting the up-arrow on your keyboard to repeat the previously run command?
Not a lot, but when your test environment is on a remote server (because it replicates the production DB and you can't have that on your dev machine for security reasons), you tend to do it a lot. Add fast compilation to the mix, and you have a very productive workflow. `make && zip && scp && ssh unzip` is way slower than `go build && scp`
There's also xar[1] (and similar things like par), which provide similar functionality in python in a managed way.

Usage would be

python foo/setup.py bdist_xar && scp foo/dist/foo.xar user@server

if my understanding is correct.

[1]: https://github.com/facebookincubator/xar

Then you don't have any metadata about what's installed nor a way to roll back. I'd rather have automation that runs "sudo yum update $MYPACKAGE".
Good point. We found ways to overcome these issues, but that's a tradeoff, for sure.
Everyone seem to agree the generation of a static binary is easy with go. That's not my experience as soon as CGO and the glib is involved. I would gladly pay 100$ to make it work on my open source project using libvips to anyone who can create the static build: https://github.com/mickael-kerjean/nuage
Such a tool now exists and is called nuitka. See my other comment. It's very mature and can be trusted even with compiled extensions.
Of course you can do the same with python - that's what docker is for.
Unless your application also contains other assets like images etc. Now you suddenly need to package and install/deploy your Go application.
Or you just bundle them all in to the binary like a monster.
Is there an accepted way to do this?
(comment deleted)
Hello Win32 rc files all over again.
I do miss editing those with Resource Hacker or XN Resource Editor. I translated tons of Japanese and Chinese software into English that way back in high school and early college.

Good times.

And TBH properly packaging Go program is a nightmare if you want packager to build it and the build machine has no access to net.
Statically linked binaries are hardly new. As with everything they come with tradeoffs.
Speaking from the linux distro perspective, this is not true.

You can develop an application in python that utilizes distro-supplied python libraries. If you package your library in .deb or .rpm, it's trivial to utilize the dependencies from the distro.

Otherwise, you can install from pypi via pip, and especially into a virtualenv. You can declare each dependency's specific version if you so choose. Modern distros support python2.7 and python 3.x.

If you're on some other operating system, yeah, using python sucks for you.

While it's fair to say python is not statically compiled, it's no different than managing dependencies in other dynamically linked software.

Unlike python, go applications have to be cross compiled. For the most part, native python code will run anywhere the interpreter runs.

In the world of containerized applications, vendoring dependencies is a moot point anyway. We're all shipping around big tarballs with everything baked in anyway, so go doesn't offer any advantages in that space other than smaller binary size.

Not to say that go doesn't have some advantages over python, but I think the dependency management problem is vastly overstated, at least when it comes to python.

> You can develop an application in python that utilizes distro-supplied python libraries.

And unless you're aggressively tracking your distro's package releases you'd better hope that the new libdep doesn't introduce any breaking bugs. Also your app is no longer for Linux it's for Ubuntu 18.04 updated roughly on 2018-08-03.

The same goes for the actual python. Distributions apply lots of downstream patches and backport fixes so your python 2.7 is really redhat-python2.7-13 so you might want to test against that -- or you can bundle your own and be done with it.

Unless you have the wall clock time to actually define and test supported distributions you probably want to pretend the system python doesn't exist.

> pypi via pip, and especially into a virtualenv

I would never subject my users to this workflow. What you're describing are source distribution channels which are (ab)used to distribute applications. I have no idea if some dependency's native extension will even compile on their system. Do they even have build tools? Why would they?

> And unless you're aggressively tracking your distro's package releases you'd better hope that the new libdep doesn't introduce any breaking bugs.

Or use a distribution that does not break shit left and right, like Debian or Red Hat (CentOS).

> Unless you have the wall clock time to actually define and test supported distributions you probably want to pretend the system python doesn't exist.

If you write software that will be run by others (which usually means open source, probably libraries), yes. If you write software that will only be run by you (pretty much all dynamic websites a.k.a. webapps land in this category), you don't want to have three different distributions in half a dozen different versions anyway, so you can pin yourself to the target environment just as well.

> Or use a distribution that does not break shit left and right, like Debian or Red Hat (CentOS).

Sadly, there are a community of devs who want to rely on libflakey 0.0.1-beta, released an hour ago, and think that waiting for APIs to stabilise & distros to securely package things is unprofessional (!).

Using libflakey (lol!) is not necessarily a bad thing; The proper course would be to pull out what you need, make sure the interface you care about is sane and works well; you can revisit the upstream in the future.

Raw npm-style clone-from-master seems to be prolific, though.

> If you write software that will be run by others (which usually means open source, probably libraries), yes. If you write software that will only be run by you (pretty much all dynamic websites a.k.a. webapps land in this category), you don't want to have three different distributions in half a dozen different versions anyway, so you can pin yourself to the target environment just as well.

Yeah, my primary software development mode is 1) gather opensource software, 2) build web app or platform-specific app.

> Or use a distribution that does not break shit left and right, like Debian or Red Hat (CentOS).

It's crazy that this even has to be said in this day and age. I suppose distro specific patches to maintain a secure and stable api has gone away in favor of some vendored static dependencies that may or may not ever be upgraded, and rebasing your dependencies introduces the same set of problems.

I foresee dynamically linked go with platform specific binary packaging becoming the future. Recompiling the same bits of software ad infinitum is probably going to get old.

My industry, vfx, isn't as ubiquitous as web development, but it is fairly large with commercial interests to motivate making the "right" decisions and not too odd of a use-case. I've heard some second-hand stories about Guido crashing on the couch of some guys at ILM back in the "early days" helping to spur adoption. However, we're still pretty solidly on Python2 with proposals to start adopting Python3 next year. Most of the commercial applications run Python inside them and the apps are very, very scriptable.

Houdini is a fairly ubiquitous tool for creating dynamics; fire, water, destruction, dust and has been around since 1996. In general, Houdini users tend to prefer Ubuntu and most medium to large vfx houses use CentOS/RHEL (Pixar, DreamWorks, ILM).

For Linux and macOS, Houdini uses your system's Python but it has to monkey patch parts of the standard library in order to work smoothly. You can set a flag (and on Windows this is the default) where it will use the Python shipped with the application.

At my current job we're using slightly out of date versions of everything, which is quite common in production. We're running CentOS 7.2, Houdini 15.5 (released Nov 2017), and Python 2.7.5. If you call "httplib.HTTPSConnection('google.com')" it throws an exception because of ssl library changes across 2.7. I haven't tried any other versions, but the requirements for the current version of Houdini says "CentOS 6+ (64-bit)" (among other OSes).

This isn't unique to Python, but it's very hard to properly support real-world clients and dynamic libraries among operating systems, commercial applications, and less well funded open source or independent efforts.

My M.O. is if a technology is core to your business, you should control it. Which means not using the OS' Python. Upgrading Python and OS independently makes so many things much easier when you have a large codebase.

> I would never subject my users to this workflow. What you're describing are source distribution channels which are (ab)used to distribute applications. I have no idea if some dependency's native extension will even compile on their system. Do they even have build tools? Why would they?

If the users of your software are end-user desktop types, I can see your sentiment. My software is typically for building software (aka, libraries) or users that know how to compile things.

Not entirely true anymore.

Nuitka (nuitka.net) has been able to compile python programs to a single stand alone executable reliably, robustly and easily for a while now.

It's a real gem and deserves to be more popular. We are very far from the quirks of pyinstaller or cxfreeze : it just works, even with numpy or qt.

Yes, go is still superior for deployment :

- it can cross compile

- executables are smaller

- it's built in so it's just an easier experience

- it doesn't depend on libc

But it's important to spread the word: the time were your python program was hard to ship is over.

Just use nuitka.

op literally said:

> To be safe, you need to carry the vm and other dependencies with you.

You do the equivalent with Go too, it's just much smaller and the norm, rather than something remarkable.
Absolutely true.

You should make it easy as possible for users of your software .

Wow, that seems like a pretty big jump. Good for the author.
> I make stupid mistakes in Python constantly. I misname a variable or a function or I pass in the wrong arguments. Devtools can catch some of these, but they usually require special setup. I’ve never been able to configure pylint easily, and I’m not a fan of full blown IDEs that require their own configuration. The worst is if you mistype a variable that’s hidden behind conditional logic. Your script might run for hours before triggering the error, and then everything blows up, and you have to restart it.

This this this, a million times this. I work mostly in Clojure where such problems are equally as bad, maybe even more so. Dynamic, elegant languages can feel liberating in so many ways, but there are significant workflow drawbacks that get me nearly every day.

I program mostly in ClojureScript which will "throw" a compiler warning if I misspell a variable. But to get that in an editor is a bit more setup than to just fire up vanilla vim.
Doesn’t stop you from passing in wrong arguments, your function requires an int and you give it a string because you typed the wrong variable name there, or you mistype a keyword, which is never caught by the compiler, and you end up with nil floating through your app.
Can’t you just use type hints (in both clojure or python 3)?
They don't take care of this issue most of the time, and in Clojurescript they are meaningless. In Clojure, only hints on primitive Java types (integer, etc) do something meaningful, so any custom Clojure types or your own "types" have no such checking, and in particular keywords are always valid if they are mistyped which results in a running program with no errors that creates nil and null pointer exceptions and undefined and other things all over the place.

It's a tradeoff for using an expressive dynamic language, I guess.

Type hints are not taken into account by the Python interpreter. They do help an IDE like pycharm, but that won't catch all errors.
First off, I get that it's unfortunate they aren't detected by Python itself in some form of compile step, but that's a cost of using a dynamic language.

But they don't use the tools created to catch these kinds of errors, and complain that they aren't protected against these errors? There's a fairly large disconnect there. Hell, I have protection against this built into my VIM install. Not running these basic tools is tying a hand behind your back.

For those who lack the generics, I just wanted to remind that you can "script" in Haskell by adding this to your header:

    #!/usr/bin/env stack
    -- stack script --resolver=lts-12.2
and run it as an executable if your machine has stack installed. If not -- compile it with a `stack ghc --resolver=lts-12.2 ./file.hs` and you'll have a `./file` executable. No project creation, no need to list the dependencies.
Good trick! Do you also know the Turtle library? I just got across it, and it looks like a neat shell library from the outside.
This makes me want to try Go, though I wonder how long it would take to develop the same "fluency" and knowledge of which library to use. It would be exiting to have things run fast though...

One minor plug `pyflakes` is very good for basic checks for mistyped variable and missing imports, etc. It's not "smart" and doesn't check type consistency, but very useful to catch stupid mistakes.

I went from Python to Go; Go is an amazingly simple and quick to learn language and ecosystem. Give it a try -- you'll probably start to feel proficient a lot faster than you'd think.
Python is regularly ranked to top 4 programming language for a reason, Go has a long way to catch up I think, if it ever happens.

I recently started using python as the primary language, the more I use, the more I like it.

Go's put everything in a static binary has its pros and cons.

I've been using Go over python for a couple months now and so far it has been incredibly effective. I never really liked Python's syntax so it's nice to go back to something c-ish.
(comment deleted)