Python used to be my go to language. For me it was clear, easy to read, had a lot of great packages. Trying ideas out in the REPL was handy. (I've since mostly abandoned it for golang, and wonder how any of my Python ever worked considering how many silly mistakes the compiler catches).
Yeah, both Golang and Swift are direct descendants of Python, and it's sort of hard to justify a niche for Python now that Google and Apple have performant in-house versions.
I have to wonder if Go and Swift would've just been Python if Guido had put more attention on the Python VM (and been willing to make some compromises to accommodate improvements).
Now Python is stuck playing catch up. Discussion around removing the GIL is finally beginning to be taken seriously instead of considered hypothetical, optional type annotations are being added, etc., but it may be too little too late.
1. There's no reason that Python has to be "the language of ML", especially not with so many new Python-inspired languages, like Go. ML is a performance-sensitive sector. Python's strong/established C FFI is its selling point here, since Python gives such nice icing on the cake buried in a C module like NumPy, but this will only matter as long as the newer languages are not fast enough for the performance-sensitive bits. Once native Go is fast enough, it seems logical that everything will move to Go instead of C+Python.
2. "ML is the future" is vague and meaningless. ML is certainly not the future for many things, and it's certainly being misapplied on many other things in the cargo-cult belief that ML must be used everywhere.
Actually there are several reasons that Python has to be the language of Machine Learning and Data Science. It begins with two decades of being promoted as the most accessible language for scientists to get the job done. Then the widespread use of Python led to an explosion of libraries creating a rich ecosystem.
Nowadays there are sophisticated tools like Orange3 and Jupyter (not to mention Beaker Notebook) that make it easy to share code within teams of scientists. None of this is going away and even R users are discovering that it is easier to use R for the stats part of the problem and integrate with Python for everything else. R used to have the best presentation graphics but now Python has caught up with improved Matplotlib and Bokeh. Also some people prefer to use d3.js in the browser moving from just presenting their data to enable exploration of it.
In the ML/DS area, CPUs will never be fast enought with Go. You need to exploit the GPU which Python has good support for, or move to distributed computing which Python also does well.
No other language will dislodge Python. Instead we are making integration more prominent and using multiple languages working together. Beaker Notebook is particularly good at this and is driving Jupyter in that direction too.
there is no reason that python has to be the language of ML.
there is no reason that english has to be the language that is spoken when foreigners interact with each other. but somehow, it is. and it doesnt really change.
I'm in ops and python is pre-installed on debian and fedora so besides writing stuff in bash, python is my next best thing without installing other languages. I would like to do more advanced things in python but I don't have any big projects / automation in my head yet.
I would say that Ansible, with its all SSH baggage, is quite poor method of
running remote procedures, especially when it's mixed with maintaining
configuration. But d'oh, it's the usual way people abuse it.
Yea i'm tying python into that stuff where I can. But i'm hoping to do something not ops related with Python eventually. Maybe implement my own monitoring dashboard with django and some other front end framework.
Python used to be my "main" language, but that was only due to familiarity. For most IO intensive tasks (especially networking), Python is not the best choice [0]. Recently I've been using nodejs with promises and async/await for heavy IO. I really like the productivity gains of Javascript's functional programming style, and the event loop lends itself well to network/IO intensive tasks. (I would prefer to use golang for this, but haven't gotten around to learning it yet)
Now I use python when it's the best tool for the job, where the "job" is a part of a project. My projects usually involve a combination of three languages: python, javascript (node and/or frontend), and bash scripting. The bulk of the code is node, but for scripts I use a combination of bash and python.
I use bash scripts for anything that involves piping data between processes, simple file I/O, backup scripts, etc. I prototype almost every script in bash. When it gets too complicated, or requires parsing a lot of data, I switch to Python (sometimes I even inline the Python in bash with heredocs).
In my experience, bash is usually the best tool for the job if the "job" involves invoking lots of shell commands and piping their outputs somewhere. Using Python to call shell commands is of course possible, but I find it to be much less readable than bash for that specific use case.
Nothing beats Python for parsing and manipulating data, which is why it can be so powerful when combined with bash. Here [1] is an example bash script I wrote in ~30 minutes that uses inline Python. It deploys schema from a JSON file to parse-server. Bash handles the raw IO of reading files, network requests, etc, but inline Python does the simple manipulation of the JSON files.
[0] I haven't worked much with python3 or async/await... but that looks promising (hah) for increased throughput. I've even seen some benchmarks inline with node.
Automation. I'm building a dev environment that does various bits and pieces transparently - DNS, databases, reverse proxying, backups/restores, secrets, stacking/merging/relating containers etc.
I kinda just stumbled into it as the logical way to move forward once the shell scripts I started out with became too complex, but I've fallen in love with it. I'm sure it will be my go-to language for a lot of coding tasks in the future.
I work on a GUI application written with PyQt. It's been very productive for my small team. The downside is that Python's permissiveness has bitten us over and over again. We've become focused on managing mutability --- it's the central problem we have to solve.
The biggest reason I use Python? Honestly, because it's great for interview-type problems, comfortable for writing on a whiteboard, and suits one-man projects where I don't find myself wishing for static typing or interfaces as a way to coordinate code.
P.S.: I'm sure you can make big multi-team projects in Python with the right discipline and tooling, but that's just not the spot it occupies in my language-toolkit.
Likewise for me — it's the best general purpose, (very) high-level programming language.
I don't think it's really even playing favorites when it's a competitive language with a very diverse collection of good libraries across applications and domains.
Python is great for prototyping. I work a lot with spatio-temporal textual datasets (think geo-tagged tweets), and most of the time, I need to verify my idea actually works before working on performance improvement using C++.
I use it most for numpy, pandas, scikit-learn, and a bit of graphing. I use it because there is so much data processing, formatting, piping, munging, and so forth involved in analytical work in data analysis, and I think python brings in the ideal combination of general programming and stats/numerical libraries.
I don't use it for web programming. If I had my choice, I'd still use an integrated system like rails or Django, and bring in javascript sparingly, but unfortunately, that's not my choice, so I end up using a lot of Javascript.
Jupyter Notebooks + Theano + Lasagne + Matplotlib + [the pip package ecosystem]. For my weird little niche (Machine Learning / Computational Intelligence), it is the perfect tool. It's also the perfect language for basic management scripts - a handful of lines of Python made setting up a shared machine learning server with Slurm/Jupyter-Hub/IPFS/Theano+Lasagne+Tensorflow (the last of which I've not personally used) easy. I work with a tmux session with scripts running training nets, and a Jupyter notebook open to evaluate and explore parameters and behavior at save points each epoch. Python/Theano/Lasagne allow for concise easily readable code that documents and describes the concept at hand such that my code cells wind up containing few, if any, comments - instead writing anything that needs to be written in markdown with any inline graphic needed in cells between code cells. For real-time work like mine, scripts are likely to be run once (or rather several times, changing meta-parameters along the way), and most code written will be notebook cells either soon to be deleted or as part of a record that should ideally be in clear readable language.
Django. A great web framework with great third party libraries such as Django Rest Framework, which will help you to quickly get your app up and running.
It was easy to learn, has great libraries to do virtually anything and I haven't yet found a need to require strict static typing (I haven't explored MyPy yet though). More importantly, I know the internals of the language pretty well and I invested a lot of time in learning them. So I'm kind of reluctant to switch to another language, although I want to explore Go.
I have been using Python to interact with APIs, download data and files, then manipulate the data and put it into a database. Then I set up my own APIs (Python cgi-bin scripts) to access the data. I also use the PIL library for image thumbnailing.
Years ago I would have done all of this with Perl 5. I started using Python because so many people were talking about it. What really got me using it was that a programmer told me he thought Python pip libraries were as good as Perl CPAN libraries. He was right, although in some instances one particular library can be superior to the other's equivalent.
38 comments
[ 3.3 ms ] story [ 111 ms ] threadI have to wonder if Go and Swift would've just been Python if Guido had put more attention on the Python VM (and been willing to make some compromises to accommodate improvements).
Now Python is stuck playing catch up. Discussion around removing the GIL is finally beginning to be taken seriously instead of considered hypothetical, optional type annotations are being added, etc., but it may be too little too late.
2. "ML is the future" is vague and meaningless. ML is certainly not the future for many things, and it's certainly being misapplied on many other things in the cargo-cult belief that ML must be used everywhere.
Nowadays there are sophisticated tools like Orange3 and Jupyter (not to mention Beaker Notebook) that make it easy to share code within teams of scientists. None of this is going away and even R users are discovering that it is easier to use R for the stats part of the problem and integrate with Python for everything else. R used to have the best presentation graphics but now Python has caught up with improved Matplotlib and Bokeh. Also some people prefer to use d3.js in the browser moving from just presenting their data to enable exploration of it.
In the ML/DS area, CPUs will never be fast enought with Go. You need to exploit the GPU which Python has good support for, or move to distributed computing which Python also does well.
No other language will dislodge Python. Instead we are making integration more prominent and using multiple languages working together. Beaker Notebook is particularly good at this and is driving Jupyter in that direction too.
there is no reason that english has to be the language that is spoken when foreigners interact with each other. but somehow, it is. and it doesnt really change.
Don't roll your own monitoring solution, but tie Python scripts into Nagios, etc..
Why should a sysadmin use Nagios at all?
Recent C++ with lambda and auto is much improved but still not nearly as productive as Python.
Likewise lambda has really made Java better, but lack of type inference really makes it clunky to write.
I really want a statically typed Python. But because I don't have it, I choose Python for the syntax over c++/Java even for performance critical code.
Can golang fill this role?
Python also has optional type support since 3.5: https://docs.python.org/3/library/typing.html
What do you base that assertion on? :)
You need to give type annotations a shot! https://www.python.org/dev/peps/pep-0484/#abstract
Now I use python when it's the best tool for the job, where the "job" is a part of a project. My projects usually involve a combination of three languages: python, javascript (node and/or frontend), and bash scripting. The bulk of the code is node, but for scripts I use a combination of bash and python.
I use bash scripts for anything that involves piping data between processes, simple file I/O, backup scripts, etc. I prototype almost every script in bash. When it gets too complicated, or requires parsing a lot of data, I switch to Python (sometimes I even inline the Python in bash with heredocs).
In my experience, bash is usually the best tool for the job if the "job" involves invoking lots of shell commands and piping their outputs somewhere. Using Python to call shell commands is of course possible, but I find it to be much less readable than bash for that specific use case.
Nothing beats Python for parsing and manipulating data, which is why it can be so powerful when combined with bash. Here [1] is an example bash script I wrote in ~30 minutes that uses inline Python. It deploys schema from a JSON file to parse-server. Bash handles the raw IO of reading files, network requests, etc, but inline Python does the simple manipulation of the JSON files.
[0] I haven't worked much with python3 or async/await... but that looks promising (hah) for increased throughput. I've even seen some benchmarks inline with node.
[1] https://gist.github.com/milesrichardson/b30acef8827c53aae088...
I kinda just stumbled into it as the logical way to move forward once the shell scripts I started out with became too complex, but I've fallen in love with it. I'm sure it will be my go-to language for a lot of coding tasks in the future.
P.S.: I'm sure you can make big multi-team projects in Python with the right discipline and tooling, but that's just not the spot it occupies in my language-toolkit.
I don't think it's really even playing favorites when it's a competitive language with a very diverse collection of good libraries across applications and domains.
I don't use it for web programming. If I had my choice, I'd still use an integrated system like rails or Django, and bring in javascript sparingly, but unfortunately, that's not my choice, so I end up using a lot of Javascript.
I don't care about the language nor its ecosystem the slightest. If I need a dynamic language I stick to Perl or Chicken Scheme.
Years ago I would have done all of this with Perl 5. I started using Python because so many people were talking about it. What really got me using it was that a programmer told me he thought Python pip libraries were as good as Perl CPAN libraries. He was right, although in some instances one particular library can be superior to the other's equivalent.