84 comments

[ 504 ms ] story [ 1833 ms ] thread
That's a lot of extra character overhead for not very much gain.
It means that static analysis tools can do a much better job of finding issues in code, and will make code completion much better. I've worked on some pretty big Python codebases in my years, and both of those would be big wins. Currently people do the same thing in a much more janky and inconsistent way in docstrings.
Of the the large python code bases I've worked on (4 at current count), type errors have tended to account for a small proportion (around 4-8%) of production issues.

If this feature came at relatively little cost this would be great - 5% of bugs squashed just like that. It doesn't. It contributes significantly to the verbosity of the code - chipping away at one of python's main strengths - concision and clarity.

> Of the the large python code bases I've worked on (4 at current count), type errors have tended to account for a small proportion (around 4-8%) of production issues.

Some projects are more than eager to cut up to 8% of potential bugs with such a trivial change.

>If this feature came at relatively little cost this would be great - 5% of bugs squashed just like that. It doesn't. It contributes significantly to the verbosity of the code - chipping away at one of python's main strengths - concision and clarity.

I'm not sure you're being serious. Stating a type in a variable declaration doesn't harm concision, and obviously it does exactly the opposite of what you claim regarding clarity.

>Some projects are more than eager to cut up to 8% of potential bugs with such a trivial change.

Except it's not trivial, it's not 'free' - it adds noise to the code base and the idea it will eliminate all typing bugs is ludicrous.

>Stating a type in a variable declaration doesn't harm concision

Java is a good example of language that has explicit type declarations for every variable. That's one of the reasons why the average Java program is 2x as long as an equivalent python program.

> obviously it does exactly the opposite of what you claim regarding clarity.

No, not obviously. Are Java programs easier to read than python thanks to their explicit variable declarations and tendency towards verbosity?

> Except it's not trivial, it's not 'free' - it adds noise to the code base and the idea it will eliminate all typing bugs is ludicrous.

Except it is trivial. Just add the type declaration like it says in the PEP. There. Problem solved. Who in their right mind makes a fuss for such a trivial thing?

> That's one of the reasons why the average Java program is 2x as long as an equivalent python program.

That's a ridiculous thing to say, and is entirely oblivious to the issue of boilerplate code that is mandatory in Java.

> No, not obviously. Are Java programs easier to read than python thanks to their explicit variable declarations and tendency towards verbosity?

You're trying to misrepresent a personal (and miss-guided) opinion that you formed on a particular programming language as being representative of a single issue.

If your argument was sound and remotely rational, you would've pointed something in the PEP to discuss the merits of the PEP. Instead, you invent problems where there are none by making up stuff regarding an entirely different programming language.

It sounds like you're very opinionated about something you clearly have no ground to criticize or comment on. Therefore it's pointless to continue with this discussion.

>Except it is trivial. Just add the type declaration like it says in the PEP.

For every single variable declaration. This will increase the length of your program.

>That's a ridiculous thing to say,

Are you asserting that program length will not go up when adding explicit types? That seems a peculiarly indefensible position.

>You're trying to misrepresent a personal (and miss-guided) opinion that you formed on a particular programming language as being representative of a single issue.

Lack of verbosity is one of python's main strengths.

>If your argument was sound and remotely rational, you would've pointed something in the PEP to discuss the merits of the PEP.

Which I did which you would have picked up on if you were remotely reading what I was actually saying: it picks up type errors that would otherwise not be caught.

It does that at a cost: longer code that does the same thing.

You seem to fail to understand that this is a trade off.

> Therefore it's pointless to continue with this discussion.

Agree about that.

It's not just about that, but also about catching when people are doing type fuckery (that's really the best term I can come up with), i.e., doing silly things with the type system. I have one particular co-worker who's particularly bad when it comes to this.

Also, while it works almost miraculously well as-is, type annotations would make the likes of Jedi much better. It might not seem like a big thing, but autocompletion is really useful with larger codebases and libraries you don't necessarily use every day.

There's very little extra cost with this over and above what's currently there. All it's doing is formalising something people have been doing in docstrings for decades already.

> It contributes significantly to the verbosity of the code.

No, it doesn't. The only places this is meant to be used is at interface points, i.e., declaring object member attributes and method/function signatures. We're not talking Java or C++ levels of type annotation here: we're talking Go, Haskell, and O'Caml levels of type annotation at most.

And this is the core of the FUD around type annotations: people act like it's introducing Java and C++ levels of needless crud into the language, when the actual number of places where it's meant to be used is much smaller.

Not very much gain? Maybe you are able to write Python without introducing type errors, but I know that I can't keep track of it, and it's my impression that the rest of the world's Pythonists can't either. At least when the project reaches a certain size. Scripting is another matter.
I've worked on multiple large projects, and while type errors are a problem, they aren't in the top 5 classes of 'most commonly reported bugs in production' - not if you have tests and a sprinkling of sanity checks, anyhow.

Furthermore, the most common type errors I do see would be solved more effectively by stricter runtime error checking than they would be by slathering the language in static typing.

For example:

    if not x:
        do_something()
This example of perly weak runtime type checking is probably responsible for a larger proportion of bugs I see in production than most issues that static typing would solve.

Ironically just a month ago a junior changed some of my code from :

    if x is not None:
         do_something()
To:

    if not x:
         do_something()
Because it was "more pythonic".

Not long after he did that he hit a bug because it wasn't supposed to do_something() if x was a blank string. Whoops.

    > I've worked on multiple large projects, and while type 
    > errors are a problem, they aren't in the top 5 classes 
    > of 'most commonly reported bugs in production' - not   
    > if you have tests and a sprinkling of sanity checks, anyhow.
My claim is that Pythonists are completely unaware that almost all of the errors they experience could have been avoided by just informing the compiler about what type a value is. I find it peculiar that people prefer writing tests to verify type correctness (a lot of work), rather than just having the compiler check it through simple type annotations. I say this coming from Python, having moved to to Haskell. Two diametrically opposite worlds in terms of type strictness.

I think sprinkling type annotations all over the code is much preferable to writing test code. We have a type system such that the compiler can produce the code needed to check types, rather than us having to write this code in a test.

In my humble opinion, humans shouldn't have to write test code. If you can't get the compiler to generate the test code you want - by constructing the types carefully - you're wasting a lot of time, because you have to write both the implementation, and a second implementation which tests the first one. So every time you do a rewrite, you also have to rewrite your tests, so that the two match. Double work.

Having made the same switch from Python to Haskell I agree. However, I just wanted to temper your claim that "almost all of the errors" could be prevented by type systems. I think it's hard to measure errors in any way such that "almost all" is a valid claim, but "very many" certainly is!
>My claim is that Pythonists are completely unaware that almost all of the errors they experience could have been avoided by just informing the compiler about what type a value is.

You literally didn't have to say anything else to let me know that you're a haskell developer :)

My claim is that developers who habitually don't do TDD like the methadone hit that static type checking provides.

I actually write very almost no tests to directly verify type correctness but I always write a test for any new behaviour.I implement. 95% of the type errors I do encounter indirectly come out in the process of writing tests that I would have written anyway in any language.

>I think sprinkling type annotations all over the code is much preferable to writing test code.

And I think if you're avoiding writing tests to define what your code should do you're a shoddy programmer.

I've been sprinkling my programs with sanity checks and runtime type checks for a decade which has been more than sufficient at eliminating almost all bug-causing type issues the tests don't catch.

This isn't something I even need to do often so any alternative to this process isn't going to yield me much of a productivity gain.

>In my humble opinion, humans shouldn't have to write test code. If you can't get the compiler to generate the test code you want - by constructing the types carefully - you're wasting a lot of time, because you have to write both the implementation, and a second implementation which tests the first one. So every time you do a rewrite, you also have to rewrite your tests, so that the two match. Double work.

Wherein you just indirectly addressed the biggest complaint people have with the haskell community - it's horrendously shitty documentation. You're supposed to figure out the behavior of libraries just by looking at the type.

Good tests tell a story.

The tests I write for libraries double as readable documentation for that library. The tests I write for applications double as readable executable specifications. This is a relatively new thing hampered somewhat by tooling issues but I think it'll catch on.

For libraries that I use I often ignore the official documentation and treat the tests as documentation instead.

Compilers can't do this for you. They don't know how your code is supposed to work.

Ironically one of the best things to come out of the haskell community was quickcheck - a testing library.

    > And I think if you're avoiding writing tests to define 
    > what your code should do you're a shoddy programmer.
At the opposite end of the testing/no-testing spectrum would be the statement: if you can't look at your own code and see whether it does what you intend it to do, but need to write a test to make sure of this, your code is either of poor quality or you're not paying enough attention.

I think both viewpoints have merit. But neither one paints the whole picture. Tests are useful to a certain degree, but since time is limited, it's all about striking the right balance between testing and no testing. You can always write more tests, but at some point it becomes more useful to just look at the code you're testing. And, conversely, you can always look at your code one extra time, but sometimes it's better to formalize your doubts into a test.

Yeah, it seems it's time to abandon Python since the static typing fundamentalists seem to be getting hold of it

Extremely frustrated by this decision and even though the multiple denials that it won't become a statically typed language it's clear that idea has changed

What's wrong with optional typing? All it does is formalise docstrings into an integrated syntax.
It's a slippery slope

People will start using and it will become mandatory at some point

I can see the medium post now!

"Dynamically typed Python variables considered harmful"

As long as it remains optional I'm all for it.
Docstrings are for modules, classes, functions and methods, not variables mixed with the code.

And "optional" is a misnomer unless you only ever touch your own code. Programming is generally a social activity, so for most of us, "optional" means just means "inconsistently required".

I don't have a philosophical objection to typing, but I'm yet to see a syntax that won't muddy Python's.

> And "optional" is a misnomer unless you only ever touch your own code. Programming is generally a social activity, so for most of us, "optional" means just means "inconsistently required".

Optionally means that it's not required or mandatory.

You don't need to use it if you don't want to. If you're working on a project and the project follows a style guide then you follow the style guide, but nothing forces your bosses to require it. The code will run anyway.

That's great for my bosses. I'm not my bosses, though, at least as yet.
It isn't good or bad. It's an optional feature. No one is forced to use it unless they specifically want it.

Your bosses aren't forced to use it. This means that if they don't want to use them and they don't believe the official style guide has to impose them, you don't have to use them.

If your bosses decide to impose them and change teh official style guide to require type annotation then your personal taste doesn't have any say in the subject either way, and you only need to act like a professional and be a team player.

But the key issue is that no one is forced to use optional features unless they really want to.

It does seem odd that a module could be partially static. If it was optional I'd prefer to see it enabled by a different filetype (e.g. pys, heh piss).
It's optional until the lead developer at work forces you to use it.
Currently they can force you to use a docstring with types, how is that any different?
Docstrings are not as interspersed with the code.
> Yeah, it seems it's time to abandon Python since the static typing fundamentalists seem to be getting hold of it.

It seems it's time to abandon Erlang since static typing fundamentalists seem to be getting hold of it, with its typespecs and Dialyzer.

Oh wait, it's still dynamically typed.

Dialyzer seems to be Erlang's Pylint (which does type analysis but still lets you do dynamic typing)

And I'm all for Pylint, it also proves type hints are not needed for this kind of type analysis

Well, sort of. Dialyzer can do quite well if you run it on code with no typespecs whatsoever, but if you add those, it can detect so many more mistakes (which is the whole point of such tools).

No typespecs feels like crippling the tool, except it was designed to be as useful as possible for old code without modifying it. It just shows how useful the typespecs are.

Why not the other way around:

    my_class_attr = ClassAttr[bool]: True
This seems more familiar somehow.
A few reasons, not least of which is consistency with function/method signature declarations.

Also, that style is only more familiar if you're coming from C, C++, Java, &c. However, the style in the PEP is pretty common in languages outside of the C family, such as those inspired by Pascal, and there's at least two examples of C family languages - Go and Swift - which use the style in the PEP.

Also, putting the declaration on the LHS likely results in fewer grammar ambiguities, but it's well over a decade since I last looked at Python's grammar, so that's just speculation on my part. I know that's one of the reasons why Swift and Go went down the route they did, and Python would have even more reason, given how ':' is used in the language.

A very sad day. We were told type hints were just that, optional hints to help IDEs and such. Now they're creeping into the language proper; IDEs will start adding them for you, and before you know it all "serious codebases" will be statically typed.

The beauty of python was that it could read like pseudocode; more and more special characters and notations are putting that idea to rest. I guess it's the price to pay for mainstream success -- when banks get involved, everything becomes uber-formal and soul-crushing.

> [...] before you know it all "serious codebases" will be statically typed.

And this is bad because...?

because they will lose readability and raise the bar for contribution from novices. Every extra feature you add to the language requires extra effort to read and extra effort from novices to understand.
>>> [...] before you know it all "serious codebases" will be statically typed.

> because they will lose readability and raise the bar for contribution from novices

OK, hold for a second. Adding something to make the code easier to follow will make code harder to follow? Is this really your argument?

And why would a novice to a language want to contribute to "serious codebase"? I don't think Django, Jinja, or NumPy are popular for Python novices to contribute code.

"Readability" depends on a context. I've worked with a huge codebases where having a variable types attached to the variables there would actually improve the readability. Sure, it would take you a moment longer to skim the line, but overall it would save you a much more time to figure out what's the variable about.
I don't know about this. My feeling from working with Typescript and Javascript is that the type hints really help people understand what is happening. My background is as a Python programmer, but Typescript really convinced me that optional type hints are a super-valuable addition to a dynamic language. Most of my docstrings in python tend to be type annotations anyway -- might as well make it possible for tools to support it easily.
@dozzie, I don't think static typing does make it easier to read, it makes it more verbose.
Are you confusing “static typing” with “type annotations”? You can write large swathes of statically typed code without a single type annotation, in languages with proper type inference. This is 70's-era technology, mind you.
You're right, I'm conflating the two, though I appreciate that one doesn't require the other.
And this is good because?

You want Java you use Java. Oh it sucks? I wonder why...

Oh even better, you set a type, does it prevent you from using things with a similar interface? Like iterables in place of lists?

> And this is good because?

Better code documentation? Static analysis? Compare to Erlang with typespecs and Dialyzer.

> You want Java you use Java.

Why Java? Why not SML, OCaml, Haskell, Rust? Those are statically typed languages, too.

> Oh it [Java] sucks? I wonder why...

Because it's statically typed yet it's dynamically typed ("Object" class thing)? Because its type system is quite primitive and somewhat inconsent? Because it was designed for use in TV sets two dozens years ago, and has barely advanced as a language since then? (No, foreach loops and lambdas don't qualify as a significant step, those are merely syntax sugar.)

> Oh even better, you set a type, does it prevent you from using things with a similar interface? Like iterables in place of lists?

Type specification signifies code author's intent. This is a very good thing, and I want to see it much more in the code. I don't expect type specs/hints to do anything more for a dynamically typed language.

> Haskell, Rust? Those are statically typed languages, too.

Developers on those languages usually don't push for crap like the Java developers do

And I don't want to use Rust or Haskell where Python can solve the problem just fine. Overhead and development time is also a problem

Type hints add development time and bureaucracy. The 80 columns limit was supposed to be a guidance as well but now it has become almost mandatory

> The 80 columns limit was supposed to be a guidance as well but now it has become almost mandatory

Only for those whose pinnacle of programming editors are from a VT-100 era.

Yes, it is mandatory. People just use pep8 utility as a linter and block the code being committed/tested if it doesn't adhere to that

Because thinking and reading the first phrase of PEP 8 is too hard, cargo culting is easier.

(comment deleted)
>> Haskell, Rust? Those are statically typed languages, too.

> [...] I don't want to use Rust or Haskell where Python can solve the problem just fine. Overhead and development time is also a problem

Yet you suggested using Java. Something in your comments seems inconsistent.

> Type hints add development time and bureaucracy.

That's why they are (going to be) optional. But for larger codebases beaurocracy adds some structure, which can be beneficial.

BTW, did you know that beaurocracy was an invention of ancient times? Yes, an invention. It allowed citizens to know what to do with their requests to authorities. Beaurocracy is generally a desired thing, it's excessive beaurocracy that is awful.

(comment deleted)
Why [T]? Why not <T>? It confuses with array selection
I imagine it's to avoid to change too much the parser. [] is already used to enclosed stuff, while < and > are operators.
Just to be different (see the ternary operator for a past example)? And <T> confuses with the comparison operators.
Bah. If you want a statically typed language, get a statically typed language.

What's a type hint for if, after that, the user can actually pass a different type?

IDE code completion and linting, which is admittedly an area where Python sucks.

But I'm inclined to agree. It's a little too to add static typing to Python. You're just going to end up with something far worse than a language that was designed with it in mind, like Ceylon or Go.

Python is a language where I simply don't miss code completion.
Why not? Code completion is definitely useful in Python (when it works, which is rarely). It saves time looking up documentation.
I find I can keep most things I currently need in my head with Python.

Which is nice as some IDEs tend to offer code completion popups OVER the line I'm typing which is very annoying and breaks my thought process.

Also, for _me_, not using code completion helps me think more about what I'm actually doing and not code so much on autopilot. I actually write much better, succinct code without completion.

YMMV

Sounds very like an excuse to me.
A type checker for Python can say: "pass", "reject" or "I don't know".

Statically typed languages (1) introduce limitations so that type checking is decidable (no incertitude), or (2) or treat "don't know" as "reject", because the runtime has practically no knowledge of type. This is not the case in Python, so you don't have to check everything. That does not mean you shouldn't check anything.

So there are still benefits in Python to have type in order to "reject" code that is undoubtedly going to produce type errors at runtime (or maybe to optimize things). In case of uncertainty, you know the runtime is going to catch errors anyway (including type errors).

For structurally typed languages you really want something like C++'s concepts so you can talk about `Iterable[str]` rather than forcing it to `List[str]`. Is this possible with Python's type annotations?
I don't know exactly for Python. I would expect mixin class to be used for this. Also, maybe Python will allow you to define union types: e.g. X is either a "foo", a "bar" or a "zot" (all of them duck types the same implicit interface).
It's kind of hard because you really want to be able to use generator functions as well. Maybe that just can't be properly described Python. It would be shame to lose out on passing a generator because somebody has used `List[int]` inside a function that really takes an iterable.
And that's why it's bad. Let's throw something useful (duck typing) out of the window to please the Java people

I'm sincerely disgusted

> Let's throw something useful (duck typing)

Not sure why you say this is throwing away duck typing, you just don't add a type hint and you continue working as usual.

> I'm sincerely disgusted

I can sympathize, most of the time when there is a talk about Lisp there is a guy who comes just to say "dynamic typing is bad", etc. Well, even though there are definitely static analyzers for Lisp (see https://news.ycombinator.com/item?id=12216701), the language is still the same and doesn't prevent anyone from using dynamic features. People who really want statically typed programs won't touch Python anyway so I don't think you have much to worry.

Yup. There is a Generic type which uses duck typing and has an `Iterable` subclass for classes implementing __iter__() (https://docs.python.org/3/library/typing.html#typing.Generic). It has been pointed out though that `str` counts as an instance of Iterable[str], so there is a possibility for some amusing issues to arise from this.
If I am programming in Python, I want nothing wrong to happen because I passed the "wrong type". That's the same argument as "don't use isinstance". It's not Pythonic. If you want a dynamically typed language, you don't check types. Period.

Type hinting would be ok for documentation only. But then, a comment is fine and introduces less clutter.

Well, maybe "reject" was too strong a word.

The warnings that are emitted by SBCL (Lisp implementation) are produced when two types don't intersect at all, something like "len(3)". There is no warning when types overlap. And even if the warning is issued, the code is still compiled and can run.

> Bah. If you want a statically typed language, get a statically typed language.

Which statically typed scripting language would you suggest instead of Python?

What do you mean by "scripting language"? That's an old terminology which is lacking a clear sense nowadays.

Would you like something that you can embed? Something that has an interpreter for?

> What do you mean by "scripting language"? That's an old terminology which is lacking a clear sense nowadays.

https://en.wikipedia.org/wiki/Scripting_language

> Would you like something that you can embed? Something that has an interpreter for?

I asked about your personal preference regarding a statically typed alternative to Python. Can you propose one?

I'm not the person that you responded to, but C# has a nice scripting API and repl.
Where typing is really useful, in my experience, is in an external API.

Most heavily used libraries in most dynamic languages, not only Python, ought to have assertions with type checks for parameters. In the external API calls to the lib.

Inside of a lib you should have tests anyway, so type declarations shouldn't be important.

Do the hints increase performance at all? I would think they could.
For performance we have cython and numba
I wish mypy, cython, and numba type annotations converge to something common.
This is why Python 2.7 will just continue indefinitely.
Where can we voice our opposition to this?

If people really want their bulky IDEs to give them maximum help when introspecting a codebase, there should just be a TypeScript like superset of Python that those people can use.

This syntax is awful and will seriously cause antagonism in terms of what people consider "pythonic" and "good code".

I see a ton of comments that are dismissing this but from my experience most code has implicit typing of some sort. This looks like a nice replacement for hand written input validation, if my function expects a number and I pass a string there are 3 options AFAIK:

1) I have the ability to define a type that's required so that the function can't be called without a numeric type.

2) I add boiler plate the the top of my function to validate that I'm receiving a number.

3) I forget to do 1 or 2 and I get a runtime error when a different developer inadvertently passes an incorrect type to my function.