I haven't used python's type hints much yet but I used TypeScript a bunch about a year ago and I have the exact same complaints.
Most of the benefit of types goes away unless _everything_ is typed... my libraries, the libraries my libraries use, etc. I should be able to jump around in and out of library code following types and usages thereof in my IDE, making conclusions that are accurate based on what the types say.
When some of your libraries (or indirect dependencies) don't have types, you can no longer ever make any firm conclusions when exploring a complex codebase based on types. I want to change a method in the Widget type, and my IDE says it's used in 4 places. But in reality, that just means 4 or more places, who knows if there are others.
Adding types to a language that doesn't already have them is very hard for this reason. Until many years pass and they're in universal use, they add little value.
I think Typescript nowadays has reached critical mass where most packages at least have a @types/* definition. I can't remember the last time I had to use a package that didn't provide types. I had the types be wrong one time though.
I would rather my type checker give me the truth than gloss over it. This issue should be kept open (it is) but they should wait until a good solution is found that doesn't gloss over the discrepancy.
I use the type-annotated python at work and at hobby.
I see the point of OP, but I'd say it helps even if it's not perfect or "sound".
Though I agree on its negative impact on the language ergonomics. It removes the joy of writing "scripting language" and as a statically typed language it is far from static-type-native ones.
Now I see it as a tax for compatibility: You enjoy dynamic typing at the beginning of the project, but you have to pay it back once you've grown up to a certain scale.
TypeScript to me feels much more like a native static typed language because it is. Does it help to be a transpiler? Maybe. But I would think that Hejlsberg just has done an astonishingly good work to make it compelling. Python's type annotation is good and completely reasonable, but gradual typing is such a hard problem and being good is probably not enough here.
I love type hints. They make the code so much more readable. I wish they could do more, but knowing what the programmer intended for a variable is huge. Now, when I see code without type hints I think, "Oh man, now I have to dig into everything to know what anything is."
Yeah this is such a huge benefit that’s rarely discussed. It wildly increases the understanding of a code base which makes developers more productive and safe
100% agree - never really understood the movement behind adding types to Python. Type hints are a useless complexity that yield little return. If you want types, use something else other than Python. The whole thesis behind Python is simple is better than complex.
On the one hand yeah, I agree with this, but on the other hand you have a lot of enormous software projects written in Python that are decidedly not simple, and would be extremely difficult to completely rewrite in a different language. Type hints do offer some value in this circumstance, by making it easier for disparate groups to modify the same large codebase though static analysis.
You don't understand what "strongly typed" means... I give you a hint, `"1" == 1` is a type error in Python but "okay" in JavaScript because this one is "weakly typed", you're confusing "dynamic typing vs static typing" with "strongly typed vs weakly typed".
Python is a strongly, dynamically typed language. Most Python type errors could be caught by a static analyzer. If Pythons type hint system wasn’t a disappointment it would help more people catch more bugs when they’re written. Instead Python endlessly throws exceptions at runtime because a programmer got their types wrong.
I find it best to see type hinting as "living documentation". Yeah, you can ignore it, it is Python afterall, but if you want the IDE/tools to have better more helpful hints ... use machine readable hint documentation.
I'd argue if your functions are full of type-assert like guards, _then_ use another language!
Typechecking allows certain errors to be detected at typecheck time rather than at runtime. I don't personally consider that useless. E.g. A mistake I just made. I'm working in a language I don't know too well right now with strict static typing. The file read function takes a handle. If I pass it a string of the file name, rather than the handle result from file open, it just doesn't compile. In most dynamic languages, that error would not be detected until it executes.
The changes you need to make to a Python program to enable it to be type checked result in more bugs being added than the type checker removes leaving you at a net negative.
Type checking is just a really poor way to ensure program correctness until you start using an exceptionally strongly typed language like Haskell.
You use types in C / C++ / Java because the compiler needs them to work not because it's a good idea.
> The changes you need to make to a Python program to enable it to be type checked result in more bugs being added than the type checker removes leaving you at a net negative.
This claim doesn't make sense to me. Assuming mypy is regularly run as part of CI, why would adding type annotations cause more bugs than leaving it off?
I tried mypy when I first started into type checking in Python back when I was trying to do Python in VS Code and I discovered that mypy is a time-sucking anti-productivity disaster that needs to die in a fire.
When I started learning PyCharm though, I discovered that type checking can actually work well in Python and it is not a waste of time at all. So I advise people now that unless you're using PyCharm, do not waste any time on type checking Python. In which case, have at it. I love it. PyCharm is really the only way to do type-checking productively right now.
If you use VSCode and Pylance (Microsoft python language server) you don’t need to setup Mypy, you can enable type checking in user settings and it will use its own engine for type analysis. You have two modes, basic and strict.
I would recommend to try out if you can, the experience is quite good.
I personally find “strict” too strict for the current state of python, but “basic” is already very helpful.
The developer experience is still subpar when compared to Typescript but it’s improving fast (but I mean, typescript with VSCode has one of the best developer experience I’ve ever seen).
Could you elaborate on mypy? I'm not a Python developer (and I really have much more experience with statically typed languages), but if I ever had to maintain a Python codebase, I'd assume that adding mypy and type annotations would be among the first things I'd do, so it's a bit of a surprise to read that.
I love mypy. It's fine, if sometimes a little alarming when you aim it at a previously-untyped codebase. In my case, it turned up plenty of type errors that probably never became a problem in production, but very well could've. It's also a nice thing to put in a CI pipeline to block new code errors.
I spent a lot of time trying to configure pypy in VS Code, time that I didn't have to spend in other languages such as TypeScript, and I never did get it working as well as it does in PyCharm. PyCharm comes already configured. You don't have to do anything for typing. Works out of the box. VS Code is like that with TypeScript but not for Python. That was a couple of years back so maybe it's different today. I hear VS Code's PyLance extension works but I've already made the switch to PyCharm. I might try it out later.
I agree treating Type Hints as comments is the way to go. It's a bit nicer to use a library that is fully-hinted than one that is not. But I don't worry about type-hinting my own code, unless it's a particular complicated function signature that other people will be calling.
Only facts here. I think most large codebases eventually see type hints drift away from reality as individual contributors are more incentivized to hack in `Any` types to make things compile instead of typing every line properly. This is especially common for handling data objects which come in over the network - often they can have a couple different types but people just type it as one thing for simplicity.
Overall I do think type hints are worth it though, for maybe two benefits. They force you to look at the ridiculous types you are using, like `List[Union[None, str, List[Dict[str, str]]]]` which is the sort of thing that happens in codebases all the time. It adds just enough friction to push people to make explicit dataclasses or simplify function returns, which is good. Secondly they help with tracking functions which return None, which is a pain when following callstacks in big codebases.
My understanding was large python code bases (think Google) have a large problem. Someone makes a code change and suddenly it becomes difficult to find the scope of type errors in their monorepo. That was the driving force IIRC.
That said, I think pytype makes a lot of sense since it infers types from code, which you can edit by hand and then merge back into the python file when your code is stable.
I agree. I'm glad that they're optional. One of the main reasons to use Python is to prioritize development speed over performance AND to prioritize read/write-ability over hand jamming mundane syntax. I understand why some who have a background in typed languages might prefer to use Python with type hints, but it should be understood that they aren't very Pythonic.
> main reasons to use Python is to prioritize development speed
My experience with Python is that it slows development speed to a crawl due to dynamic typing. It’s maybe faster to write the first 1000 lines. But after that it becomes slower and more painful. At least in my experience.
> I understand why some who have a background in typed languages might prefer to use Python with type hints, but it should be understood that they aren't very Pythonic.
Completely disagree. The Python community has very rapidly adapted mypy because of widespread recognition that yes, type information is extremely helpful for any code bases larger than a few files and/or worked on by more than a few developers. Every major Python library I can think of now has mypy stubs available. If you're going to dismiss them as "not Pythonic" you may as well dismiss anything other than Python 2.7 as "not Pythonic".
Man, the idea that type hints reduce read-ability are crazy to me. It's like if someone told me that they think comments make code less readable.
Like, even if you think that type hints have ugly syntax, prior to them vast most projects just didn't bother with documenting the shape of data, so you had to read source code, guess and experiment. Unless you just don't care about knowing what you're specifically working with (which strikes me as living on the edge), how is it not an ergonomics gain?
Code is way less readable without types. If I see this:
def create_user(user):
other_function(user)
Then how do I know what `user` is? Is it a dictionary? An object? If it's an object, where's its class definition? To find the answer I need to search for all the code that calls `create_user`, and then code that calls that code, and then code that calls that code, ad nauseam.
Onboarding into a huge, untyped codebase is brutal
> I understand why some who have a background in typed languages might prefer to use Python with type hints, but it should be understood that they aren't very Pythonic.
Okay, this myth that Pythonic == loosey-goosey duck-type-everything slinging dicts and strings nothing static cause that's too slow, really needs to die.
Pythonic is all about pragmatism and parsimony. If that means not typing anything cause it's a 50 line script, do that. If it means the most efficient way for large dev teams to communicate on sprawling code bases is to use rich types, then do that.
Personally, I use types even in the short scripts, because then I can offload keeping track of what type everything is, instead of having to remember yet another user_dict with whatever keys the producer felt like using at the time. It makes autocomplete faster and I'm less mentally fatigued. Less mental fatigue == more pythonic.
I've annotated Python typing with a 20/80 approach.
I annotate strings, integers, floats, simple list and dict.
I don't try to produce complex type specs for lists, dicts, functions.
My company runs some kind of static analysis that checks things.
And I get 80% of the benefit with 20% of the work.
So start small, and don't expect perfection (yet).
I find type hinting useful as an additional level of documentation, especially for complex projects, and it helps my future self understand my own code more quickly.
Also in my experience it can provide static analysis tools more information and provide autocomplete suggestions in a wider range of coding scenarios.
They feel like someone told the Python dev team about Julia's multiple dispatch based on types, and they felt like they had to respond, even though they don't really provide the actual speed and flexibility that motivates their use in Julia.
this article is so full of... not-very-educated thoughts on gradual typing in a dynamic language that it's hard to know where to start.
A few concrete criticisms:
1. If you're using a dynamic language, then _by definition_ the language will not enforce your static hints at runtime. However, good news! Python has always been strongly typed, and _does_ enforce types at runtime!
2. A number of the examples given would be _impossible_ to statically type in most compiled languages (those without dependent types). It's hard to know where to go with a critique saying that you can't fully represent heterogeneous values in a dict, given that you can't do this _at all_ in many statically compiled languages.
It seems to fall back on saying "use of the type system requires too much discipline to be useful". This might be an interesting criticism on its own; however many current users of Python can say, from experience, that the required discipline is not a high enough hurdle that it prevents us from using types consistently and correctly.
There is plenty to critique about Python's static typing, but this article would have been better titled "I'm confused about Python's static type system and want somebody to show me how it's actually used in real-world scenarios".
Duck typing and heterogeneous dictionaries are (and have been) standard Python.
Adding a type system which doesn’t respect duck typing by trying to access the member even when the types don’t match or which can’t express standard idioms used in Python seems a poor architectural choice.
It’s not saying that Python’s type system is “too much discipline”, but rather that the type system doesn’t encode typical Pythonisms.
I read 0544 and the proposal for protocols fails to support the main benefit of duck typing, as it requires the substituted-for class to be defined as a protocol.
Traditionally, duck typing is used to inject types into a library that isn’t expecting extension at that particular point — eg, substituting a test class for a real class in a data object that normally wouldn’t be a protocol.
I’m not seeing how that PEP addresses that use case.
- - - -
Similarly, the heading on the dictionary says “for a fixed set of keys” — but what if I want a dynamic heterodox dict? Eg, unpacking JSON.
You just end up shoving “any” all over the place. At which point, are the types helping?
Though, protocols do help the dict case for typing:
Your link appears not to work (for me) — can you cite what you believe I have incorrect?
This section seems to agree with me, where it explains why normal classes can’t be subclassed to protocols:
> Now, C is a subtype of Proto, and Proto is a subtype of Base. But C cannot be a subtype of Base (since the latter is not a protocol). This situation would be really weird. In addition, there is an ambiguity about whether attributes of Base should become protocol members of Proto.
I’m not sure why you think it’s “obvious” that you can use a third party library to solve the problem — or why that addresses my complaint that the built-in type system doesn’t work for that.
If anything, the existence of a third party library hints the standard library doesn’t cover the use case.
No offense, but have you actually given it a shot? Duck typing is supported and you can get virtually all use cases of heterogenous dictionaries by using union types and duck typing. The article even touches on this.
edit: To be more precise: The article specifically mentions that accessing invalid members give you a type error, but posits that people will probably not bother and just use 'any' instead. How is that not saying 'too much discipline'?
Yep — I use mixed annotations on my Python code because as the person I’m responding to pointed out, they catch many small type errors. Dataclasses have been awesome.
I’m also generally pro-types, but I think it’s worth having a discussion about this type system in the context of Pythonisms.
- - - - -
If you’re using “any” for most of your types, then it’s not providing value — an untyped statement implicitly has the type “any”.
There’s a reason I brought up the JSON example: loading and manipulating JSON of varying structure is something I do a lot at work.
Sure, but the idea behind gradual typing is that 'Any' should only be temporary. IMO if you're serious about it, stable code should at least pass strict type checking, if not completely forbid even explicit Any.
>There’s a reason I brought up the JSON example: loading and manipulating JSON of varying structure is something I do a lot at work.
Do you have a specific example that you find troublesome? For JSON with a fixed, regular structure (i.e. no subtyping) something like Pydantic works well, otherwise using unions + protocols with custom validation code has covered even my most esoteric use cases so far.
If we're talking about unstable JSON, a recursive union type works well enough in my experience, though defining that type gets a bit ugly if your type checker doesn't directly support recursive types.
Your suggestion to “use a recursive union type” and that being “ugly … if your type checker doesn’t … support recursive types” being required to support varying JSON is my point:
Python types don’t support a common Pythonism used frequently in my work.
In fact, you admit that even fixed JSON can be difficult without a 3rd party library.
You’re lecturing me like I don’t understand types without realizing that you repeated my point about a gap in the current type system.
I think I might not have been clear: this is not a limitation of the type system, but of a specific type checker (mypy), that will hopefully be fixed soon. Both Pyright and Pyre support recursive type aliases right now. Open one of their playgrounds and you'll see that the following, intuitive definition
>In fact, you admit that even fixed JSON can be difficult without a 3rd party library.
Sure, but that's just because the standard library (for now?) doesn't offer deserialization with runtime validation. Java doesn't have that either, if you want to parse JSON you'll have to either roll your own or rely on a third party library.
Don't get me wrong, the situation isn't perfect, but especially if you use the 'right' tooling, you can in my experience already get a perfectly serviceable experience, which to me is indicative that the type system itself is fine and pythonic, it's just the tooling that is lagging a bit behind at the moment.
Your initial statement was that the type system was a poor architectural choice because it doesn't support standard python idioms. To the best of my ability I've tried to communicate that there's an important difference between the type system and the standard library, that one can make it work and that the situation is not worse than most other typed languages.
If you think that's violently agreeing and talking down to you, then there must be some communication barrier that I don't know how to overcome.
A decade or so ago JSON was changed so that JSON text doesn't have to be an object, it can also be a value. 'null' (or say, '5') is therefore legal JSON.
It seems to me like Python needs to get Typescript's capabilities (if Python wants to go further this way of course). It solves all these problems very well, and has no problem with 2 and most of the author's objections.
Typescript does absolutely fantastic type inference, which mypy definitely does not. I think that's a huge advantage for Typescript over mypy, and it's really foundational to the value it provides.
However, even in Typescript it is nontrivial to _annotate_ these complex types, and in most cases it's still possible to do in Python - you just need to commit to using something like dataclasses rather than pretending that what you have is a `dict`.
Basically, to get the most out of static typing in the world of Python, you do have to write more boilerplate than you would in Typescript, and in particular you need to avoid dicts and prefer various sorts of class-based data containers (again, dataclasses, attrs, namedtuples, etc).
> However, even in Typescript it is nontrivial to _annotate_ these complex types
The only one which stands out to me as non-trivial in TypeScript is thrown exceptions… and even then only because you can only type them in a catch clause, and only with unknown, and they’re invisible to callers. But then this is why people who would care for checked exceptions quite reasonably just use something like a Result type.
But I agree with your conclusion completely, and it’s basically the same one I came to reading the post.
I was curious about how the current state of things compares to TS, as I may soon have a foray back into Python soon, and would be delighted to bring some static types into the mix. Pretty much as I expected, it’ll probably be more effort/less valuable than TS, but more valuable to me than to the author.
I had to come back to Python after working with TypeScript for a while. After going from Python 2.x to TypeScript and being very impressed with static typing, I didn’t want to go back to untyped so I had to get up to speed with the ecosystem. My findings:
— Consider using Pydantic (either its drop-in dataclasses replacement, or its models). It has issues and documentation is lacking in places, but in many use cases it’s a boon when it comes to being confident about data you read and/or pass around.
— Don’t use TypedDict if you foresee needing anything like optional keys, especially with defaults. I went with TypedDict and it came back to bite me later (though was relatively easy to refactor).
— If you use VS Code (which I started after switching to TS), there’re some shenanigans to be aware of. Unlike TypeScript’s elegant approach where you’re in control of installing typing versions, VS Code’s official Python extension will force-install unvetted third-party typings, not caring if they don’t match library versions in your use and not allowing to opt out. In later versions you can unset mypy in Python extension settings, and use a separate Mypy extension that doesn’t do this behind your back and doesn’t send you deep into some rando’s typings when you hit F12 to jump to definition.
— That aside, VS Code can provide development experience is not that far from TypeScript. You can keep a virtualenv nearby and select its Python bin as interpreter in your workspace, giving Mypy access to dependencies (and if VS Code’s interpreter selection GUI resolves symlinks, you can enter interpreter path by hand so that it doesn‘t).
— Sphinx documentation is good at picking up typing hints and works reasonably well with Pydantic’s models, too.
You can type Python dicts, see typing.TypedDict. It’s still lacking though, totality is all or nothing until py311, that is to say, TypedDict’s are either Required<T> or Partial<T> in TypeScript parlance. And you don’t get Pick<T>, Omit<T>, etc.
The ability to slowly add typed code to an existing database has been a lifesaver. I always hated JS because of callback hell (and the notions around its community) but since I started working with typescript my mind has been changed. I love it
This seems like an inaccurate and condescending put-down. What is the definition of "educated" in this context? Is there a book or generally well-known resource?
The author is clearly thoughtful and curious. Near the top of the post he says "what I’m hoping for is that someone will come along and tell me that I got it all wrong. “When you do it as follows, the system works: $explanation” Because, you know, I’d really like to have static typing in Python."
> 1. If you're using a dynamic language, then _by definition_ the language will not enforce your static hints at runtime. However, good news! Python has always been strongly typed, and _does_ enforce types at runtime!
And yet, the author points out that this is a working program:
foo: int = 'hello'
print(foo)
In what reasonable sense can Python be said to "enforce types at runtime" here?
Can a string be printed? Yes! Then Python is (correctly) allowing typesafe behavior here. Your (incorrect) annotation does not in any way contradict the typesafe-ness of printing a string (or an int). They're both equally printable.
This is just a very, very bad example, plain and simple. It 'looks' bad to a superficial reading, but in practice it demonstrates only what is already known about static typing in Python - it is enforced (or not) separately from the interpreter.
Agreed. Static typing is usually enforced during the compile stage in other languages. It's specifically meant to catch problems before running the code (hence the "static" in its name).
This said, I find the linting/type-hint-checking stage of Python cumbersome and confusing :(
It seems that your point boils down to "Python enforces types at runtime if you define 'enforce types at runtime' not to include 'enforcing that data you declare to be of a certain type actually is of that type'".
Am I the only one baffled by this way of thinking?
Python does and always has enforced types at runtime. This is called duck-typing. If you're not familiar with the concept of strong+dynamic, it is easy to see how this could be confusing. This may help. https://stackoverflow.com/questions/2351190/static-dynamic-v...
Static type annotations by definition are not enforced at runtime. This has been true of every language that has ever used static typing, including C, C++, Java, Rust, Typescript... you name it, statically typed languages only enforce their static typing at compile/analysis time.
Yes, it is possible to annotate types in Python incorrectly. It's possible to do this in all other languages that allow type-unsafe behavior (whether natively or via reflection, etc). This may be less common in some languages than in others, but it is fundamentally possible in the vast majority of languages that perform static typing, because those types are fundamentally enforced at analysis time, not runtime.
The author of the article seems to be unfamiliar with these distinctions, and maybe you are too. It's fair to complain that these distinctions are confusing and make things harder for programmers. Nevertheless, very few languages have ever been designed with a type system that acts exactly the same at analysis time and runtime. It's a very difficult problem, partly because these are all abstractions from the perspective of the underlying computer, which only understands boolean logic and integer/floating point math, and has virtually no other notion of types in any formal sense.
Type systems are a complex topic, and as someone who has been paying attention to them for a while, it's frustrating to see uninformed discussion of them show up on Hacker News. That said, it's perfectly understandable that people are confused by these distinctions, and rest assured there's a lot of effort going into developing better languages that suffer _less_ from these issues.
For the everyday working programmer, however, there are currently lots of tradeoffs to be made, and Python's approach to static+dynamic typing is actually pretty usable, all things considered, which is why the community overall has embraced the new static typing despite its blemishes.
> Yes, it is possible to annotate types in Python incorrectly. It's possible to do this in all other languages that allow type-unsafe behavior (whether natively or via reflection, etc). This may be less common in some languages than in others, but it is fundamentally possible in the vast majority of languages that perform static typing, because those types are fundamentally enforced at analysis time, not runtime.
It is pretty damn hard to make this mistake in languages that enforce static typing. I mean, you would have to go out of your way to do so. In Python, however, it is trivial to write the wrong type signature or to modify the body of a function so that it no longer matches the signature.
"Fundamentally possible but terribly unlikely, as opposed to the Python way of doing it" would be a more accurate description.
I think your (and the article's) argument could be summarized as "static type annotations without automated enforcement by actually running a type checker considered harmful".
Since this argument is not meaningfully different from "comments that are lies considered harmful", it seems fair to expect reasonable people to dismiss it as uninteresting.
> it seems fair to expect reasonable people to dismiss it as uninteresting.
Do you think that's a fair treatment of someone who disagrees with you but has been respectful of your opinion so far?
I realize proglang debates are flamewar territory. But I have been very careful to state I find Python type hints puzzling and less useful than they should be. It is obviously an interesting opinion shared by many others, not just me or the article's author.
As a fan of statically typed languages, I do indeed find some Python programmers engage in a form of Stockholm's syndrome. It usually takes the form of the assertion "I never found a bug that was a type error". Have you or anyone you know ever said this?
In my mind, the opposite statement of the one you attribute to me would be "wishful thinking and a positive attitude is enough to catch mistakes, it's not necessary to have help from automated tooling". In this day and age, I cannot disagree more.
> I think your (and the article's) argument could be summarized as "static type annotations without automated enforcement by actually running a type checker considered harmful"
It illustrates your mindset, in my opinion. I think type annotations without checking them (or without a satisfying implementation of said type checking) are mostly pointless.
You also left out the part where I remarked on the dismissive tone of your reply.
It is meaningfully different because, as the article says, comments are known to not be machine verified and thus to potentially be wrong. With type hints, they’re usually verified, but not always, so you’re more likely to trust them and get a nasty surprise when they turn out to be wrong.
However, there are other aspects to the article’s argument, such as the fact that it’s easy to end up with type annotations going silently unenforced even if you do run the checker.
Duck typing is really enforcement of interfaces, not types themselves. Everyone who has passed a string into a Python method expecting a list of strings has run into this. It works, but did you really want to process a character at a time?
Also, some languages like Java do actually enforce types at runtime. If you've ever called a Java method dynamically with the wrong types, you'd know. You can run into ClassCastException, or worse (ClassNotFoundException if you have the wrong classloader.) This is a runtime error.
> Duck typing is really enforcement of interfaces, not types themselves
A type system is just a set of rules that must be followed. The rules can be as weird or as simple as you want. Just because <int> + <string> is allowed doesn't mean it's not enforcing types.
Not really a criticism, just trying to push to expand your idea of what a type system is.
No, you're not the only one. All of these gymnastics to justify the behavior of the string-declared-int baffle me.
Is it "legally" baffling, i.e., do I understand why this behaves in the way it does purely mechanically, and do I understand the arguments the gymnasts are making? Yes.
But from an idiomatic, colloquial, linguistic, syntactic, programming-historical, or programming-cultural standpoint? Baffled.
Even calling it gradual typing is baffling. It's just adding metadata-that-doesn't-look-like-metadata that sophisticated programs that aren't part of the Python bytecode compiler can use. The definition I know is the same one from Wikipedia:
> Gradual typing is a type system in which some variables and expressions may be given types and the correctness of the typing is checked at compile time [...]
Maybe not by default but there is tooling that makes it work that way — that’s exactly how the python build system works at my company. I don’t know the details but when I “build” python code (creating bytecode pyc files) it produces compiler errors and fails to build if I e.g. try to pass an int to an f(x: str). This has usefully caught bugs before I ran some expensive/time-consuming scripts.
I guess you could say this doesn’t count as it’s a separate program doing the type checking but IMO there’s not a clear distinction between that and e.g. having a first type checking pass in the “same” compiler program.
> While these [type] annotations are available at runtime through the usual __annotations__ attribute, no type checking happens at runtime. Instead, the proposal assumes the existence of a separate off-line type checker which users can run over their source code voluntarily.
Right, it’s definitely still a voluntary feature. What I meant is if you opt in and make it part of your project’s standard build procedure for python code, you effectively get the behavior and benefits of “compile-time type checking” (to the extent that you actually use type annotations in your code).
>> Gradual typing is a type system in which some variables and expressions may be given types and the correctness of the typing is checked at compile time [...]
> This is definitely not what Python is doing.
When I read "compile time", I mentally replace it with "before runtime". There isn't a real compilation step in Python, so it's reasonable to accept static code analysis as part of "compile time" in that definition.
The point is that type annotations will let a static analysis tool make sure that the program is correctly typed. This can be enabled easily in most of the modern IDEs and editors. You could just decide to block the code's deployment in CI/CD if the type checker raises issues, for example.
typically works fine (or worse, works fine in debug mode and causes unpredictable UB-related bugs in release mode).
std::mem::transmute here is a no-op at runtime; all it does is subvert the static type checker.
The difference between statically-typed languages and things like Python with type hints is _not_ that the former has any runtime type checking; it’s that subverting the (static!) type system is more difficult and requires more obvious ceremony, and that static type checking is required whereas with Python it’s optional.
If you hacked rustc to disable static typechecking you would get a similar experience to Python with type hints but without any mypy-like tools. (Of course, this would be hard to do in practice, because Rust, like most statically-typed languages, uses types not just for typechecking but also for code generation and method dispatch.)
I agree with most of your comment, but I think there's a miscommunication problem here.
When Python "enforces types at runtime" this is the actual types, not type hints. Type hints are not a runtime artifact. The "actual type" here is "string", and enforcing it means Python would not allow invalid operations on it without error'ing. A programming language that will let you do basically anything to any value because it doesn't enforce type safety is of course C. Python is safer than C.
It also won't let you call thing.method without thing.method definitely existing.
Best of all, all of these are enforced at compile time.
Now, C absolutely will let you convert an int to a pointer, or a char to an int, or an array to a pointer, or... well, it will let you convert lots of things to lots of things. Some of those things are unsafe unless you are quite sure what you're doing.
> A programming language that will let you do basically anything to any value because it doesn't enforce type safety is of course C.
False on several grounds. You can't call something that isn't a function. You can't call a function on something that doesn't have it. You can't use an int as a pointer or an array without casting it. You can't use an int as a dict (not that C has any built-in idea of what a dict is...)
> Python is safer than C.
Depends on what you're doing, and what kinds of errors you are more prone to.
You know what I meant and what error I was clarifying for the comment I was replying to. Yet you went out of your way to point all sorts of irrelevant mistakes in my post, when the gist of it was right.
Do you feel it was more important to correct me on these trivial things, or was my reply more or less correct when fixing the conceptual error in this sentence?:
> "In what reasonable sense can Python be said to "enforce types at runtime" here?"
Here's a nitpick of my own to your post:
> "Some of those things are unsafe unless you are quite sure what you're doing."
Wrong. Type (un)safety does not depend on you "being quite sure of what you're doing".
PS: assume I know C and that I've programmed complex things using it. No need to catch yourself with statements like "not that C has any idea of what a dict is". Assume we are both C programmers. It will make you sound more polite.
1. You might need to re-read the site guidelines about assuming good faith.
2. Your last two sentences were, I think, not needed for the point you were making. They were also somewhere between gross generalizations and flat-out wrong, and felt like a gratuitous slam on a language that wasn't even the topic. I thought that deserved a response, even if it wasn't your main point.
3. I may assume that you know C. I don't assume that everyone reading this exchange knows C.
Sure you can. That's exactly what you do when you call a function returned by `dlopen` -- it might be a function, but it might also just be garbage. It might even be garbage that's coincidentally marked as executable and does some stuff before eventually crashing!
> You can't call a function on something that doesn't have it.
This is also very easy to do -- you can default-initialize a structure containing function pointers, and then call one of them. You'll be calling some random crap on the stack, which might or might not do anything of interest. But either way, C will happily let you do it.
C isn't completely untyped (there are, as you've observed, things it will forbid at compile time), but it's about as weak as a static typing system can be.
I think you're talking past the comment you're replying to.
In both examples, you have tell the compiler exactly what the types are. In the first one you have to cast the return value of `dlsym` to a function pointer, and at that point, as far as the compiler is concerned, it's a function. In the second example you have to explicitly declare the struct containing function pointers, so of course you can call its members.
So it's strange to suggest that C is somewhat untyped; it's not untyped at all. C just makes it very easy to force the compiler to assume different type for a value (that's necessary for low-level code like the examples in the comment you replied to; and of course that makes it very easy to do unsafe things, I don't think anyone disputes that).
And about the "weak typing system" comment, note that nobody agrees on what exactly that means. Just as an example, see how this page[1] defines "weaker" vs "stronger" types, and how it's completely different from the way you're using here. In general, I find people call type systems that don't do what they expect or want "weak", but that's not an useful discriminator: at that point you might just call it "bad typing" to dispel any illusion that it has any objective meaning.
I didn't say it was "somewhat untyped." I said it has a very weak static typing system.
Just because "weak typing" means something different in Haskell doesn't mean it can't be used meaningfully (and beyond "bad") in the context of C. C's typing is historically referred to as "weak" because C's notion of casting doesn't distinguish between type and value transmutation: pointer-to-pointer casts don't convert the referent (because they can't), which in turn gives the C compiler very little leeway in proving that the program's types as declared have any particular meaning at runtime.
Compare this to Python, which is "strongly" typed in the same sense: doing `y = str(x)` on `x` means that `type(y)` actually is `str`, and not merely a promise to the runtime. It's enforced, which makes it strong.
My point about "weak typing" is this: if someone says "hey, I'm designing a new language, its type system is going to be weak", they gave you no information whatsoever about the type system other than maybe "some people on the Internet will probably complain about it".
That's why it's useless to say things like "it's about as weak as a static typing system can be" and "has been historically considered weak".
If you had explained your objection to C's type system (like you have now), I wouldn't have said anything. And I've been on the Internet long enough to have seen C's typing is historically referred to as "weak" because X for *many* values of X, so I disagree that that's the only or even main objection.
For 1, there's some unfortunate but important semantics. Python does not enforce type _Hints_ at runtime, as Hints are in many ways fancy comments. But as pointed out, python is a strongly typed language (at least on the sliding scale of strong to weak), and types are known at runtime (call `print(type(var))` to see them). And calling `1 + 'a'` will result in a TypeError exception, unlike say JavaScript.
I believe this is the relavent passage in the article the GP is referencing, which is incorrect if you take "runtime" to mean something like when the line is executed: "Python is a dynamically typed language. By definition, you don’t know the real types of variables at runtime."
It _is_ however correct to say you don't know the real type _before_ runtime, at least python does not.
I think the top comment was less polite than it should be, but I would echo that the post seems to have a bit of a weird understanding of gradual typing?
The Any type is to allow incremental typing. You start out with dynamic code, and progressively 'typify' it by adding more and more annotations, with Any working as a stopgap where for the moment no valid type exists, until you finally have a fully typed program. Inserting Anys because you're lazy is like casting things to Object in Java because you're lazy, abusing it like that is just incredibly sloppy and bad code.
If someone is really struggling with this, simply use a linting rule to ban Any in 'mature' code.
Like the author said, the weird thing is that it goes both ways. I'm pretty sure if you try to return `any` from a function with an explicit return type in TypeScript you'll get pinged for it.
noImplicitAny is different than what's being discussed (I think). Returning something typed as `any` in a function that has a different return type is totally fine: it's not implicit (you've cast it to any), and it passes (you're saying the type is literally anything after all).
Python enforces its runtime types at runtime. 'hello' is a perfectly valid argument to print. Python will raise a TypeError if you try to use it to index a list though.
> In what reasonable sense can Python be said to "enforce types at runtime" here?
Python is both strongly typed and protocol-typed. In the case of `print(...)`, the protocol is that the received parameter responds to `str(...)` (i.e., has `__str__`).
(What Python isn't is statically typed. But "strong" and "static" are completely different typing dimensions.)
> In what reasonable sense can Python be said to "enforce statically declared types at runtime" here?
It can't, because Python doesn't have statically declared types. It's a strong, dynamically typed language.
Python also doesn't allow you to declare static types. It allows you to declare type hints, which are a sort of adjacent and potentially unrealistic (or just plain incorrect) typechecking proof that doesn't correspond at all to runtime behavior.
If your statement were absolutely true then a product like mypy would neither exist nor provide value.
Python has the capacity to have some form of gradual typing built-in at the language level, but it does not, which has led to the enormous confusion we have before us. The SBCL implementation of Common Lisp is a great example of a strong, dynamically typed language that is able to, when possible, detect errors statically at compile time. That tech is 30 years old now.
There is no reason why the Python bytecode compiler could not statically or dynamically detect this programmer error at either compile-time or run-time present in this statement:
x: int = "x"
Compare to Common Lisp, well known for being dynamically typed (so dynamic that it's possible to change the class of an object at run-time!):
(declaim (type integer x))
(defvar x "oops")
; ==>
Unhandled SIMPLE-TYPE-ERROR in thread #<SB-THREAD:THREAD "main thread" RUNNING {10005D05B3}>:
Cannot set SYMBOL-VALUE of X to "oops", not of type INTEGER.
quitting
compilation unit aborted
caught 1 fatal ERROR condition
Instead, this job in Python is haphazardly relegated to the program readers and IDE implementers.
> If your statement were absolutely true then a product like mypy would neither exist nor provide value.
I don't see why that would be the case. The value of Mypy is that it's essentially a proof language over something that resembles Python[1]: it's a way to do some amount of static typing without having to maintain a separate copy and representation of the program.
And sure: Python could detect inconsistent type hints at runtime. But why would it? It's already strongly typed, and the presence of type hints usually means that someone is already running Mypy or another checker during development. It's not clear that there's a significant advantage to be gained, particularly one that justifies the additional overhead.
[1]: "Resembles Python" because mypy does not actually evaluate any Python. It doesn't know what types your program has at runtime; it only knows type hints and a few small rules (for things like string literals) and trusts the developer to reconcile those rules with Python's runtime behavior.
Do you think other languages should do the same, because very few do any kind of runtime type checking (e.g. bounds checking) due to the performance penalty. Why focus specifically on python's lack of runtime type checking?
Anecdotally, my code in python is functionally fully typechecked and runtime type checking would solely penalize me by slowing down my code. There's no need for it because the type checker correctly proves the soundness of the program's types, in exactly the same way that Java's or C++'s do.
The point is precisely that you don't need to suffer the costs of runtime type checking if you have a solid typechecker. Lisp's approach here is worse than the one adopted by python/js/ts/go/etc.
What other languages do that though? C doesn't, Java doesn't outside of particular cases. If you can trick or mislead the compiler, the runtime often has no type information at all, so you'll get, at best, a ClassCastException, and more often than not get a segfault.
Python can't be said to enforce types at runtime. AFAIK, it doesn't say that. In fact, the original article starts off with "I'm expecting to be able to run a static analyzer", then it goes on to say "This isn't caught at runtime".
Yes, but is it caught at static analyzer run? Yes, I typed it into vi and ran mypy on it and it said "error: Incompatible types in assignment (expression has type "str", variable has type "int")."
But I didn't have to run mypy on it, my editor told me: "Expression of type "Literal['hello']" cannot be assigned to declared type "int"".
Maybe the word "educated" wasn't the right choice in this case, can you offer a better one? I honestly couldn't make it through the entire article, despite several attempts. It is full of inconsistencies and what seem like wilful misunderstandings to justify the authors conclusion.
Particularly as the author starts off saying "run a static analysis tool" as if knowing that's how you go about it, and then saying "it doesn't catch it at runtime".
> In what reasonable sense can Python be said to "enforce types at runtime" here?
In [1]: foo: int = 'hello'
In [2]: foo + 1
TypeError
Input In [2], in <cell line: 1>()
----> 1 foo + 1
TypeError: can only concatenate str (not "int") to str
>>> foo: int = 'hello'
>>> type(foo)
<class 'str'>
Python's type annotations are annotations. Types are being enforced at runtime, it's just that some are thinking that annotations have some effect on the interpreter, when they do not.
This is a demonstration that it has not enforced what many would reasonably believe is a static type declaration.
But we have (to quote another commenter) "uneducated" people making a supposedly boneheaded mistake to assume
x: T = y
is a declaration that x will only contain values of type T, or otherwise something will be detected by cpython as invalid (at either bytecode-compile-time or run-time).
Instead, this line is more properly interpreted as "documentation reasonably believed to represent a type saying something about values that x can be assigned at runtime, that's not in a docstring, that some programs may query."
I can totally forgive anybody who believes the former over the latter, and expresses their belief in a statement like that which you've quoted.
Type hints are not static type declarations. Python does not refer to them as such, nor does Mypy or any other typechecker for Python type hints.
Absent of active machine checking (e.g., via mypy), type hints should be considered exactly what you said: another form of documentation, one that might just be wrong.
This is my point precisely: it looks like a static type declaration to anybody who has seen what a static type declaration looks like in any statically or gradually typed program in the last 50 years.
But in Python, it's not (and documented in a PEP as not), and it causes massive confusion.
That's the way it works in Ruby with RBS, and it will work the same way in JavaScript should Microsoft's proposal for type hints make it into ECMAScript[1].
Python is strongly typed. It respects types at runtime e.g., `"1"+1` causes TypeError.
Python is not statically typed.
For comparison, C is statically typed but it is weakly typed e.g., printf function has no idea about actual types of its arguments (in particular, errors in the format arg may produce non-sensical results silently).
The "print" function will take any Python object and call its "__str__" method. Not sure if it will do it to a string, but the outcome would be the same.
As far as I remember, the "__str__" method is present on all objects in some form, so print will happily take any type.
Come on. It’s uneducated because it’s just a naive hot take. The education in this case comes from experience and basic understanding of the underlying issues. The author does not demonstrate possessing either of these attributes.
Most people I know writing Python don't use type hints because they are too much of a hurdle for very little payoff. The tooling that pays attention to type hints is slow as molasses or difficult to use or understand. The type hints themselves are of dubious use.
As a fan and advocate of static typing, I find myself advocating for type hints anyway, but I must agree I often can't reply anything to my coworkers' objections, because Python type hints are truly not that useful.
> The tooling that pays attention to type hints is slow as molasses or difficult to use or understand.
I use mypy daily on big codebases, it is fine, not fast, but fine.
> The type hints themselves are of dubious use.
They tell you when you make type errors, they help you understand what type things should be. The same as in literally every other language with static typing. There is nothing special here, nothing different. python with a static type checker running in strict mode is not fundementally different from Java's static type checking.
> As a fan and advocate of static typing, I find myself advocating for type hints anyway, but I must agree I often can't reply anything to my coworkers' objections, because Python type hints are truly not that useful.
> The same as in literally every other language with static typing
No, obviously not the same, otherwise I wouldn't be complaining. They are not even on par with Typescript, which I'm not a fan of either.
> [type hints] tell you when you make type errors
Not according to other comments I seem to be getting here. Other people are arguing type hints are not primarily for checking, but a form of notation for documentation. Seems wasteful, and I wish the Python community and tooling decided instead that they are for actually checking them.
> What do they lack that would make them useful?
Standardized, go-to tools that work in the build pipeline and that catch most errors without taking a long time to do so.
I haven't found mypy to fill these requirements. It's so bad I cannot convince my coworkers to make the effort to write more type hints.
> No, obviously not the same, otherwise I wouldn't be complaining.
What is the difference?
> They are not even on par with Typescript, which I'm not a fan of either.
Go is not on par with Typescript or Python, I still don't think it is okay for people to just say fuckit and `interface{}` it all and it is still shit to work with code that does use `interface{}`. At least Python with mypy has null safety, something that Java and Go does not have. There are some places it is worse than other statically typed languages, others where it is better.
> Standardized, go-to tools that work in the build pipeline and that catch most errors without taking a long time to do so.
It is mypy. What actual type errors does mypy not catch that you want it to catch? Why can't you use it in the build pipeline? I do it every day, it catches all the errors it should. The one complaint I can maybe see is the duct type compatibility complaint, and it is not really something that comes up that often for me, and definitely not something I would say invalidates the whole concept.
I'm not going to repeat myself, I already told you.
I find mypy slow, unsatisfying, inconsistent, and it fails to catch many type errors. No, I'm not going to go look in my work laptop to give you an example.
> There are some places it is worse than other statically typed languages, others where it is better.
In most places it is way worse, and I'll find it very hard to find common ground with anyone who disagrees on this.
Feel free to disagree, but I don't find this conversation useful.
How big is your code base? I’ve worked with really large auto generated codes and mypy never took more than a minute or two to run. On more normal sized code bases it’s just a few seconds. It’s also incremental so subsequent runs are faster.
Compare that to a c++ project where it’s not unusual to have 10, 15, 30 min build times.
Or even worse, the time to wait for discovering the problem runtime in prod.
The type hints also help in IDE navigation, where there is zero time to wait, pycharm can go to definition or find references in under a second.
Have you looked at the tools recently? LSP + Pyright is very fast and plugins exist for many editors. Or maybe it's slow on larger codebases or very large files? I don't have that broad an experience yet, but so far it's been very good.
What speed are you looking for in your CI pipline that you are not getting from mypy? Perhaps you are working on a much larger codebase than I am but the app I use it in takes under a minute. And all the unit tests take longer to run.
Personally, I took exception to the following from TFA:
> Python is a dynamically typed language. By definition, you don’t know the real types of variables at runtime. This is a feature.
In "you don’t know the real types of variables at runtime", this is just flat wrong. One doesn't know the real type of a variable until runtime. It seems to me the author maybe has a bit of confusion between weak typing and dynamic typing here.
I get what they mean though. In a typed language you know the type of a variable at compile time. In dynamically typed languages there is nothing to enforce this and a variable can be passed with any type. So there wording may be off but the point stands.
> One doesn't know the real type of a variable until runtime.
In the program
def x(f):
return f(3)
what is the "real type" of `f` at runtime; say, at the instant just before the call to `f`? How can you tell?
If your answer is that `f` has type "function-like", then that's a much weaker kind of a type than even type hints offer, let alone a real type system.
> what is the "real type" of `f` at runtime; say, at the instant just before the call to `f`? How can you tell?
We can tell by looking at what python triggers TypeErrors for.
>If your answer is that `f` has type "function-like", then that's a much weaker kind of a type than even type hints offer, let alone a real type system.
f implements the Callable interface - meaning it's either a function/method or an instance of a class with a `__call__` method.
So you can pass e.g. `print` or an object with a `__call__` method. If you try to do `x(5)` that's an immediate runtime TypeError because `5` can't be called.
f must also accept one int argument. It can have additional optional arguments, and it can also accept other types (e.g. `f("foo")` can also work!), but it must work on one int. The one int can already be optional - the function can also work with zero arguments.
Calling e.g. `"foo".join(5)` fails with a TypeError because 5 isn't Iterable, and calling `"foo".join([1], [2])` fails with a TypeError because it gets more arguments than expected.
Right, "function-like". I thought OP was meaning the static type of a variable, which genuinely can't be known fully at runtime in general. But it turns out that they meant to write "... until runtime", not "... at runtime", which makes this all a bit moot. And invites more confusion about the two disjoint usages of the word "type" in the PL community, as seen liberally spattered about elsethread, but that can't be helped :-)
You're assuming that type hints are by definition static-only. There's no particular reason why type hints can't be both statically and dynamically enforceable, as they are in PHP.
> 1. If you're using a dynamic language, then _by definition_ the language will not enforce your static hints at runtime. However, good news! Python has always been strongly typed, and _does_ enforce types at runtime!
So PHP is not a dynamically typed programming language according to you? It enforces type hints at runtime just fine.
There is nothing in the definition of dynamically typed languages that says they can't use type hints to perform runtime checks.
Was going to say the same for some implementations of Common Lisp, notably SBCL, which by default treats type declarations additionally as assertions. Subject to limitations it can use declared types and infer more types at compile time to produce more optimal assembly and give warnings if it detects type errors, or interestingly warnings if it has a chance to use e.g. a fast assembly add on a fixed sized int if you help narrow its inferred type with an explicit declaration one way or another. But unless it can prove a type always holds, it will also by default check declared types at runtime.
> you can't do this _at all_ in many statically compiled languages
What statically compiled language doesn't provide a way (sometimes clunky) to have heterogeneous values in a dict? You can do that with void* in C, even
> If you're using a dynamic language, then _by definition_ the language will not enforce your static hints at runtime
What definition is that? Raku is a dynamically language with gradual typing which will enforce those types at runtime (if it cannot do so at compile time).
> If you're using a dynamic language, then _by definition_ the language will not enforce your static hints at runtime.
I don't think this follows by definition. It's common not to check the static hints at runtime, but a dynamic language could choose to treat them as contracts or assertions. Over in Lisp-land, SBCL will enforce static type declarations at runtime [1], contrary to most other Lisp compilers, at least under the default compilation settings (you can force it to skip the checks by compiling with a low "safety" level). If the compiler can prove that a static type declaration always holds, it will omit the runtime check; otherwise it will compile it into a runtime assertion.
> can't fully represent heterogeneous values in a dict, given that you can't do this _at all_ in many statically compiled languages.
This isn't really true. For example, in Java it's Map<K,Object>. For languages without subtyping, existential types are often used. It's quite common to see in infrastructural code, e.g. caching.
Opinions like these are frustrating because, although they make valid points, it comes off as "they didn't do 100% exactly what I want, so I'm not using them at all." With many tools, there is a middle ground between using them for everything, and not using them at all. Python's type hints is one of those tools.
When I'm writing code, and a variable or signature is easy to annotate, I'll annotate it. And guess what? MyPy occasionally catches issues with my types and saves me some work. If a type is very complex, I won't bother spending time annotating. The end result is that I have some code that is annotated and prevents simple errors, and other code that isn't, and I didn't spend much time or effort doing it. To me, that's a clear net win.
Fully agree with this. Type hinting (not checking) is idiomatic to Python. MyPy is a great way to enforce checking. It's great to have the option to use the hints for intellisense only.
I've found that even when I write code intended to be statically checked from the beginning, mypy often misses stuff. I was enthusiastic about mypy at first and I still use it, but I'm not sure at this point that it is worth it.
Same here. And I come from (and am a fan of) statically typed languages. The type hinting story for Python seems confusing and not completely useful to me.
I just started using type hinting in Python and I love it. Maybe it's your tooling? I'm using neovim with LSP and pyright and the experience is very much like writing code in a Java IDE. As I code, it informs me where there are mistakes or pieces of code that need to be updated, catches typos, reminds me about forgotten imports, etc. It makes refactoring way easier because as you change signatures or object types it flags all the places in your code that need to be updated. Seems like a real game changer to me.
I want automated type checking outside the IDE. I want something that will catch other programmer's mistakes, not just mine. Without enforcing a specific IDE.
LSP support exists for many editors so you are not tied to a specific IDE. It can catch other programmers mistakes if you run it against their code. It doesn't turn Python into a statically typed language, but it's a lot better than nothing.
Sorry, I wasn't clear. I don't want to mess with each individual developers IDE or setup. I want to enforce type checking at the same place we also run mandatory tests: in the automated build pipeline (think CI/CD).
No it's not the tooling, unless by tooling you mean mypy. The issue is that you can write code with type errors that mypy doesn't catch. In what I usually think of as static typing, that would be impossible: type errors are simply not allowed, even if that means the compiler rejects some otherwise-valid code.
I think mypy has to let some potential errors through because otherwise it would spew 1000's of spurious error messages on perfectly good, unannotated legacy code. Something similar happens with the Erlang dialyzer. The same thing applies to tools like Coverity, that aim to find potential bugs in legacy C code.
You could instead imagine a version of mypy that makes no concessions at all to legacy code, and insists that your code be 100% free of type errors, even if that means that some older constructs and styles no longer work. Would that be a good thing? Probably not: we have Haskell for that. Mypy's leaking errors is probably a practical necessity in retrospect. But, I wasn't expecting the leakage, so it surprised and disappointed me when I encountered it. I had thought I was getting something more like Haskell. Mypy still has attractions, but it's less great than I had hoped.
I think this is only going to happen in reality if you're doing funky type hacks in the first place. Mypy definitely has rough edges, but it works perfectly in 95% of every codebase I've seen.
Type hints (with mypy) aren't going to save you from becomming a bad programmer. But they _do_ help you become a better one!
Honest question: what's the point of type hinting without type checking?
I must be mistaken, but I always thought "type hinting" was a synonym of "optional type annotations". But if you're not using those annotations for actual type checking at some point, what are they good for?
as a reminder for yourself/others who need to read/maintain your code?
(I am often reminded of my perl days, where something I thought idiomatic 3 days ago, is now completely incomprehensible when I just want to make a minor change)
I know because I tried setting it up. It's slow, requires too much babysitting, and fails to catch cases.
In my experience, it just doesn't work cleanly out of the box like in true statically typed languages, and so I get pushback from my coworkers, who simply can't see the point. And I can't blame them.
You get me wrong: my coworkers struggle to see the point of testing at all.
Usually they can be sold on the path of maximum-reward-for-minimum-effort. Hard to enforce/check type hints are not it (they seem like busywork for no actual payoff). With statically typed languages, the ROI would be different: they would either get the thing to compile, or they wouldn't and leave the job.
This sounds drastic but it's really not: outside the realm of purist conversations between fans of programming languages, people just want to do their job and get it over with. If the tooling is convoluted, has too many rules, or they aren't forced to use it ("...or else"), they just won't.
This is a javascript/python shop, by the way. We cannot choose the languages. We can improve the tooling and train the team with better practices though.
> If you/your team want to use a statically typed language, then use one. Python is not it.
This is a bit of a cop out though, isn't it? A response to criticism of a (possibly) flawed language feature cannot really be "well, use another language". How else will the language improve then?
> it just doesn't work cleanly out of the box like in true statically typed languages
So the response is entirely appropriate. Python is not a statically typed language, of course type hints don't make it behave exactly like a statically typed language.
Fair enough. I can see how my statement was not very clear.
I meant to say this: because Python's type hints don't work as well as true statically typed languages (based on my own experience), this makes them less useful and also makes them feel more like busywork. Because of this, my coworkers (who have no experience with statically typed languages either, and tend to dislike best practices and features unless they see a clear return of value from them) are hard to convince type hints are worth the time to learn and use them.
Also, let me restate we cannot pick the language. We can just make better or worse use of it.
That question hides the assumption that static languages are always an improvement over dynamic ones, when in reality we should think as different species that have adapted to fit into different environments.
> We can improve the tooling and train the team with better practices though.
"Better practices" are what makes you team more productive, not just blindly copying what other people are doing. If your colleagues really believe that automated tests/type checking are not worth the effort, you are not going to convince them by saying "but so-and-so said otherwise". What you can do is ask for their pain points, and see if the tooling can help with it. You can look at your past burn charts and say "look at this bug here, what caused and why did it take so long to fix? Is this the type of problem that would be easier to solve with static analysis of the code? Look at this refactor that we are planning for next quarter, should we try to increase the test coverage to make sure we have more confidence in the changes?"
> That question hides the assumption that static languages are always an improvement over dynamic ones, when in reality we should think as different species that have adapted to fit into different environments.
Let me give you a twofold answer to this.
1. I do think statically typed languages are generally better than dynamic languages for most use cases (with a few exceptions). I don't expect to convince you or anyone else of this. It may even be the case that I'm mistaken about this, but I made my mind after years of experience with both kinds of languages.
2. The alleged hidden assumption is not truly important. One should always criticize flawed features of any language, static or dynamic, and the answer should never be "well, choose another language". How else will languages improve if nobody is working on addressing their pain points?
> "Better practices" are what makes you team more productive, not just blindly copying what other people are doing. If your colleagues really believe that automated tests/type checking are not worth the effort, you are not going to convince them by saying "but so-and-so said otherwise". What you can do is ask for their pain points, and see if the tooling can help with it.
I'm both nodding in agreement and finding it very hard to think of something I said that made you think I disagreed with this.
> generally better than dynamic languages for most use cases
Shouldn't then the exercise be to figure out if these use cases are applicable to your situation?
What is your team and company optimizing for?
> How else will languages improve if nobody is working on addressing their pain points?
"See, at my work we need to travel between two islands as fast and cheap as possible. My team is used to high-speed motorboats, but my experience tells me that hydroplanes are faster. I tried putting wings on my motorboats, but it still wasn't as fast. No, we can not buy hydroplanes. No, my team is not licensed to fly. But if no one acknowledges that motorboats are slower, how will they ever be as fast as a hydroplane?
And no, I haven't really looked into the fuel costs of planes vs boats..."
The problem I see is that you are trying to peg a square in a round role and thinking that the square is at fault for not being flexible enough.
Anyone that has seen the py2 -> py3 debacle will tell you that "let's make python static" is not something that could happen unless you are willing to rewrite the entire ecosystem of libraries and applications, and quite possibly upsetting the majority of current users who were attracted to it in the first place precisely because it is so easy to get started with it.
You can not turn Python into a "fully static" language without changing it so much to the point of making into a different beast. And why should others make all this work to fit into your view of what is "best" when you can just use another language in the first place?
I don't want to turn Python into a statically typed language. I think there's valid criticism to be made about the flaws of its type hinting (as does the article's author), and I cannot help but compare its usefulness to static type checking.
I also cannot choose the language. I'm not in a position to choose languages at my current job; I seldom find myself in that position at any job.
We are going to go back in circles, but as I said in the very first comment: mypy is the tool you want [0]. It won't do everything, but progress has been steady and more and more projects are providing type libraries for it.
But given that your response to the mypy recommendation was to complain (it's slow, it doesn't catch everything) then it makes it difficult to acknowledge the "valid criticism"... as in: people are working on it, what else do you want?
> "We are going to go back in circles, but as I said in the very first comment: mypy is the tool you want [0]"
What makes you think I'm not using mypy, or that I didn't read the article you linked to or follow its guidelines (which are very sensible)?
In your mind, is it the only possibility when someone criticizes a tool that they are "using it wrong" (to paraphrase Steve Jobs)? No other possible reason?
> "But given that your response to the mypy recommendation was to complain (it's slow, it doesn't catch everything) then it makes it difficult to acknowledge the "valid criticism"... as in: people are working on it, what else do you want?"
Given that I didn't say or imply that I don't believe there are people working on improving mypy and Python type hints, "what I want" is merely to support the article's assertion that the current version of type hints Python provides is confusing and not very good.
Please, for the love of all that is holy, don't recommend to me again that I switch to a different language. That's not how this works.
> "what I want" is merely to support the article's assertion that the current version of type hints Python provides is confusing and not very good.
What I am having trouble to grasp is: if you understand that Python is not a statically typed language, and if you claim that you do not want to "turn" Python into such, why do you keep conflating type hinting with type checking?
They are two separate things. That is the nature of the game. Do you have criticisms about limitations from mypy, fine, but then you are not talking about issues with type hinting but merely the type checker tool.
> "What I am having trouble to grasp is [...] why do you keep conflating type hinting with type checking?"
It's not my business to help you with your trouble grasping things, but why do you think I'm conflating type hinting with type checking? I specifically asked in this comments section what type hinting meant if not what I thought, and was told by several people "it's just standardized comments", which I then explained was unsatisfying (and other people agreed with me).
> "you are not talking about issues with type hinting but merely the type checker tool"
How about both?
Let me suggest you a more productive use of your time: address the article's complaints as a top-level comment (which I see you haven't yet). I suppose you're more interested in the article's topic than in correcting my alleged misconceptions?
Because the "issues" with type hinting that you are talking about are only based on what you wished it was, not on what it is or what it could become!
Ask yourself this: what about the python's type hinting story you think could be improved without turning python into a statically typed language? What about the python's type checking story that is missing or broken and that is attributable to the language and not to the tool?
> I suppose you're more interested in the article's topic.
Quite frankly, no. My motivation for the conversation now is mostly to see how long is going to take you to realize that you are merely wishing that your motorboat could fly like a hydroplane. It's not about "misconceptions", it's about you complaining about something not meeting your unfounded expectations .
Please don't read it as snark. It was not my intention.
What I am trying to say is that perhaps the lesson you could take from this it's to not think that something is "inferior" or "needs fixing" just because it doesn't immediately meet your expectations.
It isn't Python's fault that you have to use at the job even though you don't like it.
It isn't Python's fault that you have to use even though you are absolutely certain that dynamically typed languages are worse.
It isn't Python's fault that type hints are just hints and can not become a full-fledged type system.
It isn't Python's fault that mypy is still not mature enough to work reliably for you.
And it certainly isn't Python's fault that your coworkers don't care about all that as much as you do.
> "What I am trying to say is that perhaps the lesson you could take"
I think there's a lesson here, but it's for you rather than me.
This is tiring. You keep making stuff up for the sake of argument. I never claimed I didn't like Python; in fact I like it. I never claimed it was Python's fault.
I thought you were going to "wait until I realize". Go on waiting, then. Off you go!
I am sorry for my assumption, I just wouldn't think that someone so certain of the superiority of statically typed languages and so disappointed (with a feature that is barely a defining aspect of it) would also claim to like it.
Anyway, you also had the chance to respond to other questions I made, but instead you preferred to stick to defensive retorts.
Your questions weren't very reasonable, were they? You made lots of claims about what you supposed I thought. I'm under no obligation to answer, for example, what "my company and team are optimizing for", that's an open ended question which is not reasonable to ask in this context, nor would it benefit me to answer it in any way, nor is it related to the question at hand. It looks like cross-examination, something explicitly discouraged by HN guidelines.
I'm not interested in being "educated". In general, you sound very condescending regardless of your claims that you are not snarky.
It's easier to pick on someone's comments than to make a comment of your own, because that would open you to correction and criticism. I think that's what's going on with you in this case.
Regardless, I suggest you stick to commenting about TFA in the future. It will go better for you. You'll be criticized, but that's par for the course.
I don't see what is unreasonable about prodding for the "why" of the reasons you gave (which was basically the point of asking what your company wanted to optimize by using Python), and I don't see what is unreasonable about asking what would be your way to solving what "disappoints" you about the typing story without changing the language.
> I'm not interested in being "educated" by knowitalls.
From where I am standing, you are trying to get your coworkers to adopt some practice without a strong case for how it would help them or the bottomline, purely out of your belief that statically languages are better. If you think I am being the know-it-all here, fine.
> It's easier to pick on someone's comments than to make a comment of your own, because that would open you to correction and criticism.
Someone else in the thread already pointed out that your original comment and TFA are basically a statement of opinion, not of fact. I do not see what kind of comment we can have except the meta-commentary.
> I don't see what is unreasonable about prodding [...]
Yes, I can see that you fail to acknowledge the unreasonability of your prodding. Time to change your tack, maybe voice an opinion of your own, on the actual article, and expose yourself to critique?
Maybe drop the condescending tone and try to avoid the cross-examination? It's against the rules on HN.
> I do not see what kind of comment we can have except the meta-commentary.
I can't help you with what you "don't see"; I cannot make you fix your blind spots. I've already told you your meta-commentary is unwanted. I do not welcome it, it's unhelpful, and every single assumption you've made has been wrong (and smug) so far. It's not my job to present my case to you, you're not doing a consultancy here, nor do you demonstrate particular expertise on the subject.
I foresee another pointless reply of your own. I'll let you have the last word.
In regards to "type hinting/type checking" in Python, my "opinion" is somewhat similar to automated testing: "use it when it can help increase your confidence about the soundness of your program and its design, but don't rely on it as a guarantee of it being bug-free."
In regards to the article, my opinion is that if you want to claim "disappointment" with a language your expectations should be aligned with the language developers and the general motivations of the community at large. Another opinion is that if the author took the time to understand these underlying motivations there would be no "disappointment" and consequently no article, perhaps?
> expose yourself to critique
Hum, that's interesting. I don't think of "expressing an opinion" as something that "exposes oneself to critique". Maybe I am too accustomed with the idea of "keeping my identity small" and "strong opinions, loosely held", that it's natural to me to differentiate an "attack" on the argument vs the person?
Any chance is this why you are so defensive?
> try to avoid the cross-examination? It's against the rules on HN.
First, not rules but guidelines. Second, you have been responding with passive-aggressive insults and retorts which also makes for unpleasant conversation. Third and most important, there is no cross-examination. I am not challenging your motives or trying to invalidate you as a way to invalidate your opinion. The "prodding" is because I don't think your argument has any merit yet, but perhaps this could change if you provided some underlying reason?
> I've already told you your meta-commentary is unwanted.
It's not lost on me that I when I asked "what you would like to change about python's type hinting without changing the language, and what is wrong about python's type checking that is not just a matter of improving the tool", you completely ignored it and preferred to continue with this grating back-and-forth.
Anyway, you are right when you say it's time to lay this one to rest. I hope the next one gets to be more enlightening to both of us.
> First, not rules but guidelines. Second, you have been responding with passive-aggressive insults and retorts which also makes for unpleasant conversation.
You continue nitpicking! You would do better if you followed those "guidelines". Indeed this conversation has been unpleasant, but it's all on you. You have been condescending and insulting. I suggest you focus your energies on better enterprises next time, maybe engage with the article itself instead of picking on others. It will do you good.
> The "prodding" is because I don't think your argument has any merit yet, but perhaps this could change if you provided some underlying reason?
Surely you can see there's nothing to gain by explaining myself to someone who thinks "my argument doesn't have any merit" (and who doesn't think that's insulting) and who keeps misrepresenting everything I say? I just said something, shared by many others, and you felt triggered by the opinion: that's fine, I don't want to convince you of anything, nor do I feel like being "educated" by you.
> Maybe I am too accustomed with the idea of "keeping my identity small" and "strong opinions, loosely held", that it's natural to me to differentiate an "attack" on the argument vs the person?
Well, you haven't done a good job at it, then!
What do you hope to gain, at this point? Are you waiting for me to "see reason"? That's not going to happen. Surely you see sooner or later dang will intervene here and tell us to shut it down?
re: insulting, passive-aggressive behavior: I re-read our exchange from the start. I was polite, even agreed with you sometimes ("I'm nodding in agreement, why do you think I think otherwise") while your tone kept getting more condescending with each reply, failing to accept my agreements and prodding in what I guess you felt was an "educational" tone. Honestly, hand on your heart, do you believe your responses were made in the best possible tone and were the best way to conduct an honest conversation? Do you believe the default position of "this argument has no merit, but maybe if this person explains his team's and company's motivations to my satisfaction I might change my mind" is a mindset that is conducive to amicable conversation? I think, if you're honest, you should admit maybe a tiny bit of misbehavior here.
I also read replies you've made in other articles and you seem more reasonable. I'll chalk it up to you not being able to let go at this point, and so you must bite and nitpick at everything I say. I really think you should let it go, make your peace with not being able to convince me of your point of view, and move on to other enterprises.
I thought you were going to let me have the last word. ;)
> someone who thinks "my argument doesn't have any merit" (and who doesn't think that's insulting)
You forgot the yet. No, it's not nitpicking. Without it, it would be a mere judgmental sentence and it would make sense if you felt dismissed. With it, it is a sign that I am trying to reach for a point of agreement or an understanding.
You are claiming I am the one "biting and nitpicking on everything", but you also could've given a much more charitable interpretation to this, much like you could have done it for the last 6-7 exchanges we had.
And sorry, but I really don't see what is "insulting" about it. Well, at least I don't see how this is insulting if we are to have a rational conversation. This would be insulting only for those who'd be tying their self-image to any of the things being discussed.
> Honestly, hand on your heart, do you believe your responses were made in the best possible tone and were the best way to conduct an honest conversation?
Yes...?
Maybe I am relying too much on the Socratic method, but my questions were mostly to dig in for Truth, not to grieve you. But given the way that old man died, perhaps I should try a different path to not end on the same way as he did.
it's a comment which can trigger an action (if you run mypy).
So, unlike a comment, which does nothing, if you run mypy and get some warnings, you can then do something about it (whether it's just # type: ignore or you realise you made a mistake with allocating to variable new_value versus newvalue/new_valu/etc)
(btw, in case someone objects, some comments DO do something, e.g. golang's godoc examples)
Now I'm confused because the root comment I was replying to asserted that type hints are for documenting, not for checking.
If they are for checking, then my other opinion stands: that they are not very good at it (compared to my experience with statically typed languages, even with Typescript!).
They are for documenting and checking. Obviously, if static typing is your only concern, you wouldn't have picked Python to start with, but if you did have a reason to pick Python, you can get a large subset of the value of static typing for correctness and dev-tine information via Python’s support for type hints and the typecheckers (mypy keeps getting pointed to, but it's not the only one) that support it.
Yes, obviously. There are outside factors why Python was chosen. I tend to work with whatever language is needed, and try to make the best of it.
I've tried mypy and pytype, and found both to be unsatisfying and hard to sell to the team (which as I mentioned, don't even like writing tests). If you're thinking "well, that's a culture/professionalism rather than a technical problem", you are absolutely right! But I'm left wondering if a language that was better at static typing would be better because, a- the checking would be mandatory, and b- it would be demonstrably more effective than Python's "optional" checking/hinting. At least it'd remove one variable from the problem. Or maybe not, who knows.
> So is it simply a standardized comment then? I will have a really bad time convincing my team mates, if that's the case.
If your team doesn't understand the value of things that are essentially standardized comments exposed through the IDE (even if it wasn't for validation, which is also available with IDE integration), you have very bad teammates.
As I said in another reply to a comment of yours: maybe so. I cannot do anything about my team mates, but I can push for tools that make the upside of their usage more evident. "Standardized comments", when working with inexperienced/resistant team mates, is not such obviously good tool.
It is good for programmers. Programmers think in many different ways, but it seems to me that a sizable portion of programmers think of code in terms types. Type hinting makes it easy to convey type ideas. Where are we going from which type.
I should say that anecdotally I find type hinting very useful when I'm reviewing a PR from a part of the code I'm not intensely familiar with.
when types are not checked and enforced, they get out of date just like comments and docstrings.
i agree with the author, if the hints can't be trusted, even just once, there's no point littering the code with them, and are actually harmful when trying to debug something using faulty hints.
i am a big fan of static typing but not in python. pick a language that was designed around it. you can't make a duck bark.
For libraries, with many consumers of the code, type hinting seems like a good thing. Many times I download a third-party library and waste tons of time looking at sample code. For application code (internal to the service or whatever), it feels less valuable.
I wish the tooling outside IDEs worked better. There's obviously some magic going on with tools like PyCharm and VSCode + plugins.
I wish this was the case with command line tools I can plug into the build/deploy pipeline. I know they exist, it's just they are unsatisfying and there are lots of cases where they miss stuff that is trivial to catch in statically typed languages.
As someone who lives in an IDE, I don't understand this. Trying to get all of the functionality of the default IDE, along with the trivially added plugins, to work in a visually digestible and sane way would be a curses nightmare of 30 command line tools. If you made it so they played well together, where a human could interpret what they were seeing, you would have something indistinguishable from an IDE.
It's simply not true. Every statically typed language works like I described. Even Typescript works like this. You can have your IDE plugins, but you definitely don't need them to perform type checks. It's not true this requires "a nightmare of 30 command line tools", you just need one: the type checker (built into the language in most statically typed languages, but sometimes split into a separate tool).
Besides, your IDE doesn't live in the build pipeline (in your CI/CD tool). So you cannot rely on it.
I view them as similar to python's "private" functions, which are really just functions starting with an underscore. The interpreter will let anyone call them like any other function, but the general rule is don't do it, unless you know what you're doing and are willing to deal with the internals changing.
Python typing is like that. If I say a function takes a List[int], but you know I'm just calling a for loop, you can ignore it and hand me a Set[int]. Maybe it breaks some day, but you're allowed to take that risk, if you have a need.
Do either of these things do anything comments couldn't? Not really. But they're ways of indicating intended semantics without formally documenting your stuff, and when most of what you use the language for is scripts, that's actually pretty helpful. People actually use type hints in a way they didn't with comments, and that's caused a major improvement in code readability. Or at least that's been my experience
You can absolutely look at all this and say python's a crazy, terrible language you never want to touch, but if you've got no choice on language for whatever reason, or you're throwing something together and don't feel like writing `private static final synchronized` forty time, type hints are great.
> I view them as similar to python's "private" functions, which are really just functions starting with an underscore
Good analogy! I wish they were more like "private" class methods, which are prefixed by a double underscore and result in name-mangling: you can still access them from outside if you want, but the code will really look ugly. And you absolutely cannot access them accidentally.
Which is what type checking should be all about, right? Preventing accidental misuse?
> Python typing is like that. If I say a function takes a List[int], but you know I'm just calling a for loop, you can ignore it and hand me a Set[int]. Maybe it breaks some day, but you're allowed to take that risk, if you have a need.
That's what drives me crazy. I come from the statically typed world. This bit of Python's philosophy really clashes with my world view. "These types are just something someone wrote, they may or may not accurately describe the code" seems so wasteful and unhelpful to me...
> That's what drives me crazy. I come from the statically typed world. This bit of Python's philosophy really clashes with my world view.
That’s the kind of the culture shock you get learning dynamic type languages with a static type pov. Type hinting is not really the problem here, but can seem that way because it makes dynamically typed code too superficially similar that it enters the uncanny valley if you treat it like statically typed.
One way that might make this easier is to forget about type hinting entirely at first and learn the language “from scratch”, and add the types back after you get used to write dynamically typed code. Dynamically typed code have its advantage and isn’t bad without type hints (well, at least you have to convince yourself on this, or you’ll never be able to learn a dynamic type language), and the type hints just add back some of the nice things static type provides without compromising dynamic type benefits.
Agreed, but it also clashes with the world view that brought us Typescript, which I also find better than Python's type hinting.
In addition, it makes it harder for me to argue in favor of type annotations with my coworkers. My coworkers come from neither world, static or dynamic; they are learning the ropes. And they just can't see the point. I'm confident I would convince them were this a statically typed language, but with Python I'm lost.
The TypeScript comparison is on an entirely different axis though, the compiled <-> interpreted one. One of the reasons languages go with the type hinting route is actually to avoid a separate compilation step like TypeScript. That may be totally fine for you (again coming from a statically typed language where this is more or less required), but it’s a hill many are willing to die on. It’s all tradeoffs and design decisions.
Agreed it's a different axis, that's why I said "also clashes". I of course think Typescript's choice in this tradeoff was the better one, even though I'm not particularly in love with Typescript either.
At some point everything in proglang is preferences and design decisions, but for me, there are better and worse tradeoffs to make.
The typing (>= python 3.6?) and collections (>= python 3.8?) packages have definitions for a bunch of protocols (basically interfaces for structural typing).
So for that List[int] example, you probably want to take a(n) Iterable[int], Iterator[int], or Collection[int] instead, depending on exactly how you use it.
> don't feel like writing `private static final synchronized` forty time, type hints are great.
We should think about whether there's a reason why we want to be writing `private static final synchronized` over and over again, after looking at the state of the software engineer these days.
They are useful for catching issue before code is committed or deployed. At my work we run mypy as part of our test suite, so failing type checks will block a merge or deploy.
How will they catch issues if they are not checked? The comment I'm replying to insinuated type hinting is just for documentation, and not necessarily meant to be actually checked by the tooling.
You seem to be describing actual type checking, which I understand (though in my opinion, mypy is not a satisfying tool for this).
I actually find that mypy is pretty weak and lacking, even within the python ecosystem. pyright seems to do a much better job at the type checking portion of things.
For me, I'm not writing huge programs in python, I may not need type hints/safety right away, or at all, not that I'm against them in any capacity. The problem I see with opinions like these is the assumption that type checking/hinting/safety are a silver bullet.
> Despite all that, what I’m hoping for is that someone will come along and tell me that I got it all wrong. “When you do it as follows, the system works: $explanation” Because, you know, I’d really like to have static typing in Python.
I want X. Y (which is designed for somthing other than X) doesn't do X. Thus, I don't like Y.
Letting perfect be the enemy of the good is not the same thing as doing a cost benefit analysis. In the authors estimation, the costs outweigh the benefits. You don't seem to be considering the cost of type checking at all.
> Even if the Python runtime did check all the type hints at runtime, then it would still be too late. I don’t want a fancy type exception at runtime. That already exists (most of the time). I want to know about type mismatches in advance.
You should check out Typeguard [1], which lets you add a @type_checked decorator to anything and get runtime type checking that's far better than e.g. a random blowup when you split() on an integer. It does introduce a significant amount of overhead, but it's still useful during development alongside type hints to avoid some of the pitfalls from this article.
In another vein, Any should definitely be used sparingly. For the foo() function outlined in the article, Union[str, int] offers more precision. In fact, you could even argue that using Any is a code smell.
Overall, though, I agree with most of the sentiment in this article. I find myself using type hints more as documentation than anything else, since their true ability to prevent type-related runtime bugs is very limited. Documentation has the same qualms the author outlines, anyway:
> You can not be sure that they are correct. [...] They waste mental energy when reading code, they create new maintenance burdens, and they are potentially deceiving: You cannot trust them.
But they're still slightly better than docstrings...
Type inference has been around (as in "used in programming languages") since the early 1970s.
In the beginning python hit a sweet spot: glue code that wasn't exactly write once, too difficult to do in bash, and too small to go all the way and use C++.
However, it was not without its own design issues.
Do you know if there is a way to do runtime type checking in the whole program with typeguard, beartype or something else? As far as I know you have to go through and add decorators manually. Typeguard had a profiler hook that almost got it right, but is being removed. Ideally I would want to say 'python3 -m typecheck myprogram.py' and it would run typechecking everything in my code (but maybe not in library code).
MyPy is great but it checks "offline" or "at compile time" if you know what I mean. It is like a super linter, and doesn't actually run the program (AFAIK).
What I'm looking for is more like a debugger or profiler. It actually runs the program, and then reports if at any time the type annotations deviate from the actual types. (I guess the general case is too costly - think of typechecking a huge list - but for most cases it should be possible.) There are runtime type checkers for python, but the all require modifying the code, and they don't work properly on the main module.
Yeah they're pretty bad but they are still better than nothing. If you are in the unfortunately position of having to use Python I would recommend using them.
I would also strongly recommend using Pyright (the default checker in VSCode) over MyPy. It's so much better, and the author is really responsive on Github.
But yeah, in general trying to use type hints in Python is like trying to discuss philosophy in a mental asylum.
The whole reason Python and JS surpassed most all other languages is specifically because you can write code fast, without worrying about strict definition of data structures.
The issues with python code bases aren't from a lack of strict typing or type support, but from a lack of a good testing framework.
JS is popular because it's literally the only language web browsers support.
Python is popular because it's very beginner oriented, so it's the first language a lot of people learn. It's modern BASIC.
Neither of those things mean that writing dynamically typed code is a good idea. The idea that you can write dynamically typed code faster isn't even true once you get past a couple of hundred lines.
This concept of "programer is generally dumb, so languages must have features to prevent them from making mistakes", with strong typing falling under those features, is solely an academic exercise that is taught in academia, and perpetuated by people who just wanna fit the mold in FAANG and other big players.
Language adoption directly depends on how quickly people can build stuff in that language, and its an exponential effect, because with speed of development comes more libraries, which in turn allows other people to build stuff that uses those libraries quicker. And the initial speed of development directly depends on the programmer having to manage less things. Static typing, in real world, is often a hinderance because code bases are dynamic, and having to go and refactor code because data definition changed takes time. And if you have competent programmers that write clean code and a good code review and testing framework, static typing gives you no advantage since the time spent fixing issues due to failing tests will be equivalent to spending time fixing issues with failure to compile.
> This concept of "programer is generally dumb, so languages must have features to prevent them from making mistakes", with strong typing falling under those features, is solely an academic exercise that is taught in academia, and perpetuated by people who just wanna fit the mold in FAANG and other big players.
Type hints work fine. They are hints (that are checkable through tools)
> But as a general rule: You don’t know if there really is a tool in place to check them.
Cool. So you just go and do stupid stuff if no one is looking?
Check it yourself. They're probably there for a reason. If you think they're not needed, then sure, take them out.
> Most of this turned out to be wrong and it threw me off the track during my debugging session.
Here's an idea: Then why don't you fix it. If I see code that's wrong or bad, I go and fix it. It's your responsibility as well
Now maybe try acting like they're anything but type hints (and I'm very glad Guido insisted it is optional and flexible - the last thing the world needed is for Python to get consumed by Java type checking pedantry where the compiler needs you to annotate every single thing)
And every time I do something in Python that would be "impossible" in Java I'm glad I'm not bound by the small-minded type pedantry of Java.
I find it reasonable that Python doesn't enforce the type checking. The owners of each project can choose how much to enforce it and how.
Defining such style rules and validations is necessary with almost every language. A codebase where developers are allowed to use any C++ feature or browser API will become very messy as it grows.
* fork the language again? We just got out of Python2 hell.
* Docstrings, like we had before they were introduced?
They won’t catch all bugs, but with minimal effort they’ll catch some bugs, and if you make an effort to really dig in, it’ll catch most bugs. To me, to accomplish that overnight on a language with millions of existing lines, that’s a big win.
Idiomatic C and Go programs aren't labeling everything with those types, and the language implementation enforces those types at compile time.
Since types are optional in Python, it's necessarily true that an "Any"-like type is prioritized since it's the default. Moreover, cpython doesn't check anything when it bytecode-compiles.
Any-by-default and no-built-in-checking feels pretty defanged to me.
> Since types are optional in Python, it's necessarily true that an "Any"-like type is prioritized since it's the default. Moreover, cpython doesn't check anything when it bytecode-compiles.
mypy is pretty whiny about types, and sure, if you add Any everywhere then you have just defeated the type checker, but this is no different from putting interface{} everywhere for Go.
> Any-by-default and no-built-in-checking feels pretty defanged to me.
Is running mypy on the code base that much to ask?
Don't let perfect be the enemy of good. Type hints are usefull especially in large codebases. I find myself using them with minimal effort and they can catch bugs that otherwise wouldn't be catched.
Exactly. The article has some valid criticisms of the weirdness of type hinting, but type hinting is a pretty large paradigm shift added to a 20+ year old language. Yes it will have shortcomings, but in my opinion has made working with large Python codebases actually possible. It’s a good thing.
In my experience, people with less experience with object oriented programming will end up creating lots of objects only to create types. These objects have no business use case, nor do they aid in any encapsulation. These objects are made just to help typing. Nuts!
Python type hinting is like moving backwards in time because the amount of time devs take to "hint", takes away the main reason for using python, that is faster development. Might as well code in Java at this point.
This has to be a troll comment. The overhead of typing is literally _nothing_ compared to the amount of time it takes to sift through code to figure out
a) what types a function/class accepts as arguments and
b) what types are returned.
If you don't check it then I agree it's useless. However, working with libraries that are typed is an _insane_ productivity boost over working with untyped libraries.
Typing encourages a large number of bad coding practices. If you can apply typing to a Python program then you are not taking advantage of the dynamic nature of the language to write simpler and shorter code.
In Python world, unit tests cover what is normally handled by typing.
> Typing encourages a large number of bad coding practices.
I'm sorry, what? What exactly is the bad practice here? Not passing random dicts to my coworkers only to surprise them because the key was userid and not user_id? Or wasting time bugging an overworked dev to ask what their crazy 8 levels of dynamic indirection magic are doing, vs just being able to Cmd-B and step through the code path?
> If you can apply typing to a Python program then you are not taking advantage of the dynamic nature of the language to write simpler and shorter code.
Yeah, no, flat out wrong. You can use typing with dynamicism to do magical things. FastApi works a near miracle, generating serializers, deserializers, openapi docs, automatic api deprecation warnings, not to mention type safety.
Properly typed python is faster to write and safely magical.
The bad practice is treating a dynamically typed language as if it was a statically typed language.
There are shorter more concise ways to write your programs if you are not using static typing.
If you are not writing your Python programs in the Pythonic way then you are not really using Python correctly and you do not get most of the benefits of the language.
Unit tests exist for catching bugs in Python. Type hints are for documentation not for ensuring code correctness.
Yes, there are libraries that take advantage of type hints, through they could of used other annonations and it's valid to use type hints along with those libraries.
But as soon as you are attempting to prove your program doesn't have bugs via typing rather than unit testing. You have gone wrong.
Typed Python is far more bug prone, why? Because structuring your Python code to support typing means writing considerably more lines of code.
It not the type hints themselves but it's the way you will structure your program at a high level to support them.
The more lines of code the more bugs it has. Bugs are well known to be proportion to the number of lines of source code regardless of language used or whether the language is typed or not.
> Typed Python is far more bug prone, why? Because structuring your Python code to support typing means writing considerably more lines of code.
That does not bear out in practice. In my experience, bugs have almost nothing to do with LoC. A short bit of complex, I/O heavy code with lots of state is going to be way more bug prone, and far harder to unit test correctly, than 10x the LoC of purely functional, well-typed, stateless code.
Mutation, parallelism, IO, and complex state space are the four horsemen of the bugpocalypse.
But in fact, I find functional, well-typed, properly abstracted code ends up with less LoC in the codebase, because it's more reusable and composes well.
WTF!! If we have to see a codebase littered with TypedDicts just to satisfy type annotations and then have to cast everything to that type - why not just wrap it in a class and call it a day?
When I see garbage like this, I seriously question the productivity improvement brought about by python type annotations. I certainly end up spending an inordinate amount of time unraveling nested unions of types all over now. Considering switching to a team that uses Golang.
> It feels really good to see a program compile without warnings or errors. You then know that you got it right.
I don't know about that. So many C/C++ programs have core dumped in my life and needed to be valgrinded to debug memory issues, or went past the bounds of a pointer, or overwrote the stack, or... whatever.
And you still don't get correctness without unit tests.
I'm not saying types aren't valuable, but they're not a crutch to program correctness like people seem to think they are.
The test for PyCharm inspections only passes when there are no warnings.
Although, I have to admit, we explicitly exclude type warnings because here we have a couple of false positives. So in this respect, it actually agrees with the article.
But then we also do code review and there we are strict about having it all correct.
Yes, I see the argument of the article that the typing in Python is not perfect and you can easily fool it if you want, so you cannot 100% trust the types. But given good standard practice, it will only rarely happen that the type is not as expected and typing helps a lot. And IDE type warnings, or mypy checks still are useful tools and catch bugs for you, just not maybe 100% of all typing bugs but still maybe 80% of them or so.
> Isn’t it better to detect at least some errors than to detect none at all?
> You can not be sure that they are correct. As such, you must always treat them as if they were wrong.
I don't get this argument. Isn't this the case for all other code as well? Most code has not been formally verified to work 100% correct. So you assume always all code is wrong? This doesn't make sense.
> They [type hints] waste mental energy when reading code
How? The author even acknowledges that you could just treat them as code comments if you like. By that argument, all code comments waste mental energy?
> I don't get this argument. Isn't this the case for all other code as well? Most code has not been formally verified to work 100% correct. So you assume always all code is wrong? This doesn't make sense.
It isn't that you have to assume the code is wrong (you always have to assume that code is wrong). It's that even though you write type hints and they pass mypy, you still have to assume that the code has type errors. That is, mypy doesn't rigorously check that the program's types match its annotations. It only approximately checks that.
In (some) other languages, the type checker is 100% sound, so if your program passes type checking, you can be completely sure that the types all match. Opportunities for the code to be wrong are thus limited to mismatches between the types and the desired behaviour.
594 comments
[ 3.4 ms ] story [ 421 ms ] threadMost of the benefit of types goes away unless _everything_ is typed... my libraries, the libraries my libraries use, etc. I should be able to jump around in and out of library code following types and usages thereof in my IDE, making conclusions that are accurate based on what the types say.
When some of your libraries (or indirect dependencies) don't have types, you can no longer ever make any firm conclusions when exploring a complex codebase based on types. I want to change a method in the Widget type, and my IDE says it's used in 4 places. But in reality, that just means 4 or more places, who knows if there are others.
Adding types to a language that doesn't already have them is very hard for this reason. Until many years pass and they're in universal use, they add little value.
[1]: https://github.com/python/mypy/issues/3186
https://stackoverflow.com/questions/69334475/how-to-hint-at-...
I would rather my type checker give me the truth than gloss over it. This issue should be kept open (it is) but they should wait until a good solution is found that doesn't gloss over the discrepancy.
Though I agree on its negative impact on the language ergonomics. It removes the joy of writing "scripting language" and as a statically typed language it is far from static-type-native ones.
Now I see it as a tax for compatibility: You enjoy dynamic typing at the beginning of the project, but you have to pay it back once you've grown up to a certain scale.
TypeScript to me feels much more like a native static typed language because it is. Does it help to be a transpiler? Maybe. But I would think that Hejlsberg just has done an astonishingly good work to make it compelling. Python's type annotation is good and completely reasonable, but gradual typing is such a hard problem and being good is probably not enough here.
Python is still strongly typed.
Me: what type does this argument need to be?
Python: you have to figure that out
Me: I just pass whatever I want?
Python: Oh no. Usually only a single specific type is supported. But you have to guess what it is.
Me: what if I get it wrong?
Python: the program crashes at runtime
Thanks for the hint.
I'd argue if your functions are full of type-assert like guards, _then_ use another language!
Typechecking allows certain errors to be detected at typecheck time rather than at runtime. I don't personally consider that useless. E.g. A mistake I just made. I'm working in a language I don't know too well right now with strict static typing. The file read function takes a handle. If I pass it a string of the file name, rather than the handle result from file open, it just doesn't compile. In most dynamic languages, that error would not be detected until it executes.
Type checking is just a really poor way to ensure program correctness until you start using an exceptionally strongly typed language like Haskell.
You use types in C / C++ / Java because the compiler needs them to work not because it's a good idea.
This claim doesn't make sense to me. Assuming mypy is regularly run as part of CI, why would adding type annotations cause more bugs than leaving it off?
When I started learning PyCharm though, I discovered that type checking can actually work well in Python and it is not a waste of time at all. So I advise people now that unless you're using PyCharm, do not waste any time on type checking Python. In which case, have at it. I love it. PyCharm is really the only way to do type-checking productively right now.
I would recommend to try out if you can, the experience is quite good. I personally find “strict” too strict for the current state of python, but “basic” is already very helpful.
The developer experience is still subpar when compared to Typescript but it’s improving fast (but I mean, typescript with VSCode has one of the best developer experience I’ve ever seen).
Overall I do think type hints are worth it though, for maybe two benefits. They force you to look at the ridiculous types you are using, like `List[Union[None, str, List[Dict[str, str]]]]` which is the sort of thing that happens in codebases all the time. It adds just enough friction to push people to make explicit dataclasses or simplify function returns, which is good. Secondly they help with tracking functions which return None, which is a pain when following callstacks in big codebases.
That said, I think pytype makes a lot of sense since it infers types from code, which you can edit by hand and then merge back into the python file when your code is stable.
My experience with Python is that it slows development speed to a crawl due to dynamic typing. It’s maybe faster to write the first 1000 lines. But after that it becomes slower and more painful. At least in my experience.
Completely disagree. The Python community has very rapidly adapted mypy because of widespread recognition that yes, type information is extremely helpful for any code bases larger than a few files and/or worked on by more than a few developers. Every major Python library I can think of now has mypy stubs available. If you're going to dismiss them as "not Pythonic" you may as well dismiss anything other than Python 2.7 as "not Pythonic".
Like, even if you think that type hints have ugly syntax, prior to them vast most projects just didn't bother with documenting the shape of data, so you had to read source code, guess and experiment. Unless you just don't care about knowing what you're specifically working with (which strikes me as living on the edge), how is it not an ergonomics gain?
Onboarding into a huge, untyped codebase is brutal
Okay, this myth that Pythonic == loosey-goosey duck-type-everything slinging dicts and strings nothing static cause that's too slow, really needs to die.
Pythonic is all about pragmatism and parsimony. If that means not typing anything cause it's a 50 line script, do that. If it means the most efficient way for large dev teams to communicate on sprawling code bases is to use rich types, then do that.
Personally, I use types even in the short scripts, because then I can offload keeping track of what type everything is, instead of having to remember yet another user_dict with whatever keys the producer felt like using at the time. It makes autocomplete faster and I'm less mentally fatigued. Less mental fatigue == more pythonic.
My company runs some kind of static analysis that checks things. And I get 80% of the benefit with 20% of the work. So start small, and don't expect perfection (yet).
Also in my experience it can provide static analysis tools more information and provide autocomplete suggestions in a wider range of coding scenarios.
A few concrete criticisms:
1. If you're using a dynamic language, then _by definition_ the language will not enforce your static hints at runtime. However, good news! Python has always been strongly typed, and _does_ enforce types at runtime!
2. A number of the examples given would be _impossible_ to statically type in most compiled languages (those without dependent types). It's hard to know where to go with a critique saying that you can't fully represent heterogeneous values in a dict, given that you can't do this _at all_ in many statically compiled languages.
It seems to fall back on saying "use of the type system requires too much discipline to be useful". This might be an interesting criticism on its own; however many current users of Python can say, from experience, that the required discipline is not a high enough hurdle that it prevents us from using types consistently and correctly.
There is plenty to critique about Python's static typing, but this article would have been better titled "I'm confused about Python's static type system and want somebody to show me how it's actually used in real-world scenarios".
Duck typing and heterogeneous dictionaries are (and have been) standard Python.
Adding a type system which doesn’t respect duck typing by trying to access the member even when the types don’t match or which can’t express standard idioms used in Python seems a poor architectural choice.
It’s not saying that Python’s type system is “too much discipline”, but rather that the type system doesn’t encode typical Pythonisms.
and heterogeneously typed ducts: https://peps.python.org/pep-0589/
Traditionally, duck typing is used to inject types into a library that isn’t expecting extension at that particular point — eg, substituting a test class for a real class in a data object that normally wouldn’t be a protocol.
I’m not seeing how that PEP addresses that use case.
- - - -
Similarly, the heading on the dictionary says “for a fixed set of keys” — but what if I want a dynamic heterodox dict? Eg, unpacking JSON.
You just end up shoving “any” all over the place. At which point, are the types helping?
Though, protocols do help the dict case for typing:
dict : Hashable -> Any
https://peps.python.org/pep-0544/#:~:text=Structural%20subty....
> substituted-for class to be defined as a protocol.
... which is false
> Similarly, the heading on the dictionary says “for a fixed set of keys” — but what if I want a dynamic heterodox dict? Eg, unpacking JSON.
You can quite obviously decode to recursive types (e.g. using pydantic), not sure what the problem is.
This section seems to agree with me, where it explains why normal classes can’t be subclassed to protocols:
> Now, C is a subtype of Proto, and Proto is a subtype of Base. But C cannot be a subtype of Base (since the latter is not a protocol). This situation would be really weird. In addition, there is an ambiguity about whether attributes of Base should become protocol members of Proto.
https://peps.python.org/pep-0544/#protocols-subclassing-norm...
- - - -
I’m not sure why you think it’s “obvious” that you can use a third party library to solve the problem — or why that addresses my complaint that the built-in type system doesn’t work for that.
If anything, the existence of a third party library hints the standard library doesn’t cover the use case.
edit: To be more precise: The article specifically mentions that accessing invalid members give you a type error, but posits that people will probably not bother and just use 'any' instead. How is that not saying 'too much discipline'?
I’m also generally pro-types, but I think it’s worth having a discussion about this type system in the context of Pythonisms.
- - - - -
If you’re using “any” for most of your types, then it’s not providing value — an untyped statement implicitly has the type “any”.
There’s a reason I brought up the JSON example: loading and manipulating JSON of varying structure is something I do a lot at work.
>There’s a reason I brought up the JSON example: loading and manipulating JSON of varying structure is something I do a lot at work.
Do you have a specific example that you find troublesome? For JSON with a fixed, regular structure (i.e. no subtyping) something like Pydantic works well, otherwise using unions + protocols with custom validation code has covered even my most esoteric use cases so far.
If we're talking about unstable JSON, a recursive union type works well enough in my experience, though defining that type gets a bit ugly if your type checker doesn't directly support recursive types.
Python types don’t support a common Pythonism used frequently in my work.
In fact, you admit that even fixed JSON can be difficult without a 3rd party library.
You’re lecturing me like I don’t understand types without realizing that you repeated my point about a gap in the current type system.
>In fact, you admit that even fixed JSON can be difficult without a 3rd party library.
Sure, but that's just because the standard library (for now?) doesn't offer deserialization with runtime validation. Java doesn't have that either, if you want to parse JSON you'll have to either roll your own or rely on a third party library.
Don't get me wrong, the situation isn't perfect, but especially if you use the 'right' tooling, you can in my experience already get a perfectly serviceable experience, which to me is indicative that the type system itself is fine and pythonic, it's just the tooling that is lagging a bit behind at the moment.
> Sure, but that's just because the standard library (for now?) doesn't offer deserialization with runtime validation.
> it's just the tooling that is lagging a bit behind at the moment
So… the standard library has exactly the problem I articulated, and you’re violently agreeing while speaking down to me.
I think that reflects poorly on you.
If you think that's violently agreeing and talking down to you, then there must be some communication barrier that I don't know how to overcome.
This does not define the type of a valid json object. (fe None is not a valid json object, neither is '')
However, even in Typescript it is nontrivial to _annotate_ these complex types, and in most cases it's still possible to do in Python - you just need to commit to using something like dataclasses rather than pretending that what you have is a `dict`.
Basically, to get the most out of static typing in the world of Python, you do have to write more boilerplate than you would in Typescript, and in particular you need to avoid dicts and prefer various sorts of class-based data containers (again, dataclasses, attrs, namedtuples, etc).
The only one which stands out to me as non-trivial in TypeScript is thrown exceptions… and even then only because you can only type them in a catch clause, and only with unknown, and they’re invisible to callers. But then this is why people who would care for checked exceptions quite reasonably just use something like a Result type.
But I agree with your conclusion completely, and it’s basically the same one I came to reading the post.
I was curious about how the current state of things compares to TS, as I may soon have a foray back into Python soon, and would be delighted to bring some static types into the mix. Pretty much as I expected, it’ll probably be more effort/less valuable than TS, but more valuable to me than to the author.
— Consider using Pydantic (either its drop-in dataclasses replacement, or its models). It has issues and documentation is lacking in places, but in many use cases it’s a boon when it comes to being confident about data you read and/or pass around.
— Don’t use TypedDict if you foresee needing anything like optional keys, especially with defaults. I went with TypedDict and it came back to bite me later (though was relatively easy to refactor).
— Keep an eye on https://pypi.org/project/typing-extensions/, it has some useful utility types. I wish I knew about this package a couple of months back.
— If you use VS Code (which I started after switching to TS), there’re some shenanigans to be aware of. Unlike TypeScript’s elegant approach where you’re in control of installing typing versions, VS Code’s official Python extension will force-install unvetted third-party typings, not caring if they don’t match library versions in your use and not allowing to opt out. In later versions you can unset mypy in Python extension settings, and use a separate Mypy extension that doesn’t do this behind your back and doesn’t send you deep into some rando’s typings when you hit F12 to jump to definition.
— That aside, VS Code can provide development experience is not that far from TypeScript. You can keep a virtualenv nearby and select its Python bin as interpreter in your workspace, giving Mypy access to dependencies (and if VS Code’s interpreter selection GUI resolves symlinks, you can enter interpreter path by hand so that it doesn‘t).
— Sphinx documentation is good at picking up typing hints and works reasonably well with Pydantic’s models, too.
This seems like an inaccurate and condescending put-down. What is the definition of "educated" in this context? Is there a book or generally well-known resource?
The author is clearly thoughtful and curious. Near the top of the post he says "what I’m hoping for is that someone will come along and tell me that I got it all wrong. “When you do it as follows, the system works: $explanation” Because, you know, I’d really like to have static typing in Python."
> 1. If you're using a dynamic language, then _by definition_ the language will not enforce your static hints at runtime. However, good news! Python has always been strongly typed, and _does_ enforce types at runtime!
And yet, the author points out that this is a working program:
In what reasonable sense can Python be said to "enforce types at runtime" here?Can a string be printed? Yes! Then Python is (correctly) allowing typesafe behavior here. Your (incorrect) annotation does not in any way contradict the typesafe-ness of printing a string (or an int). They're both equally printable.
This is just a very, very bad example, plain and simple. It 'looks' bad to a superficial reading, but in practice it demonstrates only what is already known about static typing in Python - it is enforced (or not) separately from the interpreter.
This said, I find the linting/type-hint-checking stage of Python cumbersome and confusing :(
Am I the only one baffled by this way of thinking?
Static type annotations by definition are not enforced at runtime. This has been true of every language that has ever used static typing, including C, C++, Java, Rust, Typescript... you name it, statically typed languages only enforce their static typing at compile/analysis time.
Yes, it is possible to annotate types in Python incorrectly. It's possible to do this in all other languages that allow type-unsafe behavior (whether natively or via reflection, etc). This may be less common in some languages than in others, but it is fundamentally possible in the vast majority of languages that perform static typing, because those types are fundamentally enforced at analysis time, not runtime.
The author of the article seems to be unfamiliar with these distinctions, and maybe you are too. It's fair to complain that these distinctions are confusing and make things harder for programmers. Nevertheless, very few languages have ever been designed with a type system that acts exactly the same at analysis time and runtime. It's a very difficult problem, partly because these are all abstractions from the perspective of the underlying computer, which only understands boolean logic and integer/floating point math, and has virtually no other notion of types in any formal sense.
Type systems are a complex topic, and as someone who has been paying attention to them for a while, it's frustrating to see uninformed discussion of them show up on Hacker News. That said, it's perfectly understandable that people are confused by these distinctions, and rest assured there's a lot of effort going into developing better languages that suffer _less_ from these issues.
For the everyday working programmer, however, there are currently lots of tradeoffs to be made, and Python's approach to static+dynamic typing is actually pretty usable, all things considered, which is why the community overall has embraced the new static typing despite its blemishes.
It is pretty damn hard to make this mistake in languages that enforce static typing. I mean, you would have to go out of your way to do so. In Python, however, it is trivial to write the wrong type signature or to modify the body of a function so that it no longer matches the signature.
"Fundamentally possible but terribly unlikely, as opposed to the Python way of doing it" would be a more accurate description.
Since this argument is not meaningfully different from "comments that are lies considered harmful", it seems fair to expect reasonable people to dismiss it as uninteresting.
Do you think that's a fair treatment of someone who disagrees with you but has been respectful of your opinion so far?
I realize proglang debates are flamewar territory. But I have been very careful to state I find Python type hints puzzling and less useful than they should be. It is obviously an interesting opinion shared by many others, not just me or the article's author.
As a fan of statically typed languages, I do indeed find some Python programmers engage in a form of Stockholm's syndrome. It usually takes the form of the assertion "I never found a bug that was a type error". Have you or anyone you know ever said this?
In my mind, the opposite statement of the one you attribute to me would be "wishful thinking and a positive attitude is enough to catch mistakes, it's not necessary to have help from automated tooling". In this day and age, I cannot disagree more.
It's precisely because I do like and use them that I find Python's static type hints to be extremely useful for large Python codebases.
What I hear you saying is "Python does not offer these things". What I hope you hear me saying is "Python does, in fact, offer these things".
> I think your (and the article's) argument could be summarized as "static type annotations without automated enforcement by actually running a type checker considered harmful"
It illustrates your mindset, in my opinion. I think type annotations without checking them (or without a satisfying implementation of said type checking) are mostly pointless.
You also left out the part where I remarked on the dismissive tone of your reply.
However, there are other aspects to the article’s argument, such as the fact that it’s easy to end up with type annotations going silently unenforced even if you do run the checker.
Also, some languages like Java do actually enforce types at runtime. If you've ever called a Java method dynamically with the wrong types, you'd know. You can run into ClassCastException, or worse (ClassNotFoundException if you have the wrong classloader.) This is a runtime error.
A type system is just a set of rules that must be followed. The rules can be as weird or as simple as you want. Just because <int> + <string> is allowed doesn't mean it's not enforcing types.
Not really a criticism, just trying to push to expand your idea of what a type system is.
Is it "legally" baffling, i.e., do I understand why this behaves in the way it does purely mechanically, and do I understand the arguments the gymnasts are making? Yes.
But from an idiomatic, colloquial, linguistic, syntactic, programming-historical, or programming-cultural standpoint? Baffled.
Even calling it gradual typing is baffling. It's just adding metadata-that-doesn't-look-like-metadata that sophisticated programs that aren't part of the Python bytecode compiler can use. The definition I know is the same one from Wikipedia:
> Gradual typing is a type system in which some variables and expressions may be given types and the correctness of the typing is checked at compile time [...]
This is definitely not what Python is doing.
Maybe not by default but there is tooling that makes it work that way — that’s exactly how the python build system works at my company. I don’t know the details but when I “build” python code (creating bytecode pyc files) it produces compiler errors and fails to build if I e.g. try to pass an int to an f(x: str). This has usefully caught bugs before I ran some expensive/time-consuming scripts.
I guess you could say this doesn’t count as it’s a separate program doing the type checking but IMO there’s not a clear distinction between that and e.g. having a first type checking pass in the “same” compiler program.
> While these [type] annotations are available at runtime through the usual __annotations__ attribute, no type checking happens at runtime. Instead, the proposal assumes the existence of a separate off-line type checker which users can run over their source code voluntarily.
When I read "compile time", I mentally replace it with "before runtime". There isn't a real compilation step in Python, so it's reasonable to accept static code analysis as part of "compile time" in that definition.
The point is that type annotations will let a static analysis tool make sure that the program is correctly typed. This can be enabled easily in most of the modern IDEs and editors. You could just decide to block the code's deployment in CI/CD if the type checker raises issues, for example.
E.g. this kind of code: https://play.rust-lang.org/?version=stable&mode=debug&editio...
typically works fine (or worse, works fine in debug mode and causes unpredictable UB-related bugs in release mode).
std::mem::transmute here is a no-op at runtime; all it does is subvert the static type checker.
The difference between statically-typed languages and things like Python with type hints is _not_ that the former has any runtime type checking; it’s that subverting the (static!) type system is more difficult and requires more obvious ceremony, and that static type checking is required whereas with Python it’s optional.
If you hacked rustc to disable static typechecking you would get a similar experience to Python with type hints but without any mypy-like tools. (Of course, this would be hard to do in practice, because Rust, like most statically-typed languages, uses types not just for typechecking but also for code generation and method dispatch.)
When Python "enforces types at runtime" this is the actual types, not type hints. Type hints are not a runtime artifact. The "actual type" here is "string", and enforcing it means Python would not allow invalid operations on it without error'ing. A programming language that will let you do basically anything to any value because it doesn't enforce type safety is of course C. Python is safer than C.
Best of all, all of these are enforced at compile time.
Now, C absolutely will let you convert an int to a pointer, or a char to an int, or an array to a pointer, or... well, it will let you convert lots of things to lots of things. Some of those things are unsafe unless you are quite sure what you're doing.
> A programming language that will let you do basically anything to any value because it doesn't enforce type safety is of course C.
False on several grounds. You can't call something that isn't a function. You can't call a function on something that doesn't have it. You can't use an int as a pointer or an array without casting it. You can't use an int as a dict (not that C has any built-in idea of what a dict is...)
> Python is safer than C.
Depends on what you're doing, and what kinds of errors you are more prone to.
You know what I meant and what error I was clarifying for the comment I was replying to. Yet you went out of your way to point all sorts of irrelevant mistakes in my post, when the gist of it was right.
Do you feel it was more important to correct me on these trivial things, or was my reply more or less correct when fixing the conceptual error in this sentence?:
> "In what reasonable sense can Python be said to "enforce types at runtime" here?"
Here's a nitpick of my own to your post:
> "Some of those things are unsafe unless you are quite sure what you're doing."
Wrong. Type (un)safety does not depend on you "being quite sure of what you're doing".
PS: assume I know C and that I've programmed complex things using it. No need to catch yourself with statements like "not that C has any idea of what a dict is". Assume we are both C programmers. It will make you sound more polite.
2. Your last two sentences were, I think, not needed for the point you were making. They were also somewhere between gross generalizations and flat-out wrong, and felt like a gratuitous slam on a language that wasn't even the topic. I thought that deserved a response, even if it wasn't your main point.
3. I may assume that you know C. I don't assume that everyone reading this exchange knows C.
Pointless correction. Gross generalizations are not wrong in this particular situation, which is not meant to teach anyone C.
Let me try again.
Do you think that your correction was more important than helping the poster of this comment correct their conceptual mistake?
> "In what reasonable sense can Python be said to "enforce types at runtime" here?"
Sure you can. That's exactly what you do when you call a function returned by `dlopen` -- it might be a function, but it might also just be garbage. It might even be garbage that's coincidentally marked as executable and does some stuff before eventually crashing!
> You can't call a function on something that doesn't have it.
This is also very easy to do -- you can default-initialize a structure containing function pointers, and then call one of them. You'll be calling some random crap on the stack, which might or might not do anything of interest. But either way, C will happily let you do it.
C isn't completely untyped (there are, as you've observed, things it will forbid at compile time), but it's about as weak as a static typing system can be.
In both examples, you have tell the compiler exactly what the types are. In the first one you have to cast the return value of `dlsym` to a function pointer, and at that point, as far as the compiler is concerned, it's a function. In the second example you have to explicitly declare the struct containing function pointers, so of course you can call its members.
So it's strange to suggest that C is somewhat untyped; it's not untyped at all. C just makes it very easy to force the compiler to assume different type for a value (that's necessary for low-level code like the examples in the comment you replied to; and of course that makes it very easy to do unsafe things, I don't think anyone disputes that).
And about the "weak typing system" comment, note that nobody agrees on what exactly that means. Just as an example, see how this page[1] defines "weaker" vs "stronger" types, and how it's completely different from the way you're using here. In general, I find people call type systems that don't do what they expect or want "weak", but that's not an useful discriminator: at that point you might just call it "bad typing" to dispel any illusion that it has any objective meaning.
[1] http://book.realworldhaskell.org/read/types-and-functions.ht...
Just because "weak typing" means something different in Haskell doesn't mean it can't be used meaningfully (and beyond "bad") in the context of C. C's typing is historically referred to as "weak" because C's notion of casting doesn't distinguish between type and value transmutation: pointer-to-pointer casts don't convert the referent (because they can't), which in turn gives the C compiler very little leeway in proving that the program's types as declared have any particular meaning at runtime.
Compare this to Python, which is "strongly" typed in the same sense: doing `y = str(x)` on `x` means that `type(y)` actually is `str`, and not merely a promise to the runtime. It's enforced, which makes it strong.
That's why it's useless to say things like "it's about as weak as a static typing system can be" and "has been historically considered weak".
If you had explained your objection to C's type system (like you have now), I wouldn't have said anything. And I've been on the Internet long enough to have seen C's typing is historically referred to as "weak" because X for *many* values of X, so I disagree that that's the only or even main objection.
I believe this is the relavent passage in the article the GP is referencing, which is incorrect if you take "runtime" to mean something like when the line is executed: "Python is a dynamically typed language. By definition, you don’t know the real types of variables at runtime."
It _is_ however correct to say you don't know the real type _before_ runtime, at least python does not.
The Any type is to allow incremental typing. You start out with dynamic code, and progressively 'typify' it by adding more and more annotations, with Any working as a stopgap where for the moment no valid type exists, until you finally have a fully typed program. Inserting Anys because you're lazy is like casting things to Object in Java because you're lazy, abusing it like that is just incredibly sloppy and bad code.
If someone is really struggling with this, simply use a linting rule to ban Any in 'mature' code.
There are lint rules to avoid typing as `any`.
Python is both strongly typed and protocol-typed. In the case of `print(...)`, the protocol is that the received parameter responds to `str(...)` (i.e., has `__str__`).
(What Python isn't is statically typed. But "strong" and "static" are completely different typing dimensions.)
> In what reasonable sense can Python be said to "enforce statically declared types at runtime" here?
which is certainly an extremely pertinent question for a language that allows declaring supposedly static qualities of a program.
It can't, because Python doesn't have statically declared types. It's a strong, dynamically typed language.
Python also doesn't allow you to declare static types. It allows you to declare type hints, which are a sort of adjacent and potentially unrealistic (or just plain incorrect) typechecking proof that doesn't correspond at all to runtime behavior.
Python has the capacity to have some form of gradual typing built-in at the language level, but it does not, which has led to the enormous confusion we have before us. The SBCL implementation of Common Lisp is a great example of a strong, dynamically typed language that is able to, when possible, detect errors statically at compile time. That tech is 30 years old now.
There is no reason why the Python bytecode compiler could not statically or dynamically detect this programmer error at either compile-time or run-time present in this statement:
Compare to Common Lisp, well known for being dynamically typed (so dynamic that it's possible to change the class of an object at run-time!): Instead, this job in Python is haphazardly relegated to the program readers and IDE implementers.I don't see why that would be the case. The value of Mypy is that it's essentially a proof language over something that resembles Python[1]: it's a way to do some amount of static typing without having to maintain a separate copy and representation of the program.
And sure: Python could detect inconsistent type hints at runtime. But why would it? It's already strongly typed, and the presence of type hints usually means that someone is already running Mypy or another checker during development. It's not clear that there's a significant advantage to be gained, particularly one that justifies the additional overhead.
[1]: "Resembles Python" because mypy does not actually evaluate any Python. It doesn't know what types your program has at runtime; it only knows type hints and a few small rules (for things like string literals) and trusts the developer to reconcile those rules with Python's runtime behavior.
Because the programmer explicitly intended a contract which is being violated, which would in turn indicate a bug.
Anecdotally, my code in python is functionally fully typechecked and runtime type checking would solely penalize me by slowing down my code. There's no need for it because the type checker correctly proves the soundness of the program's types, in exactly the same way that Java's or C++'s do.
The point is precisely that you don't need to suffer the costs of runtime type checking if you have a solid typechecker. Lisp's approach here is worse than the one adopted by python/js/ts/go/etc.
Yes, but is it caught at static analyzer run? Yes, I typed it into vi and ran mypy on it and it said "error: Incompatible types in assignment (expression has type "str", variable has type "int")."
But I didn't have to run mypy on it, my editor told me: "Expression of type "Literal['hello']" cannot be assigned to declared type "int"".
Maybe the word "educated" wasn't the right choice in this case, can you offer a better one? I honestly couldn't make it through the entire article, despite several attempts. It is full of inconsistencies and what seem like wilful misunderstandings to justify the authors conclusion.
Particularly as the author starts off saying "run a static analysis tool" as if knowing that's how you go about it, and then saying "it doesn't catch it at runtime".
It does: in form of TypeError exceptions. It doesn't enforce type hints, sure, but that's why they're called hints and not declarations.
But we have (to quote another commenter) "uneducated" people making a supposedly boneheaded mistake to assume
is a declaration that x will only contain values of type T, or otherwise something will be detected by cpython as invalid (at either bytecode-compile-time or run-time).Instead, this line is more properly interpreted as "documentation reasonably believed to represent a type saying something about values that x can be assigned at runtime, that's not in a docstring, that some programs may query."
I can totally forgive anybody who believes the former over the latter, and expresses their belief in a statement like that which you've quoted.
Absent of active machine checking (e.g., via mypy), type hints should be considered exactly what you said: another form of documentation, one that might just be wrong.
But in Python, it's not (and documented in a PEP as not), and it causes massive confusion.
[1] https://devblogs.microsoft.com/typescript/a-proposal-for-typ...
Python is strongly typed. It respects types at runtime e.g., `"1"+1` causes TypeError. Python is not statically typed.
For comparison, C is statically typed but it is weakly typed e.g., printf function has no idea about actual types of its arguments (in particular, errors in the format arg may produce non-sensical results silently).
As far as I remember, the "__str__" method is present on all objects in some form, so print will happily take any type.
Most people I know writing Python don't use type hints because they are too much of a hurdle for very little payoff. The tooling that pays attention to type hints is slow as molasses or difficult to use or understand. The type hints themselves are of dubious use.
As a fan and advocate of static typing, I find myself advocating for type hints anyway, but I must agree I often can't reply anything to my coworkers' objections, because Python type hints are truly not that useful.
I use mypy daily on big codebases, it is fine, not fast, but fine.
> The type hints themselves are of dubious use.
They tell you when you make type errors, they help you understand what type things should be. The same as in literally every other language with static typing. There is nothing special here, nothing different. python with a static type checker running in strict mode is not fundementally different from Java's static type checking.
> As a fan and advocate of static typing, I find myself advocating for type hints anyway, but I must agree I often can't reply anything to my coworkers' objections, because Python type hints are truly not that useful.
What do they lack that would make them useful?
No, obviously not the same, otherwise I wouldn't be complaining. They are not even on par with Typescript, which I'm not a fan of either.
> [type hints] tell you when you make type errors
Not according to other comments I seem to be getting here. Other people are arguing type hints are not primarily for checking, but a form of notation for documentation. Seems wasteful, and I wish the Python community and tooling decided instead that they are for actually checking them.
> What do they lack that would make them useful?
Standardized, go-to tools that work in the build pipeline and that catch most errors without taking a long time to do so.
I haven't found mypy to fill these requirements. It's so bad I cannot convince my coworkers to make the effort to write more type hints.
What is the difference?
> They are not even on par with Typescript, which I'm not a fan of either.
Go is not on par with Typescript or Python, I still don't think it is okay for people to just say fuckit and `interface{}` it all and it is still shit to work with code that does use `interface{}`. At least Python with mypy has null safety, something that Java and Go does not have. There are some places it is worse than other statically typed languages, others where it is better.
> Standardized, go-to tools that work in the build pipeline and that catch most errors without taking a long time to do so.
It is mypy. What actual type errors does mypy not catch that you want it to catch? Why can't you use it in the build pipeline? I do it every day, it catches all the errors it should. The one complaint I can maybe see is the duct type compatibility complaint, and it is not really something that comes up that often for me, and definitely not something I would say invalidates the whole concept.
I'm not going to repeat myself, I already told you.
I find mypy slow, unsatisfying, inconsistent, and it fails to catch many type errors. No, I'm not going to go look in my work laptop to give you an example.
> There are some places it is worse than other statically typed languages, others where it is better.
In most places it is way worse, and I'll find it very hard to find common ground with anyone who disagrees on this.
Feel free to disagree, but I don't find this conversation useful.
Compare that to a c++ project where it’s not unusual to have 10, 15, 30 min build times.
Or even worse, the time to wait for discovering the problem runtime in prod.
The type hints also help in IDE navigation, where there is zero time to wait, pycharm can go to definition or find references in under a second.
That's locally on a fast (ish) laptop. GH CI takes 10 minutes to run tests, so I'm a little hesitant to see how long mypy takes to run.
in my experience it's a far better type checker than mypy, which tends to silently not check things without you ever realising
> Python is a dynamically typed language. By definition, you don’t know the real types of variables at runtime. This is a feature.
In "you don’t know the real types of variables at runtime", this is just flat wrong. One doesn't know the real type of a variable until runtime. It seems to me the author maybe has a bit of confusion between weak typing and dynamic typing here.
I read the sentence as
as opposed to So I understood the author to mean what you said. Could have been clearer you're right.In the program
what is the "real type" of `f` at runtime; say, at the instant just before the call to `f`? How can you tell?If your answer is that `f` has type "function-like", then that's a much weaker kind of a type than even type hints offer, let alone a real type system.
We can tell by looking at what python triggers TypeErrors for.
>If your answer is that `f` has type "function-like", then that's a much weaker kind of a type than even type hints offer, let alone a real type system.
f implements the Callable interface - meaning it's either a function/method or an instance of a class with a `__call__` method.
So you can pass e.g. `print` or an object with a `__call__` method. If you try to do `x(5)` that's an immediate runtime TypeError because `5` can't be called.
f must also accept one int argument. It can have additional optional arguments, and it can also accept other types (e.g. `f("foo")` can also work!), but it must work on one int. The one int can already be optional - the function can also work with zero arguments.
Calling e.g. `"foo".join(5)` fails with a TypeError because 5 isn't Iterable, and calling `"foo".join([1], [2])` fails with a TypeError because it gets more arguments than expected.
Counterexample: PHP is a dynamic language which enforces static type declarations at runtime.
The point is that it is not expected by definition, since by definition, static != runtime.
So PHP is not a dynamically typed programming language according to you? It enforces type hints at runtime just fine.
There is nothing in the definition of dynamically typed languages that says they can't use type hints to perform runtime checks.
What statically compiled language doesn't provide a way (sometimes clunky) to have heterogeneous values in a dict? You can do that with void* in C, even
What definition is that? Raku is a dynamically language with gradual typing which will enforce those types at runtime (if it cannot do so at compile time).
I don't think this follows by definition. It's common not to check the static hints at runtime, but a dynamic language could choose to treat them as contracts or assertions. Over in Lisp-land, SBCL will enforce static type declarations at runtime [1], contrary to most other Lisp compilers, at least under the default compilation settings (you can force it to skip the checks by compiling with a low "safety" level). If the compiler can prove that a static type declaration always holds, it will omit the runtime check; otherwise it will compile it into a runtime assertion.
[1] http://www.sbcl.org/manual/#Declarations-as-Assertions
This isn't really true. For example, in Java it's Map<K,Object>. For languages without subtyping, existential types are often used. It's quite common to see in infrastructural code, e.g. caching.
When I'm writing code, and a variable or signature is easy to annotate, I'll annotate it. And guess what? MyPy occasionally catches issues with my types and saves me some work. If a type is very complex, I won't bother spending time annotating. The end result is that I have some code that is annotated and prevents simple errors, and other code that isn't, and I didn't spend much time or effort doing it. To me, that's a clear net win.
I think mypy has to let some potential errors through because otherwise it would spew 1000's of spurious error messages on perfectly good, unannotated legacy code. Something similar happens with the Erlang dialyzer. The same thing applies to tools like Coverity, that aim to find potential bugs in legacy C code.
You could instead imagine a version of mypy that makes no concessions at all to legacy code, and insists that your code be 100% free of type errors, even if that means that some older constructs and styles no longer work. Would that be a good thing? Probably not: we have Haskell for that. Mypy's leaking errors is probably a practical necessity in retrospect. But, I wasn't expecting the leakage, so it surprised and disappointed me when I encountered it. I had thought I was getting something more like Haskell. Mypy still has attractions, but it's less great than I had hoped.
Type hints (with mypy) aren't going to save you from becomming a bad programmer. But they _do_ help you become a better one!
I must be mistaken, but I always thought "type hinting" was a synonym of "optional type annotations". But if you're not using those annotations for actual type checking at some point, what are they good for?
(I am often reminded of my perl days, where something I thought idiomatic 3 days ago, is now completely incomprehensible when I just want to make a minor change)
It's especially bad because, like any comment, it can say one thing but the code may do something different (actually happened to me).
In my experience, it just doesn't work cleanly out of the box like in true statically typed languages, and so I get pushback from my coworkers, who simply can't see the point. And I can't blame them.
If you/your team want to use a statically typed language, then use one. Python is not it.
Usually they can be sold on the path of maximum-reward-for-minimum-effort. Hard to enforce/check type hints are not it (they seem like busywork for no actual payoff). With statically typed languages, the ROI would be different: they would either get the thing to compile, or they wouldn't and leave the job.
This sounds drastic but it's really not: outside the realm of purist conversations between fans of programming languages, people just want to do their job and get it over with. If the tooling is convoluted, has too many rules, or they aren't forced to use it ("...or else"), they just won't.
This is a javascript/python shop, by the way. We cannot choose the languages. We can improve the tooling and train the team with better practices though.
> If you/your team want to use a statically typed language, then use one. Python is not it.
This is a bit of a cop out though, isn't it? A response to criticism of a (possibly) flawed language feature cannot really be "well, use another language". How else will the language improve then?
> it just doesn't work cleanly out of the box like in true statically typed languages
So the response is entirely appropriate. Python is not a statically typed language, of course type hints don't make it behave exactly like a statically typed language.
I meant to say this: because Python's type hints don't work as well as true statically typed languages (based on my own experience), this makes them less useful and also makes them feel more like busywork. Because of this, my coworkers (who have no experience with statically typed languages either, and tend to dislike best practices and features unless they see a clear return of value from them) are hard to convince type hints are worth the time to learn and use them.
Also, let me restate we cannot pick the language. We can just make better or worse use of it.
That question hides the assumption that static languages are always an improvement over dynamic ones, when in reality we should think as different species that have adapted to fit into different environments.
> We can improve the tooling and train the team with better practices though.
"Better practices" are what makes you team more productive, not just blindly copying what other people are doing. If your colleagues really believe that automated tests/type checking are not worth the effort, you are not going to convince them by saying "but so-and-so said otherwise". What you can do is ask for their pain points, and see if the tooling can help with it. You can look at your past burn charts and say "look at this bug here, what caused and why did it take so long to fix? Is this the type of problem that would be easier to solve with static analysis of the code? Look at this refactor that we are planning for next quarter, should we try to increase the test coverage to make sure we have more confidence in the changes?"
Let me give you a twofold answer to this.
1. I do think statically typed languages are generally better than dynamic languages for most use cases (with a few exceptions). I don't expect to convince you or anyone else of this. It may even be the case that I'm mistaken about this, but I made my mind after years of experience with both kinds of languages.
2. The alleged hidden assumption is not truly important. One should always criticize flawed features of any language, static or dynamic, and the answer should never be "well, choose another language". How else will languages improve if nobody is working on addressing their pain points?
> "Better practices" are what makes you team more productive, not just blindly copying what other people are doing. If your colleagues really believe that automated tests/type checking are not worth the effort, you are not going to convince them by saying "but so-and-so said otherwise". What you can do is ask for their pain points, and see if the tooling can help with it.
I'm both nodding in agreement and finding it very hard to think of something I said that made you think I disagreed with this.
Shouldn't then the exercise be to figure out if these use cases are applicable to your situation?
What is your team and company optimizing for?
> How else will languages improve if nobody is working on addressing their pain points?
"See, at my work we need to travel between two islands as fast and cheap as possible. My team is used to high-speed motorboats, but my experience tells me that hydroplanes are faster. I tried putting wings on my motorboats, but it still wasn't as fast. No, we can not buy hydroplanes. No, my team is not licensed to fly. But if no one acknowledges that motorboats are slower, how will they ever be as fast as a hydroplane?
And no, I haven't really looked into the fuel costs of planes vs boats..."
Also, do you honestly expect me to detail what my team and company is optimizing for here? What for? What does it have to do with Python type hinting?
Anyone that has seen the py2 -> py3 debacle will tell you that "let's make python static" is not something that could happen unless you are willing to rewrite the entire ecosystem of libraries and applications, and quite possibly upsetting the majority of current users who were attracted to it in the first place precisely because it is so easy to get started with it.
You can not turn Python into a "fully static" language without changing it so much to the point of making into a different beast. And why should others make all this work to fit into your view of what is "best" when you can just use another language in the first place?
I also cannot choose the language. I'm not in a position to choose languages at my current job; I seldom find myself in that position at any job.
But given that your response to the mypy recommendation was to complain (it's slow, it doesn't catch everything) then it makes it difficult to acknowledge the "valid criticism"... as in: people are working on it, what else do you want?
[0] https://mypy.readthedocs.io/en/stable/existing_code.html
What makes you think I'm not using mypy, or that I didn't read the article you linked to or follow its guidelines (which are very sensible)?
In your mind, is it the only possibility when someone criticizes a tool that they are "using it wrong" (to paraphrase Steve Jobs)? No other possible reason?
> "But given that your response to the mypy recommendation was to complain (it's slow, it doesn't catch everything) then it makes it difficult to acknowledge the "valid criticism"... as in: people are working on it, what else do you want?"
Given that I didn't say or imply that I don't believe there are people working on improving mypy and Python type hints, "what I want" is merely to support the article's assertion that the current version of type hints Python provides is confusing and not very good.
Please, for the love of all that is holy, don't recommend to me again that I switch to a different language. That's not how this works.
What I am having trouble to grasp is: if you understand that Python is not a statically typed language, and if you claim that you do not want to "turn" Python into such, why do you keep conflating type hinting with type checking?
They are two separate things. That is the nature of the game. Do you have criticisms about limitations from mypy, fine, but then you are not talking about issues with type hinting but merely the type checker tool.
It's not my business to help you with your trouble grasping things, but why do you think I'm conflating type hinting with type checking? I specifically asked in this comments section what type hinting meant if not what I thought, and was told by several people "it's just standardized comments", which I then explained was unsatisfying (and other people agreed with me).
> "you are not talking about issues with type hinting but merely the type checker tool"
How about both?
Let me suggest you a more productive use of your time: address the article's complaints as a top-level comment (which I see you haven't yet). I suppose you're more interested in the article's topic than in correcting my alleged misconceptions?
Because the "issues" with type hinting that you are talking about are only based on what you wished it was, not on what it is or what it could become!
Ask yourself this: what about the python's type hinting story you think could be improved without turning python into a statically typed language? What about the python's type checking story that is missing or broken and that is attributable to the language and not to the tool?
> I suppose you're more interested in the article's topic.
Quite frankly, no. My motivation for the conversation now is mostly to see how long is going to take you to realize that you are merely wishing that your motorboat could fly like a hydroplane. It's not about "misconceptions", it's about you complaining about something not meeting your unfounded expectations .
Oh, I see. Well, I have no interest in that kind of conversation.
Goodbye.
What I am trying to say is that perhaps the lesson you could take from this it's to not think that something is "inferior" or "needs fixing" just because it doesn't immediately meet your expectations.
It isn't Python's fault that you have to use at the job even though you don't like it.
It isn't Python's fault that you have to use even though you are absolutely certain that dynamically typed languages are worse.
It isn't Python's fault that type hints are just hints and can not become a full-fledged type system.
It isn't Python's fault that mypy is still not mature enough to work reliably for you.
And it certainly isn't Python's fault that your coworkers don't care about all that as much as you do.
I think there's a lesson here, but it's for you rather than me.
This is tiring. You keep making stuff up for the sake of argument. I never claimed I didn't like Python; in fact I like it. I never claimed it was Python's fault.
I thought you were going to "wait until I realize". Go on waiting, then. Off you go!
Anyway, you also had the chance to respond to other questions I made, but instead you preferred to stick to defensive retorts.
I'm not interested in being "educated". In general, you sound very condescending regardless of your claims that you are not snarky.
It's easier to pick on someone's comments than to make a comment of your own, because that would open you to correction and criticism. I think that's what's going on with you in this case.
Regardless, I suggest you stick to commenting about TFA in the future. It will go better for you. You'll be criticized, but that's par for the course.
> I'm not interested in being "educated" by knowitalls.
From where I am standing, you are trying to get your coworkers to adopt some practice without a strong case for how it would help them or the bottomline, purely out of your belief that statically languages are better. If you think I am being the know-it-all here, fine.
> It's easier to pick on someone's comments than to make a comment of your own, because that would open you to correction and criticism.
Someone else in the thread already pointed out that your original comment and TFA are basically a statement of opinion, not of fact. I do not see what kind of comment we can have except the meta-commentary.
Yes, I can see that you fail to acknowledge the unreasonability of your prodding. Time to change your tack, maybe voice an opinion of your own, on the actual article, and expose yourself to critique?
Maybe drop the condescending tone and try to avoid the cross-examination? It's against the rules on HN.
> I do not see what kind of comment we can have except the meta-commentary.
I can't help you with what you "don't see"; I cannot make you fix your blind spots. I've already told you your meta-commentary is unwanted. I do not welcome it, it's unhelpful, and every single assumption you've made has been wrong (and smug) so far. It's not my job to present my case to you, you're not doing a consultancy here, nor do you demonstrate particular expertise on the subject.
I foresee another pointless reply of your own. I'll let you have the last word.
Bye.
In regards to "type hinting/type checking" in Python, my "opinion" is somewhat similar to automated testing: "use it when it can help increase your confidence about the soundness of your program and its design, but don't rely on it as a guarantee of it being bug-free."
In regards to the article, my opinion is that if you want to claim "disappointment" with a language your expectations should be aligned with the language developers and the general motivations of the community at large. Another opinion is that if the author took the time to understand these underlying motivations there would be no "disappointment" and consequently no article, perhaps?
> expose yourself to critique
Hum, that's interesting. I don't think of "expressing an opinion" as something that "exposes oneself to critique". Maybe I am too accustomed with the idea of "keeping my identity small" and "strong opinions, loosely held", that it's natural to me to differentiate an "attack" on the argument vs the person?
Any chance is this why you are so defensive?
> try to avoid the cross-examination? It's against the rules on HN.
First, not rules but guidelines. Second, you have been responding with passive-aggressive insults and retorts which also makes for unpleasant conversation. Third and most important, there is no cross-examination. I am not challenging your motives or trying to invalidate you as a way to invalidate your opinion. The "prodding" is because I don't think your argument has any merit yet, but perhaps this could change if you provided some underlying reason?
> I've already told you your meta-commentary is unwanted.
It's not lost on me that I when I asked "what you would like to change about python's type hinting without changing the language, and what is wrong about python's type checking that is not just a matter of improving the tool", you completely ignored it and preferred to continue with this grating back-and-forth.
Anyway, you are right when you say it's time to lay this one to rest. I hope the next one gets to be more enlightening to both of us.
You continue nitpicking! You would do better if you followed those "guidelines". Indeed this conversation has been unpleasant, but it's all on you. You have been condescending and insulting. I suggest you focus your energies on better enterprises next time, maybe engage with the article itself instead of picking on others. It will do you good.
> The "prodding" is because I don't think your argument has any merit yet, but perhaps this could change if you provided some underlying reason?
Surely you can see there's nothing to gain by explaining myself to someone who thinks "my argument doesn't have any merit" (and who doesn't think that's insulting) and who keeps misrepresenting everything I say? I just said something, shared by many others, and you felt triggered by the opinion: that's fine, I don't want to convince you of anything, nor do I feel like being "educated" by you.
> Maybe I am too accustomed with the idea of "keeping my identity small" and "strong opinions, loosely held", that it's natural to me to differentiate an "attack" on the argument vs the person?
Well, you haven't done a good job at it, then!
What do you hope to gain, at this point? Are you waiting for me to "see reason"? That's not going to happen. Surely you see sooner or later dang will intervene here and tell us to shut it down?
re: insulting, passive-aggressive behavior: I re-read our exchange from the start. I was polite, even agreed with you sometimes ("I'm nodding in agreement, why do you think I think otherwise") while your tone kept getting more condescending with each reply, failing to accept my agreements and prodding in what I guess you felt was an "educational" tone. Honestly, hand on your heart, do you believe your responses were made in the best possible tone and were the best way to conduct an honest conversation? Do you believe the default position of "this argument has no merit, but maybe if this person explains his team's and company's motivations to my satisfaction I might change my mind" is a mindset that is conducive to amicable conversation? I think, if you're honest, you should admit maybe a tiny bit of misbehavior here.
I also read replies you've made in other articles and you seem more reasonable. I'll chalk it up to you not being able to let go at this point, and so you must bite and nitpick at everything I say. I really think you should let it go, make your peace with not being able to convince me of your point of view, and move on to other enterprises.
> someone who thinks "my argument doesn't have any merit" (and who doesn't think that's insulting)
You forgot the yet. No, it's not nitpicking. Without it, it would be a mere judgmental sentence and it would make sense if you felt dismissed. With it, it is a sign that I am trying to reach for a point of agreement or an understanding.
You are claiming I am the one "biting and nitpicking on everything", but you also could've given a much more charitable interpretation to this, much like you could have done it for the last 6-7 exchanges we had.
And sorry, but I really don't see what is "insulting" about it. Well, at least I don't see how this is insulting if we are to have a rational conversation. This would be insulting only for those who'd be tying their self-image to any of the things being discussed.
> Honestly, hand on your heart, do you believe your responses were made in the best possible tone and were the best way to conduct an honest conversation?
Yes...?
Maybe I am relying too much on the Socratic method, but my questions were mostly to dig in for Truth, not to grieve you. But given the way that old man died, perhaps I should try a different path to not end on the same way as he did.
Oh. My. God.
A Socrates complex.
I give up.
I guess you were right all along, this conversation should've ended two days ago. You win the xkcd 386 today. Good night and good luck.
So, unlike a comment, which does nothing, if you run mypy and get some warnings, you can then do something about it (whether it's just # type: ignore or you realise you made a mistake with allocating to variable new_value versus newvalue/new_valu/etc)
(btw, in case someone objects, some comments DO do something, e.g. golang's godoc examples)
If they are for checking, then my other opinion stands: that they are not very good at it (compared to my experience with statically typed languages, even with Typescript!).
I've tried mypy and pytype, and found both to be unsatisfying and hard to sell to the team (which as I mentioned, don't even like writing tests). If you're thinking "well, that's a culture/professionalism rather than a technical problem", you are absolutely right! But I'm left wondering if a language that was better at static typing would be better because, a- the checking would be mandatory, and b- it would be demonstrably more effective than Python's "optional" checking/hinting. At least it'd remove one variable from the problem. Or maybe not, who knows.
If your team doesn't understand the value of things that are essentially standardized comments exposed through the IDE (even if it wasn't for validation, which is also available with IDE integration), you have very bad teammates.
In my opinion, of course.
I should say that anecdotally I find type hinting very useful when I'm reviewing a PR from a part of the code I'm not intensely familiar with.
Useful, but way less useful than if it was actually checked.
i agree with the author, if the hints can't be trusted, even just once, there's no point littering the code with them, and are actually harmful when trying to debug something using faulty hints.
i am a big fan of static typing but not in python. pick a language that was designed around it. you can't make a duck bark.
I wish this was the case with command line tools I can plug into the build/deploy pipeline. I know they exist, it's just they are unsatisfying and there are lots of cases where they miss stuff that is trivial to catch in statically typed languages.
It's simply not true. Every statically typed language works like I described. Even Typescript works like this. You can have your IDE plugins, but you definitely don't need them to perform type checks. It's not true this requires "a nightmare of 30 command line tools", you just need one: the type checker (built into the language in most statically typed languages, but sometimes split into a separate tool).
Besides, your IDE doesn't live in the build pipeline (in your CI/CD tool). So you cannot rely on it.
A type hint would have answered their question immediately (as they were reading the code).
An unenforced type hint is just like any comment: likely to get out of sync with the code, and a human must do all the work of keeping it up to date.
Python typing is like that. If I say a function takes a List[int], but you know I'm just calling a for loop, you can ignore it and hand me a Set[int]. Maybe it breaks some day, but you're allowed to take that risk, if you have a need.
Do either of these things do anything comments couldn't? Not really. But they're ways of indicating intended semantics without formally documenting your stuff, and when most of what you use the language for is scripts, that's actually pretty helpful. People actually use type hints in a way they didn't with comments, and that's caused a major improvement in code readability. Or at least that's been my experience
You can absolutely look at all this and say python's a crazy, terrible language you never want to touch, but if you've got no choice on language for whatever reason, or you're throwing something together and don't feel like writing `private static final synchronized` forty time, type hints are great.
> I view them as similar to python's "private" functions, which are really just functions starting with an underscore
Good analogy! I wish they were more like "private" class methods, which are prefixed by a double underscore and result in name-mangling: you can still access them from outside if you want, but the code will really look ugly. And you absolutely cannot access them accidentally.
Which is what type checking should be all about, right? Preventing accidental misuse?
> Python typing is like that. If I say a function takes a List[int], but you know I'm just calling a for loop, you can ignore it and hand me a Set[int]. Maybe it breaks some day, but you're allowed to take that risk, if you have a need.
That's what drives me crazy. I come from the statically typed world. This bit of Python's philosophy really clashes with my world view. "These types are just something someone wrote, they may or may not accurately describe the code" seems so wasteful and unhelpful to me...
That’s the kind of the culture shock you get learning dynamic type languages with a static type pov. Type hinting is not really the problem here, but can seem that way because it makes dynamically typed code too superficially similar that it enters the uncanny valley if you treat it like statically typed.
One way that might make this easier is to forget about type hinting entirely at first and learn the language “from scratch”, and add the types back after you get used to write dynamically typed code. Dynamically typed code have its advantage and isn’t bad without type hints (well, at least you have to convince yourself on this, or you’ll never be able to learn a dynamic type language), and the type hints just add back some of the nice things static type provides without compromising dynamic type benefits.
In addition, it makes it harder for me to argue in favor of type annotations with my coworkers. My coworkers come from neither world, static or dynamic; they are learning the ropes. And they just can't see the point. I'm confident I would convince them were this a statically typed language, but with Python I'm lost.
At some point everything in proglang is preferences and design decisions, but for me, there are better and worse tradeoffs to make.
So for that List[int] example, you probably want to take a(n) Iterable[int], Iterator[int], or Collection[int] instead, depending on exactly how you use it.
We should think about whether there's a reason why we want to be writing `private static final synchronized` over and over again, after looking at the state of the software engineer these days.
You seem to be describing actual type checking, which I understand (though in my opinion, mypy is not a satisfying tool for this).
https://pypi.org/project/typeguard/
> Despite all that, what I’m hoping for is that someone will come along and tell me that I got it all wrong. “When you do it as follows, the system works: $explanation” Because, you know, I’d really like to have static typing in Python.
I want X. Y (which is designed for somthing other than X) doesn't do X. Thus, I don't like Y.
> Even if the Python runtime did check all the type hints at runtime, then it would still be too late. I don’t want a fancy type exception at runtime. That already exists (most of the time). I want to know about type mismatches in advance.
You should check out Typeguard [1], which lets you add a @type_checked decorator to anything and get runtime type checking that's far better than e.g. a random blowup when you split() on an integer. It does introduce a significant amount of overhead, but it's still useful during development alongside type hints to avoid some of the pitfalls from this article.
In another vein, Any should definitely be used sparingly. For the foo() function outlined in the article, Union[str, int] offers more precision. In fact, you could even argue that using Any is a code smell.
Overall, though, I agree with most of the sentiment in this article. I find myself using type hints more as documentation than anything else, since their true ability to prevent type-related runtime bugs is very limited. Documentation has the same qualms the author outlines, anyway:
> You can not be sure that they are correct. [...] They waste mental energy when reading code, they create new maintenance burdens, and they are potentially deceiving: You cannot trust them.
But they're still slightly better than docstrings...
[1]: https://typeguard.readthedocs.io/en/latest/userguide.html
What I'm looking for is more like a debugger or profiler. It actually runs the program, and then reports if at any time the type annotations deviate from the actual types. (I guess the general case is too costly - think of typechecking a huge list - but for most cases it should be possible.) There are runtime type checkers for python, but the all require modifying the code, and they don't work properly on the main module.
I would also strongly recommend using Pyright (the default checker in VSCode) over MyPy. It's so much better, and the author is really responsive on Github.
But yeah, in general trying to use type hints in Python is like trying to discuss philosophy in a mental asylum.
The issues with python code bases aren't from a lack of strict typing or type support, but from a lack of a good testing framework.
Python is popular because it's very beginner oriented, so it's the first language a lot of people learn. It's modern BASIC.
Neither of those things mean that writing dynamically typed code is a good idea. The idea that you can write dynamically typed code faster isn't even true once you get past a couple of hundred lines.
And Python being popular because its basic/beginner oriented is a laughable statement. http://highscalability.com/blog/2012/3/26/7-years-of-youtube.... Not to mention that all the bleeding edge ML stuff is Python first.
Language adoption directly depends on how quickly people can build stuff in that language, and its an exponential effect, because with speed of development comes more libraries, which in turn allows other people to build stuff that uses those libraries quicker. And the initial speed of development directly depends on the programmer having to manage less things. Static typing, in real world, is often a hinderance because code bases are dynamic, and having to go and refactor code because data definition changed takes time. And if you have competent programmers that write clean code and a good code review and testing framework, static typing gives you no advantage since the time spent fixing issues due to failing tests will be equivalent to spending time fixing issues with failure to compile.
Ah the classic "I don't make mistakes" delusion.
> But as a general rule: You don’t know if there really is a tool in place to check them.
Cool. So you just go and do stupid stuff if no one is looking?
Check it yourself. They're probably there for a reason. If you think they're not needed, then sure, take them out.
> Most of this turned out to be wrong and it threw me off the track during my debugging session.
Here's an idea: Then why don't you fix it. If I see code that's wrong or bad, I go and fix it. It's your responsibility as well
Now maybe try acting like they're anything but type hints (and I'm very glad Guido insisted it is optional and flexible - the last thing the world needed is for Python to get consumed by Java type checking pedantry where the compiler needs you to annotate every single thing)
And every time I do something in Python that would be "impossible" in Java I'm glad I'm not bound by the small-minded type pedantry of Java.
Defining such style rules and validations is necessary with almost every language. A codebase where developers are allowed to use any C++ feature or browser API will become very messy as it grows.
* fork the language again? We just got out of Python2 hell.
* Docstrings, like we had before they were introduced?
They won’t catch all bugs, but with minimal effort they’ll catch some bugs, and if you make an effort to really dig in, it’ll catch most bugs. To me, to accomplish that overnight on a language with millions of existing lines, that’s a big win.
In what way do they lack more teeth than Java with Object, Go with interface{}, C++/C with void*?
Since types are optional in Python, it's necessarily true that an "Any"-like type is prioritized since it's the default. Moreover, cpython doesn't check anything when it bytecode-compiles.
Any-by-default and no-built-in-checking feels pretty defanged to me.
mypy is pretty whiny about types, and sure, if you add Any everywhere then you have just defeated the type checker, but this is no different from putting interface{} everywhere for Go.
> Any-by-default and no-built-in-checking feels pretty defanged to me.
Is running mypy on the code base that much to ask?
Python type hinting is like moving backwards in time because the amount of time devs take to "hint", takes away the main reason for using python, that is faster development. Might as well code in Java at this point.
a) what types a function/class accepts as arguments and b) what types are returned.
If you don't check it then I agree it's useless. However, working with libraries that are typed is an _insane_ productivity boost over working with untyped libraries.
Typing encourages a large number of bad coding practices. If you can apply typing to a Python program then you are not taking advantage of the dynamic nature of the language to write simpler and shorter code.
In Python world, unit tests cover what is normally handled by typing.
> Typing encourages a large number of bad coding practices.
I'm sorry, what? What exactly is the bad practice here? Not passing random dicts to my coworkers only to surprise them because the key was userid and not user_id? Or wasting time bugging an overworked dev to ask what their crazy 8 levels of dynamic indirection magic are doing, vs just being able to Cmd-B and step through the code path?
> If you can apply typing to a Python program then you are not taking advantage of the dynamic nature of the language to write simpler and shorter code.
Yeah, no, flat out wrong. You can use typing with dynamicism to do magical things. FastApi works a near miracle, generating serializers, deserializers, openapi docs, automatic api deprecation warnings, not to mention type safety.
Properly typed python is faster to write and safely magical.
There are shorter more concise ways to write your programs if you are not using static typing.
If you are not writing your Python programs in the Pythonic way then you are not really using Python correctly and you do not get most of the benefits of the language.
Unit tests exist for catching bugs in Python. Type hints are for documentation not for ensuring code correctness.
Yes, there are libraries that take advantage of type hints, through they could of used other annonations and it's valid to use type hints along with those libraries.
But as soon as you are attempting to prove your program doesn't have bugs via typing rather than unit testing. You have gone wrong.
Typed Python is far more bug prone, why? Because structuring your Python code to support typing means writing considerably more lines of code.
It not the type hints themselves but it's the way you will structure your program at a high level to support them.
The more lines of code the more bugs it has. Bugs are well known to be proportion to the number of lines of source code regardless of language used or whether the language is typed or not.
That does not bear out in practice. In my experience, bugs have almost nothing to do with LoC. A short bit of complex, I/O heavy code with lots of state is going to be way more bug prone, and far harder to unit test correctly, than 10x the LoC of purely functional, well-typed, stateless code.
Mutation, parallelism, IO, and complex state space are the four horsemen of the bugpocalypse.
But in fact, I find functional, well-typed, properly abstracted code ends up with less LoC in the codebase, because it's more reusable and composes well.
SessionInfoType = Union[AuthzQuerySessionInfo, AuthnReqSessionInfo]
Notice how it is doing a union of two objects. Do you know what those 2 objects are? Those are types as well. They have no business use case.
AuthzQuerySessionInfo = TypedDict( "AuthzQuerySessionInfo", {"name_id": Any, "came_from": Any, "issuer": Any, "not_on_or_after": Any, "authz_decision_info": Any,}, total=False, )
AuthnReqSessionInfo = TypedDict( "AuthnReqSessionInfo", { "ava": Union[dict, None], "name_id": Any, "came_from": Any, "issuer": Text, "not_on_or_after": Union[int, bool], "authn_info": list, "session_index": Any, }, total=False, )
RequestCookieType = TypedDict( "RequestCookieType", { "request_id": Text, "came_from": Optional[Text], "public_id": Text, "is_mobile": Union[Literal[True], Text], "c_challenge": Text, "c_challenge_method": Text, }, total=False, )
So now. not only do we have dicts that represent above data, we also have TypedDicts created specially for types.
To rub salt on the wound, the callsite with these types now looks like
session_info = authresp.session_info() session_info = cast(SessionInfoType, session_info)
WTF!! If we have to see a codebase littered with TypedDicts just to satisfy type annotations and then have to cast everything to that type - why not just wrap it in a class and call it a day?
When I see garbage like this, I seriously question the productivity improvement brought about by python type annotations. I certainly end up spending an inordinate amount of time unraveling nested unions of types all over now. Considering switching to a team that uses Golang.
I don't know about that. So many C/C++ programs have core dumped in my life and needed to be valgrinded to debug memory issues, or went past the bounds of a pointer, or overwrote the stack, or... whatever.
And you still don't get correctness without unit tests.
I'm not saying types aren't valuable, but they're not a crutch to program correctness like people seem to think they are.
> warnings of IDEs are simple to ignore
This is unusual. In my experience, of codebases I have worked with or have seen, when there are type hints, there are almost all perfectly correct.
Also, you can setup the CI to check also for IDE warnings. For example, we use this script for PyCharm: https://github.com/rwth-i6/returnn/blob/master/tests/pycharm...
The test for PyCharm inspections only passes when there are no warnings.
Although, I have to admit, we explicitly exclude type warnings because here we have a couple of false positives. So in this respect, it actually agrees with the article.
But then we also do code review and there we are strict about having it all correct.
Yes, I see the argument of the article that the typing in Python is not perfect and you can easily fool it if you want, so you cannot 100% trust the types. But given good standard practice, it will only rarely happen that the type is not as expected and typing helps a lot. And IDE type warnings, or mypy checks still are useful tools and catch bugs for you, just not maybe 100% of all typing bugs but still maybe 80% of them or so.
> Isn’t it better to detect at least some errors than to detect none at all?
> You can not be sure that they are correct. As such, you must always treat them as if they were wrong.
I don't get this argument. Isn't this the case for all other code as well? Most code has not been formally verified to work 100% correct. So you assume always all code is wrong? This doesn't make sense.
> They [type hints] waste mental energy when reading code
How? The author even acknowledges that you could just treat them as code comments if you like. By that argument, all code comments waste mental energy?
It isn't that you have to assume the code is wrong (you always have to assume that code is wrong). It's that even though you write type hints and they pass mypy, you still have to assume that the code has type errors. That is, mypy doesn't rigorously check that the program's types match its annotations. It only approximately checks that.
In (some) other languages, the type checker is 100% sound, so if your program passes type checking, you can be completely sure that the types all match. Opportunities for the code to be wrong are thus limited to mismatches between the types and the desired behaviour.