I’m not sure I understand the “infrastracture” label, but certainly for the cases the author listed as “infra” typing can be cumbersome. Famously just json deserialization can be rather painful in statically typed languages, especially ones without complex compile time features.
Serialization/deserialization can be really gross in something like Java, but TypeScript gives you some really good tools (zod, typebox, etc.) to validate and type-guard your inputs that I'm sure could be implemented in Python--if they're not already.
Yeah, I assumed as much. I've only looked at Pydantic in the context of FastAPI, as Python's not really my thing, but it looks really well-thought-out.
I've written a lot of code using Jackson and I strongly disagree that it's a solved problem. If you're Java on both ends, it's probably fine, but then your serde style is an implementation detail. And that's great, but if you're not, you are in a bear pit. The lingua franca for JSON specification is JSON Schema, and there is significant impedance mismatch between the standard JSON tools folks use in Java and with JSON Schema as used by most other folks. (Jackson has a nice JSON Schema module for exporting it, but importing/codegen is pretty gross.)
I find Java pretty nice for handling JSON serialization/deserialization these days. Just define some records for the expected structure, and Jackson's object mapper handles the rest.
It's the "defining some records" part that's slow and gritty. JSON is usually an interchange format, and Java class construction is either slow, tedious, and manual, or involves massaging a code generator (which, at least the last time I checked a few months ago, weren't very good in Java).
You're assuming you only have to define it in one language. JSON is an interchange format. The canonical way to express it is JSON Schema, and the tools are unpleasant in Java. So is dealing with structural typing of polymorphic responses. In an ideal world we wouldn't have those, but we live in this one.
These things are just better when you aren't in a nominatively typed language with some pretty crusty assumptions.
Hum... Almost all of the problems of serialization/deserialization in statically typed languages turn into testing problems in dynamically typed languages.
In this case it’s more about shifting where the static typing occurs, since the author is still arguing for a typed business layer.
Since I/O is usually untyped, you need to cram it into a typed schema somewhere. In strictly typed languages, that must occur upon ingestion. The author is saying you can delay it by having a layer on top of the I/O transform the data in untyped fashion before being typed later on.
What's the benefit of delaying typing? By typing it at ingestion, you guarantee that invalid (with respect to structure) untrusted data will not break your code down the line. Doing otherwise means that you're manipulating untrusted data with a strong risk of forgetting that it is untrusted.
I have seen large applications (think hundreds of millions of daily users) break because of such oopses.
> Famously just json deserialization can be rather painful in statically typed languages, especially ones without complex compile time features.
That's funny, because last time I had to write json-related code in Python, I really missed Rust's serde, which makes deserialization + validation much easier to write in Rust than in Python.
I think from the perspective that "infrastructure" code being that which rummages around in objects, where the only distinction that matters being dict vs primitive.
Yup! For me it's precisely the opposite of that, the deeper in the systems code, the more types I write. Same with JS, the bulk of libraries IMO should be written in TypeScript.
Not the writer, but based on what the author wrote, I think it's based on the author's experience with Java. It can be a hideous nightmare of java.util.Date versus java.sql.Date versus java.time.LocalDate (just to pick one very well known fiasco in the language- Java defenders please don't come after me) and all of them have to exist to avoid breaking things but you have to have different ways to handle each of them, so dealing at the boundaries between business logic and "infrastructure" is always a mess of converting types- oh this library is old enough that it needs a utilDate, so I need to convert it back down, then convert it back up at every boundary.
By contrast, duck-typing means that the intricacies of which exact type you are deserializing into is a lot less important in python.
Duck Typing always struck me as utterly absurd wishful thinking. Two different types built for the same task will almost certainly not happen to be API compatible.
I have bounced back and forth between typed and untyped languages in my two decades of employment history, and I prefer typed languages myself, even when I've been writing libraries nested deep in what I think the author would describe as the "infrastructure layer," but this is what I believe the original author of the linked piece is arguing.
Yes, interfaces aren't just bags of data and methods. They have semantic meaning, too. An Employee, a Gun, and a ClayPot might all have a method named `fire()`, but they all mean different things. With structural typing, we can load our Employees into a Kiln and give HR a Gun to downsize the company, and nothing will stop us.
There are cases where it can be useful. One thing that comes to mind is numerical code, where your algorithms are generic over things that have *,/,+,-,==,!=, conversion from 0 or 1 [or some subset thereof] do the "right thing". Trying to define proper interface types for what constitutes the "right thing" is somewhat tedious without much relative gain.
Its worked for us when dealing with local operations vs. cloud bucket operations. Protocols also help when you really don’t care to litter your code with some vendor specific object but instead just declare what functionality should be expected from the object passed in. It helps describe what you’re doing with it anyway, it’s why typing with Iterable can be helpful if you don’t care if it’s a list, set, or tuple.
Well, the article is the author's attempt to answer that question. I'm not sure that it's convincing to me, but then again I have a preference for strongly-typed languages so I'm probably not the correct audience. I'm just trying to help someone else understand the author's argument, as I understand it.
Not the OP but I explain it as former Java/C# programmers coming to Python for one reason or another.
I’ve come to Python 20 years ago after a short stint with PHP and I don’t really care if the incoming stuff I have to handle is a list or a tuple (let’s say) as long as it quacks and walks like the proverbial duck.
More generally, you could say Python the original language is the victim of its own success (the same goes for JavaScript in relation to TypeSctipt).
Might not be the best example since Python made quite a mess of dates and date times (in fairness, like most languages), and duck typing didn't help at all.
No. The people make the mess. For persistence, only UTC without timezone makes sense. For UI, with few exceptions, only local time without time zone in ISO 8601 makes sense.
The only not so easy decision is: is it better to store as Unix epoch or as ISO string. That is a tradeoff between compactness and easy searching of date/time ranges.
I think it would have been better to say "internal library code for foundational projects." That code that takes advantage of all the freedom in Python to mess with types, generate classes on the fly, use type annotations at runtime, add and remove attributes dynamically. Code that is basically a notch away from `eval` (or sometimes literately for dataclasses). There's no way you'll ever be able to fully type this stuff outside of Any Any Any Any. But to be useful, the boundary between your library and it's user should ideally be fully typed so the type checker can help your users use the library correctly.
> Allow me to propose a model of thinking about code: there's infrastructure code and there's business logic code.
> Infrastructure code is exciting, powerful code that exposes easy-to-use interfaces that solve difficult and tricky problems, like talking to browsers (Flask), talking to databases (the Django ORM, SQLAlchemy), dependency injection frameworks (incant), serialization (cattrs) or defining classes (attrs, dataclasses).
> Business logic code is boring and unexciting code that enables you to solve problems and finish tickets and sprints at your day job.
What a weird take, unironically asserting that the only "exciting" code is the glue, and all of the actual things software does is just an excuse to build more frameworks.
If you only worked in companies producing web applications, writing CRUD routes all day is indeed incredibly boring. The glue at least gives you the opportunity to work with a larger range of technical subjects.
I found those definitions to be a weird line. But, I believe the overall premise is you use types for large projects that many people will be interfacing with and making small changes to. You go cowboy when it's some fun scripts.
This is not limited to type annotations. The same typically applies to comments, documentation, tests, etc. It's just one of the many code quality tools available to you (at a slight cost) , and one typically applies those tools where they have the most benefit: code that is often reused or reread.
I like to say something along the lines is “80% of all development should be really boring otherwise you didn’t do enough planning or you’re working on < 1 year old start up”. To me the real excitement isn’t from the code working it’s from a whole team making something that works because that is way more challenging.
The author wrote cattrs, which is “Complex custom class converters for attrs.” (attrs is a library that preceded dataclasses and is more powerful). Guess he just likes the stuff.
It rings true though doesn’t it? At least in my professional experience, I primarily have run into the “fun” technical problems that require theoretical knowledge when working on infrastructure. It makes sense, because lower on the stack means higher leverage and fewer abstractions from the hardware.
calculating that 38302 + 8204 = 46506 is pretty boring
noticing that ∀x,y: x + y = y + x is more exciting, and it's a lot more exciting when you can prove it
group theory? continuous braingasm
changing the report code so that an invitee who rsvps to an event even though the host has marked them as 'not coming' still shows up as 'coming'? that's pretty boring; it solves today's problem
writing a practical extraction and report language so that you need only a third as much code to define that report, but also all the future reports you write? that's exciting as fuck, it makes the rest of your life better
generality in software is highly desirable, just as in older branches of math; it's the difference between subsistence and progress
> I suppose you can look at Rust macros as a different, infrastructure language on top of Rust.
That's a strange take, not one I've heard anywhere else. Rust macros are code generators - they're run once at build time and generate regular Rust code that's strongly typed. Maybe it's because the author has a different interpretation of "infrastructure" than the commonly accepted one.
I can understand this take, Rust macros run at a different time, with different constraints, and are used to solve different problems. In many ways that may feel like a different language.
However, a more technically correct interpretation would likely divide based on hygienic vs unhygienic macros. The C Preprocessor is most definitely a different language to C – it has the same differences as Rust/Rust Macros do, but additionally is implemented by a different binary, is de-coupled from the C language to some extent, and most importantly, has different syntax and semantics making it actually a different _language_ in the strict definition of the word.
I think the author really means "library code" (i.e. code where the "user" is another developer). He also alludes to such code using metaprogramming which makes internal type hints hard to implement. In this context, macros can be seen analogously to meta programming, or "infrastructure code" to use the terminology of OP.
Just checked PEP3107 and PEP484 and type hints are not taken into account at runtime[1].
> While these annotations are available at runtime through the usual __annotations__ attribute, no type checking happens at runtime.
At first I thought it was disproving the commenter's "strange take" assertion but it actually proves him right. While it is true that python's type hints are a build time feature like in Rust, they do not generate code for the interpreter in the way Rust does.
Actually they do not generate anything besides metadata that can be interpreted (at build time) in any way undefined way. The undefined is explicit.
I personally liked the article and agree that eventually the undefined specification of the PEP will be removed and a new language a-la Typescript will emerge.
There's still usually only one way to do things in python, there's just the option to annotate it. Annotations are clearly deliniated in th code, and if you don't understand them you can safely ignore them.
The author's take that annotated code and unannotated code are different languages is a bit silly. It's like saying code with comments is a different language than code without. While it's true that different camps favor annotations differently, but the same is true for code documentation, and we don't treat it like different languages, even if the comments are in a language you don't speak.
Just an hour ago I had to downgrade my npm install to build a project from four years ago because some random npm dependency used the print statement from 2 instead of 3. So annoying, especially since I'll need to switch back over to run a more modern app.
Having print as a function is actually useful if you want to use it like any other function, e.g. `def whatever(x, y, z, , output=print):` then you could use the default or put in whatever that's like `print`.
I also don't get what you mean by "dumb byte strings", it's just that the default* string type, i.e. when you have a literal `"foo"` in source, it's a sequence of codepoints vs. a sequence of octets. Assuming file systems use UTF-8 was broken in very early 3.x (I think before 3.3?) but it's something I've never struggled with (pure ASCII filenames for me). Forcing the programmer to know if they're dealing with characters & code points vs. bytes/structs/network buffers is a good thing, IMHO.
Manage multiple npm versions at once using a node package manager manager such as:
Node Version Manager - POSIX-compliant bash script to manage multiple active node.js versions
https://github.com/nvm-sh/nvm
Volta - The Hassle-Free JavaScript Tool Manager
https://volta.sh/
asdf - Manage multiple runtime versions with a single CLI tool
https://asdf-vm.com/
Eventually we might need a node package manager manager manager to make sure we are using the right one for a project lol
Just two? I've gotten used to the fact that there are just enough breaking changes between the versions that each major release is different. Now if we add another axis that measures the number of type hints, that's twice as many languages. That's easily getting close to 20. Luckily there's plenty of overlap in the syntax.
This is an interesting argument that Python can be both flexible and strict. I think the author made a good argument here that this is more of a benefit to the language than a risk.
The problem I have currently with Python is that all these options are increasing the number of "pythonic" ways to do something, and making the language more complicated.
Python was exciting because it was quick to write, and elegantly simple. It didn't make exciting and risky choices on syntactic sugar -- it took a moderate approach.
But I get the impression that Python is trying to evolve into more and more use cases that it isn't well suited for. I think this runs the real risk of becoming too generalized -- doing many things, but not doing them well.
My concern about this is that typed Python doesn't seem nearly as much of a feat of engineering as Typescript is. It seems like a "simplistic" typing system as opposed to a fully thought out compile-time programming language (Rust probably also is a fully thought out one, but Rust doesn't have the untyped equivalent).
Good thing is that you don't have to do anything to combine them. If you use types, they will help you, if not, business as before. Same with library code, it's just nice to have annotations, so you don't need to look up documentation all the time.
I think that adding an optional type system while allows you to write python in two different ways or "languages" it also allows you to incrementally make your application be eventually transitioned.
You would first add types to things that are obvious as you implement features or fix bugs and leave the rest to Any and then add more complex types or even construct types for your application.
"Transitioning" an incredibly stupid goal, which unfortunately is becoming trendy among engineers who have nothing better to do.
Typed Python is not better or worse than untyped Python, it has different tradeoffs.
If you're setting a goal to "transition" your application, it means you don't understand what those tradeoffs are. You have a non-nuanced thinking is "types equals good" or "new equals good".
> then add more complex types or even construct types for your application
In other words, your intention is to add complexity to your application, while preserving it's functionality. Neither engineering nor business as a whole is going to benefit from additional complexity.
The "edge case" where complexity can be beneficial is where the ratio of library maintainers to library users differs by orders of magnitude. E.g. if you have a library with 5 maintainers and 1,000,000 users, then types can be a net positive.
Perhaps a different way of looking at this: there exists a boundary between first party code and 3rd party code, in that most developers do not actively maintain the 3rd party code they use, and interact with it via a documented API boundary. Types at this boundary are valuable as another form of documentation with automation of enforcement.
ELI5: The difficulty and resources involved in adding explicit types to the language aside, besides a little more typing, what added burden is placed on the developer? It seems to me that the improvements to performance, code correctness, and code readability farrrr outweigh the cost of the added keystrokes.
I have had times where I have had to spend a great deal of effort to make the type checker happy. Finding exactly where to import the correct type from the library you are interacting with can be quite annoying at points. And it can be painful making sure everything lines up perfectly so the code (which is already well tested and functionally correct) can make it through the type checker without having to just `# type: ignore` or use `Any`. I have had a few times where it took significantly longer to get the types working than the actual code.
I like typed Python, but it's not as simple as just adding types with a few keystrokes. It is a rabbit hole: if you're typing simple types, you will eventually want to type more complex types: functions, iterables, generators, use type variables, etc (Remember what the difference is between an iterable and an iterator? You'll need to.) And Python's type system is its own thing, with its own idiosyncracies. Furthermore, invoking the typechecker itself has complexities: are you doing it in an IDE, or as part of a pre-commit hook, or CI test? The default mypy options aren't great; there's one called `--check-untyped-defs` that IMO you should usually use. Anyway, I am not an expert and not currently using it professionally, but my point is it's much more complex than you're suggesting.
I've had lots of success using pyright [1] for Python projects, it has sensible defaults and can be configured with a pyproject.toml file so everyone's using the same settings. I use the Pylance VSCode extension to catch errors earlier, but I also put it in pre-commit and as a CI check, so all contributors are committing the same quality of typed code.
With more complex types, I've found it isn't necessary to do anything more complex than specifying the element type of a list or dict most of the time. I have typehinted lambda function arguments before, but just using the plain Callable typehint is usually enough. When I forget if I need to use an Iterator or an Iterable (which is every time), then I just try one and run the type checker and change it to the other if i guessed incorrectly.
Type checking Python can become complex if you expect to be able to express everything in the typehints, but Python's type system isn't powerful enough for that. I feel its strength is that it lets you add as much detail to the types as is convenient.
The line of reasoning here seems completely backwards. Type hinting should be used in libraries and other pieces of code where type ambiguity can lead to unexpected results. In my experience, that tends to be in "infrastructure" like pieces of code. I think the author confuses the fact that many "infrastructure" libraries are relatively old, and thereby existed _before_ type hinting existed with some kind of conscious choice by the developers.
And in my experience, for most of those cases there are additional type libraries that correspond with them and it just needs to be installed separately if you want to use the types.
Infrastructure by definition is supposed to be stable with well established interfaces, this is a perfect match for types.
Business logic is constantly evolving, focusing too much on types will cause you to be disconnected from business value and you'll just waste time fighting typing system.
Going in I was expecting a different take. For me type hints are great and just as important for the 'infrastructure' as they are for the 'business logic'. My default is using them everywhere possible now. So I don't really see a difference there.
I see a potential separation as more: Application vs Data Science. Someone purely writing Python inside Jupyter notebooks for data science purposes tends to be writing a very different flavor of Python than I do while writing a web application. As least in my experience there is less concern for formatting guides, idiomatic Python, PEPs, type hinting, or things along those lines. And that is totally fine, its not something they need to really be concerned within that context. It does result in very different outputs though.
> It does result in very different outputs though.
Not really. The thing with typing is that it gets hyped, but in practice, it matters very little. Both science code and web application code don't really need typing or really type hints.
If you go back to your CS professors and ask them to explain what is the advantage to types and OOP, the overarching reason would be that they create contracts in your code that reduce the risk of errors. In practice, if you look at well set up deployment workflow and pipelines, you still have unit tests/integration test that aim to have 96%-99% code coverage with sufficient test coverage, and people don't realize that it makes all the typing system moot. You can write equivalent code in any language and make it fully correct solely by ensuring that unit tests and integration tests for your usecase pass.
There is value however in extremely strong typing, where you can essentially do static proofs on the codebase to ensure correctness without running it.
Hypothetically, given developer tooling, which would you prefer to make sure your code is as correct and robust as possible?
1. Language with strong type features.
2. Testing packages that automatically include fuzz tests and null tests to ensure correct behavior?
Granted most of my experience in production level software involves backend services, which essentially means json transformations 95% of the time, Ive rarely ever seen an error that results in the service saying everything is ok but returning a wrong result. For example in the case of null fields, something downstream fails and errors out - this is easily caught by testing.
Well I would prefer to have both. I don't think we need to choose.
However, I think tests are high-maintenance. Often they consume the same amount of time to write as the actual business code.
I don't know how automatic null checking tests would work. Wouldn't you need to specify what happens if a null is input?
In contrast, I find the cost if typing very low for the value it delivers. Unless the code is very ephemeral (like a one off script) then I highly prefer typing.
It helps avoiding typos, but most are well indicated by the code highlighting of the code editor, even for Python.
It helps to use the right data type when 'connecting' two objects. But it constraints the usefulness of code that can rely on the behaviour of the objects instead of the type.
It clutters the code: you have to read (and write) way more text. Which makes errors more probable.
It forces to think about primitive data types, so steals time maybe better spend thinking about higher data structures (list, dictionary,...) or algorithm.
> Both science code and web application code don't really need typing or really type hints.
Where "science code" is "this python script I'm working on for my PhD and no one else is ever going to use it", sure. Where "web application code" is "I need to get an MVP out for Tech Crunch Disrupt", sure. Where neither of those things are true - if your grad student project code becomes the basis for sucessive grad students' projects, or gets popular in industry, or if the web application code underlies a company for a few decades - for sanity's sake you want type hinting. Tests only go so far, and if I have to run a piece of code's test cases before I know what the code actually does, instead of just reading it, something smells rotten.
I find this sort of optimism both cute and frustrating. The reality is that almost no company is willing to foot the bill for such a level of code coverage as it makes writing the application take easily 2x as long. Sure, it'll theoretically be more provably correct, but the cost difference isn't easy to swallow. The reality is more like you have good path coverage for a small subset of business critical code with the rest of the code getting tested at a more macro level with integration and functional testing. And that's not even touching on the theory vs reality of how much better the code will be with those unit tests. Writing _good_ tests is harder than most people think. Writing tests that have good path coverage and good input fuzzing is _really_ hard. Writing a test that checks the box for 'code coverage' can be done trivially in many cases, but provide no real value.
>The reality is that almost no company is willing to foot the bill for such a level of code coverage as it makes writing the application take easily 2x as long
This is only true if the codebase uses some highly inefficient setup that makes writing/running unit tests a pain.
And a general corporate policy for many companies, including big ones like Amazon, is to aim for >90% coverage with strongly typed languages like Java, regardless.
I did not mean output as in the actual result of running the code. I mean looking at two pieces of code that functionally may do the same thing but the code itself is written in a completely different style.
One of the other differences I have seen, but didn't specifically mention, is unit tests themselves. The web app probably has a ton of them. The Jupyter notebook probably does not. Which is fine, the language is being used for different purposes.
But an advantage of having those types in place is that you can do the static analysis to ensure everything is being used correctly within your codebase. And then you can have your unit tests concentrate on the actual usage rather than other stuff. There is no need to check for / test with a null value when you already know that function is never being passed a null value. And adding those hints has helped me many times with catching edge cases before they even happen where the existing unit tests were never going to hit that edge case.
“Everyone doing Python is aware…” No. Neither me, nor anyone I’ve been helping with Python has been aware of type hints.
You can do a lot of productive work with a language without knowing everything about it. It might be safe to say everyone working with Python is aware of indentation, though.
Sometimes the disconnect between what people in the field are doing and what bloggers say is really striking.
This is very surprising to me. Type hints have been a feature of Python since 2015. How are you, and everyone you know, unaware of such a large language feature? I'm guessing you (and the "people you've been helping") either don't use Python that much, or literally do not care one single bit about any of the new features that have happened in the past almost decade.
I'm sure such people exist. I'm just surprised to see them on Hacker News.
A surprising number of people have never been happy with Python 3 and even say it was a mistake. Whether you hear this attitude probably just depends on who you're around.
A huge portion of python programmers, probably more than any other language, see python (and programming in general) purely as a means to an end, not a craft to be learned. They will happily do the bare minimum of learning required in order to get working code that solves their problem, which probably means emulating a lot of python code without type hints, and not reading documentation if they can help it.
To be clear, I don’t fault these people at all. They probably have other priorities in their work.
I have been using Python for a year. I’ve written about 4000 lines of productive code by now. Before that I’ve been in Perl land for 24 years. Before that I was on the Borland C++ team. Before that I was in the Apple Development Systems Group. Before that I wrote Assembly language.
The answer to the mystery is: I don’t write production code. I write code that supports my testing and data analysis consulting work. There are hundreds of thousands of hackers, data scientists, AI people, DevOps people, etc., who get a lot of value out of programming languages just by using a subset of what they offer. They don’t study them as Zorro studies fencing.
Lots of people in the testing field dabble with Python without becoming experts in it.
It just strikes me, from time to time, how people really into something can lose touch with the majority of humans who are doing similar work.
I kind of agree with the title, but for very different reasons. I think the python world as bifurcated into two camps, those programs that always start with something like
import numpy
import pandas
...
and those that don't. Everything the follows from there might as well be two different languages. I write some 'python' code that starts with "import django" and some code that starts with "import numpy", and they're completely different in both structure and semantics. Being familiar in one style won't really help you much in understanding the other style.
107 comments
[ 2.8 ms ] story [ 187 ms ] threadLibraries like Jackson make it entirely painless.
I would assume JSON parsing is a solved problem in any typed language.
These things are just better when you aren't in a nominatively typed language with some pretty crusty assumptions.
That's not a gain in any way.
Since I/O is usually untyped, you need to cram it into a typed schema somewhere. In strictly typed languages, that must occur upon ingestion. The author is saying you can delay it by having a layer on top of the I/O transform the data in untyped fashion before being typed later on.
I have seen large applications (think hundreds of millions of daily users) break because of such oopses.
That's funny, because last time I had to write json-related code in Python, I really missed Rust's serde, which makes deserialization + validation much easier to write in Rust than in Python.
YMMV, of course.
By contrast, duck-typing means that the intricacies of which exact type you are deserializing into is a lot less important in python.
Am I missing something?
How do you explain the popularity of type hinting in Python then? Type hinting arose _because_ duck typing is horrible for large systems.
I’ve come to Python 20 years ago after a short stint with PHP and I don’t really care if the incoming stuff I have to handle is a list or a tuple (let’s say) as long as it quacks and walks like the proverbial duck.
More generally, you could say Python the original language is the victim of its own success (the same goes for JavaScript in relation to TypeSctipt).
The only not so easy decision is: is it better to store as Unix epoch or as ISO string. That is a tradeoff between compactness and easy searching of date/time ranges.
What a weird take, unironically asserting that the only "exciting" code is the glue, and all of the actual things software does is just an excuse to build more frameworks.
The vast majority of truly impactful code is really boring at a technical level.
So, yes, that’s sometimes why people write frameworks: to get some excitement in their code.
calculating that 38302 + 8204 = 46506 is pretty boring
noticing that ∀x,y: x + y = y + x is more exciting, and it's a lot more exciting when you can prove it
group theory? continuous braingasm
changing the report code so that an invitee who rsvps to an event even though the host has marked them as 'not coming' still shows up as 'coming'? that's pretty boring; it solves today's problem
writing a practical extraction and report language so that you need only a third as much code to define that report, but also all the future reports you write? that's exciting as fuck, it makes the rest of your life better
generality in software is highly desirable, just as in older branches of math; it's the difference between subsistence and progress
That's a strange take, not one I've heard anywhere else. Rust macros are code generators - they're run once at build time and generate regular Rust code that's strongly typed. Maybe it's because the author has a different interpretation of "infrastructure" than the commonly accepted one.
However, a more technically correct interpretation would likely divide based on hygienic vs unhygienic macros. The C Preprocessor is most definitely a different language to C – it has the same differences as Rust/Rust Macros do, but additionally is implemented by a different binary, is de-coupled from the C language to some extent, and most importantly, has different syntax and semantics making it actually a different _language_ in the strict definition of the word.
If anything this is just a sign of a more versatile language.
> While these annotations are available at runtime through the usual __annotations__ attribute, no type checking happens at runtime.
At first I thought it was disproving the commenter's "strange take" assertion but it actually proves him right. While it is true that python's type hints are a build time feature like in Rust, they do not generate code for the interpreter in the way Rust does.
Actually they do not generate anything besides metadata that can be interpreted (at build time) in any way undefined way. The undefined is explicit.
I personally liked the article and agree that eventually the undefined specification of the PEP will be removed and a new language a-la Typescript will emerge.
[1] https://peps.python.org/pep-0484/
But YMMV, I know very little about Python so I can be persuaded this is no big deal.
The author's take that annotated code and unannotated code are different languages is a bit silly. It's like saying code with comments is a different language than code without. While it's true that different camps favor annotations differently, but the same is true for code documentation, and we don't treat it like different languages, even if the comments are in a language you don't speak.
I also don't get what you mean by "dumb byte strings", it's just that the default* string type, i.e. when you have a literal `"foo"` in source, it's a sequence of codepoints vs. a sequence of octets. Assuming file systems use UTF-8 was broken in very early 3.x (I think before 3.3?) but it's something I've never struggled with (pure ASCII filenames for me). Forcing the programmer to know if they're dealing with characters & code points vs. bytes/structs/network buffers is a good thing, IMHO.
The author means that in the sense of there being two distinct camps in the Python community, but using type hints is not exclusive in any way.
Gradually typed [1] is the recognized term.
[1] https://en.wikipedia.org/wiki/Gradual_typing
The problem I have currently with Python is that all these options are increasing the number of "pythonic" ways to do something, and making the language more complicated.
Python was exciting because it was quick to write, and elegantly simple. It didn't make exciting and risky choices on syntactic sugar -- it took a moderate approach.
But I get the impression that Python is trying to evolve into more and more use cases that it isn't well suited for. I think this runs the real risk of becoming too generalized -- doing many things, but not doing them well.
You would first add types to things that are obvious as you implement features or fix bugs and leave the rest to Any and then add more complex types or even construct types for your application.
Typed Python is not better or worse than untyped Python, it has different tradeoffs.
If you're setting a goal to "transition" your application, it means you don't understand what those tradeoffs are. You have a non-nuanced thinking is "types equals good" or "new equals good".
> then add more complex types or even construct types for your application
In other words, your intention is to add complexity to your application, while preserving it's functionality. Neither engineering nor business as a whole is going to benefit from additional complexity.
The "edge case" where complexity can be beneficial is where the ratio of library maintainers to library users differs by orders of magnitude. E.g. if you have a library with 5 maintainers and 1,000,000 users, then types can be a net positive.
> TypeScript… JavaScript… Going to guess it's similar.
> I haven't touched Java for almost a decade
> I don't really know enough Rust to be able to comment with any confidence
With more complex types, I've found it isn't necessary to do anything more complex than specifying the element type of a list or dict most of the time. I have typehinted lambda function arguments before, but just using the plain Callable typehint is usually enough. When I forget if I need to use an Iterator or an Iterable (which is every time), then I just try one and run the type checker and change it to the other if i guessed incorrectly.
Type checking Python can become complex if you expect to be able to express everything in the typehints, but Python's type system isn't powerful enough for that. I feel its strength is that it lets you add as much detail to the types as is convenient.
[1] https://github.com/microsoft/pyright
Infrastructure by definition is supposed to be stable with well established interfaces, this is a perfect match for types.
Business logic is constantly evolving, focusing too much on types will cause you to be disconnected from business value and you'll just waste time fighting typing system.
I see a potential separation as more: Application vs Data Science. Someone purely writing Python inside Jupyter notebooks for data science purposes tends to be writing a very different flavor of Python than I do while writing a web application. As least in my experience there is less concern for formatting guides, idiomatic Python, PEPs, type hinting, or things along those lines. And that is totally fine, its not something they need to really be concerned within that context. It does result in very different outputs though.
Not really. The thing with typing is that it gets hyped, but in practice, it matters very little. Both science code and web application code don't really need typing or really type hints.
If you go back to your CS professors and ask them to explain what is the advantage to types and OOP, the overarching reason would be that they create contracts in your code that reduce the risk of errors. In practice, if you look at well set up deployment workflow and pipelines, you still have unit tests/integration test that aim to have 96%-99% code coverage with sufficient test coverage, and people don't realize that it makes all the typing system moot. You can write equivalent code in any language and make it fully correct solely by ensuring that unit tests and integration tests for your usecase pass.
There is value however in extremely strong typing, where you can essentially do static proofs on the codebase to ensure correctness without running it.
That is, did you remember to check for null? Did you remember to write a test for floating points in languages that just have generic number types?
I don't see the value in relying on unit tests to enforce types.
It might be possible, but it relies on the programmer to:
1. Remember all of the types (was this an int? Or a float? Or a int | null union?
2. Implement exhaustive tests for each type
3. Remember to update the tests when something changes.
4. When reading code, if the types emitted are not specified, having to examine the unit tests to determine the type limits.
And the advantage of this is that it saves me a few keystrokes so i can write:
X = 5
Instead of
int X = 5
1. Language with strong type features.
2. Testing packages that automatically include fuzz tests and null tests to ensure correct behavior?
Granted most of my experience in production level software involves backend services, which essentially means json transformations 95% of the time, Ive rarely ever seen an error that results in the service saying everything is ok but returning a wrong result. For example in the case of null fields, something downstream fails and errors out - this is easily caught by testing.
However, I think tests are high-maintenance. Often they consume the same amount of time to write as the actual business code.
I don't know how automatic null checking tests would work. Wouldn't you need to specify what happens if a null is input?
In contrast, I find the cost if typing very low for the value it delivers. Unless the code is very ephemeral (like a one off script) then I highly prefer typing.
What static typed language, apart from SQL, does automatic null checking?
I guess typing is moot when all the code and the tests are written, but having types sure does help while you're writing that code and those tests.
It helps to use the right data type when 'connecting' two objects. But it constraints the usefulness of code that can rely on the behaviour of the objects instead of the type.
It clutters the code: you have to read (and write) way more text. Which makes errors more probable.
It forces to think about primitive data types, so steals time maybe better spend thinking about higher data structures (list, dictionary,...) or algorithm.
YMMV.
Where "science code" is "this python script I'm working on for my PhD and no one else is ever going to use it", sure. Where "web application code" is "I need to get an MVP out for Tech Crunch Disrupt", sure. Where neither of those things are true - if your grad student project code becomes the basis for sucessive grad students' projects, or gets popular in industry, or if the web application code underlies a company for a few decades - for sanity's sake you want type hinting. Tests only go so far, and if I have to run a piece of code's test cases before I know what the code actually does, instead of just reading it, something smells rotten.
I find this sort of optimism both cute and frustrating. The reality is that almost no company is willing to foot the bill for such a level of code coverage as it makes writing the application take easily 2x as long. Sure, it'll theoretically be more provably correct, but the cost difference isn't easy to swallow. The reality is more like you have good path coverage for a small subset of business critical code with the rest of the code getting tested at a more macro level with integration and functional testing. And that's not even touching on the theory vs reality of how much better the code will be with those unit tests. Writing _good_ tests is harder than most people think. Writing tests that have good path coverage and good input fuzzing is _really_ hard. Writing a test that checks the box for 'code coverage' can be done trivially in many cases, but provide no real value.
This is only true if the codebase uses some highly inefficient setup that makes writing/running unit tests a pain.
And a general corporate policy for many companies, including big ones like Amazon, is to aim for >90% coverage with strongly typed languages like Java, regardless.
One of the other differences I have seen, but didn't specifically mention, is unit tests themselves. The web app probably has a ton of them. The Jupyter notebook probably does not. Which is fine, the language is being used for different purposes.
But an advantage of having those types in place is that you can do the static analysis to ensure everything is being used correctly within your codebase. And then you can have your unit tests concentrate on the actual usage rather than other stuff. There is no need to check for / test with a null value when you already know that function is never being passed a null value. And adding those hints has helped me many times with catching edge cases before they even happen where the existing unit tests were never going to hit that edge case.
You can do a lot of productive work with a language without knowing everything about it. It might be safe to say everyone working with Python is aware of indentation, though.
Sometimes the disconnect between what people in the field are doing and what bloggers say is really striking.
I'm sure such people exist. I'm just surprised to see them on Hacker News.
To be clear, I don’t fault these people at all. They probably have other priorities in their work.
The answer to the mystery is: I don’t write production code. I write code that supports my testing and data analysis consulting work. There are hundreds of thousands of hackers, data scientists, AI people, DevOps people, etc., who get a lot of value out of programming languages just by using a subset of what they offer. They don’t study them as Zorro studies fencing.
Lots of people in the testing field dabble with Python without becoming experts in it.
It just strikes me, from time to time, how people really into something can lose touch with the majority of humans who are doing similar work.