PEP 810 – Explicit lazy imports (peps.python.org)

Feels like a good feature, with a simple explanation, real world use cases, and a scoped solution (global only, pretty simple keyword). I like it!

I wonder how much things would break if all imports were lazy by default.

That seems really great, .moving imports in-line for CLI tools startup and test discovery has always been a pain.

For posterity, as the submitted link seems temporary: https://github.com/python/peps/pull/4622

Wake me up when we can import a module by relative file path.

If anyone's interested I've implemented a fairly user friendly lazy import mechanism in the form of context managers (auto_proxy_import/init) at https://pypi.org/project/lazyimp/ that I use fairly heavily. Syntactically it's just wrapping otherwise unmodified import statements in a with block, so tools 'just work' and it can be easily disabled or told to import eagerly for debugging. It's powered primarily by swapping out the frame's f_builtins in a cext (as it needs more power than importlib hooks provide), but has a lame attempt at a threadsafe pure python version, and a super dumb global hook version.

I was skeptical and cautious with it at first but I've since moved large chunks of my codebase to it - it's caused surprisingly few problems (honestly none besides forgetting to handle some import-time registration in some modules) and the speed boost is addictive.

I'm a fan because it's something you can explicitly turn on and off. For my Docker based app, I really want to verify the completeness of imports. Preferably, at build and test time. In fact, most of the time I will likely disable lazy loading outright. But, I would really appreciate a faster loading CLI tool.

However, there is a pattern in python to raise an error if, say, pandas doesn't have an excel library installed, which is fine. In the future, will maintainers opt to include a bunch of unused libraries since they won't negatively impact startup time? (Think pandas including 3-4 excel parsers by default, since it will only be loaded when called). It's a much better UX, but, now if you opt out of lazy loading, your code will take longer to load than without it.

Does this fix the circular imports problem that comes up if you don't structural your programs in a hierarchical way?

YES!!!

Make Python more like Haskell, LFG!

Lazy imports have been proposed before, and were rejected most recently back in 2022: https://discuss.python.org/t/pep-690-lazy-imports-again/1966.... If I recall correctly, lazy imports are a feature supported in Cinder, Meta's version of CPython, and the PEP was driven by folks that worked on Cinder. Last time, a lot of the discussion centered around questions like: Should this be opt-in or opt-out? At what level? Should it be a build-flag for CPython itself? Etc. The linked post suggests that the Steering Council ultimately rejected it because of the complexity it would introduce to have two divergent "modes" of importing.

I hope this proposal succeeds. I would love to use this feature.

I don’t want lazy imports. That’s just makes performance shitty later and harder to debug. It’s a hacky workaround.

What I want is for imports to not suck and be slow. I’ve had projects where it was faster to compile and run C++ than launch and start a Python CLI. It’s so bad.

I wish all imports were lazy by default.

I know/heard there are "some" (which I haven't seen by the way) libraries that depend on import side effects, but the advantage is much bigger.

First of all, the circular import problem will go away, especially on type hints. Although there was a PEP or recent addition to make the annotation not not cause such issue.

Second and most important of all, is the launch time of Python applications. A CLI that uses many different parts of the app has to wait for all the imports to be done.

The second point becomes a lot painful when you have a large application, like a Django project where the auto reload becomes several seconds. Not only auto reload crawls, the testing cycle is slow as well. Every time you want to run test command, it has to wait several seconds. Painful.

So far the solution has been to do the lazy import by importing inside the methods where it's required. That is something, I never got to like to be honest.

Maybe it will be fixed in Python 4, where the JIT uses the type hints as well /s

I don't like the idea of introducing a new keyword. We need a backward compatible solution. I feel like Python needs some kind of universal annotation syntax such as in go (comments) or in Rust (macros). New keyword means all parsers, lsps, editors should be updated.

I’m pretty sure there will be new keywords in Python in the future that only solve one thing.

Really excited about this - we've recently been struggling with making imports lazy without completely messing up the code in DeepInverse https://deepinv.github.io/deepinv/

I love the feature but I really dislike using the word lazy as a new language keyword. It just feels off somehow. I think maybe defer might be a better word. It is at least keeps the grammar right because it would be lazily.

    lazily import package.foo
    vs
    defer import package.foo

Also the grammar is super weird for from imports.

    lazy from package import foo
    vs.
    from package defer import foo.

I like the approach of ES6 where you pull in bindings that are generally lazily resolved. That is IMO the approach that should be the general strategy for Python.

Love this. My https://llm.datasette.io/ CLI tool supports plugins, and people were complaining about really slow start times even for commands like "llm --help" - it turned out there were popular plugins that did things like import pytorch at the base level, so the entire startup was blocked on heavy imports.

I ended up adding a note to the plugin author docs suggesting lazy loading inside of functions - https://llm.datasette.io/en/stable/plugins/advanced-model-pl... - but having a core Python language feature for this would be really nice.

We tend to prefer explicit top-level imports specifically because they reveal dependency problems as soon as the program starts, rather than potentially hours or days later when a specific code path is executed.

Remember mercurial? Me neither. But what I remeber is this article I've read about all the hacks they had to do to achieve reasonable startup time for CLI in python. And the no #1 cause was loading the whole world you don't ever need. As I recall they somehow monkeypatched the interpreter to ignore imports and just remember their existence until they were actually needed, at which point the import happened. So all the dead paths were just skipped.

This is needed, but I don't like new keywords. What I would love, for many reasons, is if we could decorate statements. Then things like:

import expensive_module

could be:

@lazy

import expensive_module

or you could do:

@retry(3)

x = failure_prone_call(y)

lazy is needed, but maybe there is a more basic change that could give more power with more organic syntax, and not create a new keyword that is special purpose (and extending an already special purpose keyword)

why does this have to be a syntax feature and not a lazy code loader at the intepreter level?

Ugh...I like the idea, but I wish lazy imports were the default. Python allows side effects in the top level though so that would be a breaking change.

Soooo instead now we're going to be in a situation where you're going to be writing "lazy import ..." 99% of the time: unless you're a barbarian, you basically never have side effects at the top level.

> The standard library provides the LazyLoader class to solve some of these inefficiency problems. It permits imports at the module level to work mostly like inline imports do.

The use of these sorts of Python import internals is highly non-obvious. The Stack Overflow Q&A I found about it (https://stackoverflow.com/questions/42703908/) doesn't result in an especially nice-looking UX.

So here's a proof of concept in existing Python for getting all imports to be lazy automatically, with no special syntax for the caller:

  import sys
  import threading # needed for python 3.13, at least at the REPL, because reasons
  from importlib.util import LazyLoader # this has to be eagerly imported!
  class LazyPathFinder(sys.meta_path[-1]): # <class '_frozen_importlib_external.PathFinder'>
      @classmethod
      def find_spec(cls, fullname, path=None, target=None):
          base = super().find_spec(fullname, path, target)
          base.loader = LazyLoader(base.loader)
          return base
  sys.meta_path[-1] = LazyPathFinder

We've replaced the "meta path finder" (which implements the logic "when the module isn't in sys.modules, look on sys.path for source code and/or bytecode, including bytecode in __pycache__ subfolders, and create a 'spec' for it") with our own wrapper. The "loader" attached to the resulting spec is replaced with an importlib.util.LazyLoader instance, which wraps the base PathFinder's provided loader. When an import statement actually imports the module, the name will actually get bound to a <class 'importlib.util._LazyModule'> instance, rather than an ordinary module. Attempting to access any attribute of this instance will trigger the normal module loading procedure — which even replaces the global name.

Now we can do:

  import this # nothing shows up
  print(type(this)) # <class 'importlib.util._LazyModule'>
  rot13 = this.s # the module is loaded, printing the Zen
  print(type(this)) # <class 'module'>

That said, I don't know what the PEP means by "mostly" here.

I don't hate it but I don't love it. It sounds like everyone will start writing `lazy` before essentially every single import, with rare exceptions where eager importing is actually needed. That makes Python code visually noisier. And with no plan to ever change the default, the noise will stay forever.

I would have preferred a system where modules opt in to being lazy-loaded, with no extra syntax on the import side. That would simplify things since only large libraries would have to care about laziness. To be fair, in such a design, the interpreter would have to eagerly look up imports on the filesystem to decide whether they should be lazy-loaded. And there are probably other downsides I'm not thinking of.