Talks - Ludovico Bianchi: Using Python's import machinery to handle API deprecations

For any software project with an established user base, introducing breaking changes in its API can be daunting. To minimize disruptions for users, projects are incentivized to plan these transitions carefully, which may include API deprecations, where messages warning users of upcoming changes are added to the affected APIs while they’re still functional. However, this imposes extra workload for the project’s maintainers, as both old and new versions of the API must be kept functional throughout the transition period. As a maintainer of a software project undergoing preparations for a major version release, I recently found myself in a similar situation: our goal was to provide backward compatibility with the previous version for as long as possible, without impacting the development of new features. Practically, this included dealing with a radical restructuring of the Python codebase, resulting in hundreds of modules being relocated, split, or removed. Was there any way to ensure that the deprecated import paths could still be used without errors, without having to maintain two separate versions of the package? Fortunately, the answer to “can you do that in Python?” is more often than not “yes!”; for this particular case, the path to success turned out to be through the importlib package of the standard library. For something so close to Python’s internals, importlib is both accessible and extensible, allowing ordinary code to customize almost completely how and what modules can be imported---including modules that are not there anymore! This intermediate-level talk will present a complete solution based on Python’s importlib machinery that allows to redirect modules or module attributes with deprecations in a simple, robust, and scalable way. While the context of the solution is especially relevant for project maintainers, the focus is on importlib techniques that are generally applicable.