Meta Solves Python’s Problem of the Not-so-Immutable Objects
Cue “The more you know” graphic: Python objects are not immutable. In fact, all Python objects contain a small amount of mutable state. And this can turn out to be bad news for memory usage.
This problem is at the core of how the Python runtime works and one of the reasons why the Global Interpreter Lock (GIL) is important. This problem has a large negative impact on CPU and memory performance, especially for approaches to Python’s scalability, because it impacts how objects scale in memory.
A recent blog post written by Eddie Elizondo details how and why social networking conglomerate Meta created Immortal Objects, PEP-683 for Python.
Meta’s engineers wanted to improve the memory and CPU efficiency of handling their requests. The frontend server was Python (Django). They achieved parallelism with a multiprocess architecture paired with asynchio for per-process concurrency. To further mitigate memory inefficiencies, Meta engineers prefer shared memory over private memory whenever possible.
To prioritize shared memory, developers lean on a pre-fork web server architecture to cache as many objects as possible. Separate processes use these caches as read-only, structured through shared memory. Resource metrics show this method was helpful, but private memory usage still grew over time while shared memory usage decreased.
Uh-oh!
The Problem with Not-so-Immutable Objects
Python and other programming languages use reference counting as a memory management approach. The reference count is a piece of state on the object that’s mutable by the runtime. The reference count is incremented whenever the object is referenced and decremented when dereferenced. When the reference count is 0, the object’s memory is deallocated (garbage collection).
Meta’s engineers analyzed the Python heap and found that most of their Python objects lived throughout the entire runtime, but the runtime still modified the reference counts and garbage collection (GC) operations on every read and GC cycle. This triggered a copy-on-write in the server process and up, up, up goes the private memory.
Immortal Objects for Python
Immortal Objects are objects with unchanging core object state. A special value in the object’s reference count field lets the runtime know when it can and can’t mutate both the referenced count fields and GC header. Immortal objects bypass reference count checks and live throughout the entire runtime execution and remove the need for GIL safety.
Standard objects versus Immortal Objects.
Implementing and releasing Immortal Objects within Instagram was a relatively straightforward process but this was not the case with the general release.
Challenges with Immortal Objects
Implementing Immortal Objects brought a few obstacles such as performance degradation, backward compatibility, and platform compatibility. Immortal objects cause a performance degradation. The core implementation relies on adding explicit checks in the reference count increment and decrement routines. This led to a performance degradation in the service. The smart use of register allocations managed to keep this to around a 2% regression.
Another large implementation hurdle was making sure applications didn’t crash after changing reference count implementations. Meta also made sure Immutable Objects works across all the different platforms, compilers, architectures, and hardware types.
Immortal Objects Reduced Private Memory Use
By developing Immortal Objects, Meta’s engineers increased shared memory usage and decreased private memory usage though reducing copy-on-writes.
But wait, there’s more! Now there’s true immutability of objects on the heap.
Before immutable objects, both the GC and reference count mechanism had unrestricted access to both fields. The contribution of Immortal Objects to Python means these objects can be shared across threads without needing the safety provided by the GIL. This is another step toward a multicore Python run time.
