🔗 Found our Django app was losing 2 seconds per request to GIL and GC. Built an eBPF tool to catch it.

We run Django + gunicorn + DRF in production. Our API latency was consistently higher than what our traces showed. OpenTelemetry showed database and third-party API time, but there were always 1-2 seconds unaccounted for.

I built an eBPF profiler that hooks into CPython from kernel space. Here's what we found in our Django app:

GIL contention from Django URL resolution. Django's URL resolver was holding the GIL while resolving routes. With a large urlconf, this added up. Since our URLs are static, we added URL caching and cut GIL wait by 80%.

GC gen-2 pauses. Django loads a lot of objects at startup (models, middleware, URL patterns). Gen-2 GC was walking 1.5M objects on each collection. We added gc.freeze() in our gunicorn post_fork hook. Worst-case GC pause went from 1.27s to 88ms.

Gunicorn handoff latency. Measures how long a request sits in gunicorn's queue before a worker picks it up. When workers are busy with GIL contention or GC, handoff time spikes. After our fixes, handoff dropped from seconds to single-digit milliseconds.

Off-CPU stacks. Found that a Kafka push in one of our DRF views was blocking the thread for 130ms. Invisible in OpenTelemetry because it wasn't instrumented. Our tool showed the full Python stack: views.py:post > sessions.py:post > socket.py:readinto.

The tool uses eBPF uprobes on CPython's take_gil/drop_gil and gc_collect_main. No code changes to your Django app. Just point at the gunicorn PIDs and it shows you everything.

CPython 3.11 only for now. Open source.

GitHub: https://github.com/deepanshu406/python-ebpf-profiling