Lidi Zheng, Pau Freixes - gRPC Python, C Extensions, and AsyncIO

"gRPC Python, C Extensions, and AsyncIO
EuroPython 2020 - Talk - 2020-07-24 - Microsoft
Online

By Lidi Zheng, Pau Freixes

Goal - Encourage Python developers to understand C extensions by sharing gRPC Python’s practice, and advocate the adoption of AsyncIO.

Prerequisite
- Understand thread vs. process;
- Interested in asynchronous programming.

gRPC Brief
- What’s gRPC Core? And what is gRPC Python?

Cython To The Rescue
- Why we picked Cython among all other available tools (e.g., pybind11, ctypes)
- Debuggability: pdb & gdb

The GIL Friction
- How to delegate work to C extension
- How to make multithreading work

AsyncIO Topic
- Not blocking the loop, the main headache.
- Non-blocking I/O solution 1: replacing C libraries’ I/O operations
- Non-blocking I/O solution 2: dedicated background poller thread
- Performance improvement (10k - 20k for client, 4k - 16k for server)

Migration to AsyncIO
- Tolerate multithreading and AsyncIO in the same application
- Make both API co-existable in the same application

"