User API & C++ Implementation of a Multi Producer, Multi Consumer, Lock Free, Atomic Queue - CppCon

---

User API and C++ Implementation of a Multi Producer, Multi Consumer, Lock Free, Atomic Queue - Erez Strauss - CppCon 2024
---

This presentation introduces a multi-producer, multi-consumer, lock-free queue with unique characteristics. We will cover the C++17 implementation and the std::atomic features required for this queue based on the CPU atomic instructions and discuss the queue's portability across various CPU architectures, beyond just X86_64 and runtime environments.

Efficient message queue-based communication between threads is crucial for optimal performance in multi-threaded applications. Queues are fundamental data structures that interact with various aspects of the application environment, including schedulers, memory allocation systems, and CPU hardware architectures.

Many use cases such as trading platforms, games, audio processing and other fields have strict latency and scaling requirements and this queue implementation has proved to reduce system latencies.

The presentation will feature the design of the queue, the required template language features, bandwidth and latency consideration, and multiple demo applications. Comparing this queue implementation’s benchmark results with other existing queues will follow. Finally, we will discuss potential future work and areas for improvement.

Join us to explore how this innovative queue implementation can improve your multi-threaded application performance.

All presentation materials, including the C++17 source code, slides, benchmarks, and demo applications are available on GitHub.
---

---

Erez Strauss

Erez Strauss worked in Banks and Hedge Funds while focused on low latency systems.
---

CppCon is the annual, week-long face-to-face gathering for the entire C++ community. The conference is organized by the C++ community for the community. You will enjoy inspirational talks and a friendly atmosphere designed to help attendees learn from each other, meet interesting people, and generally have a stimulating experience. Taking place this year in Aurora, Colorado, near the Denver airport, and including multiple diverse tracks, the conference will appeal to anyone from C++ novices to experts.
---

---

#cpp #cplusplus #cppcon #cppprogramming #cplusplusprogramming #softwaredevelopment #softwareengineering #coding #code #computerscience #technology #technews #programming #programmer

[TysonBrown-hy]

I almost spat may coffee out when I heard 73 million messages per second! Interested in what's preventing usage on Linux/ARM architecture, as I want to use it on my RaspberryPi 5! :)
Well done Erez.

[Roibarkan]

18:41 [slide 14] Great intro. The slide says it’s pseudo code, a few typos/issues: there’s some missing modulo or bitwise-and after calling fetch_add (unless the queue is meant to hold UINT_MAX elements); the unique_ptr should be the ‘array version’ for allowing operator[]; lastly the code uses the pair objects in a different order (assumes it’s pair<T, unsigned>)

[Roibarkan]

44:57 this illustration shows that there’s some redundancy in the Seq values of each cell. For every i, (elements[i].seq % N) is always equal to i. This means we don’t have to store the “bottom bits” of seq. Similarly, the “top bits” are never too far away from the global indexes. Thus, I think it might be enough to keep 1 or 2 bits of ‘seq’ in each array entry (together with the ‘data’ bit). If the value type of the array is a pointer (an approach discussed earlier in the talk) perhaps the pointer will have some “free bits” (due to alignof(T) or due to O/S virtual-memory considerations), to allow us to reduce each entry to sizeof(T*)

[Roibarkan]

32:24 [slide 24] this idea of wrapping a queue of pointers with some unique_ptr functionality is nice. I think if pointer_mpmc_queue also accepts a ‘Deleter’ template argument, to allow for better (non blocking) allocation scheme of the actual objects

[Roibarkan]

18:41 [slide 14] I think the ‘unsigned’ member in each pair might need to be atomic<unsigned> to prevent this from being UB

[Roibarkan]

34:52 the use of union here seems to imply some ‘type punning’ might be used. Hopefully std::bit_cast can be used instead to be standard conformant