SIMD Libraries in C++ - Jeff Garland - CppNow 2023

---

SIMD Libraries in C++ - Jeff Garland - CppNow 2023
---

Vector processing to accelerate computation developed more than forty years in the 1970's. At the time limited to extremely expensive machines dedicated to large mathematical problems. By 2016, single instruction multiple data (SIMD) registers and pipelines started occupying the silicon of processors available on every desktop. While the early promise of SIMD seemed to rely on the idea of the compiler vectorizing loops automatically, that mostly has not happened. Instead, over time programmers started exploring how to directly utilize SIMD by altering algorithms to directly exploit parallelism. The performance results were staggering with some SIMD algorithms producing trouncing the performance of highly optimized code. In 2023, a new era is dawning where portable SIMD applications can be built on top of libraries targeted at application developers. Most notably for c++ std::simd.

The goal of the session is to provide a starting point for developers interested in exploiting SIMD in c++. This session will be a tour through various c++ libraries utilizing SIMD. We will overview application level libraries for json parsing, compression, and crc processing that keep all the SIMD details below the interface. As well as 'developer libraries' that provide abstractions to enable the development of new parallel algorithms. We'll discuss the design choices and trade-offs made by these libraries.

The final part of the talk is an introduction to the proposed for c++26 std::simd and all of it's components. This includes data-parallel types via std::simd and it's related traits and Abi facilities. In addition to types the talk will cover algorithms and the role of std::simd_mask. We will also cover the nuts and bolts of getting access to the preliminary implementation via gcc11 and up. This session will be an adjunct to Library in a Week where we will be working on writing parallel algorithms using std::simd.

---

Jeff Garland

---

Video Sponsors: think-cell and Bloomberg Engineering
Audience Audio Sponsors: Innoplex and Maryland Research Institute
---

---

CppNow 2024
---

#boost #cpp #cppprogramming

[pkboy]

Now this gets me excited. Glad to see SIMD is getting some love in cpp 26. I probably won't see that at work for a couple of years after release but I'm still excited. I can only imagine how wonderful it'd be to have a native portable SIMD solution

[sephorusFR]

Good talk :)
As for EVE code:
- wide v{4} is the value set in the wide register
- the second parameter is the cardinal of the wide in case you want to so stuff like size - i or something. This needs to be made clearer.
- wide size is fixed and is usually assert. There is no dynamic sized wide.

:) Thanks for the shout out.

[chriselrod]

1:17:50
I strongly prefer `==` returning a mask for data parallel types, not a bool. This is necessary for generic code.
For code that is generic between scalar and SIMD-vector to work, each lane of the vector has to execute the same way as the scalar (but e.g. using masks instead of branches).
This is what comparison operators returning masks enables.
Returning bools suddenly changes the meaning of the code when you go from scalar to vector: the result of one lane suddenly depending on the neighboring lanes, violating the basic tenant of data parallelism: data should effectively execute independently in parallel.
Only people who haven't really done any SIMD programming think returning `bool` is a good idea.

[denisyaroshevskiy]

Appreciate you trying eve! I'm sorry it wasn't very intuitive.

You can find my and Joel's talk:
SIMD in C++20: EVE of a new Era
I also think that my "My First Simd" talk is a good place to get a first overview of how to do cool things with simd.
I show strlen, find, reduce, insclusive_scan and remove.

Regardless of confusions:

Size of the register is known at compile time. wide<float> will select the default one for the architecture in question.
You can override it: wide<float, fixed<4>>

Note: there is a big problem of what's "default size" on avx512, we are working on detecting compiler flags but not done yet.

The second parameter in the lambda is size. Otherwise getting it is very clumsy. [](int i, int size) { return size - i - 1; } will generate you values in reverse order.

wide<float>(float*) is same as load(const float*). This is maybe too cute.

[Theawesomeking]

I really hope this gets added, finally a reason to upgrade c++ to a new version
c++ is supposed to be fast, with this it will make it run 8 to 16 times faster.
currently the main issue with simd is platform dependency, so if the standard is able to provide us with this and be able to automatically switch intrinsic instructions based on the platform without us worrying that would be a really big favor to everyone.

Also i disagree with this guy at 1:17:25 complaining about the == mask, anyone who has programmed with shaders would already know that every operator or instruction you do will be applied to every element, thats why its called single instruction multiple data, it also helps us make the code more seamless, organized and more similar to shading languages.

[fenokineza]

You know... it's a good talk, you know

[dgo]

The key to getting good auto factorization - use a static size (preferably cache line aligned) frame to iterate unknown size data, finish off any remaining items at the end in a separate loop, this way most of the ops are vectorized and only the unpadded remained is done in scalar.

[LucasHartmann]

Regarding the operator==, I believe it should return the mask to keep SIMD mentality, but maybe the mask could provide an operator bool that implicitly uses all_of. Code that includes branches like if(a==b) is clearly not expecting SIMD, and will take the usual meaning, while branchless code will never ask for boolean conversion.

[sinom]

"Don't operator overload that isn't the same as everywhere else" ranges already do it with the | operator. But yes in general that is a very valid point

[dexterman]

What surprises me a lot is even with the existence of such amazing libraries, which people have used, and has features, the standard proposal cuts these down and also seems to produce worse assembly. If that's the case, why not focus on divert effort to language features instead? I definitely am ignorant of the constraints, so please do educate me! Reflection is still up in the air, and the slow evolution is pushing me to use other languages. I definitely am ignorant of the constraints, so please do educate me, and I do not mean that in a sarcastic way.
Great talk! Nice to see simd native support. Hopefully the problems are ironed out and it makes it for C++26!

[yuan.pingchen]

The SIMD instructions should be added into C standard, rather than the CPP standard, because some systems, eg. trueNAS scale, using sse4.2/AVX to evaluate ZFS performance at booting time, or just simply keep inline assembly language capability in both C/CPP compilers... anyway this feature comes too late..