The Math Behind Font Rasterization | How it Works

#SoME

If you have ever wondered how a font is rasterized, look no further. In this video I go into depth about the necessity for font rasterization, the math behind it, and how we convert the math to pixels on the screen. I also go over how 1st and 2nd order Bezier curves work. I talk about how to gain an intuition for Bezier curves, and how to formulize that intuition into Mathematical formulae. I also go over how to test if a point will intersect with a 2nd order Bezier curve. This is my entry into 3Blue1Brown's SoME competition and I hope you enjoy it!

0:00 Intro
0:38 A Brief History of Fonts
3:48 Using Math to Draw Letters
5:42 What are Bezier Curves?
9:09 How do Bezier Curves Really Work?
12:41 Testing Pixels
15:13 Edge Cases

---------------------------------------------------------------------

Here are some books I recommend if you want to learn about game engine development more thoroughly. I do not profit off any of these sales, these are just some books that have helped me out :)

My Recommended Game Engine Books:

My Recommended Beginning Game Programming Books:

My Recommended Java Books:

[LookingGlassUniverse]

You have a great style of explaining things and making them feel friendly. I especially loved the two blob characters 🥰 I hope you make many more maths videos!

[SimonBuchanNz]

A few notes on what most actual implementations do (from what I've been able to read), just in case anyone thought this was simple:
* Solving quadratics is (probably much) slower and harder than just subdividing until you're smaller than pixels then testing against the x coordinates of the points closest in y coordinates.
* Generally the letters (or, more generally, glyphs) are only rendered once per font size when needed and then that copy is stamped at all the places it's needed.
* It's common to use the fact that PC monitors layout their pixels in vertical RGB stripes to get more horizontal resolution.
* At small sizes, avoiding blurring as lines cross through pixels is more important than exactly representing the outline. The solution is pretty crazy: TTF fonts include programs that execute to adjust the coordinates based on size... and potentially other things like weight and surrounding characters!
* This doesn't work if you're animating, as you can see the outlines jumping around (and it's slow). In this and other situations, like very large text or 3d, signed distance fields (or SDF), which are basically fancier bitmaps that are quite good at approximating the outline but still very fast to render.

[nikki-deprecated]

I am seriously unable to comprehend why you have such a small following... Your content is amazing and super high effort! You should be in the millions!

[krytharn]

Might be worth mentioning that there is an alternative solution for fast, real-time rendering of glyphs pioneered and patented by NVIDIA, which is based on using the stencil buffer to determine coverage per pixel.

Basically, a pixel is subdivided into subpixels and a shader determines for each subpixel whether it is inside the glyph or not. This can be quickly determined if you are smart in how you create your geometry. Quadratic curves, for instance, can be evaluated by creating a triangle from points p0, p1 and p2 and using barycentric coordinates for the vertices. It then becomes trivial to determine whether the subpixel is inside or outside the curve.

The "winding" path is then drawn first, adding values to the stencil buffer. Next, the "non-winding" paths are drawn, subtracting the values in the stencil buffer. You end up with a stencil buffer containing ones and zeroes for every subpixel.

Finally, in the so-called coverage step, you calculate the coverage of each pixel by adding all subpixel values for that pixel; if 16 out of 16 subpixels are set to one, you have 100% coverage, but if only 8 pixels are set to one, you have 50% coverage and so on. The final color will be determined by using coverage as an alpha value, or by multiplying the red, green and blue colors by the coverage.

[elraviv]

2:14 just for comparison, I had a Commodore 64 back in the '80s.
it used 8x8 pixels for each character and only 1 bit per pixel.
so the WHOLE ASCII table of 256 characters took 256*8*8*1bits = 2KB
well it only had 64KB of memory anyway.

[asherhaun]

Very interesting... the more I learn about font rendering/rasterization, the less I take it for granted :D

[CarlSmithNZ]

This is great. Would love to see you go to the next step and talk about anti-aliasing etc.

[TorMatthews]

Distance field fonts are worth a mention as well because of their awesome utility, though getting into distance fields is itself a whole other topic

[gokselkucuksahin]

Future 3Brown1Blue channel.

Keep going.

[kosmiksausagezz]

What an amazing video - great graphics, really informative and easy to understand. Props!

[HJfod]

This video is absolutely fantastic, such a clear and simple explanation for a thoroughly interesting topic!

[ASalfity]

I think a video on anti-aliasing would be interesting as well. The idea behind anti-aliasing is that if the edges contrast too much with the background you get a jagged looking texture along the edges on a (low resolution) bitmap screen. So what you do is you fill in the background near the edges with shades that transition between the shades of the letter and the background so that shades transition between the letter and the background more gradually. This gives less jagged looking edges. Thank for this video.

[roy]

the size of the channel isn't representative of the production quality of this video. subbed, really good video

[Nikki-]

Wow, that's a lot more complicated than I thought! Great video!

[oliviers]

Just wow. I had to work with a small OLED display and hexfonts a while ago. Since then l have been wondering how more advanced responsive font systems work and this video was the perfect answer. Thank you for this bit of knowledge.

[ashfvt]

This was very interesting. You did a great job. Hope to see more from you in future.

[Just_Moh_it]

This channel is underrated, and my day is *made

[hiyata]

I'm looking for an interesting topic for my maths homework right now, and your way of explaining this concept made it tremendously easy! Thank you so much for the video and your beginner-friendly explanations!

[kipchickensout]

nice video, i kind of expected the part with the raster, like how it's turned into pixels, anti aliasing and stuff like that

[wellisntthatnice]

I'm not up on my quadratic equations, but overall I enjoyed the video. I was a little distracted at the beginning, though, due to some errors. Not big things, but big enough to make the problem of raster fonts seem overwhelming.

At 1:26 you introduce a 32 x 32 grid, but it's actually only 16 x16. 

At 2:10 you present the formulas for determining the size of a raster file for uppercase and lowercase English letters in a monospaced typeface. 1 pixel is certainly 8 bits of data for grayscale, but fonts weren't grayscale--they were bitmaps; thus, 1 pixel was 1 bit. 1 letter in your font would have occupied 16 x 16 bits. The final equation should have been 26 x 2 x 16 x 16, or 13, 312 bits--not 425, 000 bits. This font would take up about 1, 664 bytes or 1.625 KB (K-bytes), not 53 Kb (K-bits).

At 2:37 you bring in BMP files, but fonts weren't stored as BMPs. Also, Apple didn't invent TrueType to solve the problem. The problem was already being solved with Adobe Type 1 and Type 3 fonts, introduced in 1984, the same year as Macintosh. Apple made TrueType to avoid licensing fees for Type 1 fonts. Side-note: Too bad for consumers because TrueType fonts (quadratic B-spline) were inferior to Adobe Type 1 fonts (cubic B-spline). Anyway.

At 3:23 you present a 12 px font at 53 KB, referring to the previous 32 x 32 px grid that was actually 16 x 16. The 512 px font is presented as being 13.3 MB. But using bits, not grayscale pixels, the 512 px font would only be 1.664 MB.

Again, I really liked the description and animation of the Bézier splines--top notch work.