Getting Started With Java Profiling - Richard Startin

Hey Everyone,

In this episode I talk to Richard Startin on Java Profiling. Richard has an amazing experience in profiling performance sensitive applications and is currently working with Datadog as a Software Engineer focused on profiling.

We have talked about Richard's experience with Profiling applications, best practices, tools available and much more.

Chapters:
00:00 Introduction
02:02 What is Profiling?
04:09 How frequently should we profile an application?
08:00 Continuous profiling - heap dump example.
10:36 Approaching Memory leaks
17:40 Different tools to profile an application
23:46 Concepts to know for profiling an application
31:15 Getting started with understanding a profile
35:53 IO, Resource contention and Memory issues with an example LSM implementation
52:49 Memory requirements of an application
57:06 Examples where Profiling helped Richard solving critical performance problems
01:02:18 Best practices in Developing applications and Profiling
01:04:16 Limitations in current profiling tools
01:09:23 Resources to learn more

References:
Andrei Pangin @AndreiPangin

Corrections:
1) While talking about LSM bloom filter, Richard said "False negatives" while he meant "False positives".
2) Instead of "getcalltrace" it should be "getstacktrace".

If you like this episode please like, share and subscribe to the channel.

Cheers,
The GeekNarrator

[TheGeekNarrator]

Some corrections:
1) While talking about LSM bloom filter, Richard said "False negatives" while he meant "False positives".
2) Instead of "getcalltrace" it should be "getstacktrace".

[michaeldunlavey]

Thanks for this.
Below is a comment I put on another vid about profiling. It does not address multi-threading and memory leaks. I have ways of going after those as well.


Speed problems are bugs. Not the kind of bug that makes your program wrong, but the kind that wastes time by doing stuff you don't need.

A speed problem wastes a certain fraction of time X, like 20%, 50%, or 90%. The problem you have is to FIND the problem, NOT MEASURE it. The importance of X is, the bigger it is, the easier it is to FIND the problem.

How do you FIND it? Easy. You take a *_SMALL_* number, like 10 (not thousands), of random-time samples of the call stack. You LOOK CAREFULLY at each one (you can't do that with thousands), to understand what the program was doing at that time, and, especially, WHY. This way, you will see if there is a way to avoid that activity.

Now, the important thing is, if you only see it wasting time on ONE sample, that may not mean anything. But, if you see it wasting time the same way on TWO OR MORE samples, guess what? You've FOUND the problem! So fix it, and collect your speedup!

Then, do it all again, until you can't anymore. (Every speed bug you fix amplifies the ones that are left.) That's how you can make code as fast as possible. (You may make it so fast that you have to artificially slow it down, by putting in a temporary outer loop, so it will run long enough to take stack samples.)
(This is sometimes called the "poor man's profiler". The point is, it actually works, and any "rich man's profiler" won't, because they are more interested in measuring what you can't fix than finding what you can. Of course, some programmers like being told there's not much they can fix!)

[chiragr]

Please create more Java profiling videos on topics like async profiler. It's advanced and not enough content online. Guidance on how to master it will be very very appreciated 😭 🙏

[chiragr]

Finally the wait is over since Kaivalya posted this a month ago.