Apple Watch : Scientific Sleep Test

🟡 Twitter: @QuantifiedRob
* Affiliate (paid) links support the channel by earning me a commission on any sale, and it doesn't cost you any more! My opinions are completely my own, and this content is not sponsored.

Conflict of interest note: Dreem employees are co-authors.
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
👇 $30 off any new WHOOP membership, affiliate* (paid) link, supports the channel!👇
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
👇Amazon Affiliate* (paid) links. Supports the channel, doesn't cost you any more!👇
▬▬▬▬▬▬▬▬▬▬BEST WATCHES▬▬▬▬▬▬▬▬▬▬
($30 off any new WHOOP membership)

▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
📸My Gear* (paid links, supports the channel and doesn’t cost you any more)📸
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬

Timestamps:
00:00 Intro
00:52 How Apple Tracks Sleep
02:24 Test Design
03:49 Deep Sleep Test
06:21 Light Sleep Test
06:48 REM Sleep Test
09:05 Sleep Cycles Test
09:53 Awake Detection
11:53 37 Watches Tested
13:25 Sleep Timing Test
14:34 Limitations
15:29 Missing Features
16:06 Am I Biased?
16:52 Conclusions

Intro: When Apple first announced their new sleep tracking functionality, they did not provide much detail as to how it actually works, but let’s briefly summarize what they did tell us, and what this means.The best way to measuring sleep stages, what scientists call the gold standard, is called polysomnography. This basically means measuring the electrical signals coming from the brain while you are sleeping, and in addition also measuring the movements of the eyes and of some muscles. This way scientists can get an accurate picture of what sleep stages you went through. So, to make sleep tracking more comfortable and readily available, Apple trained a computer algorithm that takes in the heart rate and movement data from the Apple Watch, and uses that to predict your sleep stages. To create this algorithm, they collected both polysomnography data and heart rate and movement data with the Apple Watch, on what they say is “the largest and most diverse study population for a wearable”. That algorithm then uses the sleep stages measured with polysomnography, and tries to predict the same sleep stages with just heart rate and movement data from the Apple Watch. If Apple has enough data and good machine learning tools, they could have potentially created a very good sleep tracking algorithm, making sleep tracking available to the general public.

So, how did I test this? Well, I evaluated the sleep stage tracking of the Apple Watch for 18 nights by comparing it to the Dreem 2 EEG headband, a device that can actually measure my brainwaves, and was specifically designed for sleep stage tracking. The EEG device should therefore give us an impression if the Apple Watch is good enough for sleep stage tracking. In this case I used the Apple Watch 7 as the test device, but I think both older and newer Apple Watches will give more or less the same result, since the heart rate and movement measurements are roughly equally reliable between generations, as we saw in previous videos. Measuring heart rate and movement is especially easy during the night, when you are generally relatively still with a low heart rate. I should mention that I am using the beta version of WatchOS 9, but I suspect nothing will change with regards to sleep tracking in the official release, since the algorithm was already changed.

[TheQuantifiedScientist]

🔵 Newsletter: www.robterhorst.com
🟡 Twitter: @QuantifiedRob
* Affiliate (paid) links support the channel by earning me a commission on any sale, and it doesn't cost you any more! My opinions are completely my own, and this content is not sponsored.

Conflict of interest note: Dreem employees are co-authors.
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
👇 $30 off any new WHOOP membership, affiliate* (paid) link, supports the channel!👇
($30 off any new WHOOP membership)
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
👇Amazon Affiliate* (paid) links. Supports the channel, doesn't cost you any more!👇
▬▬▬▬▬▬▬▬▬▬BEST WATCHES▬▬▬▬▬▬▬▬▬▬
Whoop Strap 4.0:
($30 off any new WHOOP membership)

[MrJeremydb]

How can anyone accuse this guy of bias lol this has to be one of the most scientific testers I’ve ever seen. Nice work sir!

[yashchawla]

Thanks to you, I have realised which smartwatches are meretricious and which ones are worth buying.

[xander]

Apple has a proven history of putting their health tracking programs through rigorous scientific vetting before release. In fact, they often have the hardware and software in the watch without enabling it until they’ve gotten all approvals. This feature will be great once it launches. Apple has been at the forefront for health features for a long time and this is another chapter in their history.

[Tony_YT]

We need more folks like you in the tech review community. Thank you for the videos!

[syreetanolan]

This is the first time watching one of your videos and I love the way that you present and verbally describe data on graphs to be fully accessible in this video. I love how it is basically organized as a clear and concise scientific paper, do you ever publish papers after all the work you put forth?

[joel_pnk]

It blows my mind on how detailed this is and how i managed to pull through all of this scientific analysis... Well done man! Another great video, visually and content/data wise! <3

[craigward]

Really amazed and happy with Apple, just shows how serious they are with developing existing hardware and software features

[yandan]

This video is amazing! The amount of work you did. The thoroughness of the analysis. The comparison to medical devices. Really helpful. Thank you.

[yildiraykomurcu]

Great effort by Apple ! Now they almost solved sleep tracking. I wasn't expecting such an achievement. This came out of nowhere. Awesome ! I hope the rest follows. Thanks Rob for bringing it up. So if Apple is able to extract brain status from heart rate and movement data, then it should be easier to extract blood pressure from the same data. Hope they will do it as well.

[greenfoodcrush]

Awesome review as always! My husband and I wear Fitbit because in our opinion it had the best sleep tracking. This morning he showed me the new Apple Watch and I immediately said “let me see what the Quantified Scientist has to say about it!”
You hold a lot of clout with me and I am always amazed at your ability to test and report, seems like so much work! It’s greatly appreciated!!

[weekendwarrior]

I like how you had to make the graph bigger just for the Apple Watch…it’s nearly half the reason I switched to the apple eco system. I’ve been using the “auto sleep” app and it has been showing the things you mentioned

[josephskeller]

I've been anticipating your analysis of these new features and I'm glad to see them. Great work!

[anneskeen]

Thank you Rob! I was very curious about the accuracy as I recently started wearing the watch and amazed by the data but was curious how accurate it is. I am 51 years old and have had sleep issues all my life, did home studies and PSGs but finally got a doctor who ordered a MSLT (which stupid insurance took almost a year to approve because of all the other things they insisted be done first to rule out). I finally was diagnosed with Narcolepsy w/cataplexy in the last 1 1/2 years. I spent almost my entire adulthood being diagnosed and treated for depression, anxiety, ADD and being told it was all in my head and nothing was wrong. Most people don't understand the impact it has on every day life.

May sound silly but it is gratifying to finally have a diagnosis that makes sense, that I can see test results that support the diagnosis. So now having the ability to track without full sleep studies is invaluable!! I have only been using the Apple watch for a short time but already finding the information extremely insightful. Using the data, making a few changes and already seeing some results (albeit just a little) that I am hoping can lead to significant improvements.

Thank you for the detailed analysis and helping me determine how much assurance I can have in in the data!

[Johnny-ldbc]

Thanks for the super professional reviews and comparisons!!

[TStudios]

Wow, this is just a super high quality video! I sometimes find it difficult to interpret graphs, but your explanations made it super clear! There was so much info but you presented it in a very concise engaging, and friendly way. Not to mention how thorough and credible your research is. Absolutely top tier, and I’ll definitely be watching more!

[ankurdeo]

Just an observation: Apple Watch seems to show a lot more transients (or spikes) in readings compared to the Dreem2. From a software perspective, this might be because of the sensitive transient and suppressing techniques used. If you do actually filter the short duration spikes, the accuracy of Apple Watch is remarkly high!

Great video by the way.

[StephenWithaPH]

Thank you Rob! I've been waiting for this exact video since the beta release of WatchOS 9! An excellent video once again and thank you for all the amazingly helpful reviews and work that you do!

[mazkebar]

Dr. Happy & ALWAYS Smiling Scientist! That’s rare to find! 👏🏼😅☺️😊👍🏼

[kpudv]

Finally! Now, the Apple Watch 8 had announced. Guess we all hope you can put that review on the new watch as priority. We are really desperate to see the results on its exercises and sleep tracking accuracy. Once again, thank you for your effort and passion! Still the best reviewer I'd ever seen.