Menu

How do Sleep Trackers Work and How Accurate are They?

By

How do Sleep Trackers Work?

The eight-hour-per-day sleep mantra no longer holds true in the busy lifestyle of today. With all the anxiety and chaos surrounding our lives, I have always believed that the quality of sleep is as important as the quantity.

A good night’s sleep not only makes you feel less groggy, but it also improves the efficiency with which you perform your daily tasks, which culminates in a better performance both at work and at home.

But how do we measure the quality of sleep? To quantify sleep, all you need to do is to look at the clock and calculate the number of hours. However, measuring the quality is a far more complex ordeal.

Sleep like a baby

Are crude statements like “I slept like a baby” or “I slept terribly” sufficient to judge the quality of sleep or do we need actual measurements to ascertain the different factors affecting our sleep?

Fortunately, sleep experts have realized that accurately monitoring different metrics and phases of sleep is a key determinant in leading a healthy life.

In this article, I will explain the different elements of sleep and how scientific devices, as well as personal gadgets, track each of them to judge the overall quality of sleep.

I will also compare different methodologies of measuring sleep and will walk you through a myriad of wearable and non-wearable sleep devices at your disposal, citing relevant research and upcoming projects where possible.

The Science behind Sleep Monitoring

Sleep Cycles

Sleep is a far more intricate process than merely lying on the bed, dreaming and waking up. Researchers divide sleep into two broad phases: Non-REM and REM.

Body parameters such as heart rate, brain activity, temperature, breathing pattern, eye movements, and muscle activation vary during different stages and can be measured to record the contribution of different stages in a person’s sleep.

The non-REM stage is important as tissue repair and growth occurs in the deeper phases of this stage. It can be further divided into four sub-stages: stage I, II, III and IV.

During stage I, eye movements diminish, muscle activity decreases (muscle jerks/twitches may occur), breathing slows, and heart rate becomes regular. The brain produces alpha and theta waves which result in the relaxation of the body.

During stage II, eye movements stop, slow muscle activity alternates with fast activity, breathing slows, body temperature falls and heart rate/blood pressure decrease. Brain activity diminishes.

During stages III and IV, the body becomes motionless, and parameters such as blood pressure, breathing rate, and body temperature drop even lower. The brain produces slow waves during this stage.

During REM sleep, a person is immobile, but their eyes start exhibiting rapid movements. The breathing rate increases and becomes irregular, and heart rate and blood pressure approach waking levels. This is when dreams occur, and brain activity is similar to when the person is awake.

Scientists and doctors can get valuable information by studying the variability of body parameters during each of these stages. We will discuss more how experts use them in the next section.

Protocols and Concepts Used in Clinical Practice

Sleep researchers may have disagreements over different sleep theories, but the one thing they all agree on is that polysomnography is the gold standard of sleep monitoring in the clinics.

Polysomnography (PSG) is derived from three words: poly meaning many, somnus meaning sleep and graphy meaning study. As such, it is a combination of several studies:

  • Detection of brain activity via EEG (Electroencephalography)
  • Detection of eye movements via EOG (Electrooculography)
  • Measurement of skeletal muscle activity via EMG (Electromyography)
  • Measurement of heart activity via EKG (Electrocardiography)
  • Measurement of sleep positions
  • Detection of breathing rate, blood oxygen, and carbon dioxide levels

All these studies collectively yield results that can be used to judge a person’s sleep quality or to diagnose sleep disorders such as sleep apnea.

However, it is essential to bear in mind that PSG requires complex machinery that needs technical expertise to operate. This makes it highly impractical as a tool for personalized medicine.

Thankfully, scientists have come up with another method that is far more portable and is called actigraphy. This non-invasive method allows scientists to measure your rest and activity cycles.

But what has measuring activity cycles got to do with sleep quality? As mentioned above, the movements you make while you sleep are anything but random, and actigraphy can be used to ascertain if they are normal or abnormal. More on this in the next section.

Polysomnography vs. Actigraphy

PSG is an expensive and cumbersome procedure that requires you to take time out of your busy schedule to book an appointment and is reliant upon the availability of the specialized machinery used for the procedure.

The administrator of this test will attach several wires to different parts of the body for which you may have to shave the excess hair in the region.

This is highly inconvenient as the environment where you usually sleep and the clinic where PSG is being done may differ significantly enough to yield any valid results.

This makes actigraphy all the more important. Although not as accurate as PSG, actigraphy is highly convenient in that it is usually performed using a portable device (called a sleep tracker) worn on a person’s wrist.

The movements a person makes are recorded on the device, and specialized computer algorithms are used to find out any deviation from normalcy.

This eliminates the need to visit the doctor’s clinic for a 2-3 hour test (or setup bulky equipment at home), as the doctor can just download the data from the device and then diagnose any anomalies.

A number of studies have sought to compare the accuracy and validity of wrist actigraphy with that of PSG, behavioral responses, sleep diaries and other subjective measures.

In a study published in 2011 on depressed insomniacs, McCall and McCall compared actigraphy with PSG and sleep logs [1]. They found out that actigraphic measurements were closer to PSG for almost all factors studied when compared with sleep diaries.

Blackwell and colleagues conducted a study on older women that compared sleep parameters from actigraphy and polysomnography [2]. They found out that the two procedures corresponded well with each other as long as the sleep was not disrupted.

Marino et al. also studied the accuracy of wrist actigraphy and compared it with PSG [3]. They concluded that actigraphy did have its perks when compared to PSG. Actigraphy was found to be even more valuable in patients undergoing treatment for insomnia, by providing an objective measurement of their sleep quantity.

The Rise of Portable Sleep Tracking Gadgets

As we have moved from computers occupying the whole room to ones that can fit in our palms, we have realized the importance of fitting good things in small packages.

Sleep trackers are portable devices that can either be worn on the wrist, torso or ankle or can be placed in close proximity. As their name suggests, they track different variables related to sleep which can then be used to judge overall sleep quality.

Currently, sleep trackers are available in three flavors: wearable, non-wearable and software-based. All three have their pros and cons which will be highlighted later in this article.

What do Sleep Trackers Monitor?

Modern sleep trackers monitor the following variables:

  • Sleep duration: These devices ascertain sleep duration by measuring the time you’re inactive.
  • Sleep quality: These devices can detect actions such as tossing and turning, as well as waking in the middle of the night.
  • Sleep phases: Some advanced systems keep tabs on the different phases of your sleep and adjust the alarm to coincide with a stage that makes you less tired when you wake up.
  • Environmental factors: A few of these devices measure surrounding temperatures and the amount of light in your room.

Although actigraphy and accelerometry remain at the heart of most sleep trackers, new technologies are threatening to take their place. These include measurement of vitals such as:

Heart Rate

Since heart rate varies throughout different sleep stages, it can be used to infer which phase you are in at a particular point of time.

A plethora of different methodologies can be used to measure heart rate:

  • light-based sensors (Basis, Pulse) for measuring blood flow using optical technology
  • skin-based electric sensors (Jawbone) for measuring bioimpedance
  • the pressure exerted on a mattress (Beddit, Nokia(Withings) Sleep) due to the contraction of the heart (ballistocardiography).
  • biomotion sensors using radio waves (ResMed S+ and SleepScore Max by ResMed).

Brain Activity

Unfortunately, EEG based sleep tracking is still in its primitive stages, although a few devices have already hit the market:

  • Neuroon is a mask that you wear on (or over) your eyes. It is a fascinating device as it closely mimics PSG by employing sensors that record EEG, EMG and EOG [4]. The creators have graciously open-sourced everything related to the device, which means you can manufacture and code the device at home [5].
  • Other devices include Sleep Shepherd [6], Kokoon, Dreem and Smart Sleep by Philips.

The Case for Wearable Sleep Trackers

Wearable sleep trackers usually come as an all-in-one package, bundling together fitness monitoring, calorie intake and heart activity recording in one stylish device.

An overwhelming majority of wearable trackers come in the form of smart watches that have inbuilt accelerometers that can be used to track your rest and activity cycles.

If for some reason, you are unable (or unwilling) to wear a device on your wrist, alternatives are available that can be worn on the ankle, chest, hips, scalp or as eye masks. Some popular wearable options are given below:

Fitbit Ionic, Charge, Alta-HR and Flex

The Fitbit series of wearables packs sleep tracking abilities into sleek looking devices that also measure other fitness parameters. The Alta-HR also includes a heart rate monitor and a whopping 5-day battery life.

Scholars have conducted several studies to ascertain the accuracy of Fitbit Flex and Charge HR [7, 8]. These devices generally showed good correlation with gold standard protocols in normal adults. Differences arose in individuals suffering from medical conditions such as asthma or depression [9, 10].

Jawbone UP3

The latest device by Jawbone has received mixed reviews from users. These devices are fairly accurate when measuring activity, but seem to have difficulty distinguishing sleep from awake states.

The de Zambotti lab found out that the results of sleep tests in adolescents wearing UP closely approximated those from PSG in measures of total sleep time and wake after sleep onset [11].

The authors, however, recommended exercising caution when extrapolating these results to other age groups. The same lab also reported errors in reporting on nights where sleep was disrupted [12].

In a head-to-head battle between Jawbone UP3, Fitbit Charge HR, four other products and sleep diary, Jawbone’s device seemed to edge Fitbit in reporting outcomes when compared to sleep diaries [13].

Non-wearable Sleep Trackers

Non-wearable sleep trackers provide valuable alternatives to the more conventional wearable trackers. Some popular non-wearable sleep trackers include:

Emfit QS, Beddit, and Withings Aura

These sleep trackers use a particular technique called ballistocardiography, which monitors heart activity by measuring the force exerted on the mattress. They are contactless gadgets that consist of a measuring strip that slides under the side of your mattress.

The Emfit QS is considered one of the most accurate non-wearable sleep trackers. In fact, a study by Kortelainen even suggested it as an alternative to more expensive procedures used for sleep staging [14].

The only drawback is that it can yield unreliable results if you sleep with a partner. They can roll over the measuring strip and mess your readings.

Beddit works on similar principles of ballistocardiography. Dr. Paalasmaa, who works at Beddit, believes that measurement taken using their device is unobtrusive, sleep staging is more precise, and the technique yields respiratory and cardiac information in addition to activity. The drawback compared to actigraphy is that the device cannot measure daytime activity [15].

Beddit was recently acquired by Apple, whereas Withings Aura was acquired in 2016 by Nokia (but sold back to Withings). This shows that the future is promising for these technologies.

SleepMinder, SleepDesign, and SleepScore Max

These devices use specialized low-frequency radio waves to monitor your movements and breathing. Unlike bed-based technologies, they rest on your bed’s side table and take measurements from a distance.

O'Hare and colleagues found such radio-frequency biomotion sensors to be as accurate for determination of sleep/wake cycle as actigraphs. However, they provided slightly improved estimates of total sleep time, sleep-onset latency, and wake-after-sleep onset [16].

Tracking Your Sleep via Mobile Apps

Over the past few years, we have seen mobile applications take over tasks previously only thought possible with tangible devices.

The trend has also crept into sleep tracking, with a number of apps being released for both iPhone and Android devices.

These applications use the array of add-on input modules that come with your mobile devices such as the accelerometer and microphone and use data gathered from them to make certain judgments about your sleep.

To use them, you first have to download them from the relevant software store. Then, every night you have to turn on the app when you lie down to sleep and keep the phone under your pillow or nearby.

Overall, such apps can give you a gist of what sleep tracking is all about, and are perfect for new entrants into the arena. However, there is a huge question mark on their reliability.

A few popular ones are given below:

Sleep Cycle Alarm Clock (Free, iOS, and Android)

This app uses the accelerometer in your mobile phone to track your movements and use them to make assumptions about your sleep. The app also has an added feature that uses the phone’s microphone to monitor breathing.

MotionX 24/7 ($0.99, iOS)

This app uses a patented technology that is generally used in sleep clinics. Like Sleep Cycle, it also uses your phone’s microphone to records breathing patterns.

Researchers from Australia compared Jawbone UP, Actiwatch2 and MotionX 24/7 against PSG in children [17]. They concluded that Jawbone and Actiwatch gave similar results, but advised caution against using 24/7 as it gave highly inaccurate results.

Sleep as Android (Free, Android)

This app integrates with several smartwatches and uses your microphone feature to record snoring, sleep talking or apnea.

Sleep Trackers: To Wear or Not to Wear?

In the past, the general rule of the thumb used to be: the closer the device is to you, the better it will be able to measure your sleep activity.

This notion has now become outdated as new non-wearable technologies hitting the market pass most accuracy tests with flying colors.

But the fact remains: wearable sleep tracking technology, especially in the form of wristwatch-based fitness trackers, becomes a part of your identity and a style statement. Non-wearable technology rarely has that luxury.

Another factor that holds non-wearable devices back is the lack of automatic sleep detection. Most devices in this category have to be turned on before going to sleep, whereas some automatic trackers are already available in the wearable category.

But proponents of non-wearable technology still argue: who wears a wristwatch or any other device when sleeping? They also claim that people often lose interest in wearable technology due to low battery life, and this claim seems to hold with most devices currently in the market.

Depending on your level of sensitivity, wearable sleep trackers can negatively affect your sleep quality depending on your comfort level. This, in turn, defeats the purpose of using portable sleep trackers.

No matter what critics say, wearable sleep tracking is where exciting research is currently focused. For example, researchers from New Zealand have even found out that there is absolutely no difference whether you wear the tracker on your left or right wrist [18]!

Dr. Christopher Winter, a sleep specialist published a personal study in the Huffington Post in early 2014. He designed the study in a way that allowed him to wear Actiwatch Spectrum, Basis Chrome, Jawbone Up, Fitbit Flex, and turn on Motion 24/7 iPhone app while undergoing a PSG [19].

The results of his tests concluded that the 24/7 was the worst regarding performance. Basis tracker was found to be most accurate, while Jawbone and Fitbit devices performed less well.

How Accurate and Reliable are Sleep Trackers?

The accuracy and reliability of sleep trackers is a highly debated topic in the sleep research universe, with some experts considering them the next generation of devices in sleep devices, while others doubt their utility.

The general rule of the thumb is the more modalities a device employs, the more reliable it is. This rule pushes sleep tracking applications to the bottom of the list, as they only use accelerometry to measure movements crudely.

The benchmark study that has been cited quite often by those in favor of these devices is the one by Martin and Hakim, published in 2011 [20].

They concluded that sleep trackers were not meant to replace clinical interviews, sleep diaries, or PSG, but instead could be used to harvest useful insight about sleep in the natural sleep environment.

They further inferred that sleep trackers became especially relevant when doctors prescribed monitoring over an extended period of time.

Cellini et al. conducted a study in 2013 where participants were undergoing PSG concurrently also wore AW-64 and GT3X+ devices. They found both actigraphs to be valid and reliable devices for detecting sleep/wake patterns [21].

Mantua's group went a step further and decided to test five wearable devices: Fitbit Flex, Basis Health Tracker, Withings Pulse O2, Misfit Shine, and a research-based actigraph, Actiwatch Spectrum [22].

Fitbit and Misfit were found to be the least performant of the devices. All devices gave similar readings to PSG in terms of total sleep time.

In contrast, only Actiwatch gave similar readings to PSG when measuring sleep efficiency. None of the devices was able to approximate the PSG concerning light sleep time, whereas just the Basis came close for measuring deep sleep time.

Upcoming Technologies and Crowdfunding Projects

With the US sleep market worth over $28 billion in 2017, innovators are becoming more and more interested in measuring and improving sleep quality.

For example, Karl McCarrison and his friends have come up with a smart pillow that not only measures different parameters related to sleep but is marketed as a sleep management tool and an alarm clock [23].

The guys at Knit health have successfully raised funds for a mountable bedside monitor for babies [24]. Conveniently called the “Knit Monitor", it uses a camera to record a baby’s breathing and movements and uses a smart algorithm to alert the parents if there seems to be a problem.

Another project that has recently entered the production stage is the Oura Ring, a piece of jewelry worn on the fingers that can measure temperature, pulse volume, and movements. Dr de Zambotti’s lab has recently validated the accuracy of Oura ring [25].

Conclusion

Personal sleep trackers have been growing in popularity. However, there are mixed opinions regarding their reliability and accuracy when compared with gold standard procedures such as PSG.

Majority of sleep trackers depend heavily on sensors that detect movement, which itself is a crude indicator of sleep-wake patterns.

However, this does not mean that such devices are useless. They have their own utility in personalized medicine and extended monitoring of sleep parameters.

I would advise all users of these devices to have realistic expectations regarding their use. If you want some insights into your sleep patterns, then sleep trackers are your best bet.

However, if you want sleep trackers to diagnose sleep disorders for you, then you will be massively disappointed as these devices are inferior at doing so.

If you believe you may have a sleep condition that may be negatively affecting your daily life, I would advise you to seek a medical opinion as soon as possible.

References

[1] McCall, C., & McCall, W. V. (2012). Comparison of Actigraphy with Polysomnography and Sleep Logs in Depressed Insomniacs. Journal of Sleep Research, 21(1), 122–127. https://doi.org/10.1111/j.1365-2869.2011.00917.x

[2] Blackwell, T., Redline, S., Ancoli-Israel, S., Schneider, J. L., Surovec, S., Johnson, N. L., … for the Study of Osteoporotic Fractures Research Group. (2008). Comparison of Sleep Parameters from Actigraphy and Polysomnography in Older Women: The SOF Study. Sleep, 31(2), 283–291.

[3] Marino, M., Li, Y., Rueschman, M. N., Winkelman, J. W., Ellenbogen, J. M., Solet, J. M., … Buxton, O. M. (2013). Measuring Sleep: Accuracy, Sensitivity, and Specificity of Wrist Actigraphy Compared to Polysomnography. Sleep, 36(11), 1747–1755. https://doi.org/10.5665/sleep.3142

[4] Neuroon: Personal Sleep Architect

[5] Inteliclinic

[6] Sleep Shepherd Blue: A Sleep Tracker that HELPS You Sleep

[7] Validity of a commercial wearable sleep tracker in adult insomnia disorder patients and good sleepers Kang, Seung-Gul et al. Journal of Psychosomatic Research , Volume 97 , 38 - 44

[8] Lee, H.-A., Lee, H.-J., Moon, J.-H., Lee, T., Kim, M.-G., In, H., … Kim, L. (2017). Comparison of Wearable Activity Tracker with Actigraphy for Sleep Evaluation and Circadian Rest-Activity Rhythm Measurement in Healthy Young Adults. Psychiatry Investigation, 14(2), 179–185. https://doi.org/10.4306/pi.2017.14.2.179

[9] Jessica Castner, Manoj J. Mammen, Carla R. Jungquist, Olivia Licata, John J. Pender, Gregory E. Wilding & Sanjay Sethi (2018) Validation of fitness tracker for sleep measures in women with asthma, Journal of Asthma, DOI: 10.1080/02770903.2018.1490753

[10] Cook, J. D., Prairie, M. L., & Plante, D. T. (2017). Utility of the Fitbit Flex to Evaluate Sleep in Major Depressive Disorder: A comparison against polysomnography and wrist-worn actigraphy. Journal of Affective Disorders, 217, 299–305. https://doi.org/10.1016/j.jad.2017.04.030

[11] De Zambotti, M., Baker, F. C., & Colrain, I. M. (2015). Validation of Sleep-Tracking Technology Compared with Polysomnography in Adolescents. Sleep, 38(9), 1461–1468. https://doi.org/10.5665/sleep.4990

[12] De Zambotti, M., Claudatos, S., Inkelis, S., Colrain, I. M., & Baker, F. C. (2015). Evaluation of a Consumer Fitness-Tracking Device to Assess Sleep in Adults: Evaluation of Wearable Technology to Assess Sleep. Chronobiology International, 32(7), 1024–1028. https://doi.org/10.3109/07420528.2015.1054395

[13] Lee, J.-M., Byun, W., Keill, A., Dinkel, D., & Seo, Y. (2018). Comparison of Wearable Trackers’ Ability to Estimate Sleep. International Journal of Environmental Research and Public Health, 15(6), 1265. https://doi.org/10.3390/ijerph15061265

[14] J. M. Kortelainen, M. O. Mendez, A. M. Bianchi, M. Matteucci and S. Cerutti, "Sleep Staging Based on Signals Acquired Through Bed Sensor," in IEEE Transactions on Information Technology in Biomedicine, vol. 14, no. 3, pp. 776-785, May 2010. doi: 10.1109/TITB.2010.2044797

[15] J. Paalasmaa, M. Waris, H. Toivonen, L. Leppäkorpi and M. Partinen, "Unobtrusive online monitoring of sleep at home," 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, San Diego, CA, 2012, pp. 3784-3788. doi: 10.1109/EMBC.2012.6346791

[16] O’Hare, E., Flanagan, D., Penzel, T. et al. Sleep Breath (2015) 19: 91. https://doi.org/10.1007/s11325-014-0967-z

[17] Toon, E., Davey, M. J., Hollis, S. L., Nixon, G. M., Horne, R. S. C., & Biggs, S. N. (2016). Comparison of Commercial Wrist-Based and Smartphone Accelerometers, Actigraphy, and PSG in a Clinical Cohort of Children and Adolescents. Journal of Clinical Sleep Medicine: JCSM: Official Publication of the American Academy of Sleep Medicine, 12(3), 343–350. https://doi.org/10.5664/jcsm.5580

[18] Driller, M. W., O’Donnell, S., & Tavares, F. (2017). What wrist should you wear your actigraphy device on? Analysis of dominant vs. non-dominant wrist actigraphy for measuring sleep in healthy adults. Sleep Science, 10(3), 132–135. https://doi.org/10.5935/1984-0063.20170023

[19] Personal Sleep Monitors: Do They Work?

[20] Nicola Cellini, Matthew P. Buman, Elizabeth A. McDevitt, Ashley A. Ricker & Sara C. Mednick (2013) Direct comparison of two actigraphy devices with polysomnographically recorded naps in healthy young adults, Chronobiology International, 30:5, 691-698, DOI: 10.3109/07420528.2013.782312

[21] Martin, J. L., & Hakim, A. D. (2011). Wrist Actigraphy. Chest, 139(6), 1514–1527. https://doi.org/10.1378/chest.10-1872

[22] Mantua, J., Gravel, N., & Spencer, R. M. C. (2016). Reliability of Sleep Measures from Four Personal Health Monitoring Devices Compared to Research-Based Actigraphy and Polysomnography. Sensors (Basel, Switzerland), 16(5), 646. https://doi.org/10.3390/s16050646

[23] ThinkPillow :The World’s First Adjustable Smart Pillow

[24] Knit Monitor —Tracks Sleep & Breathing w/o Wearables

[25] De Zambotti, M., Rosas, L., Colrain, I. M., & Baker, F. C. (2017). The Sleep of the Ring: Comparison of the ŌURASleep TrackerAgainst Polysomnography. Behavioral Sleep Medicine, 1–15. Advance online publication. https://doi.org/10.1080/15402002.2017.1300587