Some people who rely on fitness trackers to see how hard they work out may want to rethink this approach, according to a small study that suggests the increasingly popular devices may get more accurate heart rate readings when users are at rest than during exercise.

The study tested four popular wristbands, each of which has a light-emitting diode (LED) that measures heart rate from tiny changes in skin blood volumes by using light reflected from the skin.

Participants in the study – 40 healthy adults – wore two trackers on each wrist and compared resting and exercise heart rate readings on the devices to the gold standard used by doctors: an electrocardiogram (ECG or EKG) test.

At rest, the Fitbit Surge got heart rate measurements that most closely matched the ECG results, and the Basis Peak was furthest off. In tests that also included the Fitbit Charge and Mio Fuse, none of the trackers got exercise heart rate readings that came close to the ECG.

These results suggest that while the trackers may help monitor daily activity, it’s not clear the heart rate readouts would be accurate enough to help patients with certain health problems make medical decisions, the authors note in Annals of Internal Medicine.

“At any moment, the tracker could be off by a fair bit, but at most moments, it won’t be,” said lead study author Lisa Cadmus-Bertram of the University of Wisconsin in Madison.

“This is why our paper doesn’t suggest that the commercial trackers we tested would be sufficient for medical applications where high precision is needed during exercise,” Cadmus-Bertram said by email. “Yet for the typical recreational user, they may still provide feedback that’s useful and motivational.”

To assess the accuracy of the trackers, researchers examined heart rate data for participants who were 49 years old on average and slightly overweight.

First, they looked at the amount of agreement between the readings from the trackers and the ECG tests.

When participants were seated, researchers took readings for the trackers and the ECG tests at one-minute intervals for 10 minutes.

The narrowest range of differences between the trackers and the ECG, indicating the most accuracy, was for the Fitbit Surge. The range for this tracker ranged from an underestimation of 5.1 beats per minute to an overestimation of 4.5 beats per minute.

The widest range of difference at rest was for the Basis Peak, which ranged from an underestimation of 17.1 beats per minute to an overestimation of 22.6 beats per minute.

When participants exercised on a treadmill, the ranges were even wider. The Mio Fuse ranged from an underestimation of 22.5 beats per minute to an overestimation of 26 beats per minute, for example, while the Fitbit Charge range from an underestimation of 41 beats per minute to an overestimation of 36 beats per minute.

The study is small, and researchers found only limited repeatability with results for the same participant under the same conditions.

Still, the findings are an important first step in understanding the clinical validity of wrist trackers many patients already use, said Dr. Daniel Cantillon, a researcher at the Cleveland Clinic in Ohio who wasn’t involved in the study.

“We need data testing these devices among patients with specific disease states, such as heart failure, atrial fibrillation and other chronic medical problems, where it is possible that additional variation will occur with physical activity,” Cantillon said by email.

In particular, patients with the most common heart rhythm disorder, atrial fibrillation, shouldn’t rely on the trackers to detect abnormal rhythms, said Dr. Sumeet Chugh, a researcher at Cedars-Sinai Heart Institute in Los Angeles who wasn’t involved in the study.

“There is a lot at stake here,” Chugh said by email. “When it comes to the use of wrist-worn trackers, we need to be confident of accuracy comparable to treadmill testing if we are going to use the information for patient care.”

A spokesperson for Fitbit told Reuters Health that Fitbit trackers “are not intended to be medical devices” but instead “to give a more informed picture” of overall health. “Extensive internal studies . . . show that Fitbit’s PurePulse technology performs to industry standard expectations for optical heart rate on the wrist,” the spokesperson said.

Mark Gorelick, Chief Science Officer at Mio Global, said in a statement that the company’s technology “helps consumers understand the intensity of their exercise, based on their personal profile and heart rate data, and empowers them to proactively manage their health and reduce risk of lifestyle-related diseases.”