Accuracy and Validity of Commercial Smart Bands for Heart Rate Measurements During Cardiopulmonary Exercise Test
Article information
Abstract
Objective
To assess the accuracies and validities of popular smart bands for heart rate (HR) measurement in cardiovascular disease (CVD) patients during a graded exercise test (GXT).
Methods
Seventy-eight patients were randomly assigned to wear two different smart bands out of three possible choices: Samsung Galaxy Fit 2, Xiaomi Mi Band 5, or Partron PWB-250 on each wrist. A 12-lead exercise electrocardiogram (ECG) and patch-type single-lead ECG were used to assess the comparative HR accuracy of the smart bands. The HR was recorded during the GXT using the modified Bruce protocol.
Results
The concordance correlation coefficients (rc) were calculated to provide a measure of agreement between each device and the ECG. In all conditions, the Mi Band 5 and Galaxy Fit 2’ correlations were rc>0.90, while the PWB-250 correlation was rc=0.58 at rest. When evaluating the accuracy according to the magnitude of HR, all smart bands performed well (rc>0.90) when the HR was below 100 but accuracy tended to decrease with higher HR values.
Conclusion
This study showed that the three smart bands had a high level of accuracy for HR measurements during low-intensity exercise. However, during moderate-intensity and high-intensity exercise, all the three smart bands performed less accurately. Further studies are needed to find a more optimal smart band for HR measurement that can be used for precise HR monitoring during formal cardiac rehabilitation exercise training, including at high and maximal intensity (Clinical Trial Registration No. cris.nih.go.kr/KCT0007036).
INTRODUCTION
Cardiac rehabilitation (CR) is an essential treatment for patients with cardiovascular disease (CVD) for improving exercise capacity and reducing cardiovascular (CV) mortality rate [1–3]. The American Heart Association (AHA) and European Society of Cardiology (ESC) guidelines recommend that all eligible patients diagnosed with acute myocardial infarction, angina pectoris, or peripheral artery disease should participate in CR [4,5]. The CR program evaluates the patient’s condition based on cardiopulmonary exercise (CPX) test results and includes monitoring exercise, risk factor control, and psychosocial management [6].
CR is a necessary treatment for patients with CVD, but there are many barriers to CR, and the overall rate of CR participation is only approximately 30% in Western developed countries [7]. The barriers to CR include time, distance, cost, old age, comorbidities, and lack of knowledge about CR [8]. It is known that center- and home-based CR are equally effective in improving mortality, morbidity, and health-related quality of life outcomes in patients with a low risk of exercise-related CV events [6,9]. Thus, home-based CR can be substituted for center-based CR for patients with a low risk of exercise-related CV events, which could potentially increase CR uptake and adherence [10]. In home-based CR, heart rate (HR) is an important parameter for self-monitoring exercise intensity and can act as a safeguard during exercise [11,12]. Commonly, patients are instructed to check their radial artery and measure how many times the radial pulse beats for 10 seconds, but this method is cumbersome and inconvenient for patients, and tends to be less accurate.
With the recent development of information and communication technology (ICT) and health technology, commercially available HR measuring devices, such as chest straps, smart watches and smart bands have been introduced as assistive tools for HR monitoring and physical activity tracking. Home-based cardiac tele-rehabilitation is now regarded as an alternative method that uses ICT and monitors a patient’s daily physical activity, including exercise; therefore, it is important that HR monitoring devices measure HR accurately [13]. The accuracy and validity of smart watches for HR measurements have been reported [14–21]. However, smart watches are expensive, especially for low-income patients, and are somewhat heavy during exercise and complicated to use for HR monitoring during exercise training.
Newly released cheap smart bands (<50 US dollars) also have an HR measurement function, but there is no previous study on their accuracy for HR measurement during moderate-to-high-intensity exercises in patients with CVD. This study aimed to assess the accuracy and validity of various popular smart bands for HR measurements during a graded exercise test (GXT) in patients with CVD.
MATERIALS AND METHODS
Ethical statement
The protocol of this study was approved by the Institutional Review Board of the Inje University (No. SGPAIK 2021-04-016). All the participants provided written informed consent. The researcher explained the purpose, methods, benefits, and risks of the study to all the patients. Patient privacy and data confidentiality were maintained throughout the study period.
Participants
This prospective randomized controlled study was performed in the outpatient clinic of the cardiac rehabilitation center at Sanggye Paik Hospital between June 2021 and September 2021. The study recruited 78 patients, aged 20 years or older and diagnosed with CVD (myocardial infarction, angina, valvular heart disease, or heart failure) who received acute management, including medication, percutaneous coronary intervention (PCI), coronary artery bypass graft (CABG), or heart valve surgery. All the patients were scheduled for regular follow-up with a GXT as part of their CR program. Exclusion criteria included patients with contraindications to GXT, including acute myocardial infarction or unstable angina, uncontrolled arrhythmias causing symptoms or hemodynamic compromise, and acute endocarditis, myocarditis or pericarditis [22]. Patients with symptomatic severe aortic stenosis, uncontrolled heart failure, asthma, acute pulmonary embolus, suspected dissecting aneurysm, or inability to exercise on the treadmill due to physical problems (i.e., stroke, spinal cord injury, musculoskeletal diseases, or dyspnea) were excluded from the trial. In addition, patients with hypertension (systolic blood pressure >200 mmHg, diastolic blood pressure >120 mmHg), tachyarrhythmias or bradyarrhythmias, and pregnant woman were excluded from the study.
HR measuring devices
To assess the comparative HR measurement accuracy of the smart bands, a 12-lead exercise electrocardiogram (ECG) (GE Cardiac Assessment for Exercise Testing [CASE]; GE Healthcare, Waukesha, WI, USA) and a patchtype single-lead ECG, MobiCARE-MC100 (Seers Technology, Pyeongtaek, Korea) were used. The ECG was considered as the gold standard, while the MobiCARE-MC100 was used as another reference.
For this study, a Galaxy Fit 2 (Samsung, Suwon, Korea), Mi Band 5 (Xiaomi, Beijing, China), and PWB-250 (Partron, Hwaseong, Korea) were selected to verify the HR accuracy because of their popularity, affordability, and commercial availability in South Korea. All three smart bands have an exercise mode that allows real-time HR monitoring and exercise time recording. The Galaxy Fit 2 and Mi Band 5 were released in 2020, while the PWB-250 was released in 2017. All the patients were randomly assigned to wear two different smart bands. Before performing the GXT, the patient wore two different smart bands on each wrist, and the researcher confirmed that the smart bands measured HR in real-time. The HR values were displayed on the mobile phone (Android or iPhone) application for each smart band.
Graded exercise test
The patients performed the GXT on a treadmill using the modified Bruce protocol. In GXT, the slope and speed of the treadmill changed every 3 minutes, and the patients gripped the treadmill handrail to prevent loss of balance or falling when the speed increased. GXT was performed once per patient. A respiratory gas analyzer (Quark CPET; COSMED Co., Rome, Italy), automatic blood pressure (BP) and pulse monitor (TANGO M2; SunTech Medical Inc., Morrisville, NC, USA), and treadmill (T-2100; GE-Marquette Medical Systems, Milwaukee, MI, USA) were used during the GXT. The GXT was terminated at the patient’s request to stop because of subjective symptoms (i.e., severe dyspnea, dizziness, leg fatigue, chest pain, etc.) or upon the attainment of maximal HR or the occurrence of exercise-induced hypotension, exercise-induced hypertension (systolic BP >250 mmHg, diastolic BP >120 mmHg), or abnormal ECG findings (i.e., marked ST segment depression or increasing frequency of ventricular ectopy, new high-grade AV block, sustained ventricular tachycardia, or ventricular fibrillation), according to the AHA guidelines [23].
The GXT consisted of a rest period, exercise period, and recovery period (Fig. 1). The duration of rest period was 6 minutes, and the patients sat on chairs while BP was measured and ECG was monitored. The HR was recorded every two minutes during the rest period. After 6 minutes, the CPX test was begun and HR was recorded every minute during all exercise stages until termination of the test. This was followed by the recovery phase, in which the patients walked slowly for 5 minutes and HR was measured every minute. We also evaluated the metabolic equivalents (METS), oxygen consumption (VO2), rate pressure product (RPP), rate of perceived exertion (RPE), and respiratory exchange ratio (RER).
Timeline of GXT and HR measurement. HR was recorded every two minutes during the rest period. HR was recorded every minute during all exercise stages until termination of the test. HR was recorded every minute during recovery phase. GXT, graded exercise test; HR, heart rate.
Statistical analysis
All data were analyzed using SPSS version 25 (IBM SPSS, Armonk, NY, USA) and expressed as means and standard deviations or numbers and percentages. The HR difference between the ECG and each device was compared by calculating the paired difference, paired absolute difference, percent difference, and absolute percent difference, to assess accuracy. The paired differences were calculated by subtracting HR recorded by exercise ECG (HRECG) from HR recorded by each smart band (HRband), and the paired absolute differences were the absolute values of paired differences. The paired absolute differences denoted the absolute differences between HRECG and HRband. The percent differences were calculated as ([HRECG–HRband]/HRECG)×100), and the absolute percent differences were the absolute values of percent differences.
Lin’s concordance correlation coefficients (rc) were calculated and Bland-Altman analysis was performed to assess the agreement between each device and the ECG. Interpretation of Lin’s concordance correlation coefficients according to McBride was used; rc>0.99 was considered almost perfect, 0.95 to 0.99 was substantial, 0.90 to 0.95 was moderate, and <0.90 was poor [24]. Using paired differences and paired absolute differences, multiple regression analysis was performed to determine whether the HR values were affected by patient factors, including age, sex, height, weight, body mass index (BMI), diagnosed CVD, and use of the smart band on the left or right wrist.
RESULTS
Baseline characteristics of the participants
The baseline characteristics and GXT results of the participants are presented in Table 1. The mean age of the 78 patients (65 men and 13 women) was 59.9 years. The majority of the patients were diagnosed with acute coronary syndrome and underwent PCI. There were no significant differences in the characteristics of the patients when randomized to the different smart bands. In the GXT results, the average attained exercise stage was 5.8, and the average of duration of the exercise period was 15.7 minutes. The averages of maximal oxygen consumption (VO2max) and maximal metabolic equivalents (METsmax) were 24.4 and 7.0, respectively. The reasons for terminating the GXT included subjective maximal dyspnea (42 patients), attainment of maximum HR (11 patients), leg fatigue or pain (8 patients), ST depression on ECG monitoring (5 patients), inability to follow treadmill speed (5 patients), and other reasons, including chest pain, and exercise-induced hypotension or hypertension.
Characteristics of participants
HR monitoring by devices
The differences between the HR monitors and the ECG are presented in Table 2. During the rest period, the absolute percent differences of the Mi Band 5 and Galaxy Fit 2 were less than 3, but that of the PWB-250 was more than 10. During the exercise and recovery periods, the absolute percent difference of all the smart bands was less than 4. Of the three smart bands, the Mi Band 5 had the lowest paired absolute difference and absolute percent difference values in all conditions. Overall, when comparing the smart bands to ECGs, there was a tendency to underestimate HR.
Heart rate monitor differences from ECG
The Mi Band 5 and Galaxy Fit 2 correlations were rc>0.90 in all conditions, while the PWB-250 correlation was rc=0.58 at rest (Table 2, Fig. 2). However, during the exercise and recovery periods, PWB-250’s correlation coefficients were >0.90. Combining all conditions, the correlation coefficients with the HRECG were as follows: Mi Band 5 (rc=0.98), Galaxy Fit 2 (rc=0.96), and PWB-250 (rc=0.97). The MobiCARE-MC100 showed the highest correlation under all conditions (rest, exercise, recovery, rc>0.99).
Concordance correlation coefficients describing agreement of device-measured HR with ECG during GXT. (A) MobiCare-MC 100. (B) Galaxy Fit 2. (C) Mi Band 5. (D) PWB-250. ECG, electrocardiogram; GXT, graded exercise test; HR, heart rate; rc, concordance correlation coefficient.
In the Bland-Altman analysis (Fig. 3), solid horizontal lines indicate the mean of HR differences, while dashed lines indicate the 95% confidence limits of agreement. For the Mi Band 5, the mean difference was −0.1 bpm, and 95% of differences were between −6.8 and 6.6 bpm. For the Galaxy Fit 2 and PWB-250, 95% of differences were between −8.7 and 12.8 bpm (mean difference, 2.1 bpm), and −10.6 and 11.0 bpm (mean difference, 0.2 bpm), respectively. The mean difference of MobiCAREMC 100 was −0.1 bpm, with 95% of differences between −3.1 and 2.9 bpm.
Bland-Altman analysis and 95% limits of agreement with HR measured by ECG. (A) MobiCare-MC 100. (B) Galaxy Fit 2. (C) Mi Band 5. (D) PWB-250. Solid horizontal lines indicated the mean of HR differences, while dashed lines indicated the 95% confidence limits of agreement. ECG, electrocardiogram; HR, heart rate; SD, standard deviation.
The accuracy of each device was evaluated for different HR intervals (Table 3, Fig. 4). The correlation coefficients of all smart bands varied with the HR magnitude. In the HR intervals below 100, the coefficient values of all smart bands were >0.90. In the HR range of 120 to 140, the correlation coefficients with the ECG were as follows: Galaxy Fit 2 (rc=0.36), Mi Band 5 (rc=0.71), and PWB-250 (rc=0.71). The accuracy of all smart bands tended to fall off with higher HR values. The MobiCARE-MC100 showed excellent correlation in all intervals (rc>0.90), except in the HR above 160 interval. In multiple regression analysis, variables including age, sex, height, weight, BMI, and wearing the smart band on the left or right wrist did not influence the HR measurement accuracy.
Concordance correlation coefficient of each HR interval
Concordance correlation coefficient of each HR interval. The accuracy of each device was evaluated for different HR intervals. The HR between 100 and 160 was divided by 20, and a graph of the rc values for each section was depicted. HR, heart rate; rc, concordance correlation coefficient.
DISCUSSION
This study aimed to assess the accuracy and validity of three popular smart bands for HR measurements in patients with CVD during a GXT. When analysis of HR values during each phase of exercise and recovery, all the smart bands had a high level of accuracy in HR measurement (rc>0.90). The Mi Band 5 and Galaxy Fit 2 performed well during the rest period (rc>0.90), though the PWB-250 did not (rc=0.58). The MobiCARE-MC100, another reference, showed the highest correlation during the GXT (rc>0.99).
We also evaluated the accuracy of the smart bands based on the HR magnitude. Exercise intensity was classified based on the percent of the age-based maximum HR [25]. An HR value below 100 was regarded as lowintensity exercise, and all three smart bands performed well in this range (rc>0.90). When the HR of the smart bands was between 100 and 120, which was regarded as moderate-intensity exercise, all three smart bands were less accurate. An HR of 120 or higher was considered to be high-intensity exercise, and all smart bands showed low accuracy in this range. Our findings suggest that all smart bands have limited accuracy in terms of HR measurement during moderate-intensity exercise and above. The MobiCARE-MC100 showed a high level of accuracy, except when the HR was above 160.
Various wearable HR monitoring devices have been developed for exercise and training. Chest strap monitors sense cardiac electrical rhythm through disposable electrodes, and other studies have revealed that chest strap monitoring devices have an acceptable level of accuracy in healthy adults during aerobic exercise [14–17]. Etiwy et al. [19] evaluated the accuracy of a chest strap monitor (Polar H7) in patients with CVD during exercise on a treadmill or stationary cycle and found that in all exercise conditions, the chest strap monitor had the best agreement with the ECG. However, these devices tend to be relatively uncomfortable to use during exercise. MobiCARE-MC100 is an adhesive patch-type single-lead ECG monitoring device which received a certification from the Korea Ministry of Food and Drug Safety, and showed the highest correlation with the 12-lead ECG in this trial. However, this device is more expensive than a smart band, and in some cases the disposable electrodes can fall off due to sweat during exercise, so the electrodes were fixed with a plaster during the GXT in this study.
Other wearable HR monitoring devices include smart bands and smart watches, which are wrist-worn type and based on photoplethysmography (PPG). These devices have a photodetector on the skin surface that measures variations in the blood volume. Many studies associated with wrist-worn monitors have been reported, and they have found that some of the monitors were relatively accurate during exercise in healthy people [14–18]. In addition, there have been a few studies on the HR measurement accuracy of wrist-worn monitors during exercise in patients with CVD [19–21]. Etiwy et al. [19] assessed the accuracy of four wearable HR monitors in patients with CVD during monitoring exercise on a treadmill or stationary cycle. They reported that the Apple Watch was the most accurate among the wearable HR monitors, though its accuracy varied under different exercise conditions. Falter et al. [20] investigated the accuracy of Apple watch measurement of HR during CPX test using a cycle ergometer and found that the Apple Watch measured HR with clinically acceptable accuracy. Heyken et al. [21] evaluated the accuracy of seven different devices during CR training on a bicycle ergometer, and reported that Apple Watch, Mio, and Garmin showed excellent accuracy of HR measurement. Our study differs from other studies by selecting smart band types and models that have not been investigated in previous studies, and analyzing the accuracy of these smart bands according to heart rate magnitude.
In general, smart bands are bands with a sensor and are focused on fitness, while smart watches are a type of wrist-worn watches with a larger screen and more complex functions, similar to a mobile phone, including calling, texting, and emails. Smart bands tend to be cheaper, lighter, and easier to use than smart watches. However, to the best of our knowledge, there have been no studies on the accuracy and validity of using smart bands for measuring HR in patients with CVD. Therefore, we selected three smart bands that were recently released, readily available, and inexpensive. It appears that the use of recently released devices partly contributed to the high accuracy of the smart bands in our study compared to previous studies. Moreover, in this study we assumed that differences in the accuracy of the HR measurements of each smart band resulted from technological differences in the manufacturing methods and algorithms for signal processing of each company. Additional research is needed to compare the HR measurement accuracies of smart bands during monitoring exercises performed in hospitals as part of center-based CR, and further, this method has the potential to be applied to home-based CR. Additionally, wearable devices are updated every year, and the accuracy of the HR measurement function is important, especially in patients with CVD. Thus, a continuous study to verify the accuracy and validity of these new devices is necessary.
This study had several limitations. First, we excluded patients with cardiac pacemakers or arrhythmias (such as atrial fibrillation) due to the limitation of the PPG-oriented HR measurement. Second, during the test, the participants gripped the treadmill handrail. This may not reflect realistic training conditions where there is usually a free-arm motion, and the exercise intensity may be lower than in reality. Third, other types of physical activity, such as biking, climbing, or swimming, were not tested. Moreover, we did not measure the wrist circumference, which is considered an important factor in the accuracy of HR measurement. Finally, we did not measure the HR of patients with each smart band at exactly the same time point. Therefore, further studies are needed to address these limitations and to investigate more updated smart bands.
In conclusion, this study showed that the Mi Band 5, Galaxy Fit 2, and PWB-250 had a high level of accuracy for HR measurements in low-intensity exercise and in the recovery phase of a GXT in patients with CVD. However, during moderate-intensity and high-intensity exercise, the accuracy of all three smart bands tended to decrease; therefore, they should be used for HR monitoring during low-intensity, but not moderate-intensity exercise and above. Further studies are needed to identify more optimal smart bands for HR measurement, so that these can be used for precise HR monitoring during formal CR exercise training, even at high intensity.
Notes
No potential conflict of interest relevant to this article was reported.
Conceptualization: Kim C, Kim SH, Suh MR. Methodology: Kim C, Kim SH. Formal analysis: Kim C, Suh MR. Funding acquisition: Kim C. Project administration: Kim C. Visualization: Kim C, Suh MR. Writing – original draft: Kim C, Suh MR. Writing – review and editing: Kim C, Suh MR. Approval of final manuscript: all authors.