Determining the Reliability of a New Method for Measuring Joint Range of Motion Through a Randomized Controlled Trial

So Young Ahn; Hanbit Ko; Jeong Oh Yoon; Sun Ung Cho; Jong Hyun Park; Kang Hee Cho

doi:10.5535/arm.2019.43.6.707

Ann Rehabil Med > Volume 43(6); 2019 > Article

Ahn, Ko, Yoon, Cho, Park, and Cho: Determining the Reliability of a New Method for Measuring Joint Range of Motion Through a Randomized Controlled Trial

Original Article

Annals of Rehabilitation Medicine 2019;43(6):707-719.

Published online: December 31, 2019

DOI: https://doi.org/10.5535/arm.2019.43.6.707

Determining the Reliability of a New Method for Measuring Joint Range of Motion Through a Randomized Controlled Trial

So Young Ahn, MD, PhD¹

, Hanbit Ko, Dr¹

, Jeong Oh Yoon, BS¹

, Sun Ung Cho, BS¹

, Jong Hyun Park, BS²

, Kang Hee Cho, MD, PhD^1,²

¹Department of Rehabilitation Medicine, Chungnam National University Hospital, Daejeon, Korea

²Biomedical Engineering Center, Chungnam National University College of Medicine, Daejeon, Korea

Corresponding author: Kang Hee Cho Department of Rehabilitation Medicine, Chungnam National University Hospital, 282 Munhwa-ro, Jung-gu, Daejeon 35015, Korea.
Tel: +82-42-338-2460, Fax: +82-42-338-2461, E-mail: khcho@cnu.ac.kr

Received January 22, 2019 Accepted July 2, 2019

This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Objective

To compare the reliability and validity of the Korean range of motion standard protocol (KRSP) for measuring joint range of motion (ROM) with those of the conventional ROM measurement using a goniometer.

Methods

We conducted a randomized controlled trial involving 91 healthy elderly individuals. We compared two strategies of measuring joint ROM to evaluate the reliability and validity of each standardized protocol: first, the KRSP based on the Chungnam National University guidelines and second, handheld goniometric measurement. In the first strategy, 3 examiners (1 rehabilitation doctor, 1 physical therapist, and 1 physical therapy student) independently measured joint ROM in 46 randomly selected subjects; in the second strategy, another 3 examiners (1 rehabilitation doctor, 1 physical therapist, and 1 physical therapy student) measured joint ROM in 45 randomly selected subjects. The reliability of each protocol was calculated using intraclass correlation coefficient, ICC(2,1), and root mean square error (RMSE).

Results

Both protocols showed good to excellent intra-rater reliability. With goniometer use, the inter-rater reliability was low—ICC(2,1), 95% confidence interval ranged from 0.643 (0.486–0.783) to -0.078 (-0.296–0.494)— and RMSE was high. With the KRSP, the inter-rater reliability ranged from 0.846 (0.686–0.931) to 0.986 (0.972–0.994) and RMSE was low.

Conclusion

ROM measurements using the KRSP showed excellent reliability. These results indicate that this protocol can be the reference standard for measuring ROM in clinical settings as an alternative to goniometers.

Keywords: Range of motion, Reference standard, Joints, Measures

INTRODUCTION

Joint range of motion (ROM) is the most common outcome used to evaluate the effect of treatment for musculoskeletal disease. Recognizing limitations in joint ROM is essential to clinicians for making diagnoses, evaluating improvement or deterioration in mobility, and determining functional limitations. Therefore, a reliable and valid measurement tool to objectively measure disease improvement or progression, outcomes, and mobility impairment is of importance [1].

The measurement methods and types of evaluation of ROM vary among clinicians and institutions, including the American Medical Association and McBride guidelines; however, there is no consensus [2-6]. For example, the measurement method for normal ROM provided by the Disability Act and National Pension Act of Korea is based on passive ROM, in which the examiner moves the patient and offers no information about measurement position, the site of applied force, and the degree of pushing force. In a study by Cho et al. [7] regarding ankle ROM, a significant correlation was found between ankle ROM and pushing force; however, ROM was measured using different techniques according to the knee joint position and sex. Therefore, it is crucial to establish a standard reference for joint ROM measurement considering the pushing force (site and amount) and measurement position.

Classically, a standard goniometer for measuring joint ROM is the gold standard in clinical settings because it is portable and relatively inexpensive. However, it has several limitations, making it difficult for clinicians to use. Clinicians need both hands to use a goniometer, making limb stabilization difficult. Thus, the risk of a high measurement error increases [8]. An inclinometer is a tool that uses the constant of gravity as a reference point to evaluate joint ROM. Digital inclinometers are easy to use, portable, and lightweight. However, they are expensive compared to goniometers. Moreover, to minimize the risk of measurement errors, examiners have to accurately and always set the zero point before use [9,10].

Different authors have reported a range of reliability of ROM measurements. Trijffel et al. [11] reviewed previous studies of inter-rater reliability for measuring passive ROM of the lower extremities using different measurement tools. They suggested that clinicians should be cautious when relying on results obtained through measurement of passive movements in joints for making decisions because of their low reliability. Therefore, it is necessary to establish standards protocols for ROM to achieve consistent and precise values based on the measurement method, position, site, and degree of force.

The aim of this study was to compare the reliability and validity of the newly developed Korean ROM standard protocol (KRSP) developed based on the Chungnam National University guidelines in measuring ROM with those of the conventional ROM measurement using a goniometer.

MATERIALS AND METHODS

We conducted a randomized controlled trial that included healthy elderly subjects recruited from the Department of Rehabilitation Medicine at Chungnam National University Hospital, Korea. A computer-generated randomization sequence was generated. The reliability of joint ROM was evaluated with the two measurement methods. The study was approved by the Chungnam National University Ethics Committee (No. 2016-07-014) and was performed in accordance with the protocol; all subjects provided written informed consent for participating in this study. However, the subjects did not know whether they were in the experimental or comparison group (Fig. 1). The KRSP is described in detail in Supplementary Tables S1–S5.

Participants and sample size

A total of 101 healthy elderly volunteers aged ≥65 years were recruited in the initial validation study. Before measuring ROM, the body mass index of all subjects was measured, and radiology and laboratory studies were conducted to rule out diseases affecting ROM. Those with a history of orthopedic surgery due to fracture or other diseases; with limited joint ROM due to severe osteoarthritis or rheumatoid arthritis, congenital defects, structural disease, amputation, and central or peripheral nervous system injuries; who were unable to participate due to their poor general condition; or who had difficulty in voluntary decision-making due to cognitive impairment were excluded. Participants were randomly allocated to the KRSP group or goniometer group.

A sample size of 41 subjects with three observations per subject achieves 81% power to detect an intraclass correlation of 0.80 under the alternative hypothesis when the intraclass correlation under the null hypothesis is 0.65 using an F-test with a significance level of 0.05. Therefore, 101 participants were recruited in consideration of a 15% dropout rate.

Raters

Six examiners (2 rehabilitation doctors, 2 experienced physical therapists, and 2 physical therapy students) participated. Three examiners (1 rehabilitation doctor, 1 experienced physical therapist, and 1 physical therapy student) performed ROM measurements using the KRSP. The other 3 examiners used a goniometer for ROM measurement. All examiners reviewed the testing procedures of the KRSP and its use prior to actual testing. To prevent the occurrence of systematic differences between the examiners due to repeated testing, the sequence of the examiners was randomly determined. After a warm-up period and 3 familiarization trials for each measurement, which was the same in both groups, the testing procedure started at the subject’s dominant side. The interval between the measurements was at least 1 hour. Each examiner performed 3 additional measurements for further analysis of correlation. Data for each measurement were recorded by an independent recorder on separate data sheets so that the examiner was unable to view any measurements from previous encounters with each subject.

Procedure

KRSP group

ROM was measured according to change in the pushing force (upper limb: 0 kg, 2 kg of pushing force; lower limb: 0 kg, 4 kg). Zero kg of pushing force means full active ROM. All measurements were conducted according to the KRSP guidelines described in Supplementary Tables S1–S5.

Goniometer group

ROM was measured using a standard goniometer marked in increments of 1, with 2 adjustable overlapping arms. The measurement position was the same as that used in the KRSP guidelines. Only active ROM was measured because both the examiner’s hands were already occupied.

Statistical analysis

All statistical analyses were performed using IBM SPSS version 22 (IBM Corp, Armonk, NY, USA). Mean and standard deviation for ROM was calculated. All dependent variables demonstrated normal distribution (Kolmogorov-Smirnov test), and parametric tests were applied.

The independent t-test was used to compare interdevice differences. Relative reliability was assessed using intraclass correlation coefficient, ICC(2,1) and corresponding 95% confidence interval (CI). For intra-rater reliability, 3 trials for each measurement were compared. For inter-rater reliability, the mean of 3 trials was used, where the measurement from 3 independent examiners was compared to determine whether the particular instrument (in this case the inclinometer and goniometer) can be used with confidence and reliability. Absolute reliability was determined by calculating the root mean square error (RMSE) between measures and between examiners.

ICC interpretation was based on the guidelines by Fleiss, according to whom high (>0.90) values reflect excellent reliability, values between 0.80 and 0.89, good; between 0.70 and 0.79, moderate; and <0.70, low [1]. Acceptable reliability was set a priori at ≥0.70. A p-value of less than 0.05 was considered significant.

RESULTS

A total of 91 participants were enrolled. Among 101 volunteers, 5 were excluded: 2 due to surgery, 1 due to limited ROM, 1 due to cognitive impairment, and 1 failed to undergo the measurement. Another 5 did not complete the measurement. Participant characteristics are listed in Table 1. ROM measurements for all 6 examiners are listed in Tables 2 and 3. The average of the measurements taken by the two instruments by different examiners was calculated to investigate any significant differences. The independent t-test showed a statistically significant difference between the groups in all measurements except shoulder extension, abduction, wrist extension, and ankle eversion. The inter- and intra-rater reliability data for the two measurement modalities among all 6 examiners are reported in Tables 4–8.

Intra-rater reliability

Both groups showed good to excellent intra-rater reliability. In the goniometer group, ICC(2,1) (95% CI) ranged from 0.850 (0.775–0.906) for hip flexion found by examiner F to 0.984 (0.975–0.990) for elbow flexion found by examiner F (Table 4). RMSE ranged from 1.483 (hip extension found by examiner F) to 3.268 (hip flexion found by examiner D) (Table 7). In the KRSP group, ICC(2,1) (95% CI) ranged from 0.985 (0.977–0.991) for active hip abduction found by examiner B to 0.999 (0.999–0.999) for passive shoulder flexion found by examiner A (Table 5). RMSE ranged from 0.543 (passive hip external rotation found by examiner A) to 3.268 (active wrist flexion found by examiner A) (Table 7).

Inter-rater reliability

In the goniometer group, the inter-rater reliability was low and ICC(2,1) (95% CI) ranged from 0.643 (0.486–0.783) for hip external rotation to -0.078 (-0.296–0.494) for ankle dorsiflexion with knee flexion 0° (Table 6). Higher RMSE was observed, ranging from 5.650 (shoulder flexion) to 9.342 (shoulder flexion) (Table 8). The KRSP group showed excellent inter-rater reliability; ICC(2,1) (95% CI) ranged from 0.846 (0.686–0.931) for active plantar flexion with 90° knee flexion to 0.986 (0.972–0.994) for active shoulder abduction (Table 6). RMSE ranged from 1.431 (active hip internal rotation) to 2.993 (passive hip external rotation) (Table 8).

DISCUSSION

Accurate and consistent measurement of joint ROM is important for physical examination and disability evaluation. In the current healthcare scenario, accurate medical recording has become critical [9]. The ability of healthcare providers to effectively communicate a subject’s condition or improvement is significant. However, measurement methods are extremely diverse, with no consensus or guidelines regarding measurement tools, joint positions, and pushing force.

Therefore, in this study, we developed an innovative method for measuring joint ROM. A standardized protocol (KRSP) applied to each joint was completed through preliminary experiments and consultation from experts. Our randomized controlled trial involving healthy elderly participants displayed excellent reliability of the new protocol for measuring joint ROM. To our knowledge, this is the first study to perform a reliability evaluation in a randomized controlled trial.

Several factors affect joint ROM measurement. Extrinsic factors include type of measurement (active vs. passive), site, amount of pushing force in passive ROM, movement of adjacent joints or muscles, measurement instrument, and skill level of the examiner. Intrinsic factors include sex, obesity, soft tissue condition (muscle mass, tendon length, and tissue viscoelasticity), and physical activity (exercise and/or occupation) [7,9,12]. While intrinsic factors are difficult to control, extrinsic factors can be controlled using a standardized protocol. In this study, we developed a highly reliable ROM measurement protocol by defining the measurement tools, measurement method, amount of pushing force, and position of pushing force.

There have been several new attempts to accurately evaluate ROM measurements. Some studies have measured joint ROM using an optical motion capture system to obtain more objective and error-free values [13]. This device has excellent accuracy, but it is expensive and difficult to use and apply in clinical settings. More recently, new motion analysis devices such as inertial sensors and Kinect have been introduced [14-17]. However, skilled technicians are needed to operate these devices; more importantly, their reliability has not been proven. Several studies have reported acceptable reliability of ROM measurement using a smartphone application [18-22]. This method is easy to use and inexpensive but cannot control various compensatory movements that occur during joint measurements, which can lead to poor reliability in joints with multiplanar motion.

Regarding the KRSP, we tried to standardize the measurement methods. The measurement position, site of sensor attachment, and site of pushing force were clearly defined. Several straps and molded devices were fabricated to prevent motion of adjacent body parts. During shoulder rotation measurement, a fabricated acrylic panel was set to maintain shoulder abduction at 90°. In case of the hip joint, we faced many difficulties in controlling measurement due to the variety of motion directions and variables. During hip flexion evaluation, a pelvic strap was used to prevent lumbar flexion. When evaluating knee flexion, a hip strap was used to prevent the pelvis from lifting up. To evaluate ankle motion at 90° knee flexion, a rectangular acrylic plate with a string was prepared to fix the lower leg.

Another issue is that the inclinometer is attached to the patient’s arm, as was the case in our study, and both the examiner’s hands are free. Therefore, the axis of the patient’s arm can be fixed with one hand, and the other hand can deliver pressure to the arm while measuring passive ROM. This led to an excellent intra- and interrater reliability of the KRSP.

Several studies have investigated the inter-rater reliability of digital inclinometers for the measurement of joint ROM. Hoving et al. [23] reported ICCs of 0.11 to 0.80 in a study of shoulder joints of 6 healthy volunteers and 6 examiners using a digital inclinometer. A study by de Winter et al. [24] found ICCs of 0.28 to 0.90 for 155 patients with 2 examiners using a similar device. Our study showed high inter-rater ICC(2,1) ranging from 0.846 to 0.986. Regarding the intra-rater reliability of the KRSP, the average ICC(2,1) was 0.995 for the rehabilitation doctor, 0.994 for the physical therapist and 0.995 for the physical therapy student. Interpreting this with the results of inter-rater reliability, the KRSP showed excellent reliability, independent of the operator’s skill level, with high correlation coefficients for the averaged measures of all 3 examiners.

Previous studies have shown various ICCs for goniometers. For shoulder ROM measurement, Hayes et al. [25] reported inter-rater ICCs of 0.64–0.69 and intra-rater ICCs of 0.53–0.65 in standard goniometry. Shin et al. [19] reported slightly higher ICCs in standard goniometer among 3 examiners for both active (ICC range, 0.67–0.91; average ICC, 0.80) and passive (ICC range, 0.64–0.90; average ICC, 0.79) shoulder ROM. For elbow, wrist, knee, and ankle ROM measurement, authors have reported a moderate to high level of intra-rater reliability associated with the universal goniometer [26-31]. Rothstein et al. [26] showed that intra-rater reliability for knee flexion and extension and the elbow joints was high (ICC 0.91–0.99). Inter-rater reliability was also high (ICC 0.88–0.97), except for knee extension (ICC 0.63–0.70). Horger [30] found that measurement of wrist motion using a goniometer was highly reliable and that intra-rater reliability was higher than inter-rater reliability for all active and passive movements. In a study by Youdas et al. [28] focusing on the reliability of goniometric measurement of ankle ROM, in terms of the intra-rater reliability of measurements obtained with a goniometer, the ICC was 0.64 to 0.92 (median, 0.825) for ankle dorsiflexion and 0.47 to 0.96 (median, 0.865) for ankle plantarflexion. Inter-rater ICCs for measurements were 0.28 for ankle dorsiflexion and 0.25 for ankle plantarflexion. For the hip, a range of ICCs were reported. Herrero et al. [32] reported excellent (>0.80) intra-rater reliability and low (0.375 and 0.475) inter-rater reliability. Poulsen et al. [33] found moderate inter-rater reliability for goniometric hip ROM measurement.

Compared with previous studies, our analysis incorporated more examiners with varying skills and a larger sample size, further strengthening our results. The results of the studies by Youdas et al. [28] and Herrero et al. [32] are in agreement with those of our study regarding acceptable intra-rater reliability in repeated measures of joint ROM, independent of the skill level of the operator and low inter-rater reliability. However, we found that goniometer use had higher intra-rater reliability and poorer inter-rater reliability. The reason for the high intrarater reliability is that the examiners remembered every measurement, because there was not enough interval between measurements. Regarding the low inter-rater reliability; the potential sources of error include location of the axis of motion, inconsistent patient positioning and goniometric alignment, inability or disinclination of the patient to accurately reproduce active joint motion, and inconsistent identification or alteration of bony landmarks used to guide measurement [29].

Joint ROM measurement using a conventional goniometer should be made by the same examiner because considerable disagreement exists when measurements are made by different examiners. This disagreement could erroneously influence the clinical decision-making process. Due to lack of reliability of the conventional goniometer currently in use, clinicians should be cautious when interpreting results, especially those obtained by different examiners.

This study had some limitations. First, this study was performed with elderly participants. It is possible that more significant age-related effects would have become apparent if more participants were measured or if the cohort was younger. Second, this protocol is limited to the healthy participants studied, so the results cannot be extrapolated to other injured populations. Third, the intrarater relativity was relatively high. This may because the 3 sequential measurements were made at the same time in a day and the examiner memorized the measured values. If we evaluate ROM at longer time intervals, the intrarater reliability may be lowered, especially for measurements made according to a non-standardized protocol.

The newly developed highly reliable measurement method can be used as reference standard for ROM measurement in clinical settings. Moreover, standardized and consistent evaluations of ROM would be possible, regardless of examiners. Further studies for simplification as well as computerization of the developed protocol will make it more useful for application in clinical settings.

To conclude, ROM measurement using the goniometer currently in use showed poor inter-rater reliability and therefore cannot be used as an objective physical evaluation. ROM measurements according to the newly developed KRSP showed excellent inter-rater and intra-rater reliability. These results indicate that the KRSP can be the reference standard protocol for measuring ROM in the clinical setting as an alternative to a goniometer. Further research is required to enable better clinical application of this protocol.

CONFLICTS OF INTEREST

No potential conflict of interest relevant to this article was reported.

AUTHOR CONTRIBUTION

Conceptualization: Ko H, Ahn SY. Methodology: Ahn SY, Ko H, Yoon JO, Cho SU, Park JH. Formal analysis: Ahn SY. Writing – original draft: Ahn SY. Writing – review and editing: Cho KH. Approval of final manuscript: all authors.

ACKNOWLEDGMENTS

This research is supported by “The Foundation Assist Project of Future Advanced User Convenience Service” through the Ministry of Trade, Industry and Energy (MOTIE) (R0004840, 2019).

SUPPLEMENTARY MATERIALS

Supplementary materials can be found via https://doi.org/10.5535/arm.2019.43.6.707. Table S1. Measurement position. Table S2. Inclinometer measurement: sensor attachment, site of applying pushing force (upper limb). Table S3. Inclinometer measurement: sensor attachment, site of applying pushing force (lower limb). Table S4. Measurement protocol for upper limb range of motion. Table S5. Measurement protocol for lower limb range of motion.

arm-2019-43-6-707-suppl1.pdf

arm-2019-43-6-707-suppl2.pdf

arm-2019-43-6-707-suppl3.pdf

arm-2019-43-6-707-suppl4.pdf

arm-2019-43-6-707-suppl5.pdf

Fig. 1.

Flow diagram of the randomized controlled trial. LOM, range of motion; KRSP, Korean range of motion standard protocol.

Table 1.

Characteristics of participants

	KRSP (n=45)	Goniometer (n=46)
Gender
Man	18	13
Women	27	33
Age (yr)	71.8±4.9	72.4±4.0
Body mass index (kg/m²)	25.2±2.6	24.7±3.0

Values are presented as mean±standard deviation.

KRSP, Korean range of motion standard protocol.

Table 2.

Joint range of motion of upper extremity

Tested movements	KRSP (º)	Goniometer (º)	Mean difference
Shoulder
Flexion_A	176.3±9.2	167.6±7.0	8.7^**
Flexion_P	184.2±8.3	-	-
Extension_A	42.5±8.9	41.5±5.8	1.0
Extension_P	44.1±8.5	-	-
Abduction_A	166.7±16.6	171.3±7.9	4.6
Abduction_P	178.4±14.3	-	-
IR 90°_A	87.2±13.2	66.6±7.6	20.6^**
IR 90°_P	99.4±11.1	-	-
ER 90°_A	84.7±12.2	78.1±8.1	6.6^*
ER 90°_P	92.8±10.4	-	-
ER 0°_A	81.0±10.9	70.2±8.1	10.8^**
ER 0°_P	92.8±10.6	-	-
Elbow
Flexion_A	136.9±5.9	133.9±4.5	3.0^*
Flexion_P	144.5±6.8	-	-
Wrist
Flexion_A	66.5±11.8	52.2±6.2	14.3^**
Flexion_P	77.8±11.2	-	-
Extension_A	63.2±10.1	61.3±6.7	1.9
Extension_P	69.8±9.1	-	-

Values are presented as mean±standard deviation.

KRSP, Korean range of motion standard protocol; A, active; P, passive; IR n°, internal rotation with shoulder abduction n°; ER n°, external rotation with shoulder abduction n°.

^* p<0.01,

^** p<0.001.

Table 3.

Joint range of motion of lower extremity

Tested movements	KRSP (º)	Goniometer (º)	Mean difference
Hip
Flexion_A	111.9±9.6	123.3±6.3	11.4^**
Flexion_P	119.3±10.2
Extension_A	22.7±5.5	28.7±4.8	6.0^**
Extension_P	28.6±6.9
Abduction_A	36.4±5.7	45.4±6.6	9.1^**
Abduction_P	45.7±7.8
Adduction_A	36.6±5.9	41.7±5.4	5.2^**
Adduction_P	45.6±6.9
IR_A	28.7±9.8	33.9±7.6	5.1^*
IR_P	40.2±11.5
ER_A	57.5±8.7	50.7±8.7	6.8^**
ER_P	69.3±9.0
Knee
Flexion_A	115.2±10.5	130.1±6.7	14.9^**
Flexion_P	124.5±11.2
Ankle
PF 0°_A	39.7±6.5	48.4±6.4	8.7^**
PF 0°_P	46.5±7.4
DF 0°_A	28.7±6.2	22.5±3.3	6.2^**
DF 0°_P	35.6±6.7
PF 90°_A	43.3±6.8	51.7±6.2	8.4^**
PF 90°_P	51.2±6.1
DF 90°_A	35.2±6.5	30.4±5.0	4.7^**
DF 90°_P	42.1±7.0
Inversion_A	32.6±7.6	22.2±4.8	10.4^**
Inversion_P	39.9±10.0
Eversion_A	18.6±5.4	17.7±3.9	0.9
Eversion_P	24.6±5.7

Values are presented as mean±standard deviation.

KRSP, Korean range of motion standard protocol; A, active; P, passive; IR n°, internal rotation with shoulder abduction n°; ER n°, external rotation with shoulder abduction n°; PF n°, plantarflexion with knee flexion n°; DF n°, dorsiflexion with knee flexion n°.

^* p<0.01,

^** p<0.001.

Table 4.

Intra-rater reliability (upper extremity)

	KRSP			Goniometer
	Examiner A	Examiner B	Examiner C	Examiner D	Examiner E	Examiner F
Shoulder
Flexion_A	0.999 (0.998–0.999)	0.998 (0.997–0.999)	0.999 (0.998–0.999)	0.954 (0.926–0.972)	0.955 (0.929–0.972)	0.955 (0.929–0.972)
Flexion_P	0.999 (0.999–0.999)	0.998 (0.996–0.999)	0.999 (0.998–0.999)	-	-	-
Extension_A	0.995 (0.992–0.997)	0.998 (0.996–0.999)	0.998 (0.997–0.999)	0.932 (0.895–0.958)	0.914 (0.868–0.947)	0.961 (0.940–0.976)
Extension_P	0.997 (0.994–0.998)	0.997 (0.996–0.998)	0.999 (0.998–0.999)	-	-	-
Abduction_A	0.999 (0.999–1.000)	0.999 (0.998–0.999)	0.999 (0.999–1.000)	0.945 (0.914–0.966)	0.977 (0.964–0.986)	0.945 (0.914–0.966)
Abduction_P	0.999 (0.999–0.999)	0.998 (0.997–0.999)	0.999 (0.999–1.000)	-	-	-
IR 90°_A	0.998 (0.997–0.999)	0.998 (0.996–0.999)	0.999 (0.998–0.999)	0.940 (0.907–0.963)	0.950 (0.922–0.969)	0.944 (0.913–0.966)
IR 90°_P	0.997 (0.996–0.998)	0.996 (0.994–0.998)	0.999 (0.999–0.999)	-	-	-
ER 90°_A	0.998 (0.997–0.999)	0.998 (0.996–0.999)	0.999 (0.998–0.999)	0.897 (0.843–0.936)	0.969 (0.952–0.981)	0.963 (0.943–0.978)
ER 90°_P	0.998 (0.996–0.999)	0.998 (0.996–0.999)	0.999 (0.998–0.999)	-	-	-
ER 0°_A	0.998 (0.996–0.999)	0.998 (0.997–0.999)	0.999 (0.998–0.999)	0.896 (0.841–0.936)	0.971 (0.955–0.983)	0.918 (0.874–0.950)
ER 0°_P	0.997 (0.995–0.998)	0.998 (0.997–0.999)	0.999 (0.998–0.999)	-	-	-
Elbow
Flexion_A	0.998 (0.996–0.999)	0.996 (0.993–0.997)	0.995 (0.993–0.997)	0.917 (0.872–0.949)	0.919 (0.875–0.950)	0.850 (0.775–0.906)
Flexion_P	0.998 (0.997–0.999)	0.997 (0.995–0.998)	0.997 (0.996–0.998)	-	-	-
Wrist
Flexion_A	0.975 (0.960–0.985)	0.998 (0.997–0.999)	0.999 (0.998–0.999)	0.947 (0.872–0.968)	0.961 (0.939–0.976)	0.943 (0.911–0.965)
Flexion_P	0.999 (0.999–0.999)	0.998 (0.997–0.999)	0.999 (0.998–0.999)	-	-	-
Extension_A	0.999 (0.998–0.999)	0.998 (0.996–0.999)	0.999 (0.998–0.999)	0.895 (0.840–0.935)	0.935 (0.964–0.986)	0.933 (0.896–0.959)
Extension_P	0.999 (0.998–0.999)	0.997 (0.996–0.998)	0.999 (0.998–0.999)	-	-	-

Values are intraclass correlation coefficient (95% confidence interval).

KRSP, Korean range of motion standard protocol; A, active; P, passive; IR n°, internal rotation with shoulder abduction n°; ER n°, external rotation with shoulder abduction n°.

Table 5.

Intra-rater reliability (lower extremity)

	KRSP			Goniometer
	Examiner A	Examiner B	Examiner C	Examiner D	Examiner E	Examiner F
Hip
Flexion_A	0.997 (0.996–0.998)	0.996 (0.994–0.998)	0.997 (0.995–0.998)	0.985 (0.977–0.991)	0.890 (0.886–0.894)	0.984 (0.975–0.990)
Flexion_P	0.998 (0.996–0.999)	0.996 (0.994–0.998)	0.998 (0.996–0.998)	-	-	-
Extension_A	0.993 (0.987–0.995)	0.993 (0.988–0.996)	0.993 (0.989–0.996)	0.953 (0.927–0.972)	0.956 (0.932–0.973)	0.978 (0.965–0.986)
Extension_P	0.993 (0.989–0.996)	0.992 (0.987–0.995)	0.993 (0.989–0.996)	-	-	-
Abduction_A	0.992 (0.987–0.995)	0.985 (0.977–0.991)	0.992 (0.988–0.995)	0.956 (0.931–0.973)	0.923 (0.881–0.952)	0.924 (0.883–0.953)
Abduction_P	0.993 (0.988–0.996)	0.988 (0.981–0.993)	0.993 (0.989–0.996)	-	-	-
Adduction_A	0.991 (0.985–0.994)	0.991 (0.986–0.995)	0.992 (0.987–0.995)	0.956 (0.932–0.973)	0.938 (0.904–0.962)	0.928 (0.889–0.956)
Adduction_P	0.993 (0.988–0.996)	0.990 (0.985–0.994)	0.994 (0.990–0.998)	-	-	-
IR_A	0.996 (0.994–0.998)	0.996 (0.993–0.997)	0.997 (0.994–0.998)	0.963 (0.942–0.977)	0.961 (0.939–0.976)	0.966 (0.947–0.979)
IR_P	0.997 (0.994–0.998)	0.996 (0.994–0.998)	0.997 (0.995–0.998)	-	-	-
ER_A	0.996 (0.994–0.998)	0.993 (0.989–0.996)	0.994 (0.991–0.997)	0.964 (0.943–0.978)	0.936 (0.901–0.961)	0.954 (0.928–0.972)
ER_P	0.997 (0.995–0.998)	0.994 (0.991–0.997)	0.995 (0.998–0.997)	-	-	-
Knee
Flexion_A	0.997 (0.996–0.998)	0.997 (0.995–0.998)	0.998 (0.997–0.999)	0.894 (0.890–0.896)	0.991 (0.985–0.994)	0.991 (0.986–0.995)
Flexion_P	0.997 (0.995–0.998)	0.997 (0.996–0.998)	0.998 (0.997–0.999)	-	-	-
Ankle
PF 0°_A	0.993 (0.989–0.996)	0.990 (0.985–0.994)	0.993 (0.988–0.996)	0.946 (0.916–0.967)	0.921 (0.878–0.951)	0.925 (0.885–0.954)
PF 0°_P	0.993 (0.989–0.996)	0.991 (0.986–0.995)	0.992 (0.988–0.995)	-	-	-
DF 0°_A	0.995 (0.992–0.997)	0.992 (0.988–0.995)	0.993 (0.988–0.996)	0.979 (0.966–0.987)	0.964 (0.943–0.978)	0.980 (0.968–0.988)
DF 0°_P	0.995 (0.992–0.997)	0.993 (0.988–0.996)	0.993 (0.988–0.996)	-	-	-
PF 90°_A	0.993 (0.989–0.996)	0.992 (0.987–0.995)	0.993 (0.988–0.996)	0.944 (0.913–0.966)	0.881 (0.819–0.951)	0.925 (0.884–0.954)
PF 90°_P	0.992 (0.987–0.995)	0.990 (0.984–0.994)	0.991 (0.985–0.994)	-	-	-
DF 90°_A	0.993 (0.988–0.996)	0.991 (0.986–0.995)	0.992 (0.988–0.995)	0.968 (0.950–0.981)	0.938 (0.903–0.962)	0.978 (0.965–0.987)
DF 90°_P	0.994 (0.989–0.996)	0.991 (0.986–0.995)	0.993 (0.988–0.996)	-	-	-
Inversion_A	0.994 (0.990–0.996)	0.990 (0.983–0.994)	0.995 (0.993–0.997)	0.883 (0.873–0.890)	0.961 (0.938–0.976)	0.976 (0.962–0.985)
Inversion_P	0.997 (0.995–0.998)	0.994 (0.991–0.996)	0.998 (0.996–0.998)	-	-	-
Eversion_A	0.994 (0.990–0.996)	0.991 (0.986–0.995)	0.994 (0.990–0.998)	0.989 (0.983–0.993)	0.973 (0.957–0.984)	0.983 (0.973–0.990)
Eversion_P	0.993 (0.990–0.996)	0.991 (0.986–0.995)	0.989 (0.982–0.931)	-	-	-

Values are intraclass correlation coefficient (95% confidence interval).

KRSP, Korean range of motion standard protocol; A, active; P, passive; IR, internal rotation; ER, external rotation; PF n°, plantarflexion with knee flexion n°; DF n°, dorsiflexion with knee flexion n°.

Table 6.

Inter-rater reliability

	KRSP	Goniometer
Shoulder
Flexion_A	0.956 (0.911–0.980)	0.320 (-0.360–0.690)
Flexion_P	0.969 (0.936–0.986)	-
Extension_A	0.958 (0.915–0.981)	0.243 (-0.515–0.654)
Extension_P	0.973 (0.946–0.988)	-
Abduction_A	0.986 (0.972–0.994)	0.322 (-0.356–0.691)
Abduction_P	0.986 (0.971–0.994)	-
IR 90°_A	0.975 (0.950–0.989)	0.197 (-0.607–0.634)
IR 90°_P	0.968 (0.969–0.986)	-
ER 90°_A	0.975 (0.949–0.989)	0.308 (0.016–0.776)
ER 90°_P	0.978 (0.954–0.990)	-
ER 0°_A	0.979 (0.956–0.990)	0.578 (0.155–0.807)
ER 0°_P	0.974 (0.947–0.988)	-
Elbow
Flexion_A	0.901 (0.799–0.956)	0.516 (-0.768–0.757)
Flexion_P	0.920 (0.838–0.964)	-
Wrist
Flexion_A	0.968 (0.934–0.986)	0.310 (-0.281–0.648)
Flexion_P	0.978 (0.955–0.990)	-
Extension_A	0.947 (0.893–0.977)	0.355 (-0.290–0.709)
Extension_P	0.970 (0.938–0.986)	-
Hip
Flexion_A	0.961 (0.921–0.983)	0.017 (-0.435–0.217)
Flexion_P	0.955 (0.909–0.980)	-
Extension_A	0.954 (0.888–0.975)	0.293 (-0.615–0.632)
Extension_P	0.957 (0.913–0.981)	-
Abduction_A	0.941 (0.881–0.974)	0.311 (-0.378–0.686)
Abduction_P	0.960 (0.919–0.982)	-
Adduction_A	0.873 (0.741–0.982)	0.589 (0.177–0.812)
Adduction_P	0.941 (0.880–0.974)	-
IR_A	0.984 (0.966–0.974)	0.629 (0.258–0.831)
IR_P	0.985 (0.970–0.993)	-
ER_A	0.977 (0.953–0.990)	0.643 (0.486–0.783)
ER_P	0.938 (0.875–0.972)	-
Knee
Flexion_A	0.971 (0.942–0.987)	0.483 (-0.034–0.764)
Flexion_P	0.979 (0.957–0.990)	-
Ankle
PF 0°_A	0.846 (0.686–0.931)	0.177 (-0.443–0.524)
PF 0°_P	0.952 (0.902–0.979)	-
DF 0°_A	0.951 (0.901–0.978)	-0.078 (-0.296–0.494)
DF 0°_P	0.947 (0.892–0.976)	-
PF 90°_A	0.960 (0.919–0.982)	0.387 (-0.025–0.766)
PF 90°_P	0.951 (0.899–0.978)	-
DF 90°_A	0.970 (0.938–0.986)	0.224 (-0.153–0.637)
DF 90°_P	0.962 (0.923–0.983)	-
Inversion_A	0.958 (0.914–0.981)	-0.109 (-0.039–0.307)
Inversion_P	0.974 (0.947–0.988)	-
Eversion_A	0.960 (0.918–0.982)	-0.019 (-0.661–0.513)
Eversion_P	0.939 (0.876–0.970)	-

Values are intraclass correlation coefficient (95% confidence interval).

Table 7.

Intra-rater reliability using RMSE

	KRSP			Goniometer
	Examiner A	Examiner B	Examiner C	Examiner D	Examiner E	Examiner F
Shoulder
Flexion_A	0.663	0.605	0.655	2.675	1.915	2.158
Flexion_P	0.602	0.742	0.654
Extension_A	0.736	0.789	0.812	2.172	2.396	1.830
Extension_P	0.640	0.946	0.703
Abduction_A	0.687	0.921	0.799	2.207	1.774	2.756
Abduction_P	0.933	1.035	0.838
IR 90°_A	0.654	1.101	0.764	2.825	2.617	2.239
IR 90°_P	0.780	1.219	0.672
ER 90°_A	0.691	0.930	0.727	2.316	2.192	2.165
ER 90°_P	0.660	0.876	0.662
ER 0°_A	0.587	0.900	0.707	3.108	2.285	2.719
ER 0°_P	0.761	0.854	0.670
Elbow
Flexion_A	0.651	0.751	0.774	2.029	2.206	2.317
Flexion_P	0.571	0.655	0.582
Wrist
Flexion_A	2.423	0.864	0.733	2.470	1.903	2.147
Flexion_P	0.652	0.827	0.687
Extension_A	0.672	0.853	0.699	3.200	2.286	2.730
Extension_P	0.598	0.901	0.674
Hip
Flexion_A	0.746	0.974	0.880	3.268	2.038	2.895
Flexion_P	0.677	1.118	0.720
Extension_A	0.873	0.718	0.774	2.134	2.176	1.483
Extension_P	0.645	0.716	0.643
Abduction_A	0.688	1.142	0.691	2.687	3.031	2.437
Abduction_P	0.706	1.126	0.697
Adduction_A	0.799	0.809	0.721	1.882	2.062	2.383
Adduction_P	0.643	0.824	0.720
IR_A	0.616	0.706	0.636	2.073	2.012	1.820
IR_P	0.677	0.929	0.600
ER_A	0.668	0.977	0.688	2.985	2.541	1.803
ER_P	0.543	0.915	0.550
Knee
Flexion_A	0.764	0.847	0.649	2.269	2.027	2.367
Flexion_P	0.595	0.950	0.663
Ankle
PF 0°_A	0.692	0.823	0.639	2.056	1.986	2.053
PF 0°_P	0.625	0.872	0.647
DF 0°_A	0.623	0.701	0.667	1.521	1.730	1.787
DF 0°_P	0.577	0.694	0.624
PF 90°_A	0.616	0.759	0.699	2.581	2.006	1.946
PF 90°_P	0.686	0.854	0.676
DF 90°_A	0.711	0.754	0.685	1.704	2.233	2.813
DF 90°_P	0.616	0.825	0.607
Inversion_A	0.802	1.088	0.662	1.571	2.402	1.935
Inversion_P	0.712	0.872	0.673
Eversion_A	0.644	1.001	0.690	3.041	2.207	1.575
Eversion_P	0.593	0.844	1.096

RMSE, root mean square error; KRSP, Korean range of motion standard protocol; A, active; P, passive; IR n°, internal rotation with shoulder abduction n°; ER n°, external rotation with shoulder abduction n°; PF n°, plantarflexion with knee flexion n°; DF n°, dorsiflexion with knee flexion n°.

Table 8.

Inter-rater reliability using RMSE

	KRSP	Goniometer
Shoulder
Flexion_A	2.293	5.650
Flexion_P	2.017
Extension_A	2.122	5.823
Extension_P	2.075
Abduction_A	2.691	6.611
Abduction_P	2.569
IR 90°_A	2.415	7.223
IR 90°_P	2.389
ER 90°_A	2.594	6.836
ER 90°_P	2.265
ER 0°_A	2.237	7.746
ER 0°_P	2.235
Elbow
Flexion_A	2.086	6.210
Flexion_P	1.753
Wrist
Flexion_A	2.719	7.182
Flexion_P	2.375
Extension_A	2.953	7.697
Extension_P	2.079
Hip
Flexion_A	1.906	8.667
Flexion_P	1.745
Extension_A	1.640	5.808
Extension_P	1.587
Abduction_A	1.799	8.790
Abduction_P	1.639
Adduction_A	2.041	5.993
Adduction_P	1.729
IR_A	1.431	6.122
IR_P	1.705
ER_A	1.765	6.689
ER_P	2.993
Knee
Flexion_A	2.013	6.933
Flexion_P	1.799
Ankle
PF 0°_A	2.739	7.064
PF 0°_P	1.594
DF 0°_A	1.841	8.802
DF 0°_P	1.856
PF 90°_A	1.565	8.389
PF 90°_P	1.699
DF 90°_A	1.524	7.463
DF 90°_P	1.667
Inversion_A	1.827	9.342
Inversion_P	1.664
Eversion_A	1.381	9.266
Eversion_P	1.639

REFERENCES

1. Fleiss JL, Cohen J. The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability. Educ Psychol Meas 1973;33:613-9.

2. Cocchiarella L, Andersson GB. Guides to the evaluation of permanent impairment. 5th ed. Chicago: American Medical Association; 2000. p.433-455.

3. Rondinelli R. Guides to the evaluation of permanent impairment. 6th ed. Chicago: American Medical Association; 2007.

4. American Medical Association. AMA Guides to the evaluation of permanent impairment. 4th ed. Chicago: American Medical Association; 1993.

5. American Medical Association. Guides to the evaluation of permanent impairment. 3rd ed. Chicago: American Medical Association; 1990.

6. McBride ED. Disability evaluation and principles of treatment of compensable injuries. Philadelphia: J. B. Lippincott; 1963.

7. Cho KH, Jeon Y, Lee H. Range of motion of the ankle according to pushing force, gender and knee position. Ann Rehabil Med 2016;40:271-8.

8. Laupattarakasem W, Sirichativapee W, Kowsuwon W, Sribunditkul S, Suibnugarn C. Axial rotation gravity goniometer. A simple design of instrument and a controlled reliability study. Clin Orthop Relat Res 1990;(251): 271-4.

9. Gajdosik RL, Bohannon RW. Clinical measurement of range of motion. Review of goniometry emphasizing reliability and validity. Phys Ther 1987;67:1867-72.

10. Kolber MJ, Fuller C, Marshall J, Wright A, Hanney WJ. The reliability and concurrent validity of scapular plane shoulder elevation measurements using a digital inclinometer and goniometer. Physiother Theory Pract 2012;28:161-8.

11. van Trijffel E, van de Pol RJ, Oostendorp RA, Lucas C. Inter-rater reliability for measurement of passive physiological movements in lower extremity joints is generally low: a systematic review. J Physiother 2010;56:223-35.

12. Park W, Ramachandran J, Weisman P, Jung ES. Obesity effect on male active joint range of motion. Ergonomics 2010;53:102-8.

13. Herda L, Fua P, Plankers R, Boulic R, Thalmann D. Using skeleton-based tracking to increase the reliability of optical motion capture. Hum Mov Sci 2001;20:313-41.

14. Perez R, Costa U, Torrent M, Solana J, Opisso E, Caceres C, et al. Upper limb portable motion analysis system based on inertial technology for neurorehabilitation purposes. Sensors (Basel) 2010;10:10733-51.

15. El-Zayat BF, Efe T, Heidrich A, Anetsmann R, Timmesfeld N, Fuchs-Winkelmann S, et al. Objective assessment, repeatability, and agreement of shoulder ROM with a 3D gyroscope. BMC Musculoskelet Disord 2013;14:72.

16. Bonnechere B, Jansen B, Salvia P, Bouzahouene H, Omelina L, Moiseev F, et al. Validity and reliability of the Kinect within functional assessment activities: comparison with standard stereophotogrammetry. Gait Posture 2014;39:593-8.

17. Lee SH, Yoon C, Chung SG, Kim HC, Kwak Y, Park HW, et al. Measurement of shoulder range of motion in patients with adhesive capsulitis using a kinect. PLoS One 2015;10:e0129398.

18. Werner BC, Holzgrefe RE, Griffin JW, Lyons ML, Cosgrove CT, Hart JM, et al. Validation of an innovative method of shoulder range-of-motion measurement using a smartphone clinometer application. J Shoulder Elbow Surg 2014;23:e275-82.

19. Shin SH, Ro du H, Lee OS, Oh JH, Kim SH. Within-day reliability of shoulder range of motion measurement with a smartphone. Man Ther 2012;17:298-304.

20. Jenny JY. Measurement of the knee flexion angle with a Smartphone-application is precise and accurate. J Arthroplasty 2013;28:784-7.

21. Milanese S, Gordon S, Buettner P, Flavell C, Ruston S, Coe D, et al. Reliability and concurrent validity of knee angle measurement: smart phone app versus universal goniometer used by experienced and novice clinicians. Man Ther 2014;19:569-74.

22. Vohralik SL, Bowen AR, Burns J, Hiller CE, Nightingale EJ. Reliability and validity of a smartphone app to measure joint range. Am J Phys Med Rehabil 2015;94:325-30.

23. Hoving JL, Buchbinder R, Green S, Forbes A, Bellamy N, Brand C, et al. How reliably do rheumatologists measure shoulder movement? Ann Rheum Dis 2002;61:612-6.

24. de Winter AF, Heemskerk MA, Terwee CB, Jans MP, Deville W, van Schaardenburg DJ, et al. Inter-observer reproducibility of measurements of range of motion in patients with shoulder pain using a digital inclinometer. BMC Musculoskelet Disord 2004;5:18.

25. Hayes K, Walton JR, Szomor ZR, Murrell GA. Reliability of five methods for assessing shoulder range of motion. Aust J Physiother 2001;47:289-94.

26. Rothstein JM, Miller PJ, Roettger RF. Goniometric reliability in a clinical setting. Elbow and knee measurements. Phys Ther 1983;63:1611-5.

27. Watkins MA, Riddle DL, Lamb RL, Personius WJ. Reliability of goniometric measurements and visual estimates of knee range of motion obtained in a clinical setting. Phys Ther 1991;71:90-6.

28. Youdas JW, Bogard CL, Suman VJ. Reliability of goniometric measurements and visual estimates of ankle joint active range of motion obtained in a clinical setting. Arch Phys Med Rehabil 1993;74:1113-8.

29. LaStayo PC, Wheeler DL. Reliability of passive wrist flexion and extension goniometric measurements: a multicenter study. Phys Ther 1994;74:162-74.

30. Horger MM. The reliability of goniometric measurements of active and passive wrist motions. Am J Occup Ther 1990;44:342-8.

31. Armstrong AD, MacDermid JC, Chinchalkar S, Stevens RS, King GJ. Reliability of range-of-motion measurement in the elbow and forearm. J Shoulder Elbow Surg 1998;7:573-80.

32. Herrero P, Carrera P, Garcia E, Gomez-Trullen EM, Olivan-Blazquez B. Reliability of goniometric measurements in children with cerebral palsy: a comparative analysis of universal goniometer and electronic inclinometer. A pilot study. BMC Musculoskelet Disord 2011;12:155.

33. Poulsen E, Christensen HW, Penny JO, Overgaard S, Vach W, Hartvigsen J. Reproducibility of range of motion and muscle strength measurements in patients with hip osteoarthritis: an inter-rater study. BMC Musculoskelet Disord 2012;13:242.