To investigate the inter-rater agreement using the Videofluoroscopic Dysphagia Scale (VDS).
The present study was designed as a multicenter, single-blind trial. A Videofluoroscopic Swallowing Study (VFSS) was performed using the protocol described by J.A Logemann. Thick-fluid, pureed food, mechanically altered food, regularly textured food, and thin-fluid boluses were sequentially swallowed. Each participant received a 3 ml bolus followed by a 5 ml bolus of each food material, in the order mentioned above. All study procedures were video recorded. Discs containing these video recordings in random order were distributed to interpreters who were blinded to the participant information. The video recordings were evaluated using a standardized VDS sheet and the inter-rater reliability was calculated.
In total, 100 patients participated in this study and 10 interpreters analyzed the findings. Inter-rater reliability was fair in terms of lip closure (κ: 0.325), oral transit time (0.253), delayed triggering of pharyngeal swallowing (0.300), vallecular residue (0.275), laryngeal elevation (0.345), pyriform sinus residue (0.310), coating of the pharyngeal wall (0.310), and aspiration (0.393). However, other parameters of the oral phase were lower than those of the pharyngeal phase (0.06-0.153). Moreover, the summation of VDS reliability (intraclass correlation coefficient: 0.556) showed moderate agreement.
VDS shows a moderate rate of agreement for evaluating the swallowing function. However, many of the parameters demonstrated a lower rate of agreement, particularly the oral phase parameters.
Dysphagia is a frequent result of a stroke, brain tumor, or neurodegenerative disease. Many authors have tried to detect swallowing abnormalities (particularly aspiration) using non-radiographic observations, yet, these methods demonstrate poor sensitivity and specificity.
The Videofluoroscopic Dysphagia Scale (VDS) can be used to predict the long-term prognosis of dysphagia patients following stroke. Han et al.
This study was designed as a multicenter (10 rehabilitation centers), single-blind trial. Patients who exhibited any symptoms of difficulty in swallowing were recruited. The criteria for inclusion were patients with (1) a history of aspiration symptoms, such as coughing or choking; (2) symptoms clinically suspicious of dysphagia, such as reduced gag reflex or delayed swallowing reflex; and (3) a history of the use of an alternative feeding method, such as a nasogastric tube. Patients who could not sit or those who had difficulty maintaining consciousness were excluded. All of the recruited patients who agreed to participate in our study underwent VFSS from January through June in 2011. The protocol for this study was approved by the Institutional Review Board of Seoul Asan Hospital.
VFSS was conducted by two physiatrists using fluoroscopy. The first physiatrist was a professor with 15 years of experience with VFSS; the second physiatrist was a resident physician. A modified version of the protocol used in Logemann's study
All of the participating interpreters were physiatrists who had at least 5 years of experience in interpreting VFSS results. They agreed to participate after being informed of the nature of this study. All patient information, including age, sex, and underlying diseases, was withheld from the interpreters. The interpreters only observed the patients using the files on the DVD and described their findings using a standardized format (
The intra-class correlation coefficient (ICC) model 2.1 of the VDS was calculated in order to test the inter-rater reliability based on the VDS scores provided by the interpreters. The ICC model was used because it can be used not only for scale variables but also for ordinal variables. Ordinal variables equivalent to the weighted kappa ICC values over 0.80 was considered "very good", and ICC values between 0.60-0.80 were considered "good". The consistency of the other items was evaluated using Cohen's kappa (κ).
One hundred patients (59 males and 41 females) with dysphagia were enrolled, including 64 stroke patients, 13 patients with traumatic brain injury, 12 patients with head and neck cancer, 6 patients with brain tumors, and 5 patients with other diseases. The average age of the enrolled patients was 64.4±14.8 years. All of the recruited patients underwent VFSS. Inter-rater reliability of the oral phase parameters are shown in
The past two decades have brought an enormous widening of our knowledge about dysphagia research and treatment.
Overall, the VDS score demonstrated low to moderate reliability in our study (0.556 in terms of ICC). However, 14 individual sub parameters, particularly the oral phase parameters, showed low reliability. A previous study conducted by Stoeckli et al.
Regarding the pharyngeal phase, the overall reliability was higher than the oral phase (κ=0.165-0.393 vs. κ=0.060-0.325, respectively), similar to other studies that reported higher reliability for pharyngeal phase parameters than oral phase parameters.
The total VDS score demonstrated higher reliability than the individual parameters (0.556 in terms of ICC). This is due to the dilution effect of the scores of each parameter given by the interpreters.
The overall reliability is not particularly high in our study, and we believe this is because no clear definitions exist for intermediate values VDS, even though 9 of the 14 parameters have at least 3 categorical values. For example, "intact" mastication is given 0 points and "inadequate" mastication is given 4 points according to the VDS; however, depending on how each interpreter classifies the patient's mastication function, a single patient can be given any point--either 0 or 4. Therefore, the evaluation of patients showing some poor functioning of the parameters may lack consistency from interpreter to interpreter. Second, the guidelines specifying the type of food to be used as a standard for evaluation do not exist. In our study, various types of food material were tested on each patient. Depending on which type of material was used as the standard for evaluation, VFSS findings may be classified differently for each patient. For example, patients demonstrating good swallowing of solid foods but poor swallowing of liquid foods may be interpreted differently depending on whether solid or liquid foods was used for evaluation. For future studies, there should be guidelines regarding which food materials should be used as the standard for evaluating the findings related to each parameter.
This study has an obvious limitation. The interpretation was performed only via the observation of VFSS video recordings, as it was not logistically possible to have all 10 interpreters examine each patient. If the interpreters had been allowed to clinically examine their patients, this would have improved the results of the interpretations by increasing accuracy. However, the object of this study was to evaluate inter-rater reliability of VDS based on VFSS findings. If the interpreter had predicted the findings from the clinical examination, this would have acted as a bias.
This is the first study to evaluate the inter-rater reliability of VDS. For future studies, a more precise and widely accepted study protocol will be needed. The development of such a protocol can be achieved by standardized education programs, such as interactive lecture movies or formal guidelines for interpreters. These education programs may contribute to achieving higher levels of accuracy in interpretation, and subsequently, to improving the abilities to predict the long-term prognosis of dysphagia.
VDS demonstrates a moderate rate of inter-rater reliability for evaluating the swallowing function. Some of the parameters demonstrated a lower rate of agreement, particularly the oral phase parameters. VDS has some limitations in predicting the long-term prognosis of dysphagia; hence, more accurate definitions of each parameter as well as a study protocol will be essential.
Inter-rater Reliability of VDS
SE: Standard error, CI: Confidential interval, ICC: Intra-class correlation coefficient