Videofluoroscopic assessment of swallowing is widely used in clinical settings. The interpretation of such assessments depends on subjective visual judgments but the reliability of these judgments has been poorly researched. This study measured interrater reliability of judgments, made by speech pathologists, of videofluoroscopic images of subjects swallowing liquid and semisolid boluses. A 5-point rating scale was used in three conditions: individually after careful reading; together with other speech pathologists in group discussion; and individually after the group discussion. Analysis of the ratings for the three conditions revealed that the level of agreement among raters was generally higher for semisolid swallows than for liquid swallows. The highest levels of agreement occurred for ratings made after group discussions. The levels of agreement were lowest when raters worked alone, relying only on reading the scale. Individual rating after group discussion resulted in higher levels of agreement than sole reliance on reading the scale. Factors influencing the levels of interrater agreement, including the timing of observations, bolus consistency, the quality of the image, and the complexity of the task, are discussed.