Establishing Interrater Reliability for Use of the Ostomy Forum Observation Index® in Clinical Research

In order to use a clinical instrument in a research study, the issues of validity and reliability must be addressed.  In this poster presentation, we will share the methods used to establish interrater and test-retest reliability on the Ostomy Forum Observation Index^®, which is being used in a longitudinal study of stoma and peristomal skin complications. This instrument was developed by a group of experienced Enterostomal Therapy nurses from the United Kingdom, The Netherlands, Poland, Denmark, Norway, Sweden and Japan, and used in one prior study¹. Page one of the instrument consists of observations about the stoma, peristomal skin, and output, while page 2 consists of psychological, social, and sexuality evaluations.  Only the 13 items pertaining to stoma condition and 11 items pertaining to the peristomal skin are used in the current study.

The WOC Nurses participating in this study were provided with education about the purpose and use of the instrument, and then they practiced using the tool. The interrater reliability was measured by showing the data collectors a series of pictures of normal and abnormal peristomal skin and stomas.  The level of agreement for the ratings of peristomal skin and stoma was calculated. Rater training was continued as needed, with the goal of obtaining an interrater reliability of 0.8 or above. To accommodate the use of three remote clinical study sites, periodic measurement of interrater reliability were completed by sending electronic files of stoma images for the data collectors to measure and report. Actual stoma pictures collected as a part of the study were then integrated into subsequent measurements, to allow us to calculate test-retest reliability.  We will present the methods and results of this effort to establish reliability for the Ostomy Forum Observation Index^® and discuss the strengths and challenges of this important research activity.