The logistic modeling of interobserver agreement

Abstract

An approach to the logistic modeling of interobserver agreement is described that allows for the estimation of a commonly employed measure of agreement. The dependent variable is defined to be 1 if the two raters agree, and 0 otherwise. Covariates may be included in the regression equation in order to obtain adjusted or subgroup-specific estimates of percent agreement. As an empirical example, logistic models were fitted to data from a validation study of the agreement between interview information and physician records on the history of post-menopausal estrogen use, from a case-control study of breast cancer conducted on Oahu, Hawaii. Variables found to be related to agreement in previous univariate analyses were examined as covariates in the logistic model. The directly calculated estimates of percent agreement agreed well with the modeled estimates derived from the regression coefficients. Thus, the logistic model may provide a useful alternative to existing methods for the description of interobserver agreement.