Validity of Workers’ Self-Reports. Evaluation of a Question Assessing Lifetime Exposure to Occupational Physical Activity
Anne Møller *
Department of Occupational and Environmental Medicine, Køge Hospital, Lykkebækvej 1, DK-4600 Køge, Denmark and National Research Centre for the Working Environment, Copenhagen, Denmark and The Research Unit for General Practice and Section of General Practice, Department of Public Health, University of Copenhagen, Copenhagen, Denmark
Susanne Reventlow
The Research Unit for General Practice and Section of General Practice, Department of Public Health, University of Copenhagen, Copenhagen, Denmark
Johan Hviid Andersen
Danish Ramazzini Centre, Department of Occupational Medicine, Regional Hospital, Herning, Denmark
Kirsten Avlund
Section of Social Medicine, Department of Public Health, University of Copenhagen, Copenhagen, Denmark and Centre for Healthy Aging, University of Copenhagen, Copenhagen, Denmark and Danish Aging Research Centre, Universities of Southern Denmark, Aarhus and Copenhagen, Denmark
Ole Steen Mortensen
National Research Centre for the Working Environment, Copenhagen, Denmark and Department of Occupational and Environmental Medicine, Bispebjerg Hospital, Copenhagen, Denmark.
*Author to whom correspondence should be addressed.
Abstract
Aims: In epidemiological studies exposure assessment based on questionnaires is the most cost-effective method. A question about lifetime exposure to occupational physical activity (OPA) was used in a population-based survey (part of the Copenhagen Aging and Midlife Biobank, CAMB). The aim of the study was to validate this question through a three-step process.
Methodology: Firstly, the response process was studied by cognitive interviewing of 7 persons. Secondly, 64 persons participated in semi-structured interviews about their work-life, and expert judgments of exposure to OPA were compared with questionnaire-data. Exposure was 20 years of work in one of four categories of OPA: sedentary, standing and walking, moderate or high OPA. Kappa values were calculated for agreement and interpreted according to Landis and Koch’s criteria. Agreement was visualized in Bland-Altman plots. Thirdly, intra- and inter-rater reliability of expert judgments was tested.
Results: Response process: The question had a complicated instruction, and the respondents found it hard to remember, categorize, and summate exposures. Semi-structured interviews: Kappa value for exposure to sedentary work was ‘substantial’ (0.71) but ‘fair’ for the other categories of OPA (0.27-0.29). Agreement between questionnaire and interview was higher in sedentary jobs and jobs with high OPA. Intra-rater reliability of expert judgments was ‘substantial’ or ‘moderate’ (0.60-0.71). Inter-rater reliability was high in sedentary jobs but lower in the more active jobs.
Conclusion: Self-reports of lifetime exposure to sedentary work are valid in the CAMB cohort, whereas the validity of self-reports of exposure to high levels of occupational physical activity (OPA) are questionable. Thorough pre-testing of questions about lifetime OPA is recommended.
Keywords: Validity, self-reports, occupational physical activity, inter-rater reliability, intra-rater reliability, cognitive interviewing, Bland-Altman plots, qualitative methods.