Du Bois Review: Social Science Research on Race

Unpacking Racism and its Health Consequences


How Best to Ask the Questions, in One or Two Stages, Across Multiple Racial/Ethnic Groups?1

Salma Shariff-Marcoa1 c1, Nancy Breena22, Hope Landrinea3, Bryce B. Reevea4, Nancy Kriegera5, Gilbert C. Geea6, David R. Williamsa7, Vickie M. Maysa8, Ninez A. Poncea9, Margarita Alegríaa10, Benmei Liua11, Gordon Willisa12 and Timothy P. Johnsona13

a1 Division of Cancer Control and Population Sciences, National Cancer Institute

a2 Applied Research Program, Division of Cancer Control and Population Sciences, National Cancer Institute

a3 Center for Health Disparities Research, East Carolina University

a4 Applied Research Program, Division of Cancer Control and Population Sciences, National Cancer Institute

a5 Harvard School of Public Health, Harvard University

a6 School of Public Health, University of California, Los Angeles

a7 Department of Society, Human Development, and Health, Harvard School of Public Health; Departments of African and African American Studies and of Sociology, Harvard University

a8 School of Public Health, University of California, Los Angeles and Center for Research, Education, Training, and Strategic Communication on Minority Health Disparities, University of California, Los Angeles

a9 School of Public Health, University of California, Los Angeles and UCLA Center for Health Policy and Research

a10 Harvard Medical School, Harvard University

a11 Surveillance Research Program, Division of Cancer Control and Population Sciences, National Cancer Institute

a12 Applied Research Program, Division of Cancer Control and Population Sciences, National Cancer Institute

a13 Survey Research Laboratory, Department of Public Administration and Institute for Health Research and Policy, University of Illinois at Chicago


While it is clear that self-reported racial/ethnic discrimination is related to illness, there are challenges in measuring self-reported discrimination or unfair treatment. In the present study, we evaluate the psychometric properties of a self-reported instrument across racial/ethnic groups in a population-based sample, and we test and interpret findings from applying two different widely-used approaches to asking about discrimination and unfair treatment. Even though we found that the subset of items we tested tap into a single underlying concept, we also found that different groups are more likely to report on different aspects of discrimination. Whether race is mentioned in the survey question affects both frequency and mean scores of reports of racial/ethnic discrimination. Our findings suggest caution to researchers when comparing studies that have used different approaches to measure racial/ethnic discrimination and allow us to suggest practical empirical guidelines for measuring and analyzing racial/ethnic discrimination. No less important, we have developed a self-reported measure of recent racial/ethnic discrimination that functions well in a range of different racial/ethnic groups and makes it possible to compare how racial/ethnic discrimination is associated with health disparities among multiple racial/ethnic groups.

(Online publication April 15 2011)


  • Discrimination;
  • Racism;
  • Prejudice;
  • Race/Ethnicity;
  • Differential Item Functioning;
  • Surveys;
  • Split-ballot;
  • Psychometric


c1 Salma Shariff-Marco, Cancer Prevention Institute of California, 2201 Walnut Avenue, Suite 300, Fremont, CA 94538. E-mail: salma.shariffmarco@cpic.org

Salma Shariff-Marco is a Staff Scientist at the Cancer Prevention Institute of California. Dr. Shariff-Marco conducted this research at the National Cancer Institute in the Division of Cancer Control and Population Sciences (DCCPS). She joined DCCPS as a Cancer Prevention Fellow in 2006 and then as a Research Scientist in 2010. Dr. Shariff-Marco earned her MPH from Columbia University and her PhD in social and behavioral sciences from Johns Hopkins University. Since joining DCCPS, Dr. Shariff-Marco has led a multidisciplinary team in the development of a racial/ethnic discrimination measure for use in health surveys and supported the Centers for Population Health and Health Disparities. Her research interests include assessing the role of social determinants in cancer health disparities and understanding the context of health behaviors.

Nancy Breen has served as an economist at the National Cancer Institute since 1991. Her research investigates how race, social class, gender, community factors, and health service diffusion shape cancer outcomes, especially cancer screening. Dr. Breen received her PhD in Economics from the New School for Social Research Graduate Faculty in 1988. Dr. Breen focuses on cancer screening, the entry point to health services delivery where disparities first emerge along the cancer continuum. Knowledge about screening and survey data led to Dr. Breen's appointment as NCI project officer to the California Health Interview Survey (CHIS) in 1999. As a member of the CHIS Multi-Cultural Task Force, Dr. Breen initiated research to develop and test a module on CHIS to measure self-reported discrimination in multi-cultural populations.

Hope Landrine is Professor of Psychology and Director of the Center for Health Disparities Research at East Carolina University. She is internationally known for her research on health disparities, and widely-regarded as a leading authority on ethnic-minority health. She developed the well-known African-American Acculturation Scale and several measures of ethnic and gender discrimination. She has received more than $10 million in extramural funding, and five national research awards including the American Psychological Association Dalmas Taylor Award for Lifetime, Distinguished Contributions to Research on Ethnic Minorities. Her areas of expertise are African American and Latino health and health disparities, with a focus on the role of acculturation, community poverty, ethnic discrimination, and residential segregation in the physical and mental health of American ethnic-minority groups.

Bryce B. Reeve is Associate Professor in the Department of Health Policy and Management, Gillings School of Global Public Health, University of North Carolina at Chapel Hill (UNC-CH). He is also a member of the UNC Lineberger Comprehensive Cancer Center. He holds a PhD in quantitative psychology. From 2000 to 2010, he served as a Program Director at the U.S. National Cancer Institute. At UNC-CH, Dr Reeve's work focuses on enhancing the application of patient-reported outcomes (PRO)(including symptoms, functioning, and quality of life) in clinical research and practice to improve the quality of care for pediatric and adult cancer patients. This includes the development of PRO measures using qualitative and quantitative methodologies and integration of PRO data in research and healthcare delivery to inform decision-making.

Nancy Krieger is Professor of Society, Human Development, and Health at the Harvard School of Public Health and Co-Director of the HSPH Interdisciplinary Concentration on Women, Gender, and Health. She received her PhD in Epidemiology from the University of California at Berkeley in 1989. An internationally recognized social epidemiologist, her work addresses conceptual frameworks to understand, analyze, and improve population health and reduce health inequities, including the ecosocial theory of disease distribution she has been developing since 1994; etiologic research on societal determinants of population health and health inequities; and methodologic research on improving monitoring of health inequities. She is author of Epidemiology and The People's Health: Theory and Context (2011) and editor of Embodying Inequality: Epidemiologic Perspectives (2004).

Gilbert C. Gee is Associate Professor in the School of Public Health at the University of California, Los Angeles. Dr. Gee's research focuses on the measurement of racial discrimination at multiple levels, and on assessing how these forms of discrimination are related to health problems. His current projects include studies in the United States, Japan, and the Philippines. His research has received several recognitions, including the “Best Published Paper” from the Asian and Pacific Islander Caucus of the American Public Health Association; two Scientific and Technical Achievement Awards from the Environmental Protection Agency; and a Merit Award from the National Institutes of Health.

David R. Williams is the Florence and Laura Norman Professor of Public Health and Professor of African and African American Studies and of Sociology at Harvard University. His prior academic appointments were at Yale University and the University of Michigan. His research has focused on trends and determinants of socioeconomic and racial disparities in health, the effects of racism on health and the ways in which religious involvement can affect health. He is the author of more than 250 scholarly papers in scientific journals and edited collections and he was ranked as the Most Cited Black Scholar in the Social Sciences in 2008. He is an elected member of the Institute of Medicine and of the American Academy of Arts and Sciences.

Vickie M. Mays is Professor of Psychology in the College of Letters and Science and Health Services in the UCLA School of Public Health. She is the Director of the UCLA Center on Research, Education, Training, and Strategic Communication on Minority Health Disparities. She has a number of publications that address race-based discrimination, particularly as it applies to negative physical and mental health outcomes. Her work has examined the burden of discrimination in mental health outcomes and the cognitive and neural bases of the processes of discrimination. Mays is currently finishing an interview study of African American men over sixty-five, their experiences of race-based discrimination, and their coping.

Ninez A. Ponce is Associate Professor in the Department of Health Services, UCLA School of Public Health. As Co-Principal Investigator of the California Health Interview Survey (CHIS) 2001 and 2003, she led the efforts in the measurement of race/ethnicity, acculturation, physician-patient communication, and discrimination. In addition, Dr. Ponce conceptualized the rationale and implementation of CHIS 2001's Asian ethnic oversamples and the cultural and linguistic adaptation of the survey. In 2004, NCI's Special Populations Health Network recognized Dr. Ponce as a rising star in cancer disparities. In 2008, she received an NIH merit award for her work in multicultural survey research. In 2009 she was an Institute of Medicine subcommittee member on the standardization of Race/Ethnicity/Language data.

Margarita Alegría is Director of the Center for Multicultural Mental Health Research at the Cambridge Health Alliance and a Professor in the Department of Psychiatry at Harvard Medical School. She currently serves as Principal or co-Principal Investigator of three NIH-funded research studies. Dr. Alegría's published work focuses on the improvement of health care services delivery for diverse racial and ethnic populations, conceptual and methodological issues with multicultural populations, and ways to bring the community's perspective into the design and implementation of health services. She has actively mentored numerous students and junior investigators. Dr. Alegría has received numerous awards including the 2003 Mental Health Section Award of APHA, the 2008 Health Disparities Innovation Award from the NCMHD, the 2008 Carl Taube Award from APHA, and the 2009 Simon Bolivar Award from the APA.

Benmei Liu is a Mathematical Statistician in the Division of Cancer Control and Population Sciences at the National Cancer Institute (NCI). Liu received her PhD in Survey Methodology from the University of Maryland. Before joining NCI, she worked as a survey statistician at Westat for seven years.

Gordon Willis is Cognitive Psychologist in the Applied Research Program, Division of Cancer Control and Population Sciences, National Cancer Institute, National Institutes of Health. Formerly, he was Senior Survey Research Methodologist at Research Triangle Institute, and he also worked in the Questionnaire Design Research Laboratory at the National Center for Health Statistics, Centers for Disease Control. He now works mainly in the area of the development and evaluation of surveys on cancer risk factors, and focuses on questionnaire pretesting. He has published the book Cognitive Interviewing: A Tool for Improving Questionnaire Design (2005), and has taught questionnaire design at the University of Maryland and the University of North Carolina. Increasingly, his research interests have turned to cross-cultural issues in self-report surveys and research studies. Dr. Willis attended Oberlin College, and received a PhD in Cognitive Psychology from Northwestern University.

Timothy P. Johnson is Director of the Survey Laboratory at the University of Illinois at Chicago (UIC). He is also Professor of Public Administration and Research Professor of Public Health at UIC, where he is responsible for teaching courses in survey methodology and data analysis. His personal research has focused on cultural and other sources of measurement and nonresponse errors in surveys, and the social epidemiology of health behaviors. Johnson earned his PhD in Sociology from the University of Kentucky in 1988. He currently also serves as Vice Chair for UIC's Social and Behavioral Institutional Review Board, as Deputy Director for Evaluation & Tracking of UIC's Center for Clinical Translational Sciences, and as President of the Midwest Association for Public Opinion Research.


1 We would like to thank the NCI Division of Cancer Control and Population Sciences (DCCPS) and the NIH Office of Behavioral Social Sciences Research for co-funding the 2007 field test, and NCI DCCPS for providing staff for this project. We would also like to acknowledge the Cancer Prevention Fellowship Program, Office of Preventive Oncology, National Cancer Institute, for providing financial support for S. Shariff-Marco in her postdoctoral fellowship; Tim McNeel and William Waldron at IMS for their programming support; and Penny Randall-Levy of the Scientific Consulting Group, Inc., for her help with the references. We would also like to acknowledge Drs. David Grant, E. Richard Brown, and David Takeuchi for their participation in this research project. Finally, we would like to acknowledge Drs. William Klein and Rachel Ballard-Barbash for their thoughtful reviews.

2 The first two authors shared in the leadership of the project and bore equal responsibility in drafting this manuscript.