Focus Group & Scale Development
Measuring Stress Caused by COVID-19-Triggered Anti-Asian Stigma
Client: N/A (this is my personal project as a doctoral researcher in experimental social psychology)
Duration: Feb 5 – 22, 2020 (Note: this is the duration for this highlighted part of the project. The overall project went from February to March, 2020 )
My role: Research scientist
Disclosure: This is part of a research project that included multiple phases. Although this is not a UX project, I highlight this portion to showcase my expertise in focus group and survey scale development.
TL;DR: Skip to Outcome to glimpse my research toolbox and result.
Project Overview
Background & Goal
In the beginning of February 2020, right after the announcement of a travel restriction that barred entry by foreigners who had recently visited China, numerous Chinese international students and Asian American students of mine expressed to me their constant anxiety and expectations of getting harassed and discriminated in the public because of the increasing stigma that Chinese (which had been generalized to Asians) caused and spread COVID-19 and the resulting spikes in anti-Asian hate crimes. These conversations alarmed me because discrimination-related stress could take a huge toll on people's mental and physical health. Thus, I decided to investigate this phenomenon and its negative consequences.
However, one big challenge was that, because the COVID-19 outbreak was a new event at that time, there was no existing scale to measure the unique stress facing Asians due to the outbreak. I decided to develop my own scale, and this article demonstrates the process of scale development.
Goal: develop a scale that effectively measures East Asian college students' anxious expectations of receiving discriminations due to COVID-19-triggered stigma.
Team
I led all aspects of this project, with input from Dr. Cynthia Levine. I liaised with 2 Chinese international student associations (the Chinese Students and Scholars Association at University of Washington (UWCSSA) and Hua Psychology Club) and 1 Asian student association (Asian Student Commission) in participant recruitment and coordination.
Process
Research Questions & Method Selection
I envisioned the final scale to be a list of scenarios, which respondents would read and rate the extent to which they anxiously expect COVID-19-related discrimination in each scenario. To do so, I needed to first discover the common situations inducing such anxious expectations among most East Asian students.
I chose focus group over 1-on-1 interviews because 1) I needed participants to freely generate anxiety-inducing situations and focus groups would provide better opportunities for brainstorming (e.g., one could be inspired by another's answer); 2) I wanted to find common situations in which most Asian people would feel anxious and focus groups would allow me to explore consensus among participants; and 3) I had a tight schedule and with focus groups, I could hear from more people in a short period of time.
Focus Group Research Question
What are the situations in which East Asian students anxiously expect to receive race-based discrimination because of the COVID-19-triggered stigma against East Asians?
Phase 1: Planning (9 days)
Defining & Recruiting Participants
Qualified participants needed to be:
college students
people of East Asian descent: either Chinese citizens studying in the United States on student or scholar VISAs or natural-born American citizens from an East Asian ethnic background (e.g., Chinese, Japanese, Korean, Vietnamese)
had been living in the U.S. for at least 3 months at the time of the study
A screener was administered when people signed up for participating to ensure that only qualified participants were included.
I built liaisons with Chinese international student associations and an Asian student association at University of Washington. These associations helped with participant recruitment by distributing advertisements via social media and email subscription list.
In the end, I recruited 28 participants, including 14 Chinese international students and 14 East Asian American students. Each participants received $15 for attending a focus group session.
Preparing Script
I wrote a script for informed consent at the beginning that explained purpose of the focus group session, confidentiality, incentives, and ground rules such as "no side conversation" and "only one person speaks at a time."
I listed prompts pertaining to people's perceptions of and experiences with COVID-19-triggered stigma and discrimination against East Asian people, and made notes on what to follow up and probe potentially.
The script was presented to multiple interdisciplinary research teams to solicit feedback. The final script incorporated the feedback.
To guide recording relevant nonverbal data during focus group sessions, I created a table to code nonverbal communications such as facial expression and body languages that would indicate agreement, disagreement, stress, upset, anger, and confusion.
Phase 2: Execution (3 days)
I set up and moderated 4 in-person focus group sessions, each containing 7 participants. Considering Chinese international students' experiences with discrimination might likely be different from those of East Asian American students, 2 sessions exclusively had Chinese international students and the other 2 exclusively had East Asian American students. Each session was 70-min long, consisting of a 10-min introduction and informed consent, 55-min discussion, and 5-min closing remarks.
All sessions were audio recorded, consented by all participants. I also took notes of nonverbal communications using the prepared coding table.
Participants answered a list of questions from the script.
Focus Group Discussion Prompts
Do you think the COVID-19 outbreak has changed the public's perceptions of East Asians? If so, what changes?
Has the COVID-19 outbreak changed your interactions with non-Asian people? If so, how?
In the past month, has there been a situation in which you felt rejected, excluded, or discriminated simply because of your race? If so, what was the situation? Do you think it was related to the COVID-19 outbreak?
In the past month, have you witness or heard of any situations in which another East Asian individual was rejected, excluded, or discriminated? If so, what was the situation? Do you think it was related to their race? Do you think it was related to the COVID-19 outbreak?
Phase 3: Analysis (2 days)
Generating Scenarios via Thematic Analysis
Step 1: I took note of all situations brought up by participants in which they had experienced, witnessed, or heard of incidents of discrimination or rejections anxiously.
Step 2: I consolidated situations that were similar enough. For example, being worried about getting discrimination if coughing at a bus stop could be consolidated with being worried about getting discrimination if sneezing at a subway station.
Step 3: I ranked these situations by 1) how many times they were brought up across all sessions, and 2) the extent to which other participants in the same session verbally or non-verbally agreed upon the mentioned situations. Finally, the analysis generated the following 10 scenarios that were most likely to induce anxious expectations of COVID-19-related discrimination among East Asian students:
Scenarios Generated from Focus Group Data
Imagine you are waiting for the bus at a bus stop, and you sneeze. You notice some people at the bus stop looking at you.
Imagine you are in class one day. You have a common cold and cough. The professor asks students to form groups and discuss a question. You do not know anyone in the class.
Imagine you are in class. You feel under the weather, coughing and sniffling. You notice the professor glancing your way.
Imagine you are riding the bus one day. The bus is full except for two seats, one of which is next to you. As the bus comes to the next stop, you notice a woman getting on the bus.
Imagine you arrive in class early wearing a face mask. You sit down in an empty row. You notice other students walking into the class. You do not know anyone in the class.
Imagine you are walking down the street, and you are wearing a face mask for germ protection. You notice that other pedestrians are looking at you.
Imagine you are applying for a part-time job at a coffee shop. You complete an in-person interview, and the interviewer informs you that they will let you know about their decision soon.
Imagine you visit a clinic to get a flu shot. While waiting, you notice another person in the waiting room glancing your way and shifting in their seat.
Imagine you are at a pharmacy, trying to pick up some cold medicines. You bring the medicines to the checkout counter, but the clerk is by the shelves, so you try to get their attention.
Imagine you try to rent an apartment, and the property manager is giving you an apartment tour. Your nose itches so you sneeze during the tour. The property manager tells you that you will know whether your rental application is approved soon.
Phase 4: Scale Writing & Validation (10 days)
Writing Items
To refresh your memory, the goal of conducting the focus group and distilling these scenarios was to help me develop a scale that effectively measures Asian people's anxious expectations of receiving discriminations due to COVID-19-triggered stigma.
Using the 10 scenarios, I designed the scale such that respondents would first read each scenario They then would rate the extent to which they were anxious that race-based discrimination might occur in that scenario as well as the extent to which they expected that race-based discrimination would occur in that scenario.
Take the first scenario as an example:
The anxiety and expectation ratings for each scenario would be multiplied to yield an anxious expectation score for that scenario. Then, I would average across the anxious expectation scores across all scenarios to yield an overall anxious expectation score. Higher scores would indicated stronger anxious expectations of receiving discriminations due to COVID-19-triggered stigma.
Validating Scale
I conducted an experiment and a survey study using this scale. For the sake of this article, I will only discuss the findings relevant to the focus group and the scale development. Click here if you are interested in reading details of the overall project.
Factor analysis showed that all items measured the same construct.
Inferential statistics showed that Chinese international students (n=152) and East Asian American students (n=102) scored significantly higher on the developed scale compared to non-East Asian students (n=158). In other words, Chinese international students and East Asian Americans were more likely to anxiously expect receiving race-based discrimination in the given scenarios than people who are not East Asian descendants. The plot below provides a visualization of such differences.
In conclusion, these findings validated that:
the focus group sessions were successful in answering the research question to discover the common scenarios that would induce anxiety about receiving discrimination among most East Asian students.
The scale was able to uncover that the anxious expectation of COVID-19-related discrimination is a unique stressor that only East Asian students, but not students of any other race/ethnicity, experience.
Outcome
By building liaisons with community partners, I managed to recruit participants within 4 days, allowing me to conduct four 70-min 7-person focus group sessions. Using focus group, I uncovered common scenarios in which East Asian students anxiously expected to receive race-based discrimination because of the COVID-19-triggered stigma against East Asians. I then developed a scale measuring such anxious expectations using the discovered scenarios. Later studies validated that this scale captured the construct as expected, suggesting that I achieved the primary goal.
Reflection
I found it overwhelming to simultaneously moderate the sessions and take notes of non-verbal communications even though I had an organized coding table for nonverbal cues. Next time when I conduct focus groups, I will videotape the sessions -- which I could not do in this study due to IRB restrictions -- and/or have an observation coder sitting in on the sessions. This will also allow me to compare notes with other coders.
I usually recruit participants via MTurk and Prolifics, but this experience of recruiting participants through community partners shows me that liaising with organization and recruiting participants through their networks can be very efficient when I clearly define the target group of participants.