oluwatobi oyinloye
Back to all work

From Emergency to Aftermath:Tracking Vaccine Hesitancy and Anti-Vax Discourse on X/Twitter

On May 11, 2023, the U.S. Department of Health and Human Services ended the COVID-19 public health emergency, a moment that marked a transition point for online discourse and health policy. This research examines the conversation surrounding vaccine hesitancy on X before and after that date, through the sentiment and main themes of 860,280 posts from across 2023.

Individual research. Undergraduate Research Opportunities Program (UROP), Boston University. Advised by Dr. Traci Hong. Submitted to ICA's Health Communication Division.

Built with

Data collection
Brandwatch
Language
Python, with pandas and NLTK
Sentiment
VADER
Topics
LDA, in gensim
Statistics
SciPy and SPSS

The Question

Vaccine hesitancyA "delay in acceptance or refusal of vaccination despite availability of vaccination services" (MacDonald, 2015). It's the definition used by the WHO's SAGE working group, and the one this research follows.

COVID-19 public health emergencyDeclared by the U.S. Department of Health and Human Services on January 31, 2020, and renewed every 90 days for over three years. It unlocked extra federal funding and flexible rules for testing, treatment and vaccines. Ending it on May 11, 2023 signalled that COVID-19 was now being treated as an ongoing illness rather than a crisis.

The World Health Organization named vaccine hesitancy one of its ten threats to global health in 2019, before COVID-19 had even started. After the pandemic, hesitancy began showing up around other vaccines too, like flu and MMR (Eagan et al., 2023). So the end of the U.S. public health emergency marked a real turning point.

I chose social media because it's a place where people are somewhat free to share their opinions, partly thanks to anonymity. X in particular is a popular platform for sharing public opinion, which makes it a good place to see how attitudes shift in real time.

My research asked two questions.

  • How did sentiment around vaccine hesitancy on X change before and after the emergency was lifted on May 11, 2023?
  • What themes were people talking about in each period, and had anti-vax discourse spread beyond COVID-19 to other infectious diseases?

Tracking this online discussion matters for policymakers and public health communicators. It helps them build evidence-based, continuous communication strategies that combat misinformation, strengthen vaccine confidence and reduce polarization.

Tobi at the UROP symposium beside the research poster
With my research advisor, Dr. Traci Hong, at the UROP symposium.

The Data

Why a 25% samplePulling every matching post would have taken far longer than the download limits allowed. A random sample keeps the picture representative while staying manageable.

I collected posts through Brandwatch, a social listening platform, using hashtags and phrases drawn from earlier research on vaccine hesitancy (Khadafi et al., 2022). This was the search:

(vaccine AND (COVID OR choice OR hesitancy)) OR #antivaxx OR #antivax OR #NoVaccine OR "Vaccine Trust" OR #AntiVaccine OR #AntiVacc OR #StopVaccine OR #unsafevaccine OR #AntiVaxxer

At a 25% sample rate, that returned 860,280 posts from January 1 to December 31, 2023. Brandwatch caps daily downloads, so I pulled the data in batches every day from March 12 to 21, 2024.

I split the posts in two: 382,170 from before the emergency ended (January 1 to May 11) and 478,110 from after (May 12 to December 31). Then I cleaned them in Python, removing links, emojis and special characters and normalising the text so the models were reading words without the interference of noise.

Measuring the Mood

VADERA dictionary-based sentiment scorer. It knows how positive or negative common words are, and adjusts for things social posts use a lot, like capital letters, exclamation marks and emojis.

Welch's t-testA test for whether two group averages really differ. With over 860,000 posts, even a modest shift is very unlikely to happen by chance, which is why p is so small.

To examine sentiment, I used VADER, a sentiment tool built for social media. It gives every post a compound score from -1, very negative, to +1, very positive. Then I compared the average score before and after May 11.

-0.3-0.2-0.10 More negativeNeutral Before, -0.157 After, -0.218

Both periods leaned negative, which makes sense for a hesitancy conversation. But after the emergency ended, the average dropped from -0.157 to -0.218.

To test whether it was a real change, I conducted a Levene's test, which showed the two groups had different amounts of spread (F = 126.4, p < 0.001). So I used Welch's t-test, which doesn't assume equal spread. The result was t = 54.98, p < 0.001. I ran it in Python and confirmed it in SPSS.

Sentiment on X became significantly more negative after May 2023.
Line chart of average monthly sentiment in 2023
My monthly average sentiment across 2023, with the before and after averages as dashed lines and the May 11 cut-off in red.

What People Were Talking About

To examine what people were talking about, I used LDA topic modelling in gensim, which found five topics in each period. The small numbers are how much each word defines its topic. Hover over a word to find it on the other side.

Before May 11, 2023

  1. Topic 1people1.2 said1.2 get1.0 trump1.0 got0.9
  2. Topic 2died3.8 year3.0 old2.4 pfizer2.3 diedsuddenly2.2
  3. Topic 3moderna4.6 card3.9 two3.2 company2.3 billion2.2
  4. Topic 4mandate4.3 news3.9 home2.2 misleading2.0 school2.0
  5. Topic 5mrna1.6 pfizer1.3 vaccine1.2 death1.1 injury1.1

After May 11, 2023

  1. Topic 1pfizer8.2 breaking5.9 trial3.8 clinical3.5 censor3.0
  2. Topic 2people1.6 get1.4 many0.8 mandate0.8 take0.7
  3. Topic 3new2.6 12.4 public2.4 health2.0 data1.8
  4. Topic 4pfizer2.6 died1.9 mrna1.9 death1.9 year1.8
  5. Topic 5zealand4.2 process4.0 data3.1 new2.9 report2.4

Before, the conversation centred on mandates, Moderna and Pfizer, vaccine cards and political references, including President Trump. Deaths and injuries came up a lot, including the #diedsuddenly hashtag.

After, the focus moved to Pfizer's clinical trials and lawsuits, clinical data and claims of censorship, and wider public health debates.

Did it Spread to Other Diseases?

LDALatent Dirichlet Allocation, an unsupervised probabilistic topic model. It treats each post as a mixture of topics and each topic as a probability distribution over words, then iteratively estimates both from which words co-occur across the whole dataset. The output is each topic's most heavily weighted words, which I then interpreted and labelled.

No significant data indicating a rise in conversation about other infectious diseases after the emergency ended was found. There were sporadic mentions of other diseases, but COVID-19 stayed central.

Something that stood out was that "mRNA" shows up in the LDA topics from both periods, a sign that COVID-19 wasn't the only vaccine being talked about. The technology itself had become part of the story.

Poster for UROP Symposium

Research poster, From Emergency to Aftermath

Next Steps

  • Apply deeper machine learning models, like BERT, to capture nuance missed by dictionary-based tools such as VADER.
  • Extend the research to cross-platform data, like Facebook, TikTok and Reddit, to compare discourse beyond X.
  • Link sentiment to CDC or state-level vaccination rates, to test whether online negativity aligns with real-world uptake.
  • Analyse social media data after 2023, to see whether the negative sentiment around vaccine hesitancy remains.
  • Include non-English discourse and a larger dataset, to better understand the global effects of vaccine discourse.
  • Balance the two periods, since the post-emergency group had more posts, and configure the topic model to look for keywords linked to other infectious diseases.

Negativity Increased After the Emergency Ended

The conversation got more negative once the emergency ended, and it moved from mandates and injuries to lawsuits, trials and data transparency.

That's key knowledge for policymakers and public health communicators. The worry didn't go away. It found new things to be about.

Next project
Does Six Degrees of Separation Hold in the Wizarding World?