Zhengyang Shan
Zhengyang Shan headshot

Zhengyang Shan

PhD Candidate

Boston University (CDS)

I am a PhD candidate in the Faculty of Computing and Data Sciences at Boston University. I work on interpretability and evaluation in large language models, with a particular interest in understanding and mitigating bias and fairness issues in model behavior.

Interests
  • LLM Fairness & Inclusivity
  • Interpretability
  • Evaluation
  • LLMs Personalization
Education
🤯
Boston University — PhD in the Faculty of Computing and Data Sciences
🎓
University of Michigan — Ann Arbor — MSc in Quantitative Finance and Risk Management
🎓
The Pennsylvania State University — BSc in Mathematics and Computational Statistics, minor in Computer Science
News

New Dataset Release: Global-PIQA-Nonparallel (2025)

I’m excited to have participated as a contributor to the Global-PIQA-Nonparallel dataset, a large-scale multilingual benchmark for reasoning and commonsense evaluation.

📄 Paper · 🤗 Dataset · 🐦 Twitter/X · 🌐 BlueSky

Publications

Zhengyang Shan, Aaron Mueller (2026). Measuring Mechanistic Independence: Can Bias Be Removed Without Erasing Demographics? European Chapter of the Association for Computational Linguistics (EACL) 2026

▶ Abstract

We investigate how independent demographic bias mechanisms are from general demographic recognition in language models. Using a multi-task evaluation setup where demographics are associated with names, professions, and education levels, we measure whether models can be debiased while preserving demographic detection capabilities. We compare attribution-based and correlation-based methods for locating bias features. We find that targeted sparse autoencoder feature ablations in Gemma-2-9B reduce bias without degrading recognition performance: attribution-based ablations mitigate race and gender profession stereotypes while preserving name recognition accuracy, whereas correlation-based ablations are more effective for education bias. Qualitative analysis further reveals that removing attribution features in education tasks induces ``prior collapse'', thus increasing overall bias. This highlights the need for dimension-specific interventions. Overall, our results show that demographic bias arises from task-specific mechanisms rather than absolute demographic markers, and that mechanistic inference-time interventions can enable surgical debiasing without compromising core model capabilities.

PDF

Zhengyang Shan, Emily Diana, Jiawei Zhou (2025). Gender Inclusivity Fairness Index (GIFI): A Multilevel Framework for Evaluating Gender Diversity in Large Language Models. Association for Computational Linguistics 2025

▶ Abstract

We present a comprehensive evaluation of gender fairness in large language models (LLMs), focusing on their ability to handle both binary and non-binary genders. While previous studies primarily focus on binary gender distinctions, we introduce the Gender Inclusivity Fairness Index (GIFI), a novel and comprehensive metric that quantifies the diverse gender inclusivity of LLMs. GIFI consists of a wide range of evaluations at different levels, from simply probing the model with respect to provided gender pronouns to testing various aspects of model generation and cognitive behaviors under different gender assumptions, revealing biases associated with varying gender identifiers. We conduct extensive evaluations with GIFI on 22 prominent open-source and proprietary LLMs of varying sizes and capabilities, discovering significant variations in LLMs’ gender inclusivity. Our study highlights the importance of improving LLMs’ inclusivity, providing a critical benchmark for future advancements in gender fairness in generative models.

PDF GitHub

Ge Gao, Zhengyang Shan, James Crissman, Ekaterina Novozhilova, YuCheng Huang, Arti Ramanathan, Margrit Betke, Derry Wijaya (2025). Insights into Climate Change Narratives: Emotional Alignment and Engagement Analysis on TikTok. Proceedings of the Fourth Workshop on NLP for Positive Impact (NLP4PI) 2025 🏆 Best Paper Award .

▶ Abstract

TikTok has emerged as a key platform for discussing polarizing topics, including climate change. Despite its growing influence, there is limited research exploring how content features shape emotional alignment between video creators and audience comments, as well as their impact on user engagement. Using a combination of pretrained and fine-tuned textual and visual models, we analyzed 7,110 TikTok videos related to climate change, focusing on content features such as semantic clustering of video transcriptions, visual elements, tonal shifts, and detected emotions. (1) Our findings reveal that positive emotions and videos featuring factual content or vivid environmental visuals exhibit stronger emotional alignment. Furthermore, emotional intensity and tonal coherence in video speech are significant predictors of higher engagement levels, offering new insights into the dynamics of climate change communication on social media. (2) Our preference learning analysis reveals that comment emotions play a dominant role in predicting video shareability, with both positive and negative emotional responses acting as key drivers of content diffusion. We conclude that user engagement—particularly emotional discourse in comments—significantly shapes climate change content shareability.

PDF

Zhengyang Shan (2021). Nonuniqueness for a fully nonlinear, degenerate elliptic boundary-value problem in conformal geometry. Differential Geometry and its Applications

▶ Abstract

One way to generalize the boundary Yamabe problem posed by Escobar is to ask if a given metric on a compact manifold with boundary can be conformally deformed to have vanishing σ_k-curvature in the interior and constant H_k-curvature on the boundary. When restricting to the closure of the positive k-cone, this is a fully nonlinear degenerate elliptic boundary value problem with fully nonlinear Robin-type boundary condition. We prove nonuniqueness for the boundary-value problem σ_4-curvature equals zero and constant H_4-curvature by using bifurcation results proven by Case, Moreira and Wang. Our construction via products of sphere and hyperbolic space only works for a finite set of dimensions.

PDF
Teaching Experience

📚 Boston University