Just in time for Pride Month, Drs. Charles Lea and Elwin Wu led a symposium at the 2025 Society for Prevention Research Annual Conference titled “LLM Chatbots and LGBTQ+ Wellbeing: Lessons Learned for Advancing Equity and Action When Using AI in Prevention Science.” Drs. Lea and Wu were joined by Zichen “Jackson” Zhao, a recent graduate of Columbia University’s Quantitative Methods in the Social Sciences program. Drawing from gAyI, a research initiative funded by the Columbia School of Social Work Seed Grant Program, the presentation was focused on how AI technologies can be better leveraged to support LGBTQ+ mental health and wellbeing.
Mr. Zhao shared findings from a systematic evaluation of machine-driven benchmarks used to assess how various large language models (LLMs) respond to suicide-related prompts involving LGBTQ+ individuals. His analysis highlighted both the strengths and limitations of these benchmarks, as well as their implications for real-world applications and future model training. Dr. Wu introduced a human-driven evaluation tool developed through gAyI, created with input from a Community Expert and Accountability Panel made up of clinicians, scholars, and individuals with lived experience. This tool offers a more nuanced assessment of LLM outputs, focusing on validity as well as critical social work–related dimensions such as bias, cultural relevance, equity, safety, and usability. Rounding out the session, Dr. Lea presented research on prompt engineering in the context of LGBTQ+ mental health, exploring how specific input phrasing can influence chatbots to produce more equitable and inclusive responses.
Together, the presenters emphasized the importance of combining technical precision with community-informed insights to ensure AI tools are ethically and effectively integrated into behavioral health and prevention work. Congratulations to Professors Lea and Wu on receiving an Abstract of Distinction Award from the Society for Prevention Research in recognition of this impactful symposium.