Hello! I’m Irena, a CS PhD at Stanford University, advised by Carlos Guestrin. Before my PhD, I also received my BS/MS from Stanford.

My research interests center on helping humans verify the outputs of AI models, especially when the human operates with limited attention or domain knowledge.

Note: I’ve published under both “Irena Saracay” and “Irena Gao” (my maiden name).

Selected publications

  1. Beyond expert users: agents should help users construct preferences, not just elicit them
    Irena Saracay, Ludwig Schmidt, and Carlos Guestrin
    2026.
    arXiv Code Website

    We introduce a user model which constructs new preferences when agents help develop the user’s domain knowledge. We benchmark agents against this user in a shopping context and find that agents do not actively try to help users learn about their preferences.

  2. Model equality testing: which model is this API serving?
    Irena Gao, Percy Liang, and Carlos Guestrin
    In International Conference on Learning Representations 2025.
    arXiv Code Twitter

    We propose a method for helping users verify if two LLM inference API endpoints are serving the same model — a method more reliable than having humans eyeball samples.

  3. Adaptive testing of computer vision models
    Irena Gao, Gabriel Ilharco, Scott Lundberg, and Marco Tulio Ribeiro
    In International Conference on Computer Vision 2023. (Oral Presentation)
    arXiv Code Twitter

    We present a tool that helps users find bugs in object recognition and detection models. This tool uses foundation models to propose possible failure modes for the human to explore.

Show other publications Hide other publications ▾ * denotes equal contribution.
  1. OpenFlamingo: an open-source framework for training large autoregressive vision-language models
    Anas Awadalla*, Irena Gao*, Joshua Gardner, Jack Hessel, Yusuf Hanafy, Wanrong Zhu, Kalyani Marathe, Yonatan Bitton, Samir Gadre, Shiori Sagawa, Jenia Jitsev, Simon Kornblith, Pang Wei Koh, Gabriel Ilharco, Mitchell Wortsman, and Ludwig Schmidt
    2023.
    arXiv Blog Code Website Twitter
  2. CREPE: can vision-language foundation models reason compositionally?
    Zixian Ma*, Jerry Hong*, Mustafa Omer Gul*, Mona Gandhi, Irena Gao, and Ranjay Krishna
    In Conference on Computer Vision and Pattern Recognition 2023. (Highlight)
    arXiv Code
  3. Out-of-domain robustness via targeted augmentations
    Irena Gao*, Shiori Sagawa*, Pang Wei Koh, Tatsunori Hashimoto, and Percy Liang
    In International Conference on Machine Learning 2023.
    arXiv Code
  4. Extending the WILDS benchmark for unsupervised adaptation
    Shiori Sagawa*, Pang Wei Koh*, Tony Lee*, Irena Gao*, Sang Michael Xie, Kendrick Shen, Ananya Kumar, Weihua Hu, Michihiro Yasunaga, Henrik Marklund, Sara Beery, Etienne David, Ian Stavness, Wei Guo, Jure Leskovec, Kate Saenko, Tatsunori Hashimoto, Sergey Levine, Chelsea Finn, and Percy Liang
    In International Conference on Learning Representations 2022. (Oral Presentation)
    arXiv Code Website