Yijia Dai

👋 ▫️ 👍 👀 ❕

Learning is key for any type of intelligence. I love thinking about how human learns, and whether that is transferable to machines. Questions like How does sequential ordering of knowledge affect human learning? How about dataset for machines? intrigue me.

My research interests include developing self-improvement mechanisms for large models, interpreting machine intelligence, and creating AI systems that enhance people’s daily lives.

I’m fortunate to be working with Prof. Sarah Dean, Prof. Thorsten Joachims, and Prof. Jennifer Sun at Cornell. Currently, I study reinforcement learning, human-in-loop dynamical systems, and LLMs for scientific discovery. Sample-efficient low-rank representation learning for dynamical and unobserved user states, optimal learning of logistic bandits for recommendation systems, and LLMs for understanding decision trajectories are examples of what I work on.

Publications

2025

Pre-trained Large Language Models Learn Hidden Markov Models In-context

Yijia Dai, Zhaolin Gao, Yahya Satter, and 2 more authors

2025

Abs paper

Hidden Markov Models (HMMs) are foundational tools for modeling sequential data with latent Markovian structure, yet fitting them to real-world data remains computationally challenging. In this work, we show that pre-trained large language models (LLMs) can effectively model data generated by HMMs via in-context learning (ICL)–their ability to infer patterns from examples within a prompt. On a diverse set of synthetic HMMs, LLMs achieve predictive accuracy approaching the theoretical optimum. We uncover novel scaling trends influenced by HMM properties, and offer theoretical conjectures for these empirical observations. We also provide practical guidelines for scientists on using ICL as a diagnostic tool for complex data. On real-world animal decision-making tasks, ICL achieves competitive performance with models designed by human experts. To our knowledge, this is the first demonstration that ICL can learn and predict HMM-generated sequences–an advance that deepens our understanding of in-context learning in LLMs and establishes its potential as a powerful tool for uncovering hidden structure in complex scientific data.

2024

End-to-end Training for Recommendation with Language-based User Profiles

Zhaolin Gao, Joyce Zhou, Yijia Dai, and 1 more author

2024

Abs paper

There is a growing interest in natural language-based user profiles for recommender systems, which aims to enhance transparency and scrutability compared with embedding-based methods. Existing studies primarily generate these profiles using zero-shot inference from large language models (LLMs), but their quality remains insufficient, leading to suboptimal recommendation performance. In this paper, we introduce LangPTune, the first end-to-end training framework to optimize LLM-generated user profiles. Our method significantly outperforms zero-shot approaches by explicitly training the LLM for the recommendation objective. Through extensive evaluations across diverse training configurations and benchmarks, we demonstrate that LangPTune not only surpasses zero-shot baselines but also matches the performance of state-of-the-art embedding-based methods. Finally, we investigate whether the training procedure preserves the interpretability of these profiles compared to zero-shot inference through both GPT-4 simulations and crowdworker user studies.

2023

Representation Learning in Low-rank Slate-based Recommender Systems

Yijia Dai, and Wen Sun

2023

Abs paper Poster

Reinforcement learning (RL) in recommendation systems offers the potential to optimize recommendations for long-term user engagement. However, the environment often involves large state and action spaces, which makes it hard to efficiently learn and explore. In this work, we propose a sample-efficient representation learning algorithm, using the standard slate recommendation setup, to treat this as an online RL problem with low-rank Markov decision processes (MDPs). We also construct the recommender simulation environment with the proposed setup and sampling method.