Yijia Dai

👋 ▫️ 👍 👀 ❕

Learning is key for any type of intelligence. I love thinking about how human learns, and whether that is transferable to machines. Questions like How do different sequential orderings of data and knowledge affect learning? intrigue me.

My research interests include developing self-improvement mechanisms for large models, interpreting machine intelligence, and creating AI systems that enhance people’s daily lives.

I’m fortunate to be working with Prof. Sarah Dean, Prof. Thorsten Joachims, and Prof. Jennifer Sun at Cornell. Currently, I study reinforcement learning, human-in-loop dynamical systems, and LLMs for scientific discovery. Sample-efficient low-rank representation learning for dynamical and unobserved user states, optimal learning of logistic bandits for recommendation systems, and LLMs for understanding decision trajectories are examples of what I work on.

Publications

2023

Representation Learning in Low-rank Slate-based Recommender Systems

Yijia Dai, and Wen Sun

2023

Abs paper Poster

Reinforcement learning (RL) in recommendation systems offers the potential to optimize recommendations for long-term user engagement. However, the environment often involves large state and action spaces, which makes it hard to efficiently learn and explore. In this work, we propose a sample-efficient representation learning algorithm, using the standard slate recommendation setup, to treat this as an online RL problem with low-rank Markov decision processes (MDPs). We also construct the recommender simulation environment with the proposed setup and sampling method.
Sample Efficient Reinforcement Learning from Human Feedback via Active Exploration

Viraj Mehta, Vikramjeet Das, Ojash Neopane, and 4 more authors

2023

Abs paper

Preference-based feedback is important for many applications in reinforcement learning where direct evaluation of a reward function is not feasible. A notable recent example arises in reinforcement learning from human feedback (RLHF) on large language models. For many applications of RLHF, the cost of acquiring the human feedback can be substantial. In this work, we take advantage of the fact that one can often choose contexts at which to obtain human feedback in order to most efficiently identify a good policy, and formalize this as an offline contextual dueling bandit problem. We give an upper-confidence-bound style algorithm for this problem and prove a polynomial worst-case regret bound. We then provide empirical confirmation in a synthetic setting that our approach outperforms existing methods. After, we extend the setting and methodology for practical use in RLHF training of large language models. Here, our method is able to reach better performance with fewer samples of human preferences than multiple baselines on three real-world datasets.

2019

Growth Mechanisms and the Effects of Deposition Parameters on the Structure and Properties of High Entropy Film by Magnetron Sputtering

Yanxia Liang, Peipei Wang, Yufei Wang, and 9 more authors

Materials, 2019

Abs paper

Despite intense research on high entropy films, the mechanism of film growth and the influence of key factors remain incompletely understood. In this study, high entropy films consisting of five elements (FeCoNiCrAl) with columnar and nanometer-scale grains were prepared by magnetron sputtering. The high entropy film growth mechanism, including the formation of the amorphous domain, equiaxial nanocrystalline structure and columnar crystal was clarified by analyzing the microstructure in detail. Besides, the impacts of the important deposition parameters including the substrate temperature, the powder loaded in the target, and the crystal orientation of the substrate on the grain size and morphology, phase structure, crystallinity and elemental uniformity were revealed. The mechanical properties of high entropy films with various microstructure features were investigated by nanoindentation. With the optimized grain size and microstructure, the film deposited at 350 °C using a power of 100 W exhibits the highest hardness of 11.09 GPa. Our findings not only help understanding the mechanisms during the high entropy film deposition, but also provide guidance in manufacturing other novel high entropy films.