Yijia Dai

👋 ▫️ 👍 👀 ❕

Learning is key for any type of intelligence. I love thinking about how human learns, and whether that is transferable to machines. Questions like Why ML requires more data than human?intrigue me.

My research interests are broadly learning robust user representations, and aligning machine to human reasoning. I’m fortunate to be working with Prof. Sarah Dean, Prof. Wen Sun, and Prof. Thorsten Joachims at Cornell. Currently, I study reinforcement learning and human-in-loop dynamical systems. Sample-efficient low-rank representation learning for dynamical and unobserved user states, computationally-efficient weighted logistic bandits, and “algorithm-of-thoughts” styled personalized question-understanding system are examples of what I am working on.

Publications

2023

Representation Learning in Low-rank Slate-based Recommender Systems

Yijia Dai, and Wen Sun

2023

Abs paper Poster

Reinforcement learning (RL) in recommendation systems offers the potential to optimize recommendations for long-term user engagement. However, the environment often involves large state and action spaces, which makes it hard to efficiently learn and explore. In this work, we propose a sample-efficient representation learning algorithm, using the standard slate recommendation setup, to treat this as an online RL problem with low-rank Markov decision processes (MDPs). We also construct the recommender simulation environment with the proposed setup and sampling method.
Sample Efficient Reinforcement Learning from Human Feedback via Active Exploration

Viraj Mehta, Vikramjeet Das, Ojash Neopane, and 4 more authors

2023

Abs paper

Preference-based feedback is important for many applications in reinforcement learning where direct evaluation of a reward function is not feasible. A notable recent example arises in reinforcement learning from human feedback (RLHF) on large language models. For many applications of RLHF, the cost of acquiring the human feedback can be substantial. In this work, we take advantage of the fact that one can often choose contexts at which to obtain human feedback in order to most efficiently identify a good policy, and formalize this as an offline contextual dueling bandit problem. We give an upper-confidence-bound style algorithm for this problem and prove a polynomial worst-case regret bound. We then provide empirical confirmation in a synthetic setting that our approach outperforms existing methods. After, we extend the setting and methodology for practical use in RLHF training of large language models. Here, our method is able to reach better performance with fewer samples of human preferences than multiple baselines on three real-world datasets.

2019

Growth Mechanisms and the Effects of Deposition Parameters on the Structure and Properties of High Entropy Film by Magnetron Sputtering

Yanxia Liang, Peipei Wang, Yufei Wang, and 9 more authors

Materials, 2019

Abs paper

Despite intense research on high entropy films, the mechanism of film growth and the influence of key factors remain incompletely understood. In this study, high entropy films consisting of five elements (FeCoNiCrAl) with columnar and nanometer-scale grains were prepared by magnetron sputtering. The high entropy film growth mechanism, including the formation of the amorphous domain, equiaxial nanocrystalline structure and columnar crystal was clarified by analyzing the microstructure in detail. Besides, the impacts of the important deposition parameters including the substrate temperature, the powder loaded in the target, and the crystal orientation of the substrate on the grain size and morphology, phase structure, crystallinity and elemental uniformity were revealed. The mechanical properties of high entropy films with various microstructure features were investigated by nanoindentation. With the optimized grain size and microstructure, the film deposited at 350 °C using a power of 100 W exhibits the highest hardness of 11.09 GPa. Our findings not only help understanding the mechanisms during the high entropy film deposition, but also provide guidance in manufacturing other novel high entropy films.