Publications

2023

1. 

   Representation Learning in Low-rank Slate-based Recommender Systems

   Yijia Dai, and Wen Sun

   2023

   Abs paper Poster

   Reinforcement learning (RL) in recommendation systems offers the potential to optimize recommendations for long-term user engagement. However, the environment often involves large state and action spaces, which makes it hard to efficiently learn and explore. In this work, we propose a sample-efficient representation learning algorithm, using the standard slate recommendation setup, to treat this as an online RL problem with low-rank Markov decision processes (MDPs). We also construct the recommender simulation environment with the proposed setup and sampling method.

2. 

   Sample Efficient Reinforcement Learning from Human Feedback via Active Exploration

   Viraj Mehta, Vikramjeet Das, Ojash Neopane, and 4 more authors

   2023

   Abs paper

   Preference-based feedback is important for many applications in reinforcement learning where direct evaluation of a reward function is not feasible. A notable recent example arises in reinforcement learning from human feedback (RLHF) on large language models. For many applications of RLHF, the cost of acquiring the human feedback can be substantial. In this work, we take advantage of the fact that one can often choose contexts at which to obtain human feedback in order to most efficiently identify a good policy, and formalize this as an offline contextual dueling bandit problem. We give an upper-confidence-bound style algorithm for this problem and prove a polynomial worst-case regret bound. We then provide empirical confirmation in a synthetic setting that our approach outperforms existing methods. After, we extend the setting and methodology for practical use in RLHF training of large language models. Here, our method is able to reach better performance with fewer samples of human preferences than multiple baselines on three real-world datasets.

2019

1. 

   Growth Mechanisms and the Effects of Deposition Parameters on the Structure and Properties of High Entropy Film by Magnetron Sputtering

   Yanxia Liang, Peipei Wang, Yufei Wang, and 9 more authors

   Materials, 2019

   Abs paper

   Despite intense research on high entropy films, the mechanism of film growth and the influence of key factors remain incompletely understood. In this study, high entropy films consisting of five elements (FeCoNiCrAl) with columnar and nanometer-scale grains were prepared by magnetron sputtering. The high entropy film growth mechanism, including the formation of the amorphous domain, equiaxial nanocrystalline structure and columnar crystal was clarified by analyzing the microstructure in detail. Besides, the impacts of the important deposition parameters including the substrate temperature, the powder loaded in the target, and the crystal orientation of the substrate on the grain size and morphology, phase structure, crystallinity and elemental uniformity were revealed. The mechanical properties of high entropy films with various microstructure features were investigated by nanoindentation. With the optimized grain size and microstructure, the film deposited at 350 °C using a power of 100 W exhibits the highest hardness of 11.09 GPa. Our findings not only help understanding the mechanisms during the high entropy film deposition, but also provide guidance in manufacturing other novel high entropy films.