Our new paper entitled, “Analysis of Q-learning with Adaptation and Momentum Restart for Gradient Descent”, has recently been accepted by the 29th International Joint Conference on Artificial Intelligence (IJCAI), 2020. Existing convergence analyses of Q-learning mostly focus on the vanilla stochastic gradient descent (SGD) type of updates. Despite the Adaptive Moment Estimation (Adam) has been…

To enable quadruped robot to autonomously navigate through complex environment with dynamic obstacles, we have recently developed and validated two types of collision avoidance policies. The first type of policy is represented by a deep neural network trained with typical deep reinforcement learning algorithms (e.g. PPO). The input to the network is the raw Lidar…

Our new paper entitled “Analytical convergence regions of accelerated gradient descent in nonconvex optimization under Regularity Condition” is accepted by Automatica. This paper studies a class of nonconvex optimization problems whose cost functions satisfy the so-called Regularity Condition (RC). Empirical studies show that accelerated gradient descent (AGD) algorithms (e.g. Nesterov’s acceleration and Heavy-ball) with proper…

Our new paper entitled “Optimal Control Inspired Q-Learning for Switched Linear Systems” is accepted by American Control Conference 2020. This paper studies Q-learning for quadratic regulation problem of switched linear systems. Inspired by the analytical results from classical model-based optimal control, a structured Q-learning algorithm is developed. The proposed algorithm consists of a carefully designed…

Our new paper entitled “Connections between mean-field game and social welfare optimization” has been published in Automatica as a regular paper. This paper studies the connection between a class of mean-field games and a social welfare optimization problem. We consider a mean-field game in function spaces with a large population of agents, and each agent…

Our new paper entitled “Optimal Control of a Differentially Flat 2D Spring-Loaded Inverted Pendulum Model” is accepted by IEEE Robotics and Automation Letters (RA-L). We study the optimal control problem of an extended spring-loaded inverted pendulum (SLIP) model with two additional actuators for active leg length and hip torque modulation. These additional features arise naturally…

Take a look at our in-house built quadruped robot with customized motor and body designs. It is designed and built mostly by Shenggao Li (an undergraduate student in our lab). It is expected to have a high power density and is able to accomplish very agile motions. It will be a great experimental platform for…

We present a novel model-free reinforcement learning (RL) framework to design feedback control policies for 3D bipedal walking. Existing RL algorithms are often trained in an end-to-end manner or rely on prior knowledge of some reference joint trajectories. Different from these studies, we propose a novel policy structure that appropriately incorporates physical insights gained from…