Our paper entitled “Non-asymptotic Convergence of Adam-type Reinforcement Learning Algorithms under Markovian Sampling” has been accepted to AAAI 2021. This paper provides the first such convergence analysis for two fundamental RL algorithms of policy gradient (PG) and temporal difference (TD) learning that incorporate AMSGrad updates (a standard alternative of Adam in theoretical analysis), referred to…

Our paper entitled “Finite-Time Analysis for Double Q-learning” has been accepted for spotlight presentation at NeurIPS 2020! Only 280 spotlight presentations were accepted to the conference among 1900 accepted papers and a record-breaking 9454 submissions! In this paper, we provide the first non-asymptotic (i.e., finite-time) analysis for double Q-learning. We show that both synchronous and…

Our paper entitled “Underactuated Motion Planning and Control for Jumping with Wheeled-Bipedal Robots” is accepted by IEEE Robotics and Automation Letters This paper studies jumping for wheeled-bipedal robots, a motion that takes full advantage of the benefits from the hybrid wheeled and legged design features. A comprehensive hierarchical scheme for motion planning and control of…

We propose a novel perception-aware planning and control scheme for quadruepdal stair climbing. The quadruped is able to autonomously climb various real-world staircases with dynamical gaits. All computations are done online and using only on-board processor. No external inputs from human-operated remote controller and no additional parameter tuning are required for different staircases. Exploiting prior…

Our lab recently developed a new push recovery strategy for quadruped. Most existing push recovery strategies rely on Raibert heuristic regulators to adjust foot positions according to body velocities. Our proposed strategy is quite different. It is based on reachability computation, which dramatically improves the push recovery performance with theoretical support. We test our approach…

Our paper entitled “Transactive energy systems: The market-based coordination of distributed energy resources” appeared on IEEE Control Systems Magazine as the cover page paper!The increasing penetration of renewable energy poses significant challenges to the planning and operation of modern power grids. One promising solution to attenuate this challenge is by integrating the distributed energy resources…

Our new paper entitled, “Analysis of Q-learning with Adaptation and Momentum Restart for Gradient Descent”, has recently been accepted by the 29th International Joint Conference on Artificial Intelligence (IJCAI), 2020. Existing convergence analyses of Q-learning mostly focus on the vanilla stochastic gradient descent (SGD) type of updates. Despite the Adaptive Moment Estimation (Adam) has been…

To enable quadruped robot to autonomously navigate through complex environment with dynamic obstacles, we have recently developed and validated two types of collision avoidance policies. The first type of policy is represented by a deep neural network trained with typical deep reinforcement learning algorithms (e.g. PPO). The input to the network is the raw Lidar…

Our new paper entitled “Analytical convergence regions of accelerated gradient descent in nonconvex optimization under Regularity Condition” is accepted by Automatica. This paper studies a class of nonconvex optimization problems whose cost functions satisfy the so-called Regularity Condition (RC). Empirical studies show that accelerated gradient descent (AGD) algorithms (e.g. Nesterov’s acceleration and Heavy-ball) with proper…

Our new paper entitled “Optimal Control Inspired Q-Learning for Switched Linear Systems” is accepted by American Control Conference 2020. This paper studies Q-learning for quadratic regulation problem of switched linear systems. Inspired by the analytical results from classical model-based optimal control, a structured Q-learning algorithm is developed. The proposed algorithm consists of a carefully designed…

Our new paper entitled “Connections between mean-field game and social welfare optimization” has been published in Automatica as a regular paper. This paper studies the connection between a class of mean-field games and a social welfare optimization problem. We consider a mean-field game in function spaces with a large population of agents, and each agent…

Our new paper entitled “Optimal Control of a Differentially Flat 2D Spring-Loaded Inverted Pendulum Model” is accepted by IEEE Robotics and Automation Letters (RA-L). We study the optimal control problem of an extended spring-loaded inverted pendulum (SLIP) model with two additional actuators for active leg length and hip torque modulation. These additional features arise naturally…

Take a look at our in-house built quadruped robot with customized motor and body designs. It is designed and built mostly by Shenggao Li (an undergraduate student in our lab). It is expected to have a high power density and is able to accomplish very agile motions. It will be a great experimental platform for…

We present a novel model-free reinforcement learning (RL) framework to design feedback control policies for 3D bipedal walking. Existing RL algorithms are often trained in an end-to-end manner or rely on prior knowledge of some reference joint trajectories. Different from these studies, we propose a novel policy structure that appropriately incorporates physical insights gained from…