Our paper entitled "Instantaneous Capture Input for Balancing the Variable Height Inverted Pendulum" is accepted by RA-L/IROS 2021. Balancing is a fundamental ability that a legged robot, especially a bipedal robot, should possess. Throughout the history of legged locomotion literature, balance control has been extensively studied for simple models such as the linear inverted pendulum thanks to the…

Our paper entitled "Force-feedback based Whole-body Stabilizer for Position-Controlled Humanoid Robots" is accepted by IROS 2021. This paper studies stabilizer design for position controlled humanoid robots. Stabilizers are essential for position-controlled humanoids, whose primary objective is to adjust the control input sent to the robot to enhance the performance of trackingthe planned reference trajectory.Conventional stabilizer design techniques…

Our paper entitled "Quadruped Robot Hopping on Two Legs" is accepted by IROS 2021. This paper presents a control strategy for quadruped robots to hop on their rear legs in three-dimensional space. The proposed approach generates nominal center of mass (CoM) trajectories based on a template spring-loaded inverted pendulum (SLIP) model. Tracking this reference remains a challenge due to…

Our paper entitled “Encirclement Guaranteed Cooperative Pursuit with Robust Model Predictive Control” is accepted by IROS 2021. In this paper, a novel encirclement guaranteed cooperative pursuit problem involving N pursuers and a single evader in an unbounded two-dimensional game domain is studied. Throughout the game, the evader’s action is unavailable to the pursuers. Moreover, the pursuers are required to…

Our paper entitled “Robust Feedback Motion Policy Design Using Reinforcement Learning on a 3D Digit Bipedal Robot” is accepted by IROS 2021. Dynamic locomotion for humanoids is perhaps one of the most challenging problems in the legged and even the general robotics literature. Recently, reinforcement learning techniques have been attracting a considerable amount of research attentions. In this paper,…

Our paper entitled “Non-asymptotic Convergence of Adam-type Reinforcement Learning Algorithms under Markovian Sampling” has been accepted to AAAI 2021. This paper provides the first such convergence analysis for two fundamental RL algorithms of policy gradient (PG) and temporal difference (TD) learning that incorporate AMSGrad updates (a standard alternative of Adam in theoretical analysis), referred to…

Our paper entitled “Finite-Time Analysis for Double Q-learning” has been accepted for spotlight presentation at NeurIPS 2020! Only 280 spotlight presentations were accepted to the conference among 1900 accepted papers and a record-breaking 9454 submissions! In this paper, we provide the first non-asymptotic (i.e., finite-time) analysis for double Q-learning. We show that both synchronous and…

Our paper entitled “Underactuated Motion Planning and Control for Jumping with Wheeled-Bipedal Robots” is accepted by IEEE Robotics and Automation Letters This paper studies jumping for wheeled-bipedal robots, a motion that takes full advantage of the benefits from the hybrid wheeled and legged design features. A comprehensive hierarchical scheme for motion planning and control of…

We propose a novel perception-aware planning and control scheme for quadruepdal stair climbing. The quadruped is able to autonomously climb various real-world staircases with dynamical gaits. All computations are done online and using only on-board processor. No external inputs from human-operated remote controller and no additional parameter tuning are required for different staircases. Exploiting prior…

Our lab recently developed a new push recovery strategy for quadruped. Most existing push recovery strategies rely on Raibert heuristic regulators to adjust foot positions according to body velocities. Our proposed strategy is quite different. It is based on reachability computation, which dramatically improves the push recovery performance with theoretical support. We test our approach…

Our paper entitled “Transactive energy systems: The market-based coordination of distributed energy resources” appeared on IEEE Control Systems Magazine as the cover page paper!The increasing penetration of renewable energy poses significant challenges to the planning and operation of modern power grids. One promising solution to attenuate this challenge is by integrating the distributed energy resources…

Our new paper entitled, “Analysis of Q-learning with Adaptation and Momentum Restart for Gradient Descent”, has recently been accepted by the 29th International Joint Conference on Artificial Intelligence (IJCAI), 2020. Existing convergence analyses of Q-learning mostly focus on the vanilla stochastic gradient descent (SGD) type of updates. Despite the Adaptive Moment Estimation (Adam) has been…

To enable quadruped robot to autonomously navigate through complex environment with dynamic obstacles, we have recently developed and validated two types of collision avoidance policies. The first type of policy is represented by a deep neural network trained with typical deep reinforcement learning algorithms (e.g. PPO). The input to the network is the raw Lidar…

Our new paper entitled “Analytical convergence regions of accelerated gradient descent in nonconvex optimization under Regularity Condition” is accepted by Automatica. This paper studies a class of nonconvex optimization problems whose cost functions satisfy the so-called Regularity Condition (RC). Empirical studies show that accelerated gradient descent (AGD) algorithms (e.g. Nesterov’s acceleration and Heavy-ball) with proper…

Our new paper entitled “Optimal Control Inspired Q-Learning for Switched Linear Systems” is accepted by American Control Conference 2020. This paper studies Q-learning for quadratic regulation problem of switched linear systems. Inspired by the analytical results from classical model-based optimal control, a structured Q-learning algorithm is developed. The proposed algorithm consists of a carefully designed…

Our new paper entitled “Connections between mean-field game and social welfare optimization” has been published in Automatica as a regular paper. This paper studies the connection between a class of mean-field games and a social welfare optimization problem. We consider a mean-field game in function spaces with a large population of agents, and each agent…

Our new paper entitled “Optimal Control of a Differentially Flat 2D Spring-Loaded Inverted Pendulum Model” is accepted by IEEE Robotics and Automation Letters (RA-L). We study the optimal control problem of an extended spring-loaded inverted pendulum (SLIP) model with two additional actuators for active leg length and hip torque modulation. These additional features arise naturally…

Take a look at our in-house built quadruped robot with customized motor and body designs. It is designed and built mostly by Shenggao Li (an undergraduate student in our lab). It is expected to have a high power density and is able to accomplish very agile motions. It will be a great experimental platform for…

We present a novel model-free reinforcement learning (RL) framework to design feedback control policies for 3D bipedal walking. Existing RL algorithms are often trained in an end-to-end manner or rely on prior knowledge of some reference joint trajectories. Different from these studies, we propose a novel policy structure that appropriately incorporates physical insights gained from…