Research & Projects

Read my MS thesis here. 

ABSTRACT
Robots should be able to operate and fit into workflows in human-centric, unconstrained environments. In the manipulation domain, being able to perform non-prehensile contact maneuvers remains a missing piece in enabling this. Prior work has approached the problem by splitting a manipulation task into sequential subtasks which is restrictive and may not lead to optimal solutions. Moreover, this method does not scale well for more complex, contact-rich manipulation tasks. On the other hand, posing the problem as a global optimal control problem and implicitly reasoning about contact interactions is intractable for online, reactive control. Recent work uses local approximations to perform real-time contact-implicit control but is limited by the local nature of the underlying simplification. We address the need for a globally aware, contact-implicit controller capable of real-time control by breaking the problem down into making two distinct hybrid decisions: (a) finding advantageous locations from where local contact implicit control is possible by sampling in a low-dimensional subspace of the state space, and (b) performing local contact-implicit control. We present results on real-time multi-contact control of a complex non-convex toy jack in simulation.

A preliminary version of this was presented at IROS 2023 where we were invited to give a spotlight talk at the Workshop on Leveraging Models for Contact-Rich Manipulation. Read our workshop paper here.

Here is a link to the current iteration of source code. Please note this is in-development, research-grade code.

We are currently working on demonstrating our algorithm on hardware - stay tuned! 

ABSTRACT
Bipedal robots are promising for traversing through rough terrain quickly and efficiently due to their ability to make and break contact with the ground, however real-time footstep planning over rough terrain remains a challenge. Exteroceptive sensing modalities such as RGB-D cameras are usually used to capture the terrain landscape for downstream footstep planning algorithms. However, existing approaches use relatively brittle, handcrafted pipelines for elevation mapping and foothold segmentation. Therefore, we propose a project that learns to adjust the optimal footstep location depending on the terrain. In order to leverage the capabilities of deep learning for robust vision processing in a way that is composable with existing model-based approaches, we propose learning a terrain-conditioned residual to the LQR action-value function (Q-function) for a low dimensional, linear model of bipedal locomotion. By the end of the course, we have developed a complete simulation environment, data collection pipeline, and had several training, validation and testing iterations. We have also implemented an LQR controller with the learned residual, tested the closed loop behavior in simulation in real time, and had some preliminary results showing the potential improvement of bipedal locomotion on rough terrain with vision input using the proposed approach.

Please note that this work is unpublished at present. Video and report available on request.

Watch our Franka Emika Panda robot as it performs pick and place tasks using our custom arm control library. This library combines forward kinematics with Denavit-Hartenberg (DH) parameters, inverse kinematics with kinematic decoupling, and geometric Jacobian solutions for precise velocity control.

In this video, you'll see:
Advanced Control Algorithms: Seamless integration of forward kinematics, inverse kinematics, and velocity control for optimal performance.
Tag-Based Pose Estimation: Accurate detection and positioning of blocks for precise pick and place operations.
Efficient Task Execution: Watch the robot pick blocks and stack them perfectly at the designated destination stage.