About Me
Greetings! I am currently a first-year PhD student in the Department of Mechanical Engineering at The University of Tokyo, supervised by Lecturer Moju ZHAO.
My research interest lies in optimization-based control with applications in aerial manipulation, aiming to make aerial robots function as flying hands rather than just eyes.
I obtained an M.Sc. in Control Science and Engineering (2023) and a B.Eng. in Automation (2020) from Beihang University. Please refer to my CV for more information.
News
- Aug. 2024 : Invited as a reviewer for RA-L!
- Aug. 2024 : One paper has been accepted by RA-L! A great beginning for my PhD journey.
- May 2024 : Participated in ICRA 2024 (Yokohama, Japan)! One poster was presented!
- Apr. 2024 : Invited as a reviewer for IROS 2024!
Old News
- Dec. 2023 : Participated in CDC 2023 (Singapore), my first in-person conference! One paper was presented!
- Nov. 2023 : Gave a talk at CDC Workshop on Benchmarking, Reproducibility, and Open-Source Code in Controls
- Oct. 2023 : Invited as a reviewer for ICRA 2024!
- Oct. 2023 : Admitted to the University of Tokyo as a PhD student, advised by Dr. Moju ZHAO!
- July 2023 : One paper has been accepted by CDC 2023!
- June 2023 : Graduated from Beihang University with a Master's degree!
- Apr. 2023 : Invited as a reviewer for CDC 2023!
- Apr. 2023 : One paper has been accepted by ICRA 2023 workshop “The Role of Robotics Simulators for Unmanned Aerial Vehicles”!
- Jan. 2023 : One paper has been accepted by ICRA 2023!
Research
▪️ Jinjie Li, Junichiro Sugihara, Moju Zhao
▪️ IEEE Robotics and Automation Letters (RA-L)
🕗 Oct. 2023 - Aug. 2024
🔗 [paper] [video]
To the best of the authors’ knowledge, this is the first time an actuator-level NMPC is verified on a real tiltable-quadrotor.
Abstract
Utilizing a servo to tilt each rotor transforms quadrotors from underactuated to overactuated systems, allowing for independent control of both attitude and position, which provides advantages for aerial manipulation. However, this enhancement also introduces model nonlinearity, sluggish servo response, and limited operational range into the system, posing challenges to dynamic control. In this study, we propose a control approach for tiltable-quadrotors based on nonlinear model predictive control (NMPC). Unlike conventional cascade methods, our approach preserves the full dynamics without simplification. It directly uses rotor thrust and servo angle as control inputs, where their limited working ranges are considered input constraints. Notably, we incorporate a first-order servo model within the NMPC framework. Simulation reveals that integrating the servo dynamics is not only an enhancement to control performance but also a critical factor for optimization convergence. To evaluate the effectiveness of our approach, we fabricate a tiltable-quadrotor and deploy the algorithm onboard at 100 Hz. Extensive real-world experiments demonstrate rapid, robust, and smooth pose-tracking performance.▪️ Jinjie Li, Liang Han, Haoyang Yu, et al.
▪️ IEEE Conference on Decision and Control (CDC23); Master’s Thesis
🕗 Nov. 2021 - May 2023
🔗 [paper] [thesis_zh] [code]
The downwash effect caused by other agents is a unique problem for aerial robotics and is hard to model. How could aerial robots observe the downwash effect and integrate it into the state-of-the-art trajectory tracking framework?
Abstract
Swarm aerial robots are required to maintain close proximity to successfully traverse narrow areas in cluttered environments. However, this movement is affected by the downwash effect generated from other quadrotors in the swarm. This aerodynamic effect is highly nonlinear and hard to describe through mathematical modeling. Additionally, the existence of the downwash disturbance can be predicted based on the states of neighboring quadrotors. If this prediction is considered, the control loop can proactively handle the disturbance, resulting in improved performance.To address these challenges, we propose an approach that integrates a Neural network Downwash Predictor with Nonlinear Model Predictive Control (NDP-NMPC). The neural network is trained with spectral normalization to ensure robustness and safety in uncollected cases. The predicted disturbances are then incorporated into the optimization scheme in NMPC, which enforces constraints to ensure that states and inputs remain within safe limits. We also design a quadrotor system, identify its parameters, and implement the proposed method on board. Finally, we conduct a prediction experiment to validate the safety and effectiveness of the network. In addition, a real-time trajectory tracking experiment is performed with the entire system, demonstrating a 75.37% reduction in tracking error in height under the downwash effect.
▪️ Jinjie Li, Liang Han, Haoyang Yu, et al.
▪️ ICRA 2023 Workshop on
The Role of Robotics Simulators for Unmanned Aerial Vehicles
🕗 Sept. 2020 - Mar. 2023
🔗 [paper] [poster] [code]
Develop a simulation platform for mobile robotics based on data-oriented programming, supporting the simulation of over 1000 nodes.
Abstract
Large-scale simulation with realistic nonlinear dynamic models is crucial for algorithms development for swarm robotics. However, existing platforms are mainly developed based on Object-Oriented Programming (OOP) and either use simple kinematic models to pursue a large number of simulating nodes or implement realistic dynamic models with limited simulating nodes. In this paper, we develop a simulator based on Data-Oriented Programming (DOP) that utilizes GPU parallel computing to achieve large-scale swarm robotic simulations. Specifically, we use a multi-process approach to simulate different kinds of agents and leverage PyTorch with GPU to simulate one kind of agents with a large number to achieve large-scale parallel computing for swarm robotics. We test our approach using a nonlinear quadrotor model and demonstrate that this DOP approach can maintain almost the same computational speed when quadrotors are less than 5,000. We also provide two examples to demonstrate the functionality of the platform.▪️ Jinjie Li, Liang Han, Zhang Ren
▪️ IEEE International Conference on Robotics and Automation (ICRA22)
🕗 May 2021 - Feb. 2022
🔗 [oral] [paper] [video]
This work proposes a real-time centimeter-level indoor localization method based on invisible projected tags (IPT).
Abstract
Augmented reality (AR) technology has been introduced into the robotics field to narrow the visual gap between indoor and outdoor environments. However, without signals from satellite navigation systems, flight experiments in these indoor AR scenarios need other accurate localization approaches. This work proposes a real-time centimeter-level indoor localization method based on psycho-visually invisible projected tags (IPT), requiring a projector as the sender and quadrotors with high-speed cameras as the receiver. The method includes a modulation process for the sender, as well as demodulation and pose estimation steps for the receiver, where screen-camera communication technology is applied to hide fiducial tags using human vision property. Experiments have demonstrated that IPT can achieve accuracy within ten centimeters and a speed of about ten FPS. Compared with other localization methods for AR robotics platforms, IPT is affordable by using only a projector and high-speed cameras as hardware consumption and convenient by omitting a coordinate alignment step. To the authors' best knowledge, this is the first time screen-camera communication is utilized for AR robot localization.Projects
▪️ Bachelor’s Thesis, Outstanding Bachelor’s Thesis Award
🕗 Dec. 2019 - June 2020
🔗 [thesis_zh] [code]
The thesis contains three parts. (1) The fiducial tags on the ground are captured for visual positioning, and the results are combined with IMU to improve reliability and accuracy; (2) Dyna-Q reinforcement learning algorithm is implemented to train a multi-quadrotor system for formation control; (3) The algorithms are verified in a ROS/Gazebo-based simulation platform. The thesis is ranked No.1 in my major.
Abstract
In recent years, the vigorous development of computer vision and artificial intelligence has created opportunities for more intelligent drones. This thesis implements the fiducial-tag positioning method and the Dyna-Q reinforcement learning algorithm in a multi-quadrotor formation control system. This combination improves the autonomy and anti-interference of quadrotors in complex environments.The thesis contains three parts. For the first part, the basic principles of visual positioning are studied. Then, the basic knowledge of computer vision is applied to achieve camera calibration, tags detection, decoding, and pose solving to obtain pose data. Next, the pose data is integrated with an inertial measurement unit (IMU) to provide continuous and accurate positions in the sparse-tag scenario. For the second part, the formation problem is decomposed into a target tracking task and a collision avoidance task, which are designed as Markov Decision Processes (MDPs), respectively. Then the Dyna-Q algorithm is utilized for training these tasks. Finally, the positioning and formation control task of five drones are completed in a ROS/Gazebo-based simulation platform, achieving less than 3cm formation error.
▪️ China Aeromodelling Design Challenge (CADC), Leader of the Composite Material Team & Pilot
🕗 July 2017 - Oct. 2018
🔗 [BMFA News Magazine report] [tech details & video] [pilot training & video]
Designed and produced the composite part of a 5m-wingspan aircraft with the maximum load of 24kg, the maximum take-off weight of 27.5kg, and the nominal airspeed of 15m/s. Applied the carbon-PMI-carbon sandwich structure to make the 130g-weighted single wing beam, outer twined with the Kevlar line; utilized carbon and glass fiber reinforced polymer (CGFRP) to make D-box structure, increasing the torsional rigidity to 261.07%. Won the top three places in the 2018 CADC (Time-limited Airdrop Project), the best record in history. Served as the pilot of solar aircrafts as well.
▪️ Course: Fundamentals of Analog Electronics, Team Leader
🕗 Feb. 2018 - June 2018
🔗 [video] [details] [code & PCB]
Developed a physical temperature control system for a water heater from scratch, which could set and maintain the temperature between 50°C ~ 100°C. The system was controlled via Bluetooth, powered by 220V, and was able to warm up or cool down to a specified temperature in 5 minutes. Ranked No.1 in the class.
Talks
-
2023.12 - Conference Presentation - [slides]
Nonlinear MPC for Quadrotors in Close-Proximity Flight with Neural Network Downwash Prediction,
CDC, 2023 -
2023.12 - Conference Presentation - [slides] [recording]
The Open-Source Development of UTokyo Aerial Robot Team: From Hardware to Software,
CDC Workshop on Benchmarking, Reproducibility, and Open-Source Code in Controls, 2023 -
2022.04 - Introduction & Advanced Topic of Zotero - [slides]
-
2022.03 - Conference Presentation - [slides] [recording]
Indoor Localization for Quadrotors using Invisible Projected Tags, ICRA, 2022
Past Literature Sharings
-
2022.12 - Literature Sharing - [slides]
Accommodating unobservability to control flight attitude with optic flow, Nature, 2022
-
2021.12 - Literature Sharing - [slides]
Neural Lander: Stable Drone Landing Control Using Learned Dynamics, ICRA, 2019
-
2021.10 - Literature Sharing - [slides]
Lyapunov-stable neural-network control, RSS, 2021
-
2021.03 - Literature Sharing - [slides]
Glider soaring via reinforcement learning in the field, Nature, 2018
Learning to soar in turbulent environments, PNAS, 2016
-
2020.10 - Literature Sharing - [slides]
Graph Neural Networks for Decentralized Multi-Robot Path Planning, IROS, 2020
Misc.
I have loved model airplanes for over ten years, and now luckily it has become my field of study. Besides, I love travelling🌏, photography📷, skiing⛷️, snowboarding🏂, and many kinds of ball games 🎾🏓🏸🏀. If you are interested in the photos below, please email me or jump to my photography home page for more pictures.
