Research Projects on Reinforcement Learning

Deep Learning for Transportation Mode Detection

This projects introduces a light-weighted and energy-efficient transportation mode detection system using only accelerometer sensors in smartphones. The system utilizes convolutional neural networks (CNNs) to detect transportation modes with the accelerometer only, which is the first of its kind. The collected data are processed by removing gravity and smoothing, and the acceleration magnitude is used to build a CNN to recognize the transportation mode. The proposed CNN outperformed other variations of CNNs tested in terms of accuracy and convergence speed. It achieved higher accuracy compared to past studies that used only acceleration data to detect transportation modes.

rl1

Reinforcement Learning for Traffic Signal Control

Existing inefficient traffic light cycle control causes problems such as long delays and waste of energy. The proposed deep reinforcement learning model uses real-time traffic information to dynamically adjust the traffic light duration. The model quantifies the traffic scenario as states by collecting traffic data and dividing the intersection into small grids. The duration changes of a traffic light are the actions, which are modeled as a high-dimension Markov decision process. The proposed model incorporates multiple optimization elements to improve performance, such as dueling network, target network, double Q-learning network, and prioritized experience replay.

rl2

Reinforcement Learning for Safety-enhanced Traffic Signal Control

Traffic signal control is safety-critical, as a significant number of road accidents occur at intersections due to problematic signal timing. Existing studies on adaptive traffic signal control using reinforcement learning (RL) technologies have focused on minimizing traffic delay but have neglected the potential exposure to unsafe conditions. The proposed SafeLight method incorporates road safety standards as enforcement to ensure the safety of RL-based traffic signal control methods, aiming towards operating intersections with zero collisions. SafeLight employs multiple optimization techniques, such as a multi-objective loss function and reward shaping, to integrate safety into RL models and improve both safety and mobility. Extensive experiments using synthetic and real-world benchmark datasets show that SafeLight significantly reduces collisions while increasing traffic mobility, achieving over 99% reduction in collisions compared to the backbone RL model and about 30% lower average waiting time than the Fixed-time control.

rl3