à£à£Ö±²¥Ðã

In new research from South à£à£Ö±²¥Ðã State University, a novel framework for accelerating the speed at which reinforcement learning algorithms are trained has been developed by researchers in the Jerome J. Lohr College of Engineering. 

From autonomous vehicles to natural language processing to trading and finance, reinforcement learning is all around us. But reinforcement learning — a type of artificial intelligence that teaches "intelligent agents" how to take actions that maximize reward signals — has a challenge: it's time consuming and unpredictable in the early stages of learning.

"For complex tasks, reinforcement learning is time-consuming due to the need for large amounts of training data," said Jun Huang, assistant professor in South à£à£Ö±²¥Ðã State University’s Jerome J. Lohr College of Engineering. 

Huang, whose research focuses on machine learning in mobile networks, is developing a solution. In a new paper published in the academic journal Institute of Electrical and Electronic Engineers Transactions on Mobile Computing, Huang designs an acceleration framework that is three and seven times more effective and efficient than standard reinforcement learning algorithms.

Huang and Beining Wu, graduate research assistant in SDSU'sMcComish Department of Electrical Engineering and Computer Science, implemented this framework while conducting research on unmanned aerial vehicles — otherwise known as drones — and their communication systems.

UAV communications

Drones are controlled by signals transmitted from the ground level. In larger cities, these signals are often blocked by buildings, making it necessary for reconfigurable intelligent surfaces to assist in the drones' communications. Reconfigurable intelligent surfaces function as mirrors to amplify the signal from the ground terminal, where the drone is being controlled, to the sky. It also provides better connectivity and a larger area of coverage for drone usage.

"In urban areas, reconfigurable intelligent surfaces are particularly useful for overcoming obstacles, reducing signal interference, and extending the range of unmanned aerial vehicle communications," Huang said. "This new technology facilitates the implementation of space-air-ground integrated systems within the 6G technology landscape."

The unique nature of drones, which operate with limited power capacity and in complex communication spaces, requires missions and flight paths to be preplanned, especially when information gathering, or data retrieval, is the focus of the mission. The researchers refer to this as "trajectory planning."

This is where the researchers' accelerated framework comes into play. Drones rely heavily on machine learning applications — reinforcement learning in particular — for their operation in areas such as navigation, obstacle avoidance and autonomous decision-making. But as noted previously, reinforcement learning is time-consuming. Let's say a drone is flying in an urban area and signals are being obstructed or there's some sort of unforeseen challenge that throws off the initial trajectory plan. Because drones have a limited power supply, they must be able to quickly make a new trajectory plan to successfully complete the mission. If using standard reinforcement learning, this would take far too long in the air. But with Huang's and Wu's new accelerated framework, the drones can quickly adapt to challenges with faster trajectory planning.

The researchers named their framework "FedX" and believe it will allow for faster and more efficient networking innovations in the 6G technology landscape.

"By developing novel models and new algorithms, researchers and practitioners can gain not only in-depth insight into the complex nature of unmanned aerial vehicle communications but also design fast machine learning algorithms by following our outcomes," Wu said.

Funding for this research was provided by the National Science Foundation.