UNCW Researchers Advance Robotics Through Human Gaze

For someone who has lost the ability to use their hands, a glance could one day be enough to control a robotic assistant. At UNCW, intelligent systems engineering students and faculty researchers are working together to advance gaze-guided robotics, turning eye movements into commands that could increase independence and improve quality of life. 

Student researchers and recent alumni Joseph Scott ’26, Charlie Caputo ’26 and Andrew Lawson ’24 joined Assistant Professor of Computer Science Hamed Saeidi in developing a system that uses eye-tracking technology to guide robotic movement, offering new possibilities for people who cannot rely on their hands to interact with the world around them. 

“For individuals with severe mobility impairments, like paralysis or Parkinson's, regaining independence is life-changing,” Saeidi said. “Since joysticks and touchscreens aren't always options, using natural eye movement offers a direct, hands-free way to interact with the world.” 

One obstacle researchers aimed to solve was the eye's tendency to make small, rapid movements while tracking motion, a behavior that can cause shaky, uneven robotic movements.  

“Our team built a smart controller that predicts eye movement in real time,” Saeidi said. “It automatically smooths out sudden eye jumps and keeps the robot steady, creating a natural and effortless way to control the robot and move objects in the environment.” 

The researchers used a Universal Robots robotic arm for the user to control, a mixed-reality smart glasses headset that tracked the user’s gaze and voice commands, and multiple coding languages to develop and test the controller. The main form of programming uses a method called Model Predictive Control, which predicts future robot and gaze behavior and adjusts in real time, Scott said.  

To prove that the technology works, researchers had the robot carry an open cup of liquid without spilling.   

“Showing that someone can safely move liquid using only their eyes demonstrates that this technology can handle everyday activities like drinking or picking up items, giving people far greater independence,” Saeidi said.  

Tasked with improving the robot’s response to eye movements, Scott concentrated on making the technology feel intuitive for users. Through hands-on testing, he adjusted the system’s speed and reaction time, refining the robot’s movements to better match how people naturally expect it to respond, he said.  

“This culminated with us performing human subject tests, where volunteers used the system so we could collect data and user feedback,” Scott added.  ‘We tested three ways for users to move a robot arm using just their eye gaze and compared our new controllers to those from our earlier work.” 

One of the most promising aspects of the research is its potential, Scott said. He sees gaze-guided technology as a versatile tool that could improve independence, accessibility and human-machine interaction across a wide range of settings.  

“Whether it’s controlling a humanoid robot, interpreting a user's intent through eye movements or working alongside attachable prosthetic robotic limbs, these are all unique avenues I’d like to see others explore using this technology,” he added.  

For Scott, the project is about more than advancing technology. He sees engineering as an opportunity to help others, and that belief drives his efforts to create tools that make everyday tasks easier and more accessible for people with disabilities. 

“Being able to make a positive change in the world with the skills I've learned is one of my biggest motivators to keep going and keep solving problems like this one,” he said. “I think this project, along with intelligent systems engineering in general, has greatly shifted where I see myself in the future. After seeing the amount of good that could be done with engineering, that it can be a tool to invoke positive changes in people's lives, I’ve found a new drive.”