These CMU robotics research projects are tackling lanternflies and AV safety

Class may not be in session just yet at Carnegie Mellon University, but roboticists don’t get summers off.

From a robot built to get rid of spotted lanternflies to a project aiming to help computer vision systems better detect moving objects, here are a few research projects that have come out of the university this summer.

Lanternfly killers

Spotted lanternflies aren’t just a nuisance — they’re an invasive species that can wipe out important crops such as apples and grapes. But, never fear, because researchers at CMU’s Robotics Institute have developed TartanPest, an all-electric tractor that uses a robotic arm and computer vision to detect and ultimately destroy lanternfly egg masses.

While searching through forests, TartanPest, armed with hundreds of images of lanternflies, can identify lanternfly eggs and scrape them off of surfaces.

“Currently, spotted lanternflies are concentrated in the eastern portion of the nation, but they are predicted to spread to the whole country,” said Carolyn Alex, an undergraduate researcher on the TartanPest team, in a CMU article. “By investing in this issue now, we will be saving higher costs in the future.”

Other members of the TartanPest team are Robotics Institute students Simi Asher, Dominic Guri, Cole Herber, RuiJi Liu, Shrijit Singh and Srinivasan Vijayarangan.

Off-road trekkers

Away from the world of pest control, CMU roboticists drove an all-terrain vehicle through tall grass, loose gravel and mud. Why? To use a sensor attached to the vehicle to gather data about how the self-driving ATV interacted with obstacles in an off-road environment.

This resulted in the research team ranking up 200,000 interactions, which they included in a dataset they called TartanData. In the long run, researchers hope TartanData can be used to train self-driving vehicles to navigate off-road safely.

“Unlike autonomous street driving, off-road driving is more challenging because you have to understand the dynamics of the terrain in order to drive safely and to drive faster,” said Wenshan Wang, a project scientist at the Robotics Institute.

Other members of the research team are robotics students and professors Samuel Triest, Sebastian Scherer, Aaron Johnson, Sean J. Wang and Matthew Sivaprakasam (of University of Pittsburgh).

AV watchers

Lastly, one of many worries related to autonomous driving is whether or not self-driving vehicles will be able to detect people around them. With that in mind, CMU researchers have found that computer vision systems have an easier time detecting moving objects.

Alongside the Toyota Research Institute, University of California, Berkeley, and the University of Illinois Urbana-Champaign, CMU School of Computer Science Dean Martial Hebert and robotics Ph.D. student Zhipeng Bao developed MoTok. The framework enables computers to identify features of things it sees moving on its own, like a car or a person walking on the sidewalk. To make autonomous vehicles safe, a generalized method enabling computer programs to discover objects in videos on their own is needed. The researchers hope MoTok’s discoveries about using motion can help create a safer future.

“Movement also can help define an object that has multiple moving parts,” Bao said in a CMU article. “A car door might open and close and wheels might spin, but all the parts moving together as the car travels down a street can help computer programs better understand the concept of a car.”

Atiya Irvin-Mitchell is a 2022-2023 corps member for Report for America, an initiative of The Groundtruth Project that pairs young journalists with local newsrooms. This position is supported by the Heinz Endowments.

Technically Media