The study, led by students Pedro Piacenza and Keith Behrman, was published online in IEEE/ASME Transactions on Mechatronics. It demonstrates the two aspects of the underlying technology that combine to enable the new results. Firstly, in this project, the researchers use light to sense touch. Under the “skin,” their finger has a layer made of transparent silicone, into which they shined light from more than 30 LEDs. The finger also has more than 30 photodiodes that measure how the light bounces around. Whenever the finger touches something, its skin deforms, so light shifts around in the transparent layer underneath. Measuring how much light goes from every LED to every diode, the researchers end up with close to 1,000 signals that each contain some information about the contact that was made. Since light can also bounce around in a curved space, these signals can cover a complex 3D shape such as a fingertip.

“The human finger provides incredibly rich contact information--more than 400 tiny touch sensors in every square centimeter of skin!” says Ciocarlie. “That was the model that pushed us to try and get as much data as possible from our finger. It was critical to be sure all contacts on all sides of the finger were covered--we essentially built a tactile robot finger with no blind spots.”

Secondly, the team designed this data to be processed by machine learning algorithms. Because there are so many signals, all of them partially overlapping with each other, the data is too complex to be interpreted by humans. Fortunately, current machine learning techniques can learn to extract the information that researchers care about: where the finger is being touched, what it is touching the finger, how much force is being applied, etc.

“Our results show that a deep neural network can extract this information with very high accuracy,” says Kymissis, who is also a member of the Data Science Institute. “Our device is truly a tactile finger designed from the very beginning to be used in conjunction with AI algorithms.”

In addition, the team built the finger so it, and others, can be put onto robotic hands. Integrating the system onto a hand is easy: thanks to this new technology, the finger collects almost 1,000 signals, but only needs a 14-wire cable connecting it to the hand, and it needs no complex off-board electronics. The researchers already have two dexterous hands (capable of grasping and manipulating objects) in their lab being outfitted with these fingers--one hand has three fingers, and the other one four. In the next months, the team will be using these hands to try and demonstrate dexterous manipulation abilities, based on tactile and proprioceptive data.

“Dexterous robotic manipulation is needed now in fields such as manufacturing and logistics, and is one of the technologies that, in the longer term, are needed to enable personal robotic assistance in other areas, such as healthcare or service domains,” Ciocarlie adds.

Columbia Engineering

Columbia Engineering, based in New York City, is one of the top engineering schools in the U.S. and one of the oldest in the nation. Also known as The Fu Foundation School of Engineering and Applied Science, the School expands knowledge and advances technology through the pioneering research of its more than 220 faculty, while educating undergraduate and graduate students in a collaborative environment to become leaders informed by a firm foundation in engineering. The School’s faculty are at the center of the University’s cross-disciplinary research, contributing to the Data Science Institute, Earth Institute, Zuckerman Mind Brain Behavior Institute, Precision Medicine Initiative, and the Columbia Nano Initiative. Guided by its strategic vision, “Columbia Engineering for Humanity,” the School aims to translate ideas into innovations that foster a sustainable, healthy, secure, connected, and creative humanity.

About the Study

The study is titled “A Sensorized Multicurved Robot Finger with Datadriven Touch Sensing via Overlapping Light Signals.”

Authors are: Pedro Piacenza and Matei Ciocarlie, Mechanical Engineering; Keith Behrman and Ioannis Kymissis, Electrical Engineering; and Benedikt Schifferer, Computer Science.

The work was sponsored in part by the National Science Foundation, under its CAREER program (grant IIS-1551631) and a National Robotics Initiative (grant CMMI-1734557).

The authors declare no financial or other conflicts of interest.

Lead Photo Caption: Danqi Chen, assistant professor of computer science at Princeton, delivered the AI Lecture at Columbia on Dec. 6.

We celebrated our graduates’ senior design projects. We partnered with the dental school to launch a new program in dental engineering. Along with our colleagues at Columbia Business School, we graduated the first cohort of MBAxMS students. The Columbia University Formula Racing team made an impressive showing at the national competition, and the Columbia Space Institute’s rocketry team brought home a gold (and set its own records) at the inaugural FAR-OUT competition in the Mojave Desert. Our faculty partnered with collaborators across disciplines to teach courses on the social implications of AI and the political impact of algorithms and machine learning. Researchers in the storied Carleton Laboratory worked with the city to restore the pumps in the Morningside Park pond. 

Driving the Dialogue

Image

We launched The Lever, a limited-series newsletter featuring faculty perspectives on global challenges. The first series explored solutions for storing renewable energy. We also kicked off the Lecture Series in AI. In one of the first talks, the legendary deep-learning researcher Yann LeCun, who is Meta’s chief AI scientist, delivered a talk to more than 1,000 attendees. Media outlets across the world tapped our researchers’ expertise in articles and video on topics from digital twins in biomedical research to desalination technology and intelligent robots— and every aspect of AI. Kristen Myers and Christine Hendon challenged us to imagine how engineers can improve women’s health, and Pierre Gentine asked if AI could save the environment. Tal Danino dazzled readers with an art book featuring research inspired images from his lab.

Celebrating Faculty Excellence

Image

Columbia Engineering celebrated the election of faculty members Jingguang Chen and Jeannette Wing to the National Academy of Engineering and congratulated Marco Giometto, Alex Urban, and Brian Smith on their NSF CAREER awards. We commended Gordana Vunjak-Novakovic on winning a Chan Zuckerberg Biohub New York Investigator Award. We were pleased to share that Oleg Gang was named a 2024 Vannevar Bush Fellow, that Ke Cheng received the Coulter Award, that Christos Papadimitriou and Michael Weinstein were named Simons Society Senior Fellows. We congratulated Vishal Misra on his appointment to vice dean of computing and artificial intelligence and John Kymissis on being named vice dean of infrastructure and innovation, and Kymissis' election to the National Academy of Inventors.

Image

Blockchain is changing the way we do business, but its greatest asset—a transparent, shared platform—is currently its greatest weakness. While banks and businesses have built up a $100 billion market in the space over the past two years, the technology has also been the target of adversarial attacks resulting in devastating financial losses.

Ronghui Gu, assistant professor of computer science, is researching a new tool with the potential to address such vulnerabilities while helping blockchain scale up for greater adoption: smart contracts. Unlike traditional contracts that rely on the observance of each party and impose penalties for noncompliance, “smart” contracts incorporate digital code that executes contracts automatically.

“We are trying to design a new language to write contracts,” says Gu. “When developers write contracts using this language, it will be easier to use and remove potential errors from the very beginning.”

Smart contracts promise to eliminate the costs and delays due to human intervention that are associated with traditional contracts—from business agreements to securing a mortgage. But they also give rise to security challenges of their own.

“Blockchain is about trust,” says Gu. “Smart contracts are more vulnerable than traditional contracts, both because their code is open source and because they lack a centralized controller who can immediately implement fixes.”

Key to improving their reliability will be to unearth bugs in the code that could be exploited by hackers. Gu was recently awarded a grant from the new Columbia-IBM Center for Blockchain and Data Transparency to do just that. He brings his background in formal verification techniques, using mathematical methods to prove software reliability, along with augmented algorithms that can address blockchain’s immense proof space—a distributed system very different from traditional software. With time, he hopes methods to make smart contracts more trustworthy will help more organizations and individuals transfer value with greater transparency.

“You don’t want innovation to give someone a loophole to manipulate the system,” says Gu. “We can help blockchain realize that promise it was designed for.”