Campus
Unlocking the Possibilities of Data Streaming
Data streaming is helping make information more accessible when it matters.
Columbia Engineering's Lecture Series in AI returned for its third year Sept. 11 at the University’s Morningside campus. Previous lectures explored artificial intelligence (AI) architectures and computing systems. The first lecture of this school year turned to what all of those systems depend on.
“The one thing that really is at the core of any AI system is data,” said Vishal Misra, vice dean of computing and AI at Columbia Engineering.
That idea framed a talk by featured speaker, Engineering alumnus Jun Rao PhD ’00, who traced the evolution of data streaming from a problem he encountered at LinkedIn to a technology he believes will play an important role in generative and agentic AI.
Solving a data problem with a data streaming platform
When Rao joined LinkedIn in 2010, the company stored data across multiple databases and systems, and a growing number of applications needed access to it. Engineers often built custom connections between individual data sources and applications, but Rao said that he and his colleagues quickly saw that the approach would not scale. Their solution was Apache Kafka, an open-source common platform where users could publish data as continuous streams and applications could subscribe to the information they needed. The system helped simplify LinkedIn’s data architecture while making data more accessible to developers, data scientists, and product teams.
As Kafka gained wider adoption, Rao saw an opportunity to take the idea further. In 2014, he co-founded Confluent with fellow Kafka creators Jay Kreps and Neha Narkhede to build a more complete data-streaming platform. The team developed tools that made it easier to connect databases, data warehouses, and other systems, and to continuously clean, transform, and enrich data as it arrived.
Generative and agentic AI have made real-time data even more important, Rao said during the talk held in Davis Auditorium. Unlike traditional machine learning, which often relied on batch processing and historical data, AI agents increasingly make decisions and take actions in the moment.
While foundation models are powerful, they do not automatically have access to a company’s latest transactions, customer activity, or other proprietary information. To make these systems more useful, he stressed that organizations need to provide relevant business context at inference time, and the fresher that context is, the more useful it can be. Data streaming supplies that contextual layer by continuously capturing and delivering the information to the applications and agents that need it.
Improving customer service with AI
Rao pointed to one of their clients, Trust Bank, a digital bank in Singapore, as an example. The bank used an AI chatbot for customer service but also ran a second large language model to analyze customer sentiment in real time. If an interaction appeared to be going poorly, the system could alert a human representative to follow up with the customer.
The bank also used data streaming to give the chatbot more context about each customer, including the services they used and their recent interactions with the bank, which let the chatbot give more concise and relevant responses. According to Rao, the system eventually handled about 50 percent of incoming customer requests while customer complaints fell fourfold.
Building on the fundamentals
After earning his computer science PhD at Columbia, Rao joined IBM Research. Following IBM's acquisition of Confluent in March 2026, he has now returned to the company that jumpstarted his career. Although his work has shifted from databases to data streaming, Rao said the computer science fundamentals he learned while studying database systems, including designing reliable software and evaluating performance, continue to apply.
For students entering a field that is rapidly being reshaped by AI, Rao underscored the importance of those fundamentals. AI assistants may change how students learn and build software, he said, but they still need to master the skills to evaluate what those tools produce. He also stressed the importance of gaining practical experience while in school to prepare for the challenges of their careers.
“The ability to ask the right question,” said Rao, “and the critical thinking needed to determine whether the answer is correct or not, remain essential.”
