About the Study
Journal: Nature Human Behaviour
Study title: “Neuronal activity in the human amygdala and hippocampus enhances emotional memory encoding.”
Authors: Salman Ehtesham Qasim (Department of Psychiatry, Icahn School of Medicine at Mount Sinai), Uma Rani Mohan (Surgical Neurology Branch, NINDS, National Institutes of Neurological Disorders and Stroke), Joel Stein (Department of Radiology, University of Pennsylvania), and Joshua Jacobs (Departments of Biomedical Engineering, Neurological Surgery, Columbia University). Both Qasim and Mohan were graduate students in Joshua Jacobs’ lab at Columbia Engineering when beginning this project.
This work was supported by NIH grants U01-NS113198 and R01-MH104606.
COI: The authors declare no financial or other conflicts of interest.
New “camera” has incredibly fast shutter speed of around 1 picosecond
Revealing Atomic Structures with a "Neutron" Camera. Credit: Oak Ridge National Laboratory
Researchers at Columbia Engineering and Université de Bourgogne report that they have developed a new kind of "camera" that can see the local disorder. Its key feature is a variable shutter speed: because the disordered atomic clusters are moving, when the team used a slow shutter, the dynamic disorder blurred out, but when they used a fast shutter, they could see it. The new method, which they call variable shutter PDF or vsPDF (for atomic pair distribution function), doesn't work like a conventional camera--it uses neutrons from a source at the U.S. Department of Energy’s Oak Ridge National Laboratory (ORNL) to measure atomic positions with a shutter speed of around one picosecond, or a million million (a trillion) times faster than normal camera shutters. The study was published February 20, 2023, by Nature Materials.
“It’s only with this new vsPDF tool that we can really see this side of materials,” said Simon Billinge, professor of materials science and applied physics and applied mathematics. “It gives us a whole new way to untangle the complexities of what is going on in complex materials, hidden effects that can supercharge their properties. With this technique, we’ll be able to watch a material and see which atoms are in the dance and which are sitting it out.”
New theory on stabilizing local fluctuations and converting waste heat to electricity
The vsPDF tool enabled the researchers to find atomic symmetries being broken in GeTe, an important material for thermoelectricity that converts waste heat to electricity (or electricity into cooling). They hadn’t previously been able to see the displacements, or to show the dynamic fluctuations and how quickly they fluctuated. As a result of the insights from vsPDF, the team developed a new theory that shows just how such local fluctuations can form in GeTe and related materials. Such a mechanistic understanding of the dance will help researchers to look for new materials with these effects and to apply external forces to influence the effect, leading to even better materials.
Research team
Billlinge’s co-lead on this work with Simon Kimber, who was at the University of Bourgogne in France at the time of the study. Billinge and Kimber worked with colleagues at ORNL and the Argonne National Laboratory (ANL), also funded by the DOE. The Inelastic neutron scattering measurements for the vsPDF camera were made at ORNL; the theory was done at ANL.
Next steps
Billinge is now working on making his technique easier to use for the research community and applying it to other systems with dynamic disorder. At the moment, the technique is not turn-key, but with further development, it should become a much more standard measurement that could be used on many material systems where atomic dynamics are important, from watching lithium moving around in battery electrodes to studying dynamic processes during water-splitting with sunlight.
About the Study
JOURNAL: Nature Materials
STUDY: “Dynamic crystallography reveals spontaneous anisotropy in cubic GeTe”
AUTHORS: Simon A. J. Kimber, Batiment Sciences Mirande; Jiayong Zhang, Oak Ridge National Laboratory; Charles H. Liang, University of Chicago; Gian G. Guzman-Verri, Universidad de Costa Rica; Peter B. Littlewood, University of Chicago, Argonne National Laboratory; Yongqiang Cheng, Oak Ridge National Laboratory; Douglas L. Abernathy, Oak Ridge National Laboratory; Jessica M. Hudspeth, ESRF, The European Synchrotron; Zhong-Zhen Luo, Northwestern University; Mercouri G. Kanatzidis, Northwestern University; Tapan Chatterji, Institut Laue-Langevin; Anibal J. Ramirez-Cuesta, Oak Ridge National Laboratory; Simon J. L. Billinge, Columbia Engineering, Columbia University, Brookhaven National Laboratory.
FUNDING: S.J.L.B. acknowledges support from the US DOE, Office of Science, Ofice of Basic Energy Sciences, under contract no. DE- SC0012704. C.H.L. acknowledges support from NSF GRFP DGE-1746045. G.G.G.-V. acknowledges support from the Vice-Rector for Research at the University of Costa Rica (project no. 816-C1-601). Work at Argonne (P.B.L.) is supported by the US DOE, Ofice of Science, Ofice of Basic Energy Sciences, Materials Sciences and Engineering, under contract no. DE-AC02-06CH11357. At Northwestern University (M.G.K.), work on thermoelectric materials is primarily supported by the US DOE, Ofice of Science, Ofice of Basic Energy Sciences, under award no. DE-SC0014520. This work was supported by the Programme of Investments for the Future, an ISITE-BFC project (contract no. ANR[1]15-IDEX-0003) (S.A.J.K.).
The authors declare no financial or other conflicts of interest.
Smarter Robots: achieving object permanence
Dynamic scene completion: given a monocular video as input, the framework produces a 4D representation that captures the entire scene content along with all the static and dynamic objects within it over time. Credit: Basile Van Hoorick
"I have worked with images and videos before, but getting neural networks to work well with 3D information is surprisingly tricky," said Basile Van Hoorick, a third-year PhD student who worked with Vondrick to develop the framework that can understand occlusions as they occur. Unlike humans, an understanding of the three-dimensionality of our world does not come naturally to computers. The second leap in the project was not only to convert data from cameras into 3D seamlessly but also to reconstruct the entire configuration of the scene beyond what can be seen.
This work could expand the perception capabilities of home robots widely. In any indoor environment, things become hidden from view all the time. Hence, robots need to interpret their surroundings intelligently. The "soda can inside the refrigerator" situation is one of many examples. Still, it is easy to see how any application that uses vision will benefit if robots can draw upon their memory and object-permanence reasoning skills to keep track of both objects and humans as they move around the house.
Moving beyond the rigid body assumption
Most robots today are programmed with a series of assumptions for them to work. One is the rigid body assumption, which assumes that an object is solid and doesn't change shape. And that simplifies a lot of things. Roboticists can completely ignore the physics of the object the robot is interacting with and only have to think about the robot's motion.
Shuran Song
The Columbia Artificial Intelligence and Robotics (CAIR) Lab, led by Computer Science Assistant Professor Shuran Song, has been researching robotic movement in a different way. Her research focuses on deformable, non-rigid objects--they fold, bend, and change shape. When working with deformable objects, roboticists can no longer rely on the rigid body assumption, forcing them to think about physics again.
"In our work, we are trying to investigate how humans intuitively do things," said Shuran Song, also a Toyota Research Institute Young Faculty awardee. Instead of trying to account for every possible parameter, her team developed an algorithm that allows the robot to learn from doing, making it more generalizable and lessening the need for massive amounts of training data. It forced the group to rethink how people do an action, like hitting a target with a rope. We usually don’t think about the trajectory of the string--instead, we try to hit the object first and adjust our movements until we’re successful. "This new perspective was essential to solve this difficult problem in robotics,” Song noted.
Smarter Robots: Hitting the target
The robot learned how to hit the target (the yellow cup) in seven tries. Credit: Cheng Chi
"Previously, to achieve this level of precision, the robot needed to do the task maybe 100 to 1,000 times," said Cheng Chi, a third-year PhD student who worked with Song to develop IRP. "With our system, we can do it within ten times, which is about the same performance as a person."
The researchers noticed that there were still some limitations with the flinging motion that their robot could make. While the flinging motion is effective, it is limited by the speed of the robot arm, which means it cannot handle large items. Not to mention that it is dangerous to have a fast flinging motion around people.
Song’s team took this research a step further and developed a new approach to manipulating them by using actively blown air. They armed their robot with an air pump and it was able to quickly unfold a large piece of cloth or open a plastic bag. The self-supervised learning framework they call DextAIRity learns to effectively perform a target task through a sequence of grasping or air-based blowing actions. Using visual feedback, the system uses a closed-loop formulation that continuously adjusts its blowing direction.
Smarter Robots: explioting airflow
DextAIRity’s learning-based approach can quickly and reliably open the majority of bags tested. Credit: Zhenjia Xu
"One of the interesting strategies the system developed with the bag-opening task is to point the air a little above the plastic bag to keep the bag open," said Zhenjia Xu, a fourth-year PhD student who works with Song in the CAIR Lab. "We did not annotate or train it in any way; it learned it by itself."
What needs to be done to make robots more useful in our homes?
Currently, robots can successfully maneuver through a structured environment with clearly defined areas and do one task simultaneously. However, a truly useful home robot should have various skills, be able to work in an unstructured environment, like a living room with toys on the floor, and handle different situations. These robots will also need to know how to identify a task and which subtasks must be done in what order. And then, they will need to know what to do next if they fail at a job and how to adapt to the next steps needed to accomplish their goal.
“The progress that Carl Vondrick and Shuran Song have made with their research contributes directly to Toyota Research Institute's mission," says Dr. Eric Krotkov, advisor to the University Research Program. "TRI's research in robotics and beyond focuses on developing the capabilities and tools to address the socioeconomic challenges of an aging society, labor shortage, and sustainable production. Endowing robots with the capabilities to understand occluded objects and handle deformable objects will enable them to improve the quality of life for all.”
Song and Vondrick plan to collaborate to combine their respective expertise in robotics and computer vision to create robots that assist people in the home. By teaching machines to understand everyday objects in homes, such as clothes, food, and boxes, the technology could enable robots to assist people with mobility disabilities and improve the quality of everyday life for people. By increasing the number of objects and physical concepts that can be learned by robots, the team aims to make these applications possible in the future.
About the Study
JOURNAL: Nature Genetics
STUDY: “Multi-modal single-cell and whole-genome sequencing of small, frozen clinical specimens.”
AUTHORS: Yiping Wang (1,2*), Joy Linyue Fan (3*), Johannes C. Melms (1,4*), Amit Dipak Amin (1,4), Yohanna Georgis (5), Irving Barrera (6), Patricia Ho (1,4), Somnath Tagore (1,7), Gabriel Abril-Rodriguez(8), Siyu He (3), Yinuo Jin (3), Jana Biermann (1,2), Matan Hofree (6), Lindsay Caprio (1,4), Simon Berhe (4), Shaheer A. Khan (1,5), Brian S. Henick (1,5), Antoni Ribas (8,9), Evan Z. Macosko (6,10), Fei Chen (6,11), Alison M. Taylor (5,12), Gary K. Schwartz (1,5), Richard D. Carvajal (1,5), Elham Azizi (3,5,13,#), Benjamin Izar (1,2,4,5,7#)
- Department of Medicine, Division of Hematology/Oncology, Vagelos College of Physicians and Surgeons, Columbia University Irving Medical Center
- Department of Systems Biology, Program for Mathematical Genomics, Columbia University
- Department of Biomedical Engineering, Columbia University
- Columbia Center for Translational Immunology, Columbia University Irving Medical Center
- Vagelos College of Physicians and Surgeons, Herbert Irving Comprehensive Cancer Center, Columbia University
- Broad Institute of Harvard and MIT
- Department of Systems Biology, Columbia University Irving Medical Center
- Department of Medicine, Jonsson Comprehensive Cancer Center, University of California, Los Angeles (UCLA)
- Parker Institute for Cancer Immunotherapy, San Francisco
- Department of Psychiatry, Massachusetts General Hospital
- Department of Stem Cell and Regenerative Biology, Harvard University
- Department of Pathology and Cell Biology, Columbia University Irving Medical Center, New York, NY, USA
- Irving Institute for Cancer Dynamics, Columbia University, New York, NY, USA.
FUNDING: Y.W. is supported by National Institutes of Health (NIH), National Institute of Allergy and Infectious Disease training grant (no. T32AI148099). B.I. is supported by the NIH, National Cancer Institute (NCI) (grant nos. K08CA222663, R37CA258829, R01CA266446, and U54CA225088), a Burroughs Wellcome Fund Career Award for Medical Scientists, a Velocity Fellows Award, the Louis V. Gerstner, Jr. Scholars Program and a Young Investigator Award by the Melanoma Research Alliance. R.D.C., E.A., and B.I. are supported by an NCI grant (no. R21CA263381) and a Columbia University Research Initiatives in Science & Engineering Award. E.A. was supported by an NCI grant (no. R00CA230195) and NSF grant (no. CBET-2144542). J.L.F. acknowledges support from the Columbia University Van C. Mow fellowship. G.A.-R. and A.R. are supported by the Parker Institute for Cancer Immunotherapy and an NIH grant (no. P01CA168585). A.M.T. is supported by the NCI (grant no. 5K22CA237733-03). This work was supported by an NIH/NCI Cancer Center Support grant (no. P30CA013696), the Molecular Pathology Shared Resource and its Tissue Bank at Columbia University, and the Flow-cytometry Core Facility supported by a grant (no. S10OD020056).
COI: B.I. has received consulting fees from Volastra Therapeutics Inc, Merck, AstraZeneca, and Janssen Pharmaceuticals and has received research funding to Columbia University from Alkermes, Arcus Biosciences, Checkmate Pharmaceuticals, Compugen, Immunocore, and Synthekine. G.A.-R. has received honoraria from consulting with Arcus Biosciences. A.R. has received honoraria from consulting with Amgen, Bristol Myers Squibb, Chugai, Genentech, Merck, Novartis, Roche, and Sanofi, is or has been a member of the scientific advisory board, and holds stock in Arcus, Compugen, CytomX, Highlight, ImaginAb, Isoplexis, Kite-Gilead, Lutris, Merus, PACT, RAPT, Synthekine, and Tango Therapeutics. A.M.T. receives research support from Ono Pharmaceuticals. B.S.H. participated in advisory boards for AstraZeneca and Ideaya. R.D.C. is a consultant for Alkermes, Bristol Myers Squibb, Castle Biosciences, Delcath, Eisai, Hengrui, Ideaya, Immunocore, InxMed, Iovance, Merck, Novartis, Oncosec, Pierre Fabre, PureTech Health, Regeneron, Sanofi Genzyme, Sorrento Therapeutics and Trisalus, serves on clinical/scientific advisory boards for Aura Biosciences, Chimeron, and Rgenix Research, and has received research funding to Columbia University from Amgen, Astellis, AstraZeneca, BioMed Valley, Bolt, Bristol Myers Squibb, Corvus, Cstone, Foghorn, Ideaya, Immatics, Immunocore, InxMed, Iovance, Merck, Mirati, Novartis, Pfizer, Plexxikon, Regeneron, and Roche/Genentech. B.I. and J.C.M. filed a patent describing the generation of high-quality single-cell genomics data from frozen tissues. The remaining authors declare no competing interests.