Export Citations
Publications
Save this search
Please login to be able to save your searches and receive alerts for new content matching your search criteria.
- research-article
MProt-DPO: Breaking the ExaFLOPS Barrier for Multimodal Protein Design Workflows with Direct Preference Optimization
- Gautham Dharuman,
- Kyle Hippe,
- Alexander Brace,
- Sam Foreman,
- Väinö Hatanpää,
- Varuni K. Sastry,
- Huihuo Zheng,
- Logan Ward,
- Servesh Muralidharan,
- Archit Vasan,
- Bharat Kale,
- Carla M. Mann,
- Heng Ma,
- Yun-Hsuan Cheng,
- Yuliana Zamora,
- Shengchao Liu,
- Chaowei Xiao,
- Murali Emani,
- Tom Gibbs,
- Mahidhar Tatineni,
- Deepak Canchi,
- Jerome Mitchell,
- Koichi Yamada,
- Maria Garzaran,
- Michael E. Papka,
- Ian Foster,
- Rick Stevens,
- Anima Anandkumar,
- Venkatram Vishwanath,
- Arvind Ramanathan
SC '24: Proceedings of the International Conference for High Performance Computing, Networking, Storage, and Analysis•November 2024, Article No.: 7, pp 1-13• https://doi.org/10.1109/SC41406.2024.00013We present a scalable, end-to-end workflow for protein design. By augmenting protein sequences with natural language descriptions of their biochemical properties, we train generative models that can be preferentially aligned with protein fitness ...
- research-article
GaLore: memory-efficient LLM training by gradient low-rank projection
ICML'24: Proceedings of the 41st International Conference on Machine Learning•July 2024, Article No.: 2528, pp 61121-61143Training Large Language Models (LLMs) presents significant memory challenges, predominantly due to the growing size of weights and optimizer states. Common memory-reduction approaches, such as low-rank adaptation (LoRA), add a trainable low-rank matrix ...
- research-article
Equivariant graph neural operator for modeling 3D dynamics
- Minkai Xu,
- Jiaqi Han,
- Aaron Lou,
- Jean Kossaifi,
- Arvind Ramanathan,
- Kamyar Azizzadenesheli,
- Jure Leskovec,
- Stefano Ermon,
- Anima Anandkumar
ICML'24: Proceedings of the 41st International Conference on Machine Learning•July 2024, Article No.: 2265, pp 55015-55032Modeling the complex three-dimensional (3D) dynamics of relational systems is an important problem in the natural sciences, with applications ranging from molecular simulations to particle mechanics. Machine learning methods have achieved good success by ...
- research-article
Solving poisson equations using neural walk-on-spheres
ICML'24: Proceedings of the 41st International Conference on Machine Learning•July 2024, Article No.: 1513, pp 37277-37292We propose Neural Walk-on-Spheres (NWoS), a novel neural PDE solver for the efficient solution of high-dimensional Poisson equations. Leveraging stochastic representations and Walk-on-Spheres methods, we develop novel losses for neural networks based on ...
- research-article
Autoformalizing euclidean geometry
ICML'24: Proceedings of the 41st International Conference on Machine Learning•July 2024, Article No.: 1497, pp 36847-36893Autoformalization involves automatically translating informal math into formal theorems and proofs that are machine-verifiable. Euclidean geometry provides an interesting and controllable domain for studying autoformalization. In this paper, we introduce ...
- research-article
Neural operators with localized integral and differential kernels
- Miguel Liu-Schiaffini,
- Julius Berner,
- Boris Bonev,
- Thorsten Kurth,
- Kamyar Azizzadenesheli,
- Anima Anandkumar
ICML'24: Proceedings of the 41st International Conference on Machine Learning•July 2024, Article No.: 1321, pp 32576-32594Neural operators learn mappings between function spaces, which is practical for learning solution operators of PDEs and other scientific modeling applications. Among them, the Fourier neural operator (FNO) is a popular architecture that performs global ...
- research-article
DPOT: auto-regressive denoising operator transformer for large-scale PDE pre-training
- Zhongkai Hao,
- Chang Su,
- Songming Liu,
- Julius Berner,
- Chengyang Ying,
- Hang Su,
- Anima Anandkumar,
- Jian Song,
- Jun Zhu
ICML'24: Proceedings of the 41st International Conference on Machine Learning•July 2024, Article No.: 703, pp 17616-17635Pre-training has been investigated to improve the efficiency and performance of training neural operators in data-scarce settings. However, it is largely in its infancy due to the inherent complexity and diversity, such as long trajectories, multiple ...
- research-article
Physics-Informed Neural Operator for Learning Partial Differential Equations
- Zongyi Li,
- Hongkai Zheng,
- Nikola Kovachki,
- David Jin,
- Haoxuan Chen,
- Burigede Liu,
- Kamyar Azizzadenesheli,
- Anima Anandkumar
ACM / IMS Journal of Data Science, Volume 1, Issue 3•September 2024, Article No.: 9, pp 1-27 • https://doi.org/10.1145/3648506In this article, we propose physics-informed neural operators (PINO) that combine training data and physics constraints to learn the solution operator of a given family of parametric Partial Differential Equations (PDE). PINO is the first hybrid approach ...
HighlightsPROBLEM STATEMENT
Machine learning methods have recently shown promise in solving partial differential equations (PDEs) raised in science and engineering. They can be classified into two broad categories: approximating the solution function ...
- research-article
Action-conditional implicit visual dynamics for deformable object manipulation
- Bokui Shen,
- Zhenyu Jiang,
- Christopher Choy,
- Silvio Savarese,
- Leonidas J. Guibas,
- Anima Anandkumar,
- Yuke Zhu,
- Shoudong Huang,
- Kris Hauser,
- Dylan A. Shell
International Journal of Robotics Research, Volume 43, Issue 4•Apr 2024, pp 437-455 • https://doi.org/10.1177/02783649231191222Manipulating volumetric deformable objects in the real world, like plush toys and pizza dough, brings substantial challenges due to infinite shape variations, non-rigid motions, and partial observability. We introduce ACID, an action-conditional visual ...
- research-article
Symmetry-informed geometric representation for molecules, proteins, and crystalline materials
- Shengchao Liu,
- Weitao Du,
- Yanjing Li,
- Zhuoxinran Li,
- Zhiling Zheng,
- Chenru Duan,
- Zhiming Ma,
- Omar Yaghi,
- Anima Anandkumar,
- Christian Borgs,
- Jennifer Chayes,
- Hongyu Guo,
- Jian Tang
NIPS '23: Proceedings of the 37th International Conference on Neural Information Processing Systems•December 2023, Article No.: 2886, pp 66084-66101Artificial intelligence for scientific discovery has recently generated significant interest within the machine learning and scientific communities, particularly in the domains of chemistry, biology, and material discovery. For these scientific problems, ...
- research-article
Geometry-informed neural operator for large-scale 3D PDEs
- Zongyi Li,
- Nikola Borislavov Kovachki,
- Chris Choy,
- Boyi Li,
- Jean Kossaifi,
- Shourya Prakash Otta,
- Mohammad Amin Nabian,
- Maximilian Stadler,
- Christian Hundt,
- Kamyar Azizzadenesheli,
- Anima Anandkumar
NIPS '23: Proceedings of the 37th International Conference on Neural Information Processing Systems•December 2023, Article No.: 1556, pp 35836-35854We propose the geometry-informed neural operator (GINO), a highly efficient approach for learning the solution operator of large-scale partial differential equations with varying geometries. GINO uses a signed distance function (SDF) and point-cloud ...
- research-article
ClimSim: a large multi-scale dataset for hybrid physics-ML climate emulation
- Sungduk Yu,
- Walter M. Hannah,
- Liran Peng,
- Jerry Lin,
- Mohamed Aziz Bhouri,
- Ritwik Gupta,
- Björn Lütjens,
- Justus C. Will,
- Gunnar Behrens,
- Julius J. M. Busecke,
- Nora Loose,
- Charles Stern,
- Tom Beucler,
- Bryce E. Harrop,
- Benjamin R. Hillman,
- Andrea M. Jenney,
- Savannah L. Ferretti,
- Nana Liu,
- Anima Anandkumar,
- Noah D. Brenowitz,
- Veronika Eyring,
- Nicholas Geneva,
- Pierre Gentine,
- Stephan Mandt,
- Jaideep Pathak,
- Akshay Subramaniam,
- Carl Vondrick,
- Rose Yu,
- Laure Zanna,
- Tian Zheng,
- Ryan P. Abernathey,
- Fiaz Ahmed,
- David C. Bader,
- Pierre Baldi,
- Elizabeth A. Barnes,
- Christopher S. Bretherton,
- Peter M. Caldwell,
- Wayne Chuang,
- Yilun Han,
- Yu Huang,
- Fernando Iglesias-Suarez,
- Sanket Jantre,
- Karthik Kashinath,
- Marat Khairoutdinov,
- Thorsten Kurth,
- Nicholas J. Lutsko,
- Po-Lun Ma,
- Griffin Mooers,
- J. David Neelin,
- David A. Randall,
- Sara Shamekh,
- Mark A. Taylor,
- Nathan M. Urban,
- Janni Yuval,
- Guang J. Zhang,
- Michael S. Pritchard
NIPS '23: Proceedings of the 37th International Conference on Neural Information Processing Systems•December 2023, Article No.: 968, pp 22070-22084Modern climate projections lack adequate spatial and temporal resolution due to computational constraints. A consequence is inaccurate and imprecise predictions of critical processes such as storms. Hybrid methods that combine physics with machine ...
- research-article
LeanDojo: theorem proving with retrieval-augmented language models
- Kaiyu Yang,
- Aidan M. Swope,
- Alex Gu,
- Rahul Chalamala,
- Peiyang Song,
- Shixing Yu,
- Saad Godil,
- Ryan Prenger,
- Anima Anandkumar
NIPS '23: Proceedings of the 37th International Conference on Neural Information Processing Systems•December 2023, Article No.: 944, pp 21573-21612Large language models (LLMs) have shown promise in proving formal theorems using proof assistants such as Lean. However, existing methods are difficult to reproduce or build on, due to private code, data, and large compute requirements. This has created ...
- research-article
Protein Generation via Genome-scale Language Models with Bio-physical Scoring
- Gautham Dharuman,
- Logan Ward,
- Heng Ma,
- Priyanka V. Setty,
- Ozan Gokdemir,
- Sam Foreman,
- Murali Emani,
- Kyle Hippe,
- Alexander Brace,
- Kristopher Keipert,
- Thomas Gibbs,
- Ian Foster,
- Anima Anandkumar,
- Venkatram Vishwanath,
- Arvind Ramanathan
SC-W '23: Proceedings of the SC '23 Workshops of the International Conference on High Performance Computing, Network, Storage, and Analysis•November 2023, pp 95-101• https://doi.org/10.1145/3624062.3626087Large language models (LLMs) trained on vast biological datasets can learn biological motifs and correlations across the evolutionary landscape of natural proteins. LLMs can then be used for de novo design of novel proteins with specific structures, ...
- research-article
GenSLMs: Genome-scale language models reveal SARS-CoV-2 evolutionary dynamics
- Maxim Zvyagin,
- Alexander Brace,
- Kyle Hippe,
- Yuntian Deng,
- Bin Zhang,
- Cindy Orozco Bohorquez,
- Austin Clyde,
- Bharat Kale,
- Danilo Perez-Rivera,
- Heng Ma,
- Carla M. Mann,
- Michael Irvin,
- Defne G. Ozgulbas,
- Natalia Vassilieva,
- James Gregory Pauloski,
- Logan Ward,
- Valerie Hayot-Sasson,
- Murali Emani,
- Sam Foreman,
- Zhen Xie,
- Diangen Lin,
- Maulik Shukla,
- Weili Nie,
- Josh Romero,
- Christian Dallago,
- Arash Vahdat,
- Chaowei Xiao,
- Thomas Gibbs,
- Ian Foster,
- James J. Davis,
- Michael E. Papka,
- Thomas Brettin,
- Rick Stevens,
- Anima Anandkumar,
- Venkatram Vishwanath,
- Arvind Ramanathan,
- Edmond Chow
International Journal of High Performance Computing Applications, Volume 37, Issue 6•Nov 2023, pp 683-705 • https://doi.org/10.1177/10943420231201154We seek to transform how new and emergent variants of pandemic-causing viruses, specifically SARS-CoV-2, are identified and classified. By adapting large language models (LLMs) for genomic data, we build genome-scale language models (GenSLMs) which can ...
- research-article
Learning calibrated uncertainties for domain shift: a distributionally robust learning approach
IJCAI '23: Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence•August 2023, Article No.: 162, pp 1460-1469• https://doi.org/10.24963/ijcai.2023/162We propose a framework for learning calibrated uncertainties under domain shifts, where the source (training) distribution differs from the target (test) distribution. We detect such domain shifts via a differentiable density ratio estimator and train it ...
- research-article
Fast sampling of diffusion models via operator learning
ICML'23: Proceedings of the 40th International Conference on Machine Learning•July 2023, Article No.: 1784, pp 42390-42402Diffusion models have found widespread adoption in various areas. However, their sampling process is slow because it requires hundreds to thousands of network evaluations to emulate a continuous process defined by differential equations. In this work, we ...
- research-article
I2SB: image-to-image Schrödinger bridge
ICML'23: Proceedings of the 40th International Conference on Machine Learning•July 2023, Article No.: 915, pp 22042-22062We propose Image-to-Image Schrödinger Bridge (I2SB), a new class of conditional diffusion models that directly learn the nonlinear diffusion processes between two given distributions. These diffusion bridges are particularly useful for image restoration, ...
- research-article
VIMA: robot manipulation with multimodal prompts
- Yunfan Jiang,
- Agrim Gupta,
- Zichen Zhang,
- Guanzhi Wang,
- Yongqiang Dou,
- Yanjun Chen,
- Li Fei-Fei,
- Anima Anandkumar,
- Yuke Zhu,
- Linxi Fan
ICML'23: Proceedings of the 40th International Conference on Machine Learning•July 2023, Article No.: 611, pp 14975-15022Prompt-based learning has emerged as a successful paradigm in natural language processing, where a single general-purpose language model can be instructed to perform any task specified by input prompts. Yet task specification in robotics comes in various ...
- research-article
Spherical Fourier neural operators: learning stable dynamics on the sphere
- Boris Bonev,
- Thorsten Kurth,
- Christian Hundt,
- Jaideep Pathak,
- Maximilian Baust,
- Karthik Kashinath,
- Anima Anandkumar
ICML'23: Proceedings of the 40th International Conference on Machine Learning•July 2023, Article No.: 117, pp 2806-2823Fourier Neural Operators (FNOs) have proven to be an efficient and effective method for resolution-independent operator learning in a broad variety of application areas across scientific machine learning. A key reason for their success is their ability to ...