ECE Assistant Professor Zhao Zhang has received his career award from NSF. The project is “CAREER: Efficient and Scalable Large Foundational Model Training on Supercomputers for Science”. The total value is $599,707 for a five-year period starting Summer 2024.
Training foundational models at extreme scale is time consuming, prone to low utilization and limited scalability, and human effort demanding. This NSF CAREER project addresses the convergence, performance, and scalability gaps of large foundational model pre-training on supercomputers with innovative algorithm, system, and interface design. In addition to the algorithm and computer system innovation, this project contributes to translational computer science by lowering the barrier of large foundational model training and the time consumption of scientific deep learning, thus enabling significantly more scientific research to embrace large foundational models. The research results will be publicly available in the form of open source software to the wider community with comprehensive documentation on the design and usage to help users from all domains.
Training foundational models at extreme scale is time consuming, prone to low utilization and limited scalability, and human effort demanding. This NSF CAREER project addresses the convergence, performance, and scalability gaps of large foundational model pre-training on supercomputers with innovative algorithm, system, and interface design. In addition to the algorithm and computer system innovation, this project contributes to translational computer science by lowering the barrier of large foundational model training and the time consumption of scientific deep learning, thus enabling significantly more scientific research to embrace large foundational models. The research results will be publicly available in the form of open source software to the wider community with comprehensive documentation on the design and usage to help users from all domains.
Congratulations to Zhao!