Computational Systems Research Department Grant: Advancing Resilient Distributed Systems for Complex Scientific Workflows

The computational workflows associated with modern science are becoming increasingly complex, often processing an astounding amount of data generated by geographically-distributed instruments. The data is analyzed using a variety of local and remote compute resources and integrated with simulations and artificial intelligence (AI)–enhanced models to both direct the ongoing experiments and inform scientific progress. The future continuation of this trend is outlined by the 2022 report on Envisioning Science in 2050 [1].

Pursuing innovative research directions in techniques for advanced middleware and operating and runtime systems is critical to address the unprecedented challenges in implementing future workflows. These research directions may involve, but are not limited to, coordinating work on: a) billions of threads of execution on a supercomputer; b) several geographically-separated supercomputers; c) advanced experimental systems which produce hundreds of petabytes of data each day; and/or d), billions of distributed sensors monitoring the climate or other systems of interest. Recognizing that in systems of this size and complexity sporadic failures of individual components are inevitable, scientific workflows and their supporting middleware and system software must be designed with resilience in mind.

Federally affiliated entities must adhere to the eligibility standards below:

1. DOE/NNSA National Laboratories

DOE/NNSA National Laboratories are eligible to submit applications (either as a lead organization or as a team member in a multi-institutional team) under this FOA and may be proposed as subrecipients under another organization’s application. If recommended for funding as a lead applicant, funding will be provided through the DOE Field-Work Proposal System and work will be conducted under the laboratory’s contract with DOE. No administrative provisions of this FOA will apply to the laboratory or any laboratory subcontractor. If recommended for funding as a proposed subrecipient, the value of the proposed subaward will be removed from the prime applicant’s award and will be provided to the laboratory through the DOE Field-Work Proposal System and work will be conducted under the laboratory’s contract with DOE. Additional instructions for securing authorization from the cognizant Contracting Officer are found in Section VIII of this FOA.

2. Non-DOE/NNSA FFRDCs

Non-DOE/NNSA FFRDCs are eligible to submit applications (either as a lead organization or as a team member in a multi-institutional team) under this FOA and may be proposed as subrecipients under another organization’s application. If recommended for funding as a lead applicant, funding will be provided through an interagency agreement Award to the FFRDC’s sponsoring Federal Agency. If recommended for funding as a proposed subrecipient, the value of the proposed subaward may be removed from the prime applicant’s award and may be provided through an interagency agreement to the FFRDC’s sponsoring Federal Agency. Additional instructions for securing authorization from the cognizant Contracting Officer are found in Section VIII of this FOA.

3. Other Federal Agencies

Other Federal Agencies are eligible to submit applications (either as a lead organization or as a team member in a multi-institutional team) under this FOA and may be proposed as subrecipients under another organization’s application. If recommended for funding as a lead applicant, funding will be provided through an interagency agreement. If recommended for funding as a proposed subrecipient, the value of the proposed subaward may be removed from the prime applicant’s award and may be provided through an interagency agreement. Additional instructions for providing statutory authorization are found in Section VIII of this FOA.

The computational workflows associated with modern science are becoming increasingly complex, often processing an astounding amount of data generated by geographically-distributed instruments. The data is analyzed using a variety of local and remote compute resources and integrated with simulations and artificial intelligence (AI)–enhanced models to both direct the ongoing experiments and inform scientific progress. The future continuation of this trend is outlined by the 2022 report on Envisioning Science in 2050 [1].

Pursuing innovative research directions in techniques for advanced middleware and operating and runtime systems is critical to address the unprecedented challenges in implementing future workflows. These research directions may involve, but are not limited to, coordinating work on: a) billions of threads of execution on a supercomputer; b) several geographically-separated supercomputers; c) advanced experimental systems which produce hundreds of petabytes of data each day; and/or d), billions of distributed sensors monitoring the climate or other systems of interest. Recognizing that in systems of this size and complexity sporadic failures of individual components are inevitable, scientific workflows and their supporting middleware and system software must be designed with resilience in mind.