The EXDCI project helps the European HPC Community to conduct the transition from today’s petascale systems to the exascale era.
To achieve this goal, the EXDCI project was involved in the creation and updates of European Technological Roadmaps towards exascale such as the ETP4HPC Strategic Research Agenda and the PRACE Scientific Case. EXDCI also worked on specific topics such as the place of SMEs in the European HPC ecosystem.
EXDCI also issued two sets of holistic visions and recommendations reports that address the petascale to exascale challenge in a global and long-term perspective. Those reports can be downloaded on our website:
Many of these recommendations have been followed in the EXDCI-2 project or have spread through other European initiatives.
Why exascale projects matter and how to deal with HPC paradigm shift
European science and industry needs exascale capabilities, as computing has redefined the way science is produced. Among many examples, simulation is nowadays a part of the scientific process, that is used to test more complex models, adapt and improve theories or reduce significantly time for experimentation.
To take the most of those computing capabilities, EU scientists and engineers need effective access to machines suitable for their communities. With an increasing role of simulation in the scientific process and the growing amount of data to treat, buying the next generation of supercomputers is not a good enough approach to stay competitive.
The industry also needs exascale, both by its link with scientific discoveries and the skills it builds (trained PhD and engineers) but also for its own purpose in various domains such as drug discoveries, climate simulation and adaptation, advanced materials.
The European Union needs to move to the exascale era
Exascale is not an incremental change
Moving from petascale to exascale is a very complex transition, especially as it is not happening in isolation, because, at the same time, data deluge is happening.
The exascale definition can not be limited to producing a machine capable of a rate of 1018 flops, as this machine, in itself, would be of interest to very few scientific domains.
The main issue is dealing with data generated by sensors as well as numerical simulations themselves. Whilst in EESI it was clear that the data issue would be of crucial importance, during the EXDCI time frame the focus has been shifted to the convergence of extreme data and computing with new considerations such as edge computing, in-transit computing, etc.
This new focus brings many new capabilities for science (e.g. machine learning) and connections to the Big Data Market, however, at the same time, also with an enlargement of the HPC ecosystem. It is admitted in the EXDCI community that this new challenge is shaping industry and science today and will do so even more in the future.
As the Peta-Exa transition is not an incremental change, it requires the community to adapt to a new way of using computing and storage, and the later this adaptation starts, the more expensive it is, with negative impact an all the sectors that use or can use HPC : from competitiveness to skills adaptation, passing by the kind of science than can be done in the European Union.
In this context, pre-exascale machines are important
Moreover, the HPC road to exascale happens in a competitive environment, in which the European Union lacks hardware technologies, which creates strong dependencies on other countries. This means that EU access to exascale may be delayed, with a high risk for the EU to stay a step behind in terms of discovery, knowledge and competitiveness.
This is why the European Union invests in EU-based hardware for HPC capabilities
EXDCI’s main recommendation to face the exascale paradigm shift is:
To focus and organize the current European efforts in a way that is closer to an integrated industrial project rather than a set of loosely coupled research projects.
Ecosystem-level holistic recommendations
EXDCI has identified a set of recommendations for the entire European HPC ecosystem. The following paragraph gives a short overview and addresses three domains:
Better research instruments
The following recommendations aim at improving current research instruments, both computing resources and deployment of new technologies:
> R1: Design new operation policies and federations towards convergence.
> R2: Reinforce Big Data and extreme-scale international initiatives.
> R3: Improved access to advanced technologies.
The ambition of following recommendations is to ensure that the public and private investment in R&D is carried out in a coherent manner, maximising the impact of research:
> R4: IPCEI for advanced research and innovation.
> R5: Paving the way from EsD development towards applications.
> R6: Improving FETHPC and CoE result capitalisation.
The following recommendations’ ambition is to leverage R&D excellence and translating its output into industry competitiveness:
> R7: Encouraging commercial relationships between SMEs and industry via European projects
> R8: Concerted approach to HPC training in Europe
> R9: Incentives to increase the involvement of EU stakeholders in standard initiatives