“HPC and BDC ecosystems should continue to identify and research idiosyncratic challenges particular to their respective domains, but not in isolation from each other. Fostering increased collaboration between HPC and Big Data ecosystems will lead to the identification and exchange of complementary capabilities […] that will accelerate the pace of innovation for their respective stacks […]. “
In the past years, it became common for extreme computing and extreme data analytics to converge on application scenarios in many use cases – both Big Data Computing (BDC) and High Performance Computing (HPC) communities can learn from and assist one another. EXDCI (the European Extreme Data & Computing Initiative) participated to define the pathway to convergence, by coordinating the European effort in the Big Data and Extreme-scale Computing (BDEC) conference series.
At European level, differences as well as possible options for convergence of the Big Data (BD) and HPC areas were notably described in 2018 in a joint document developed by BDVA and EXDCI partner ETP4HPC.
The EXDCI-2 project contributes actively to the dialogue between HPC and Big Data, as the collaboration between both communities is a necessity to go beyond their states-of-art. Therefore EXDCI-2, building on EXDCI-1 legacy, pursues its collaboration with BDVA on finding bridges between the Big Data and Extreme Computing communities.
With BDVA representing the BD side and EXDCI focussing on the extreme computing side, such a collaboration allows to jointly propose research priorities for future work programmes in EuroHPC to master the technical challenges associated with this convergence. Among other activities, common sessions are organised, with the aim of creating linkages between the two communities.
What is BDVA?
The Big Data Value Association (BDVA) is an industry-led international non-profit association composed of 200 members all over Europe, including large, small and medium-sized industries as well as research and user organisations.
BDVA is the private counterpart to the EU Commission to implement the Big Data Value PPP program – both organisations pursue a common shared vision of positioning Europe as the world leader in the creation of Big Data Value.
The mission of BDVA consists of developing the Innovation Ecosystem that will enable the data and AI-driven digital transformation delivering maximum economic and societal benefit to Europe, its businesses and its citizens, and, achieving and sustaining Europe’s leadership on Big Data Value creation and Artificial Intelligence.
 The technology stacks of high performance computing and big data computing: what they can learn from each other – ETP4HPC, BDVA (2018).