We are a project in a group of projects ( FETHPC-2014(a)) that develop the technology needed for building the next-generation of High Performance Computing (HPC) Systems, also known as “Supercomputers” –the Exascale-level HPC systems, i.e. those achieving performance in the range of 1 ExaFLOPS = 10¹⁸ FLOPS:  one quintillion FLoating-point Operations Per Second.

HPC is, today, a tool of capital importance in the hands of humankind.  According to the ETP4HPC Strategic Research Agenda (SRA):  “HPC is a pervasive technology that strongly contributes to the excellence of science and the competitiveness of industry. Questions as diverse as how to develop a safer and more efficient generation of aircraft, how to improve the green energy systems such as solar panels or wind turbines, what is the impact of some phenomena on the climate evolution, how to deliver customized treatments to patients... cannot be answered without using HPC [...] HPC is also recognized as crucial in addressing grand societal challenges.  Today, to out-compute is to out-compete best describes the role of HPC”.  The HiPEAC Roadmap (Vision 2015) adds:  “Science is entering its 4th paradigm: data intensive discovery. After being essentially based on Theory, Empirical science, and Simulation, science is now using Data Analytics (cognitive computing) on instrument data, sensor data, but also on simulation data. Simulations are therefore becoming more and more important for science and for industry, avoiding spending millions in experiments. High performance computing is an enabling technology for complex simulations.”

Improving the Performance of Supercomputers as much as possible is an everlasting need, because that progress makes it possible to solve bigger and bigger problems, that were impossible to solve with the previous generation of supercomputers, or it allows us to solve problems at greater accuracy than what was previously possible. In previous decades, performance of supercomputers grew owing to the advances in technology that allowed individual processors to become faster using faster transistors (higher clock frequency) and more transistors (a more advanced pipelined architecture): it was possible to keep the number of processors fixed, and still come up with faster supercomputers. Not anymore, though –technology has recently reached a plateau where individual general-purpose processing “cores” cannot be made any faster because they would consume excessive electric power relative to the computational performance that they would offer. Thus, the only two ways left for achieving higher performance are to integrate a larger number of cores and to integrate special-purpose compute engines in a HPC systems. In this project, we work on enabling more cores –millions of compute cores– by improving on the network that is needed to interconnect all these cores: ordinary networks become slow and congested when they grow in size, and novel techniques are needed in order to prevent that. In the meanwhile, this increase in the number of computing cores has to be achieved...

... while keeping electric power consumption constant! Large supercomputers already consume a few tens of Mega-Watts –like a town of a few tens of thousand of people, each. Neither electric power companies nor society can afford to spend more than that for supercomputing. Thus, in order for progress to be sustained, we need each computing core to spend less Watts of electicity, if we are to integrate more cores in a supercomputer. For that reason, in this project, we use computing cores designed by ARM, as opposed to more electricity-hungry processors used by other HPC systems. ARM –a European company– is the world-leader in low-power-consumption processors, which is teh reason why it already dominates the mobile-phone market; owing precisely to this low-consumption advantage, ARM processors are now moving into Datacenters and into HPC. The computing “chiplets” that we use are provided to us by the previous EuroServer and the concurrent ExaNoDe projects.

New Storage technologies have appeared, in the meanwhile: flash memories are already out in the market, and various other kinds of non-volatile memory are little-by-little emerging out of multiple research labs. These are all (much) faster than the traditional hard-disk drives (HDD), and as their capacities grow to become comparable to HDD, they become the medium of choice for the upper level of the persistent storage hierarchy. As a result, storage now becomes much faster than it used to be, which brings along radical changes in its organization; we take part in this storage architecture revolution.

When millions of computing cores cooperate in solving a problem, they need to communicate with each other. Communicating a number from one chip to another, today, takes orders of magnitude more energy than making an arithmetic operation on this number, on-chip. A large portion of this energy is wasted in copying the communicated data from one memory buffer to another; additional energy is spent in running complex networking protocols in software.

We use the UNIMEM architecture from the EuroServer project: its Global Address Space allows communication using Remote Direct Memory Access (RDMA) operations, which deliver data in-place and avoid receiver-side copying. On the sender side, the Input/Output Memory Management Unit (IOMMU) and DMA Engine Virtualization allow user-level initiation of RDMA operations, thus avoiding expensive system calls and data copying. UNIMEM also allows remote DRAM borrowing and remote load/store instructions, which enable remote-mailbox and remote-interrupt notifications for low-latency protocols.

Network Congestion is an important and hard problem to be dealt with in all large network that aspire not to be permanently underutilized. TCP/IP solves it for the internet in an inefficient, expensive, and long-time-scale manner; other solutions, in hardware, have been tried, with better but non-ideal results. We develop novel solutions for Quality of Service (QoS) –including congestion management– in the interconnection network, always keeping the "data plane" (frequent-case) operations in hardware and only performing infrequent control-plane operations in software. We target our solutions to large networks with many flows, while keeping hardware state down to a reasonable cost.

In the domain of data persistence, we place Storage devices (flash or other non-volatile memories) with the compute nodes rather than in a centralized location, e.g. behind a network (SAN/NAS). We introduce extensions to a parallel file system in order to take advantage of such devices as cache layer. Moreover, we design cache maintenance protocols based on the concepts of UNIMEM memory consistency model. On top of that, we plan to provide replication-based resilience, protecting the filesystem by focusing on metadata integrity.

These projects collaborate on sharing of experimental platforms that are capable of working on the development and evaluation of the Unimem architecture. They aim to implement of a Unimem-based MPI runtime to support HPC applications and to utilize FPGA-based accelerators for improving computational performance and energy efficiency. Another target is the selection of application programs that can be used for evaluation. The four projects are closely collaborating for the organization of joint workshops disseminating their activities.

ExaNoDe (H2020-ICT-671578): European Exascale Processor & Memory Node Design

ExaNoDe investigates, develops, integrates and validates the building blocks for a highly integrated, high-performance, heterogeneous System-on-a-Chip (SoC) aimed towards Exascale computing. It is addressing these important challenges through the coordinated development of several innovative solutions recently deployed in HPC:

The ExaNoDe project will deliver a prototype that will enable the deployment of interposer-based SoCs and the evaluation, tuning and analysis of HPC critical kernels and the associated software stack.

ECOSCALE (H2020-ICT-671632): Energy-Efficient Heterogeneous Computing at ExaScale

Driven by the characteristics and trends of future HPC applications, ECOSCALE will co-design a) novel HPC applications and b) the novel hierarchical UNIMEM+UNILOGIC architecture in order to reach exascale performance while achieving exascale class energy-efficiency. The novel UNILOGIC (Unified Logic) architecture, introduced within ECOSCALE for the first time, is an extension of the UNIMEM architecture, proposed within the EUROSERVER project. UNIMEM provides shared partitioned global address space while UNILOGIC provides shared partitioned reconfigurable resources. The UNIMEM architecture gives the user the option to move tasks and processes close to data instead of moving data around, thus reducing significantly the data traffic and related energy consumption and delays.

EuroEXA (H2020-ICT-754337): Co-designed Innovation and System for Resilient Exascale Computing in Europe: From Applications to Silicon

EuroEXA targets to provide the template for an upcoming exascale system by co-designing and implementing a petascale-level prototype with ground-breaking characteristics. To accomplish this, the project takes a holistic approach innovating both across the technology and the application/system software pillars. EuroEXA proposes a balanced architecture for compute and data-intensive applications, that builds on top of cost-efficient, modular-integration enabled by novel inter-die links, utilises a novel processing unit and embraces FPGA acceleration for computational, networking and storage operations.

EuroEXA hardware designers work together with system software experts optimising the entire stack from language runtimes to low-level kernel drivers, and application developers that bring in a rich mix of key HPC applications from across climate/weather, physical/energy and life-science/bioinformatics domains to enable efficient system co-design and maximise the impact of the project.