The Extrae performance analysis tool helps the domain developers to understand the performance behaviour and identify the current bottlenecks of their applications developed on top of the proposed technologies. In the scope of EPEEC, Extrae has been extended to support the instrumentation of GASPI applications both standalone and in combination with other parallel runtimes (e.g, OpenMP, CUDA, OmpSs).
Extrae is the main instrumentation package from BSC. It supports many of the principal parallel programming models (e.g., MPI, OpenMP, CUDA, OmpSs), as well as combinations of them (MPI+OpenMP, MPI+CUDA, MPI+OpenMP+CUDA). The main way of working with Extrae is with library interposition through LD_PRELOAD, capturing the calls to the parallel runtimes and calling the actual implementation of the routines afterwards. Extrae generates trace files that can be analysed after application execution with Paraver, the main visualization and analysis tool from BSC.
In the scope of EPEEC Extrae has been extended to support GASPI programs. GASPI naturally supports RDMA (Remote Direct Memory Access) capabilities of the hardware. It runs one-sided, asynchronous communication instructions on an address space partitioned into segments that can be either local or remote. The asynchronous approach allows overlap of computation and communication allowing a very efficient parallel execution of the code.
We have analysed the reverse time migration (FRTM) code which is a massively parallel seismic code based on GASPI with EXTRAE. The FRTM application has taken part in the Extreme Scale Workshop at LRZ in 2015 and 2016 and has shown almost perfect strong scaling behavior over four orders of magnitude on up to 43 000 cores with over 70% of parallel efficiency. Even in such a highly optimized code Extrae could find small imbalance in the processing of the threads and show them efficiently in the Paraver tool, as can be seen in the figure.
Extrae can be checked out from the GASPI branch in the public GitHub repository for Extrae: https://github.com/epeec/extrae