HPC Servers Evolution
There existed roughly 4 choices over time:
1. SMP Fine Grain
Classic Vector choice since H/W offered very few, very fast CPUs,
scalability to 4 CPUs typically. Parallelism at the DO-LOOP
(matrix/data) level and with compiler options. OpenMP has improved this
somewhat today to about 16 CPUs, but not 100's to 1000's that is capable
today.
2. SMP Coarse Grain
Evolved with SMP programming model as way to higher levels of parallel
within an application. Rather than data level, sought parallelism at the
geometry, or domain level. There were not many tools to assist with this
effort.
3. DMP (Distributed Memory Parallel)
Began with MPP architectures and cluster of networked W/Ss, and software
tools such as PVM (parallel virtual machine) but did not achieve success
commercially until current RISC architectures and MPI. Another
breakthrough for commercial MCAE software were robust graph partitioning
schemes such as METIS that could effectively partition the geometry
domain to (1) eliminate the need for user inter-action, (2) evenly
distribute computational 'work' between each partition of the domain,
and (3) minimize boundary communication, across which messages must be
passed (again more on this below on H/W). This is the most popular
choice today for MCAE applications.
4. Hybrid SMP/DMP
This is what's now emerging, the potential to use say OpenMP within an
SMP 'node' whereby a node is typically as large as 16 CPUs and MPI
between the SMP nodes. This method seeks to leverage the programming and
performance advantages of both SMP and DMP.