Technology/ DSMP

Technology - Distributed Symmetric Multiprocessing

Symmetric Multi-Processing (SMP) computers with large shared memory and many processor cores are perfect for Big Data and high-performance computing applications that utilize large in-memory datasets. They are also ideal for hosting large database installations and other critical enterprise software. Symmetric Computing has developed large SMP computers that are a fraction of the cost of currently available mainframe computers.

Symmetric Computing creates departmental supercomputers and departmental mainframes by starting with a tightly coupled cluster of state-of-the-art, industry standard server hardware, connected by Infiniband. A cluster operating system based on a standard Linux distribution is included. This cluster operating system uses a Linux kernel with distributed symmetric multiprocessing (DSMP) extensions that transform the cluster into a single supercomputer. The Symmetric Computing architecture is a marriage of shared memory mainframe and cluster computing technology. A generalized view of this architecture is illustrated in the following diagram.

The DSMP Linux kernel transforms the cluster into a single system image, shared memory, many core supercomputer. This is done without the need of expensive proprietary hardware or using a slower, much less efficient hypervisor implementation. In the DSMP implementation, one node is configured as the head node, providing a user interface. The other nodes are worker nodes. Worker nodes can be large memory nodes that host global memory or compute nodes that host GPUs. A process launched on the head node can take advantage of the memory and cores on worker nodes, by using the SMP programming model. To programmers, the system presents as a single large memory Linux box with hundreds or thousands of cores. Programmers can use the standard Pthreads threading package to get access to all the cores and all the memory. Large datasets can be read into memory and accessed directly. Often programmers can avoid message passing interface (MPI) and complex file-access programming. This makes it simpler to port applications to a supercomputer.

DSMP adds two key extensions to the Linux kernel. They are transactional distributed shared memory (DSM) and a new kernel based global Pthreads implementation. Transactional DSM utilizes a two tiered memory organization, consisting of a local memory partition on each node, and an additional global memory partition on a subset of nodes. The global memory partitions combine to form a single global memory that is addressable by every node. Global memory pages are swapped into local memory partitions on nodes as needed by executing program threads. Modified pages are copied back to global memory. The consistency of global memory is maintained with the aid of a set of memory page locking functions that are available to application programs. A generalized view of this architecture is illustrated in the following diagram.

The DSMP Linux kernel extenstions provide large shared memory, many core SMP computing with both economy and performance.
Download the White Paper describing our DSMP Linux Kernel Extension.