CDS & MPI

By far, the most common reason people give for not considering CDS is the belief that MPI is already more than adequate, and it is a standard, while CDS is not. This motivates us to address these issues here. elepar believes that if people can make the choice based on hard facts, CDS will appear quite attractive, even to those who have their doubts due to the "non-standard" nature of CDS. A point-by-point comparison between CDS and MPI would be far too long and complex to be useful, so we restrict this discussion to philosophy and design approach.

Differences in philosophy

Perhaps the most important philosophical difference between CDS and MPI is that of overall goal. MPI, as its name implies, is a message passing interface, and message passing is a mechanism for facilitating inter-processor communication by utilizing distributed copying (i.e. where one process specifies the source of the copy and the other specifies the destination). CDS, on the other hand, is for facilitating inter-process communication, regardless of whether those processes are on different processors or on the same processor, or share memory or don't. CDS is therefore a means of expressing the data sharing policies of the individual processes, and then utilizing the most efficient mechanisms (which may include message passing) to facilitate those policies.

A lower-level philosophy which crept into MPI (and then necessarily remained there throughout) was the lack of reliance on "agents". An agent, in this context, corresponds to the idea of a daemon, a utility thread, or a communication co-processor: It is an entity which has access to some portion of a process's information (i.e. data space), and is capable of responding to external requests for that information without the direct involvement of the process (or main thread in the process) itself. As a general rule, any feature or requirement which would require the use of agents in its implementation (for one or more architectures) was excluded from MPI. CDS, on the other hand, assumes the presence of some limited access to remote processes which may require the use of agents (just as other message-passing models require agents to queue incoming messages).

This philosophical difference has motivated many other differences. It means that in MPI, no communication (whether message passing or one-sided or broadcast) can be assured to take place without the involvement of processes on both sides of the communication. It is the reason that an MPI_SEND, for example, is not guaranteed to even queue the message in the receiver, and therefore is not guaranteed to complete until the matching MPI_RECV has been performed. It is the reason that many operations in MPI are "collective", meaning that all involved processes must perform the operation (usually requiring or serving as a global synchronization, and requiring that all processes use identical arguments). It is the reason that the one-sided MPI operations are rather complex. On the other hand, in CDS, a few simple forms of one-sided communication (enq, deq, read, and write) are assumed to complete with the involvement of only one process, and therefore to require an agent (acting for the other process) on some systems (though with barely more functionality than that already supported by existing architectures to facilitate message queueing). From these basic one-sided operations springs the functionality of the rest of CDS, including shared memory, message passing, and multi-to-multi operations. elepar believes that the additional asynchronous behavior possible in CDS more than justifies the presence of an agent by creating more efficient programs--i.e. ones that require fewer synchronizations and less necessity for manual load balancing.

Another related philosophical difference is the placing of a high value on static process creation, while CDS views dynamic process creation as normal. In a similar vein, all processes in an MPI program are known to all others, by virtue of being part of the MPI_WORLD communicator, while in CDS, an information-hiding approach is taken which values the revelation of a process's identity only to those other processes which may have dealings with it.

MPI was designed to be fully functional in a Fortran environment. CDS, while arguably as functional as MPI in a Fortran environment, relies on C-like pointers to achieve the utmost in efficiency, and since such pointers are unavailable in standard Fortrans, the Fortran programmer is slightly limited relative to the C programmer in some cases (though is never completely excluded from any CDS functionality in Fortran). Interestingly, even MPI-2 found conformance to Fortran90 too difficult to attain, and therefore has not published a Fortran90 binding.

Differences in design approach

Perhaps the most apparent differences in CDS and MPI come from their very different design approaches. MPI was designed in two stages (MPI-1 and MPI-2), and each stage (especially the latter) involved a rather large number of self-selected designers from academia and industry, broken up into subcommittees, each led by a "chapter author". This approach had several impacts on the design:

The large number or designers led to an environment where it was arguably easier to add a feature (especially one that might increase performance, even if only on a few platforms) than to delete one (which essentially required everyone to re-analyze why the feature was present in the first place, and whether its deletion would impact many other decisions made since, perhaps some in other subcommittees).

The designers on each subcommittee were not expected to fully understand what what going on in other subcommittees, so to some extent, the final MPI product consists of a collection of fairly-well integrated individual products, each formidable enough to be difficult to understand on its own. As a result, the end-user of MPI, who may need to use many different parts of MPI (or at least understand them well enough to know that they will not be useful for the problem at hand) is sometimes required to understand more of the standard than even many of the designers did, and this can be daunting. This problem extends to MPI implementors, partially explaining the dearth of complete MPI-2 implementations.

New features required a vote, and vendors were unlikely to vote for functionality that didn't already exist (or couldn't easily and obviously be made to exist) on their systems, so some useful approaches were discarded relatively quickly unless they had already been implemented on a number of vendors platforms.

During MPI-2 design, it was decided early on that MPI-2 would be an extension to MPI-1 (designed years earlier), which would remain without alteration other than correcting clearly erroneous or poorly-stated behavior. This sometimes meant accepting old design for new functionality, even when the inherent problems with doing so were apparent during the later design. For example, when considering some CDS functionality in the one-sided committee during MPI-2 design, it became clear that decisions in MPI-1 regarding liveness and the absence of agents (described above) would need to be revisited, and the significant overlap between CDS functionality and that already in MPI-1 proved to be a barrier in itself.

The CDS design, on the other hand, has so far been completely controlled and led by a single person, which is not to say that good ideas and suggestions have not come from other sources. In fact, in addition to several other sources, CDS has the benefit of the MPI design rationale and experience. During this time, CDS did not have a large installed base, and so design changes were as simple as the stroke of a pen (or mouse). As a result, CDS is compact, cohesive, and easy to understand. Now that CDS has progressed to its current state of maturity, elepar believes that it is ready to be exposed to a more traditional standards process, to help assure the stability and viability which some potential users will require.