Newsgroups: comp.parallel.mpi
From: Steve Barnard <steve@megafauna.com>
Subject: Re: Shmem_get equivalent in MPI?
Organization: All USENET -- http://www.Supernews.com
Date: Tue, 30 Sep 1997 15:00:02 -0700
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-ID: <34317662.167E@megafauna.com>

Steve Huss-Lederman wrote:
> 
> Reto Koradi <kor@mol.biol.ethz.ch> writes:
> 
> > Charles Packer wrote:
> > > How would I emulate a shmem_get in MPI? We're trying to
> > > make some Cray Fortran 90 code portable.
> 
> ....
> 
> > It then discusses possible extensions which could make this
> > possible (active messages). This is mentioned as a possibility for
> > MPI 2. I understand that at least a draft standard exists in the
> > meantime, maybe somebody more knowledgable can tell us what's
> > in there?
> 
> The MPI-2 standard is done (see http://www.mpi-forum.org).  It
> contains one sided communications.  When each vendor will implement
> the MPI-2 standard is something to as the vendor.
> 
> In MPI-1 you can do something like this but you will likely use
> polling or threads.  You can post recvs with special tags and
> periodically check if it has completed.  The message recvd can tell
> you what data to send back to the processor doing the get.  Thus, a
> get becomes a send/recv pair and "majically" the processor with the
> data does a recv/send pair.  The majic occurs outside MPI (so it may
> not be portable) with the polling or threads.  Not exactly a shmem_get
> but a possible hack.  Clearly this isn't the correct solution and
> other hacks probably exist.  MPI-2 would be the easier and better way
> to get what you want.
> 
> Steve

I used the polling method in a code that implements the SPAI (Sparse Approximate
Inverse) preconditioner (used with iterative solvers and unstructured, sparse
matrices).  It actually worked pretty well.  The pattern of communication was
completely unpredictable and irregular, so I needed a one-sided communication
utility.  In fact, I first implemented this code on the T3D with SHMEM and I
thought it would be impossible with MPI-1 -- until I found someone who was
willing to pay me to do it. :-)

There are several different conditions in which one-sided communication is
needed, including fetching matrix rows, storing matrix rows, dynamic load
balancing, and testing for termination conditions.  A distributed communications
server handles these in a reasonably clean and extensible way.  It uses
MPI_Iprobe to sense when another process is requesting some action.  All the
user has to do is to call the server periodically.

A description of the application, including a discussion of the implementation,
can be found in S.T. Barnard and R.L. Clay, A Portable MPI Implementation of the
SPAI Preconditioner in ISIS++, Proc. Eigth SIAM Conference for PArallel
Processing for Scientific Computing, March 1997.  If anyone is interested I can
also send code.

	Steve Barnard