Newsgroups: comp.parallel.mpi
From: cronk@ete01-f.icase.edu (David Cronk)
Subject: asynchronous costs
Organization: ICASE/NASA Langley
Date: 13 Jan 1998 15:09:12 GMT
Message-ID: <69g02o$6cr$1@reznor.larc.nasa.gov>

I am using Lam6.1 on a network of sparcstations and am encountering
some timing numbers which I cannot explain.  I am comparing asynchronous
costs with synchronous costs for large message sizes.  Here is some
sample code to try to explain the situation better.  CODE C is closest
to how my code works for reasons that I won't get into here, so
please do not suggest I just switch to CODE A. 

CODE A:

proc 0:          proc 1:         proc 2:         proc 3: 
timer 
LOOP             LOOP            LOOP            LOOP 
MPI_Send (1)     MPI_Recv (0)    MPI_Recv (1)    MPI_Recv (2) 
MPI_Recv (3)     MPI_Send (2)    MPI_Send (3)    MPI_Send (0) 
end LOOP         end LOOP        end LOOP        end LOOP 
timer


CODE B: 

proc 0:          proc 1:         proc 2:         proc 3: 
timer 
LOOP             LOOP            LOOP            LOOP 
MPI_Isend (1)    MPI_Irecv (0)   MPI_Irecv (1)   MPI_Irecv (2) 
MPI_Wait         MPI_Wait        MPI_Wait        MPI_Wait 
MPI_Irecv (3)    MPI_Isend (2)   MPI_Isend (3)   MPI_Isend (0) 
MPI_Wait         MPI_Wait        MPI_Wait        MPI_Wait 
end LOOP         end LOOP        end LOOP        end LOOP 
timer

 
CODE C: 

proc 0:          proc 1:         proc 2:         proc 3: 
timer 
LOOP             LOOP            LOOP            LOOP 
MPI_Isend (1)    MPI_Irecv (0)   MPI_Irecv (1)   MPI_Irecv (2) 
MPI_Test(&f)     MPI_Test(&f)    MPI_Test(&f)    MPI_Test(&f)
while(!f)        while(!f)       while(!f)       while(!f)
  MPI_Test(&f)     MPI_Test(&f)    MPI_Test(&f)    MPI_Test(&f) 
MPI_Irecv (3)    MPI_Isend (2)   MPI_Isend (3)   MPI_Isend (0) 
while(!f)        while(!f)       while(!f)       while(!f)
  MPI_Test(&f)     MPI_Test(&f)    MPI_Test(&f)    MPI_Test(&f) 
end LOOP         end LOOP        end LOOP        end LOOP 
timer

 
I then divide the total time by 4*(# of loops) for average message
transfer time.  Here are the results:


                (average transfer time)
msg size    CODE A     CODE B     CODE C     
16K         12 ms      12 ms      20 ms
256K        102 ms     102 ms     135 ms

Now, I could understand why code C would take a little longer, but
I don't understand why it is 8 milliseconds longer for a 16K message
and 33 milliseconds longer for a 256k message.  Not only is this a
lot of extra time, but why the difference depending on the message
size?  I didn't present the times here, but for messages between
16k and 256k, the difference between A & B compared to C steadily
grows as the message size grows.

Does anybody know what may be causing this phenomenon?  Any help
would be greatly appreciated.

Dave.
 



-- 
=============================================================================
Dave Cronk    					Dave Cronk
College of William and Mary			ICASE
cronk@cs.wm.edu					cronk@icase.edu