From: Joachim Worringen Newsgroups: comp.parallel.mpi,comp.unix.solaris Subject: Re: MPICH on Solaris (ch_shmem) Date: Thu, 03 Dec 1998 09:48:49 +0100 Organization: Chair for Operating Systems, RWTH Aachen Message-Id: <36665071.646BC90F@lfbs.rwth-aachen.de> References: <74515u$nd6$1@news.usf.edu> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Xref: ukc comp.parallel.mpi:4385 comp.unix.solaris:164199 David Rabson (PHY) wrote: > > How should I tune shared memory for MPICH on a two-processor Ultra 60 > with loads (>1GB) of memory? I generally use MPI indirectly through > SCALAPACK and BLACS, so I haven't much control over the low-level calls. > > My current program (a matrix diagonalization) runs on four processes > with MPICH (version 1.1.1) compiled with ch_shmem. For a run that goes > on for half an hour, profiling (-xpg, gprof) shows that the program spends > 98% of its time in > > MPID_SHMEM_ReadControl(). This function polls this process' pointer for incoming messages and thus utilizes nearly 100% of the CPU cycles available while there is no message (if the yield() is to be neglected). I recommend using only 2 processes on a 2 processor machine. There also is a define of BACKOFF_LMT for the exponential backoff, this affects the periods between checking for an incoming message. You might try to lower this value an increase the value for select() in p2p_yield(). If this does not help (it should!), you have to take a look at your algorithm to find the reason for a possible load inbalance. AFAIK, there are no compiling or shared memory options that would influence this behaviour (what should these options do?). Joachim -- ---------------------------------------------------------------------------- | _ : Joachim Worringen, Lehrstuhl fuer Betriebssysteme |_|_`__ : RWTH Aachen, Kopernikusstr. 16, D-52056 Aachen | |__) : Tel. : +49-241/80-7609 | Fax : +49-241/8888-339 |__)S : e-Mail : joachim@lfbs.rwth-aachen.de