Newsgroups: comp.parallel.mpi,comp.programming.threads From: Alexandre Carissimi Subject: Mixing MPI and Threads Organization: LMC-Laboratoire de Modelisation et Calcul Date: Wed, 07 Jan 1998 18:45:37 +0100 Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <34B3BF41.5B02@imag.fr> Hi, I have a problem mixing MPI-LAM and threads, and I would like with somebody had this kind of problem, and if you know if there is some problem on LAM MPI or if my problem is due a bug on my source code. My application is very simple: It is a ring composed by "n" nodes, where "n" is the number of MPI process. Each node has two threads: a thread to read messages arriving on left side and sends to the right side, and another one that read messages arriving on right side and sends to the left side. ----- ----- | t1 | ------------------> |t1 | | t2 | <----------------- |t2 | ----- ----- | ^ | ^ | | | | V | V | ----- ----- | t1| ------------------> |t1 | | t2| <----------------- |t2 | ----- ----- To avoid problems due to fact that MPI isn't thread safe I did some equivalent functions protected by a global mutex, like: int Irecv(void *buf, int count, MPI_Datatype datatype, int dest, int tag, MPI_Comm comm, MPI_Request *request) { int r; pthread_mutex_lock(&mutex); r=MPI_Irecv(buf,count,datatype,dest,tag,comm,request); pthread_mutex_unlock(&mutex); return r; } I use to communicate only non-blocking versions to avoid blocking on MPI. So I have only MPI_Irecv, MPI_Isend and MPI_Itest calls (all protected by the global mutex). THE PROBLEM IS: When I execute this program using the -C2C option on mpirun command line I have a error message like: % mpirun -c 3 -c2c -w -nger /home/asc/threads/RingMT MPI_Irecv: internal MPI error: Invalid argument (rank 0, comm 0) MPI_Irecv: internal MPI error: Invalid argument (rank 1, comm 0) MPI_Irecv: internal MPI error: Invalid argument (rank 2, comm 0) 7543 exited with status 1 7542 exited with status 22 7544 exited with status 1 But, if I execute the program without the -C2C option it works well. My QUESTION is WHY THIS HAPPEN? Somebody knows why bypassing LAM deamon I have this problem? It is possible to solve this. Thanks in advance. Regards. ASC __________________________________________________________________ CARISSIMI, Alexandre Alexandre.Carissimi@imag.fr Laboratoire de Modelisation et Calcul - LMC Institut National Polytechnique de Grenoble - INPG 100, Rue des Mathematiques Fone: (33) 04.76.51.46.03 38041 - Grenoble - Cedex 9 FRANCE Fax : (33) 04.76.51.48.46 (International calls removes 0 from 04) "Le seul moteur pour gagner c'est innover" Renault (F1 - 1996) ________________________________________________________________