Newsgroups: comp.parallel.pvm From: manchek@thud.CS.UTK.EDU (Bob Manchek) Subject: PVM technical FAQ #3 (reposting) Organization: Computer Science Dept, University of Tennessee, Knoxville Date: 2 Nov 1993 14:26:27 GMT PVM version 3 Frequently Asked Questions - Technical 12 Oct 1993 rev. 3 ------------------------------------------------------------------------ Q1: When I run a program under PVM the memory used by the pvmd increases. After the program stops, the percentage of memory doesn't decrease. A1: The pvmd data size will grow as it runs. It uses malloc() (which uses sbrk() probably) to get buffers and malloc never returns memory to the system. However, the buffers are freed, so malloc should use the same space the next time it needs buffers. I would expect the data size to grow when your program is first run, then stay more or less constant for subsequent runs. There could be a memory leak in the pvmd, though we don't know of any big ones. ------------------------------------------------------------------------ Q2: How can I spawn a process with a nice priority value? A2: Sorry, you can't. You'll have to call setpriority() in your code or you could replace your program with a shell script, f.e.: % cd ~/pvm3/bin/SUN4 % mv prog prog- % echo 'P=$0"-"; shift; exec nice -10 $P $@' > prog % chmod 755 prog Then when you spawn "prog" the shell script will exec "prog-" (the real thing) at a nice level. ------------------------------------------------------------------------ Q3: Why does stuff printed by child tasks appear on the console (where the pvmd was started), while output from the parent task appears on the window where I start the parent? A3: The spawned tasks' output goes to their pvmds; their stdouts are not inherited from the parent task. So each pvmd collects output from its tasks through pipes and passes it to the master pvmd, which logs it in /tmp/pvml.UID and on its stdout. Additionally, stderr of spawned tasks is dup()ed to stdout - they're not separate. Finally, spawned tasks have no stdin, /dev/null is opened as file 0. Tasks started by hand may have normal stdin, stdout and stderr connections. Using PVM version 3.2, you can redirect output (stdout/err) from spawned tasks to the console, and to a file if you like. Type "help spawn" for more information. If the output is not redirected to the console, it goes to the master pvmd logfile as before. [Note it doesn't also go to the slave pvmd logfiles as in v <= 3.1] Output direction is inherited, so if you spawn a task from the console and redirect its output to the console, you'll also get output from all its children. In the future, we may support I/O redirection to the parent task. It would be implemented as described above, with the pvmds routing the output messages instead to the parent task, which would receive them and print on its stdout. [Note this probably requires the parent task to do occasional sends or receives to keep data moving.] ------------------------------------------------------------------------ Q4: Can you tell me how to install PVM in a place other than ~/pvm3? A4: I'd suggest installing v3.2 in /usr/local/pvm3 or perhaps in /usr/local/pvm3.2 (yes, that's better). Go there and setenv PVM_ROOT to /usr/local/pvm3.2 and build everything. Then any user on the system should be able to setenv PVM_ROOT to /usr/local/pvm3.2 in .cshrc and have it work. They will share files, specifically from /usr/local/pvm3.2/{lib,include,conf}. Each user's executables will still be expected to be in ~/pvm3/bin/ARCH/, unless they change that path with the ep= hostfile option. Oh, users might want symlinks from their bin directories to /usr/local/pvm3/lib/{pvm,pvmd,aimk}. For example, foreach c (pvm pvmd aimk) ln -s /usr/local/pvm3.2/lib/$c ~/bin end ------------------------------------------------------------------------ Q5: When I use pvm_mcast(), the sender doesn't get a copy of the message, even if I include its tid in the address list. A5: The spec for mcast() has changed slightly from early releases of version 3 - it now removes the sender from the list. It seems to be more often the case that the sender doesn't want to get a copy of the message, even if it is in the list of addresses. If the sender must get the message too, you'll need to send() the message explicitly with, f.e.: pvm_send(pvm_mytid(), code); Or to do it more efficiently, after the mcast() do: pvm_setrbuf(pvm_getsbuf()); It will be as though the sender had just recv()d the message. ------------------------------------------------------------------------ Q6: What does the "speed" field in the machine configuration mean? Can I set it? A6: It's currently part of the outline of a load-balancing mechanism, the idea being that if you're running more than one task per host in the virtual machine, faster machines should get more tasks. The scheduler used by pvm_spawn() should look at the speeds and current loads when assigning new tasks to hosts. You can actually set the speed of each host (which defaults to 1000) to a value from 1..1 000 000 by using the hostfile sp= option. But spawn doesn't use the values right now. ------------------------------------------------------------------------ Q7: Is message order between tasks preserved? A7: Between two given tasks, A and B, message order is preserved. That is, if A sends ten messages to B using pvm_send(), B will download them in that same order. Note messages can be picked out of the middle of the receive queue in B, though, by causing pvm_recv() or pvm_nrecv() to select a particular message. The order of arrival _between_ direct messages (pvm_send()) and multicast messages (pvm_mcast() or pvm_bcast()) is not guaranteed. I think it stays consistent right now (v3.2) because the same routing path is taken by each message. Note that the path is not the same if using PvmRouteDirect routing - direct messages will go from task to task, while multicast messages will still be routed via the pvmds. The order between a direct-routed direct message and a multicast message is not preserved. ------------------------------------------------------------------------ Q8: How can I pack a struct in a message? A8: Unfortunately, that's a hard thing to do in C without some sort of preprocessing before compilation. You'll need to write special pack and unpack routines for any struct you want to pack, for example: struct foo { int *values; /* vector of (length) elements */ short length; double perf; int flags; }; int pkfoo(foop, cnt, std) struct foo *foop; /* vector of foos */ int cnt; /* number of foos */ int std; /* stride between foos */ { int cc; while (cnt-- > 0) { if (cc = pvm_packf("%hd %*d %lf %d", foop->length, (int)foop->length, foop->values, foop->perf, foop->flags)) return cc; foop += std; } return PvmOk; } int upkfoo(foop, cnt, std) struct foo *foop; int cnt; int std; { int cc; while (cnt-- > 0) { if ((cc = pvm_unpackf("%hd", &foop->length)) || (!(foop->values = (int*)malloc(foop->length * sizeof(int))) && (cc = PvmNoMem)) || (cc = pvm_packf("%*d %lf %d", (int)foop->length, foop->values, &foop->perf, &foop->flags))) return cc; foop += std; } return PvmOk; } Note that if the struct contains only primitive types (or other structs that contain only primitive types), that is, no pointers, _and_ you're passing it to a host of identical type (not just one with the same data representations (because they might not build structs the same way)), you could get away with: struct foo foo1; pvm_pkbyte((char*)&foo1, n * sizeof(struct foo), 1); ... pvm_upkbyte((char*)&foo1, n * sizeof(struct foo), 1); ------------------------------------------------------------------------ Q9: Machines at my site share a single /tmp directory. More PVM daemons can't start up because the file /tmp/pvmd.uid# already exists. A9: Get release version >= 3.2.3. This includes a compile-time option to change the name of the file in /tmp to /tmp/pvmd.uid#.hostname, avoiding the conflict. If you want this option enabled for all machine types, edit src/Makefile.aimk and add -DSHAREDTMP to the (uncommented) OPTIONS= line near the top. If you want it set only for certain architectures, edit the config files for those machines in conf/*.def and add -DSHAREDTMP to the ARCHCFLAGS= line in each. You'll need to rebuild the distribution and relink any applications using libpvm. ------------------------------------------------------------------------ / Robert Manchek University of Tennessee / / Computer Science Department / / (615)974-8295 Ayres Hall #104 / / manchek@CS.UTK.EDU. Knoxville TN 37996-1301 /