MPI message-passing libraries
for distributed and shared memory environments, plus the ROMIO
MPI I/O package. Three different versions are available (tcp, sysv,
and usysv), corresponding to different mechanisms for local communication
within SMP nodes. Installed in both 32-bit (v8plusa, v8plusb) and 64-bit
(v9a, v9b) flavors.
/usr/local/[v8plusa,v8plusb,v9a,v9b]/[generic,ultra3cu]/lam-6.5.9-[tcp,sysv,usysv]
LAM 6.5.9 has been superceded
by LAM 7.1.2. All LAM applications should
migrate to LAM 7.1.2 as soon as practicable.
LAM/MPI 6.5.9 programs
on SciClone must be initiated via PBS using
the pbslam-6.5
command rather than with mpirun. See the SciClone
User's Guide and pbslam-6.5
man page for complete details.
A total of twelve different
variants of LAM are installed on SciClone to provide for:
- 32-bit vs. 64-bit
addressing (v8*, v9*)
- UltraSPARC II vs.
UltraSPARC III processors (v*a, v*b)
- different LAM transport
layers (tcp, sysv, usysv)
Which of these is optimum
depends on the characteristics of the application and which type(s)
of nodes it is running on. Therefore, we've made them all available
for users to experiment with. "v8plusa" and "v9a"
will run satisfactorily on any node in SciClone, but "v8plusb"
and "v9b" should only be used on nodes with UltraSPARC III
processors (c5, n2, db1, and s3 node
types).
The "v8plusa"
and "v9a" versions are built for use with a "generic"
UltraSPARC CPU, while the "v8plusb" and "v9b" versions
are compiled to take advantage of the "ultra3cu" chip found
only on our UltraSPARC III nodes. The choice between "generic"
or "ultra3cu" is determined automatically if you use the default
.cshrc file described below.
The choice of transport
layer is a little trickier. "tcp" uses TCP sockets for all
communication, both local and remote, while "sysv" and "usysv"
use shared memory to optimize local communication within a node. "usysv"
uses a combination of spinlocks and semaphores to coordinate access
to shared memory. "sysv" relies only on semaphores. If only
one process is spawned on each node, then all three transports should
provide similar performance.
Although LAM's shared
memory transports can achieve very low message latencies and high bandwidth,
some applications actually perform better on multi-cpu nodes (sometimes
much better) by using TCP for all communication. The reasons for this
are poorly understood at this time.
Environment Variables
For proper operation,
the LAMHOME environment variable must be set in the user's ~/.cshrc
file to specify the desired variant of LAM. $LAMHOME/bin should be in
the user's search path, and $LAMHOME/man should be included in the MANPATH.
The file /usr/local/etc/templates/cshrc contains the current
recommended environment settings and includes options for specifying
which version of LAM to use, and for switching between LAM and other
MPI packages.
