Configuring mpich


Up: Contents Next: Building a production  mpich Previous: Documentation

The next step is to configure mpich for your particular computing environment. mpich can be built for a variety of parallel computers and also for networks of workstations. Parallel computers supported include the IBM SP1 and SP2 (using various communication options), the TMC CM-5, the Intel Paragon, IPSC860, and Touchstone Delta, the Ncube Ncube2, the Meiko CS-2, the Kendall Square KSR-1 and KSR-2, Convex Exemplar, and IBM, SGI and Sun Multiprocessors. Workstations supported are the Sun4 family, Hewlett-Packard, DEC 3000 and Alpha, IBM RS/6000 family, and SGI. Also supported are Intel x86-based PC clones running the LINUX or FreeBSD operating systems. New ports are always pending.

Configuration of mpich is done with the configure script contained in the top-level directory. This script is automatically generated by the Gnu autoconf program from the file configure.in, but you do not need to have autoconf yourself.

The configure script documents itself in the following way. If you type

configure -usage 
you will get a complete list of arguments and their meanings. So the simplest way to document the options for configure is to just show its output here:


Configuring with args -usage 
Configuring MPICH Version 1.1.1. 
Usage: /home/MPI/mpich/configure [-arch=ARCH_TYPE] [-comm=COMM_TYPE] 
            [-device=DEVICE] 
            [-prefix=INSTALL_DIR]  [-c++[=C++_COMPILER]] 
            [-cc=C_COMPILER] [-fc=FORTRAN_COMPILER] 
            [-clinker=C_LINKER] [-flinker=FORTRAN_LINKER] 
            [-c++linker=CC_LINKER]  
            [-mpe] [-nompe] [-nof77] [-f90nag] [-opt=OPTFLAGS]  
	    [-make=MAKEPGM] 
            [-cflags=CFLAGS] [-fflags=FFLAGS] [-c++flags=CCFLAGS] 
            [-optcc=C_OPTFLAGS] [-optf77=F77_OPTFLAGS] 
            [-lib=LIBRARY] [-mpilibname=MPINAME] 
            [-no_mpegraphics] [-no_short_longs] [-memdebug] 
            [-x11_lib=X11LIB] [-x11_inc=X11INC] 
            [-mpedbg] [-nompedbg] [-cross] [-devdebug] [-nodevdebug]  
            [-debug] [-nodebug] [-tracing] 
	    [-var_pkt] [-pkt_size=LENGTH] [-adi_collective] 
            [-adi_coll_world] 
            [-wish=WISH] [-tcldir=TCLDIR] [-tkdir=TKDIR] 
            [-fortnames=FORTRANNAMES] 
            [-ar_nolocal] [-automountfix=AUTOMOUNTFIX] 
            [-noranlib] [-rsh=RSHCOMMAND] [-rshnol] 
	    [-nexusthreads] [-nexusdir=NEXUSDIR] [-nexuslib=NEXUSLIB] 
            [-nexusconfigure=NEXUSCONFIGURE] [-nexusdebug] 
where 
   ARCH_TYPE    = the type of machine that MPI is to be configured for 
   COMM_TYPE    = communications layer or option to be used 
   DEVICE       = communications device to be used 
   INSTALL_DIR  = directory where MPI will be installed (optional) 
   C++_COMPILER = default is to use g++ (optional) 
   OPTFLAGS     = optimization flags to give the compilers (e.g. -g) 
   CFLAGS       = flags to give C compiler 
   FFLAGS       = flags to give Fortran compiler 
   MAKEPGM      = version of make to use 
   LENGTH       = Length of message at which ADI switches from short 
                  to long message protocol 
   WISH         = Name of tcl/tk wish executable.  Configure will attempt 
                  to find a version of wish for you, but if there is 
                  no wish in your path or you need to use a different 
                  version, use this option.  Used only for the display 
                  tools (nupshot and upshot).  tk 3.x required for nupshot; 
                  tk 3.x or 4.x for upshot. 
   TCLDIR       = Directory containing tcl.  Must have lib/libtcl.a and 
                  include/tcl.h .  Used only for nupshot. 
   TKDIR        = Directory containing tk 3.3, 3.4, 3.5, or 3.6.  Must have  
                  lib/libtk.a and include/tk.h .  Used only for nupshot.   
                  May be the same as TCLDIR. 
   FORTRANNAMES = Form of the Fortran names.  See below. 
   X11LIB       = Full path name for libX11.a 
   X11INC       = Full path name for X11.h 
   AUTOMOUNTFIX = Command to fix automounters 
   RSHCOMMAND   = Command to use for remote shell 
   MPILIBNAME   = Name to use instead of mpi in the name of the MPI 
                  library.  If set, libMPILIBNAME will be used instead 
                  or libmpi.  This can be used on systems with 
                  several different MPI implementations. 
   NEXUSDIR     = the location of the nexus distribution 
   NEXUSLIB     = the location of the nexus library, if not in the standard 
                  place within the distribution 
 
All arguments are optional, but if 'arch', 'comm', or 'prefix' arguments 
are provided, there must be only one.  'arch' must be specified before 
'comm' if they both appear. 
 
If '-c++' is included as an option, then the C++ interface is also built. 
By default, g++ is used as the c++ compiler.  THIS IS CURRENTLY 
UNSUPPORTED. 
 
You can select a different C and Fortran compiler by using the '-cc' and 
'fc' switches.  The environment variables 'CC' and 'FC' can also provide 
values for these but their settings may be overridden by the configure 
script.  Using '-cc=$CC -fc=$FC' will force configure to use those 
compilers. 
 
If '-cross' is given, configure assumes that you are cross-compiling.  If 
it is not given, configure expects to be able to run programs.  Even if 
'-cross' is not selected, configure will try to determine if you are 
cross-compiling; this switch is needed only on systems where attempting 
to run a cross-compiled program causes the configure script to hang. 
 
If '-mpe' is included as an option, then the MPE 'helper' libraries will 
also be built.  '-nompe' causes the MPE libraries to not be built.  The 
default is '-mpe'.  If '-no_mpegraphics' is used, then the MPE routines 
that make use of X11 graphics will NOT be built; this is appropriate for 
systems that either do not have the X11 include files or that do not 
support X11 graphics (some message-passing systems cannot interoperate 
with X11).  The options -x11_inc and -x11_lib may be used to specify the 
locations of the X11 include files and libraries in the event that 
configure cannot find them (they should both be specified in that case). 
 
The option '-mpedbg' enables the '-mpedbg' command line switch in MPI 
programs.  When used with an MPI program, the default error handler 
(i.e., MPI_COMM_WORLD's error handler) tries to start xterm's running dbx 
for each process that detects an error.  This option is intended 
primarily for workstation environments but should work on some MPPs (such 
as IBM SP2). 
 
The option '-nof77' prevents the compilation of routines that require a 
Fortran compiler.  If this option is selected, you may not use the 
Fortran interface to MPI. 
 
The option '-f90nag' allows you to use the NAG Fortran 90 instead of 
Fortran 77.  This is a preliminary version and is based on the version 
for NeXTs. 
 
The option '-opt' allows you to specify optimization options for the 
compilers (both C and Fortran).  For example, '-opt=-O' chooses optimized 
code generation on many systems.  '-optcc' and '-optf77' allow you to 
specify options for just the C or Fortran compilers.  Use -cflags and 
-fflags for options not related to optimization. 
 
The option '-nexusthreads' enables multithreading within the ch_nexus 
device.  The default version of the ch_nexus device is configured as 
single-threaded. 
 
The option '-nexusdebug' enables special debugging services in the 
ch_nexus device.  The default version of the ch_nexus device is 
configured without debugging. 
 
The option '-nexusdir=NEXUSDIR' allows one to specify the location of an 
installed version of Nexus.  The default is to use a version that is 
distributed with MPICH, in the mpid/ch_nexus directory. 
 
The option '-nexuslib=NEXUSLIB' allows one to specify a specific nexus 
library to link against.  The default is -lnexus_dl.  Note that this is 
not a path name. 
 
The option '-nexusconfigure=NEXUSCONFIGURE' allows one to provide  
extra, nexus-specific, arguments to the configure to be run in the nexus 
subdirectory.  The default is "". 
 
The option '-lib' allows you to specify the location of a library that 
may be needed by a particular device.  Most devices do NOT need this 
option; check the installation instructions for those that might. 
 
The option '-make' may be used to select an alternate make program.  For 
example, on FreeBSD systems, -make=gnumake may be required because of 
bugs in the system make. 
 
The option '-no_short_longs' may be used to suppress support for ANSI C 
types 'long long int' and 'long double' when they are the same size as 
'long' and 'double' respectively.  Some systems allow these long ANSI C 
types, but generate a warning message when they are used; this option may 
be used to suppress these messages (and support for these types). 
 
The option '-fortnames=FORTRANNAMES' allows you to specify the form of 
the Fortran names.  This is used primarily to generate names with and 
without trailing underscores for those systems that support both. 
Possible values are 
    FORTRANNAMES value     if Fortran MPI_SEND looks like 
    DOUBLEUNDERSCORE       mpi_send__ 
    UNDERSCORE             mpi_send_ 
    CAPS                   MPI_SEND 
    NOUNDERSCORE           mpi_send  
 
This option should normally NOT be used; configure determines what the 
Fortran compiler generates.  This can be used to override that choice. 
 
The option '-ar_nolocal' prevents the library archive command from 
attempting to use the local directory for temporary space.  This option 
should be used when (a) there isn't much space (less than 20 MB) 
available in the partition where MPICH resides and (b) there is enough 
space in /tmp (or wherever ar places temporary files by default). 
 
The option '-noranlib' causes the 'ranlib' step (needed on some systems 
to build an object library) to be skipped.  This is particularly useful 
on systems where 'ranlib' is optional (allowed but not needed; because it 
is allowed, configure chooses to use it just in case) but can fail (some 
'ranlib's are implemented as scripts using 'ar'; if they don't use the 
local directory, they can fail (destroying the library in the process) if 
the temporary directory (usually '/tmp') does not have enough space. 
This has occured on some OSF systems. 
 
The option '-memdebug' enables extensive internal memory debugging code. 
This should be used only if you are trying to find a memory problem (it 
can be used to help find memory problems in user code as well). 
 
The option '-debug' enables the collection of additional information for 
parallel debuggers such as Totalview. 
 
The option '-tracing' enables tracing of internal calls.  This should be 
used only for debugging the MPICH implementation itself. 
 
The option '-rsh' allows you to select an alternative remote shell 
command (by default, configure will use 'rsh' or 'remsh' from your 
'PATH').  If your remote shell command does not support the '-l' option 
(some AFS versions of 'rsh' have this bug), also give the option 
'-rshnol'.  These options are useful only when building a network version 
of MPICH (e.g., '-device=ch_p4' or '-device=ch_tcp'). 
 
Special Tuning Options: 
 
There are a number of options for tuning the behavoir of the ADI 
(Abstract Device Interface) which is the low-level message-passing 
interface.  These should NOT be used unless you are sure you know what 
you are doing. 
 
The option '-nodevdebug' disables the debugging code in the MPI ADI code. 
This should be used only when you are sure that everything is working 
correctly.  (This option is also present to remind benchmarkers that the 
low level code by default may contain debugging code.)  Note also that 
some of the device code (in mpid/*) has had the debugging code removed 
from the source code.  '-devdebug' turns on the debugging code. 
'-nodevdebug' is the default. 
 
The option '-var_pkt' allows you to set the message size at which MPICH 
changes from its short to long message protocol. 
 
The option '-pkt_size=LENGTH' allows you to choose the message length at 
which the ADI (Abstract Device Interface) switches from its short to long 
message format.  LENGTH must be positive. 
 
The option '-adi_collective' allows the ADI to provide some collective 
operations in addition to the basic point-to-point operations. 
Currently, most systems do not support this option (it is ignored) and on 
the others it has not been extensively tested.  The option 
'-adi_coll_world' asks the ADI to try and use any collective operations 
that are supported only on the MPI_COMM_WORLD communicator (and any 
communicator with a similiar group).  This is also untested. 
 
 
Sample Configure Usage: 
 
To make for running on sun4's running SunOS with ch_p4 as the device, 
 and with the installation directory equal to the current directory: 
 
  ./configure -device=ch_p4 -arch=sun4 
  make 
 
Known devices are  
        ch_nx     (native Intel NX calls),  
        ch_mpl    (native IBM EUI or MPL calls), 
        ch_p4     (p4) 
	ch_nexus  (Nexus) 
        ch_meiko  (for Meiko CS2, using NX compatibility library), 
        ch_shmem  (for shared memory systems, such as SMPs),  
        ch_lfshmem(for shared memory systems, such as SMPs; uses 
                   lock-free message buffers),  
        ch_cenju3 (native NEC Cenju-3 calls) 
 
The following devices were supported with ADI-1, but are currently 
unsupported.  Please contact us if you are interested in helping us 
support these devices: 
        meiko     (for Meiko CS2, using elan tport library), and 
	nx        (for Intel Paragon), 
	t3d	  (for the Cray T3D, using Cray shmem library). 
        ch_nc     (native nCUBE calls, requires -arch=ncube), 
        ch_cmmd   (native TMC CM-5 CMMD calls),  
 
Known architectures include (case is important) 
        sun4      (SUN OS 4.x) 
        solaris   (Solaris) 
	solaris86 (Solaris on Intel platforms) 
        hpux      (HP UX) 
        rs6000    (AIX for IBM RS6000) 
        sgi       (Silicon Graphics IRIX 4.x, 5.x or 6.x) 
        sgi5      (Silicon Graphics IRIX 5.x on R4400's, for the MESHINE) 
        IRIX      (synonym for sgi) 
        IRIX32    (IRIX with 32bit objects -32) 
        IRIXN32   (IRIX with -n32) 
        IRIX64    (IRIX with 64bit objects) 
        alpha     (DEC alpha) 
        intelnx   (Intel i860 or Intel Delta) 
        paragon   (Intel Paragon) 
        meiko     (Meiko CS2) 
        CRAY      (CRAY XMP, YMP, C90, J90, T90) 
        cray_t3d  (CRAY T3D) 
        freebsd   (PC clones running FreeBSD) 
	LINUX     (PC clones running LINUX) 
        ksr       (Kendall Square KSR1 and KSR2) 
        EWS_UX_V  (NEC EWS4800/360AD Series workstation.  Untested.) 
        UXPM      (UXP/M. Untested.) 
        uxpv      (uxp/v. Untested.) 
        SX_4_float0 
                  (NEC SX-4; Floating point format float0 
                             Conforms IEEE 754 standard. 
                   C:       sizeof (int)     = 4; sizeof (float) = 4 
                   FORTRAN: sizeof (INTEGER) = 4; sizeof (REAL)  = 4) 
        SX_4_float1 
                  (NEC SX-4; Floating point format float1 
                             IBM floating point format. 
                   C:       sizeof (int)     = 4; sizeof (float) = 4 
                   FORTRAN: sizeof (INTEGER) = 4; sizeof (REAL)  = 4) 
        SX_4_float2 
                  (NEC SX-4; Floating point format float2 
                             CRAY floating point format. 
                   C:       sizeof (int)     = 4; sizeof (float) = 8 
                   FORTRAN: sizeof (INTEGER) = 8; sizeof (REAL)  = 8) 
                   !!! WARNING !!! This version will not run 
                                   together with FORTRAN routines. 
                                   sizeof (INTEGER) != sizeof (int) 
        SX_4_float2_int64 
                   (NEC SX-4; Floating point format float2 and 
                              64-bit int's) 
                    C:       sizeof (int)     = 8; sizeof (float) = 8 
                    FORTRAN: sizeof (INTEGER) = 8; sizeof (REAL)  = 8) 
 
Special notes: 
 
For SGI (-arch=IRIX) multiprocessors running the ch_p4 device, use 
-comm=ch_p4 to disable the use of the shared-memory p4 communication 
device, and -comm=shared to enable the shared-memory p4 communication 
device.  The default is to enable the shared-memory communication device. 
 
Others may be recognized. 

Normally, you should use configure with as few arguments as you can. For example, setting the C compiler with -cc=xxx may require also setting -cflags=yyy; configure will (usually) choose both the compiler and flags appropriately. If you leave all arguments off, configure will usually guess the correct architecture ( arch) unless you are in a cross-compiling environment, and will usually choose an appropriate device ( device) as well. Where TCP/IP is an appropriate mechanism for communication, the TCP device (ch_p4) will be chosen by default.

mpich is implemented using an abstract device specification (ADI), described in [3]. In some environments, this abstract device is configured to be the native communication subsystem of the machine. This is done with the device argument to configure. For the rest of the environments, a generic communication device is constructed using p4 [1,2] and that is used as the instantiation of the ADI. In these cases, use ch_p4 as the device.

The ARCH_TYPE specifies what kind of processor the compilations will take place on. Valid ones are listed above. For the IBM SP, the architecture type is rs6000. If not given, configure will attempt to determine the type.

Some machines have multiple communication options, which are specified with the comm argument. Currently, the ch_p4 device makes use of this. By selecting -comm=shared, a version of the ch_p4 device that permits the use of both shared memory and IP/TCP is built. This is particularly useful on clusters of symmetric multiprocessors.

A new device (ch_nexus, based on the Nexus run-time system is currently available, although not yet thoroughly tested. Like the ch_p4 device, the Nexus device is able to use multimethod communication on some platforms. For example, the IBM SP binary can communicate via MPL or TCP depending on which node it is communicating with. See http://www.mcs.anl.gov/nexus for more details.

Some sample invocations of configure are shown below. In most cases, the mpe libraries are also built (See User's Guide). To build without the mpe libraries, configure with -nompe. In many case, the detailed invocations below are the defaults, which you would get if you invoked configure with no arguments.

First, for massively parallel processors (MPPs) and multiprocessors:

Convex Exemplar
For a Convex Exemplar, please get the official version from Convex/HP. This is based on MPICH, but has been tuned for better performance on the Exemplar. If for some reason you want to use the shared memory version of MPICH on the Convex, use
configure -device=ch_shmem -arch=hpux 
Cray multiprocessor
(not a CRAY T3D but, for example, a 4 processor Cray YMP or C90)
configure -device=ch_p4 -arch=CRAY 
Cray t3d
(assuming you are logged into the YMP front end).
configure -device=t3d -arch=cray_t3d 
Because the T3D is cross-compiled, you should make sure that your PATH variable contains the directory with the T3D tools. We have seen /mpp/bin and /opt/ctl/bin; check with your system administrator for the paths that you should have when building T3D applications. Make sure that this path is at the head of your path list.
Intel Paragon

configure -device=ch_nx -arch=paragon 
IBM SP2
(using the high-performance switch for communication)
configure -device=ch_mpl -arch=rs6000 
To use this you must have POE installed. Note, however, that this works only for POE versions without a native MPI. If you do have IBM's native MPI, then you can try the following (this is untested)
cd include 
ln -s /usr/lpp/ppe.poe/include/mpproto.h mpproto.h 
cd .. 
configure -arch=rs6000 -device=ch_mpl -opt="-O3 -qarch=pwr2"  \ 
          -cc=cc -clinker=mpcc -f77=xlf -flinker=mpxlf \ 
          -make=/usr/local/bin/gnumake -mpilibname=mpich 
make  
This is unsupported; if you have suggestions for improving these instructions, please let us know.
Meiko CS-2

configure -device=ch_meiko -arch=meiko 
SGI multiprocessors
such as the Power Challenge and Origin 2000.
configure -device=ch_shmem  
If you need to generate a particular version that corresponds to the -32, -n32, or -64 compiler/linker options on SGI, use the architectures IRIX32, IRIXN32, or IRIX64 respectively instead of SGI. Specifically, use the following for an R10000 SGI:


configure -arch=IRIX64 \ 
        -cc="cc -64 -mips4 -r10000" \ 
        -fc="f77 -64 -mips4 -r10000" \ 
        -opt="-O2" \ 
        -device=ch_shmem 

configure -arch=IRIXN32 \ -cc="cc -n32 -mips4 -r10000" \ -fc="f77 -n32 -mips4 -r10000" \ -opt="-O2" \ -device=ch_shmem

configure -arch=IRIX32 \ -cc="cc -32" \ -fc="f77 -32" \ -opt="-O2" \ -device=ch_shmem

(The optimization level is optional; -O2 has worked for some users. Be careful of aggressive optimization, particularly in the mpid/ch_shmem code.)

See the comments under SGI workstations for different 32 and 64 bit options.


For networks of workstations (can interoperate with other types of workstations),
DEC Alpha
In order to get the full advantages of ANSI C, you may need to add -cflags="-std". For strict ANSI C, use -cflags="-std1".

IBM RS6000
In order to get the advantages of ANSI C, you may need to add -cflags="-qlanglvl=ansi". The MPICH code uses __STDC__ to check for the presence of ANSI C features; the IBM RS6000 compilers do not define this by default.

SGI

configure -device=ch_p4 
Some SGI systems support both 32 and 64 bit pointers (addresses). MPICH uses the architecture IRIX to refer to 32 bit systems and IRIX64 for 64 bit systems. You can use -arch=IRIX or -arch=IRIX64 to force a particular system. If you need to generate a particular version that corresponds to the -32, -n32, or -64 compiler/linker options on SGI, use the architectures IRIX32, IRIXN32, or IRIX64 respectively instead of sgi.

SGI multiprocessor
(such as an Onyx, Challenge, or Power Challenge), using the shared memory for fast message-passing
configure -device=ch_p4 -comm=shared 
Use -arch=IRIX to force 32 bit pointers and -arch=IRIX64 to force 64 bit pointers.

On the SGI Origin machines, it is better to let configure choose the architecture and to attach the flags to the compilers and linker as follows:


configure -cc='cc -64' -fc='f77 -64'  .... 
Sun SunOS
including the mpe libraries (See the Users Guide [5]:
configure -device=ch_p4 -arch=sun4 -mpe 
Sun Solaris

configure -device=ch_p4 -arch=solaris -mpe 
DEC Alpha

configure -device=ch_p4 -arch=alpha 
FreeBSD
For a network of PC's running the FreeBSD version of Unix:
configure -device=ch_p4 -arch=freebsd 
Nexus device
For a machine using the Nexus device, change the device in the above examples to ch_nexus. The Nexus device requires an ANSI C compiler, because the Nexus header files use the function prototypes. If the default compiler is does not support function prototypes (assuming ansicc is your compiler), add -cc=ansicc to the configure command line. Remember to point this at the correct compiler for your message passing system (e.g., mpcc for the SP, or icc for the Paragon).

You may need to set the environment variables NEXUS_INCLUDE_DIR and NEXUS_LIB_DIR to indicate the directories containing the Nexus include files and libraries respectively. Do this before running configure.

HP HPUX
For a network of HP's, including the mpe library but leaving out of it the MPE X graphics routines:
configure -device=ch_p4 -arch=hpux -mpe -no_mpegraphics 
Fujitsu
For a network of Fujitsu M780s running UXP/M, the following options have been tested:
setenv FC frt 
    configure -arch=UXPM -device=ch_p4 -fflags="-Oe,-Uep -Eml -Aabe" \ 
        -mpe -mpedbg -prefix=/usr/local/mpi \ 
        -tcldir=/usr/local -tkdir=/usr/local -wish=/usr/local/bin/wish 
NEC SX-4
For an NEC SX-4 shared-memory vector multiprocessor, use
configure -device-lfshmem 
to get the lock-free shared-memory device described in [6].

mpich can be run on a heterogeneous network of workstations of various kinds. For simple collections of workstations, the mpirun command can be used; more complex collections of heterogeneous machines require a p4 ``procgroup file'' (for the ch_p4 device) or a ``startup file'' (for the ch_nexus device). The format of these files is described in Section Special Considerations for Running on a Network of Workstations .



Up: Contents Next: Building a production  mpich Previous: Documentation