Up: Contents Next: Building a production mpich Previous: Documentation
The next step is to configure mpich for your particular computing environment. mpich can be built for a variety of parallel computers and also for networks of workstations. Parallel computers supported include the IBM SP1 and SP2 (using various communication options), the TMC CM-5, the Intel Paragon, IPSC860, and Touchstone Delta, the Ncube Ncube2, the Meiko CS-2, the Kendall Square KSR-1 and KSR-2, Convex Exemplar, and IBM, SGI and Sun Multiprocessors. Workstations supported are the Sun4 family, Hewlett-Packard, DEC 3000 and Alpha, IBM RS/6000 family, and SGI. Also supported are Intel x86-based PC clones running the LINUX or FreeBSD operating systems. New ports are always pending.
Configuration of mpich is done with the configure script contained in the top-level directory. This script is automatically generated by the Gnu autoconf program from the file configure.in, but you do not need to have autoconf yourself.
The configure script documents itself in the following way. If you type
configure -usageyou will get a complete list of arguments and their meanings. So the simplest way to document the options for configure is to just show its output here:
Configuring with args -usage Configuring MPICH Version 1.1.1. Usage: /home/MPI/mpich/configure [-arch=ARCH_TYPE] [-comm=COMM_TYPE] [-device=DEVICE] [-prefix=INSTALL_DIR] [-c++[=C++_COMPILER]] [-cc=C_COMPILER] [-fc=FORTRAN_COMPILER] [-clinker=C_LINKER] [-flinker=FORTRAN_LINKER] [-c++linker=CC_LINKER] [-mpe] [-nompe] [-nof77] [-f90nag] [-opt=OPTFLAGS] [-make=MAKEPGM] [-cflags=CFLAGS] [-fflags=FFLAGS] [-c++flags=CCFLAGS] [-optcc=C_OPTFLAGS] [-optf77=F77_OPTFLAGS] [-lib=LIBRARY] [-mpilibname=MPINAME] [-no_mpegraphics] [-no_short_longs] [-memdebug] [-x11_lib=X11LIB] [-x11_inc=X11INC] [-mpedbg] [-nompedbg] [-cross] [-devdebug] [-nodevdebug] [-debug] [-nodebug] [-tracing] [-var_pkt] [-pkt_size=LENGTH] [-adi_collective] [-adi_coll_world] [-wish=WISH] [-tcldir=TCLDIR] [-tkdir=TKDIR] [-fortnames=FORTRANNAMES] [-ar_nolocal] [-automountfix=AUTOMOUNTFIX] [-noranlib] [-rsh=RSHCOMMAND] [-rshnol] [-nexusthreads] [-nexusdir=NEXUSDIR] [-nexuslib=NEXUSLIB] [-nexusconfigure=NEXUSCONFIGURE] [-nexusdebug] where ARCH_TYPE = the type of machine that MPI is to be configured for COMM_TYPE = communications layer or option to be used DEVICE = communications device to be used INSTALL_DIR = directory where MPI will be installed (optional) C++_COMPILER = default is to use g++ (optional) OPTFLAGS = optimization flags to give the compilers (e.g. -g) CFLAGS = flags to give C compiler FFLAGS = flags to give Fortran compiler MAKEPGM = version of make to use LENGTH = Length of message at which ADI switches from short to long message protocol WISH = Name of tcl/tk wish executable. Configure will attempt to find a version of wish for you, but if there is no wish in your path or you need to use a different version, use this option. Used only for the display tools (nupshot and upshot). tk 3.x required for nupshot; tk 3.x or 4.x for upshot. TCLDIR = Directory containing tcl. Must have lib/libtcl.a and include/tcl.h . Used only for nupshot. TKDIR = Directory containing tk 3.3, 3.4, 3.5, or 3.6. Must have lib/libtk.a and include/tk.h . Used only for nupshot. May be the same as TCLDIR. FORTRANNAMES = Form of the Fortran names. See below. X11LIB = Full path name for libX11.a X11INC = Full path name for X11.h AUTOMOUNTFIX = Command to fix automounters RSHCOMMAND = Command to use for remote shell MPILIBNAME = Name to use instead of mpi in the name of the MPI library. If set, libMPILIBNAME will be used instead or libmpi. This can be used on systems with several different MPI implementations. NEXUSDIR = the location of the nexus distribution NEXUSLIB = the location of the nexus library, if not in the standard place within the distribution All arguments are optional, but if 'arch', 'comm', or 'prefix' arguments are provided, there must be only one. 'arch' must be specified before 'comm' if they both appear. If '-c++' is included as an option, then the C++ interface is also built. By default, g++ is used as the c++ compiler. THIS IS CURRENTLY UNSUPPORTED. You can select a different C and Fortran compiler by using the '-cc' and 'fc' switches. The environment variables 'CC' and 'FC' can also provide values for these but their settings may be overridden by the configure script. Using '-cc=$CC -fc=$FC' will force configure to use those compilers. If '-cross' is given, configure assumes that you are cross-compiling. If it is not given, configure expects to be able to run programs. Even if '-cross' is not selected, configure will try to determine if you are cross-compiling; this switch is needed only on systems where attempting to run a cross-compiled program causes the configure script to hang. If '-mpe' is included as an option, then the MPE 'helper' libraries will also be built. '-nompe' causes the MPE libraries to not be built. The default is '-mpe'. If '-no_mpegraphics' is used, then the MPE routines that make use of X11 graphics will NOT be built; this is appropriate for systems that either do not have the X11 include files or that do not support X11 graphics (some message-passing systems cannot interoperate with X11). The options -x11_inc and -x11_lib may be used to specify the locations of the X11 include files and libraries in the event that configure cannot find them (they should both be specified in that case). The option '-mpedbg' enables the '-mpedbg' command line switch in MPI programs. When used with an MPI program, the default error handler (i.e., MPI_COMM_WORLD's error handler) tries to start xterm's running dbx for each process that detects an error. This option is intended primarily for workstation environments but should work on some MPPs (such as IBM SP2). The option '-nof77' prevents the compilation of routines that require a Fortran compiler. If this option is selected, you may not use the Fortran interface to MPI. The option '-f90nag' allows you to use the NAG Fortran 90 instead of Fortran 77. This is a preliminary version and is based on the version for NeXTs. The option '-opt' allows you to specify optimization options for the compilers (both C and Fortran). For example, '-opt=-O' chooses optimized code generation on many systems. '-optcc' and '-optf77' allow you to specify options for just the C or Fortran compilers. Use -cflags and -fflags for options not related to optimization. The option '-nexusthreads' enables multithreading within the ch_nexus device. The default version of the ch_nexus device is configured as single-threaded. The option '-nexusdebug' enables special debugging services in the ch_nexus device. The default version of the ch_nexus device is configured without debugging. The option '-nexusdir=NEXUSDIR' allows one to specify the location of an installed version of Nexus. The default is to use a version that is distributed with MPICH, in the mpid/ch_nexus directory. The option '-nexuslib=NEXUSLIB' allows one to specify a specific nexus library to link against. The default is -lnexus_dl. Note that this is not a path name. The option '-nexusconfigure=NEXUSCONFIGURE' allows one to provide extra, nexus-specific, arguments to the configure to be run in the nexus subdirectory. The default is "". The option '-lib' allows you to specify the location of a library that may be needed by a particular device. Most devices do NOT need this option; check the installation instructions for those that might. The option '-make' may be used to select an alternate make program. For example, on FreeBSD systems, -make=gnumake may be required because of bugs in the system make. The option '-no_short_longs' may be used to suppress support for ANSI C types 'long long int' and 'long double' when they are the same size as 'long' and 'double' respectively. Some systems allow these long ANSI C types, but generate a warning message when they are used; this option may be used to suppress these messages (and support for these types). The option '-fortnames=FORTRANNAMES' allows you to specify the form of the Fortran names. This is used primarily to generate names with and without trailing underscores for those systems that support both. Possible values are FORTRANNAMES value if Fortran MPI_SEND looks like DOUBLEUNDERSCORE mpi_send__ UNDERSCORE mpi_send_ CAPS MPI_SEND NOUNDERSCORE mpi_send This option should normally NOT be used; configure determines what the Fortran compiler generates. This can be used to override that choice. The option '-ar_nolocal' prevents the library archive command from attempting to use the local directory for temporary space. This option should be used when (a) there isn't much space (less than 20 MB) available in the partition where MPICH resides and (b) there is enough space in /tmp (or wherever ar places temporary files by default). The option '-noranlib' causes the 'ranlib' step (needed on some systems to build an object library) to be skipped. This is particularly useful on systems where 'ranlib' is optional (allowed but not needed; because it is allowed, configure chooses to use it just in case) but can fail (some 'ranlib's are implemented as scripts using 'ar'; if they don't use the local directory, they can fail (destroying the library in the process) if the temporary directory (usually '/tmp') does not have enough space. This has occured on some OSF systems. The option '-memdebug' enables extensive internal memory debugging code. This should be used only if you are trying to find a memory problem (it can be used to help find memory problems in user code as well). The option '-debug' enables the collection of additional information for parallel debuggers such as Totalview. The option '-tracing' enables tracing of internal calls. This should be used only for debugging the MPICH implementation itself. The option '-rsh' allows you to select an alternative remote shell command (by default, configure will use 'rsh' or 'remsh' from your 'PATH'). If your remote shell command does not support the '-l' option (some AFS versions of 'rsh' have this bug), also give the option '-rshnol'. These options are useful only when building a network version of MPICH (e.g., '-device=ch_p4' or '-device=ch_tcp'). Special Tuning Options: There are a number of options for tuning the behavoir of the ADI (Abstract Device Interface) which is the low-level message-passing interface. These should NOT be used unless you are sure you know what you are doing. The option '-nodevdebug' disables the debugging code in the MPI ADI code. This should be used only when you are sure that everything is working correctly. (This option is also present to remind benchmarkers that the low level code by default may contain debugging code.) Note also that some of the device code (in mpid/*) has had the debugging code removed from the source code. '-devdebug' turns on the debugging code. '-nodevdebug' is the default. The option '-var_pkt' allows you to set the message size at which MPICH changes from its short to long message protocol. The option '-pkt_size=LENGTH' allows you to choose the message length at which the ADI (Abstract Device Interface) switches from its short to long message format. LENGTH must be positive. The option '-adi_collective' allows the ADI to provide some collective operations in addition to the basic point-to-point operations. Currently, most systems do not support this option (it is ignored) and on the others it has not been extensively tested. The option '-adi_coll_world' asks the ADI to try and use any collective operations that are supported only on the MPI_COMM_WORLD communicator (and any communicator with a similiar group). This is also untested. Sample Configure Usage: To make for running on sun4's running SunOS with ch_p4 as the device, and with the installation directory equal to the current directory: ./configure -device=ch_p4 -arch=sun4 make Known devices are ch_nx (native Intel NX calls), ch_mpl (native IBM EUI or MPL calls), ch_p4 (p4) ch_nexus (Nexus) ch_meiko (for Meiko CS2, using NX compatibility library), ch_shmem (for shared memory systems, such as SMPs), ch_lfshmem(for shared memory systems, such as SMPs; uses lock-free message buffers), ch_cenju3 (native NEC Cenju-3 calls) The following devices were supported with ADI-1, but are currently unsupported. Please contact us if you are interested in helping us support these devices: meiko (for Meiko CS2, using elan tport library), and nx (for Intel Paragon), t3d (for the Cray T3D, using Cray shmem library). ch_nc (native nCUBE calls, requires -arch=ncube), ch_cmmd (native TMC CM-5 CMMD calls), Known architectures include (case is important) sun4 (SUN OS 4.x) solaris (Solaris) solaris86 (Solaris on Intel platforms) hpux (HP UX) rs6000 (AIX for IBM RS6000) sgi (Silicon Graphics IRIX 4.x, 5.x or 6.x) sgi5 (Silicon Graphics IRIX 5.x on R4400's, for the MESHINE) IRIX (synonym for sgi) IRIX32 (IRIX with 32bit objects -32) IRIXN32 (IRIX with -n32) IRIX64 (IRIX with 64bit objects) alpha (DEC alpha) intelnx (Intel i860 or Intel Delta) paragon (Intel Paragon) meiko (Meiko CS2) CRAY (CRAY XMP, YMP, C90, J90, T90) cray_t3d (CRAY T3D) freebsd (PC clones running FreeBSD) LINUX (PC clones running LINUX) ksr (Kendall Square KSR1 and KSR2) EWS_UX_V (NEC EWS4800/360AD Series workstation. Untested.) UXPM (UXP/M. Untested.) uxpv (uxp/v. Untested.) SX_4_float0 (NEC SX-4; Floating point format float0 Conforms IEEE 754 standard. C: sizeof (int) = 4; sizeof (float) = 4 FORTRAN: sizeof (INTEGER) = 4; sizeof (REAL) = 4) SX_4_float1 (NEC SX-4; Floating point format float1 IBM floating point format. C: sizeof (int) = 4; sizeof (float) = 4 FORTRAN: sizeof (INTEGER) = 4; sizeof (REAL) = 4) SX_4_float2 (NEC SX-4; Floating point format float2 CRAY floating point format. C: sizeof (int) = 4; sizeof (float) = 8 FORTRAN: sizeof (INTEGER) = 8; sizeof (REAL) = 8) !!! WARNING !!! This version will not run together with FORTRAN routines. sizeof (INTEGER) != sizeof (int) SX_4_float2_int64 (NEC SX-4; Floating point format float2 and 64-bit int's) C: sizeof (int) = 8; sizeof (float) = 8 FORTRAN: sizeof (INTEGER) = 8; sizeof (REAL) = 8) Special notes: For SGI (-arch=IRIX) multiprocessors running the ch_p4 device, use -comm=ch_p4 to disable the use of the shared-memory p4 communication device, and -comm=shared to enable the shared-memory p4 communication device. The default is to enable the shared-memory communication device. Others may be recognized.
Normally, you should use configure with as few arguments as you can. For example, setting the C compiler with -cc=xxx may require also setting -cflags=yyy; configure will (usually) choose both the compiler and flags appropriately. If you leave all arguments off, configure will usually guess the correct architecture ( arch) unless you are in a cross-compiling environment, and will usually choose an appropriate device ( device) as well. Where TCP/IP is an appropriate mechanism for communication, the TCP device (ch_p4) will be chosen by default.
mpich is implemented using an abstract device specification (ADI), described in . In some environments, this abstract device is configured to be the native communication subsystem of the machine. This is done with the device argument to configure. For the rest of the environments, a generic communication device is constructed using p4 [1,2] and that is used as the instantiation of the ADI. In these cases, use ch_p4 as the device.
The ARCH_TYPE specifies what kind of processor the compilations will take place on. Valid ones are listed above. For the IBM SP, the architecture type is rs6000. If not given, configure will attempt to determine the type.
Some machines have multiple communication options, which are specified with the comm argument. Currently, the ch_p4 device makes use of this. By selecting -comm=shared, a version of the ch_p4 device that permits the use of both shared memory and IP/TCP is built. This is particularly useful on clusters of symmetric multiprocessors.
A new device (ch_nexus, based on the Nexus run-time system is currently available, although not yet thoroughly tested. Like the ch_p4 device, the Nexus device is able to use multimethod communication on some platforms. For example, the IBM SP binary can communicate via MPL or TCP depending on which node it is communicating with. See http://www.mcs.anl.gov/nexus for more details.
Some sample invocations of configure are shown below. In most cases, the mpe libraries are also built (See User's Guide). To build without the mpe libraries, configure with -nompe. In many case, the detailed invocations below are the defaults, which you would get if you invoked configure with no arguments.
First, for massively parallel processors (MPPs) and multiprocessors:
- Convex Exemplar
For a Convex Exemplar, please get the official version from Convex/HP.
This is based on MPICH, but has been tuned for better performance on the
Exemplar. If for some reason you want to use the shared memory version
of MPICH on the Convex, use
configure -device=ch_shmem -arch=hpux
- Cray multiprocessor
(not a CRAY T3D but, for example, a 4 processor Cray
YMP or C90)
configure -device=ch_p4 -arch=CRAY
- Cray t3d
(assuming you are logged into the YMP front end).
configure -device=t3d -arch=cray_t3dBecause the T3D is cross-compiled, you should make sure that your PATH variable contains the directory with the T3D tools. We have seen /mpp/bin and /opt/ctl/bin; check with your system administrator for the paths that you should have when building T3D applications. Make sure that this path is at the head of your path list.
- Intel Paragon
configure -device=ch_nx -arch=paragon
- IBM SP2
(using the high-performance switch for communication)
configure -device=ch_mpl -arch=rs6000To use this you must have POE installed. Note, however, that this works only for POE versions without a native MPI. If you do have IBM's native MPI, then you can try the following (this is untested)
cd include ln -s /usr/lpp/ppe.poe/include/mpproto.h mpproto.h cd .. configure -arch=rs6000 -device=ch_mpl -opt="-O3 -qarch=pwr2" \ -cc=cc -clinker=mpcc -f77=xlf -flinker=mpxlf \ -make=/usr/local/bin/gnumake -mpilibname=mpich makeThis is unsupported; if you have suggestions for improving these instructions, please let us know.
- Meiko CS-2
configure -device=ch_meiko -arch=meiko
- SGI multiprocessors
such as the Power Challenge and Origin 2000.
configure -device=ch_shmemIf you need to generate a particular version that corresponds to the -32, -n32, or -64 compiler/linker options on SGI, use the architectures IRIX32, IRIXN32, or IRIX64 respectively instead of SGI. Specifically, use the following for an R10000 SGI:
configure -arch=IRIX64 \ -cc="cc -64 -mips4 -r10000" \ -fc="f77 -64 -mips4 -r10000" \ -opt="-O2" \ -device=ch_shmem(The optimization level is optional; -O2 has worked for some users. Be careful of aggressive optimization, particularly in the mpid/ch_shmem code.)
configure -arch=IRIXN32 \ -cc="cc -n32 -mips4 -r10000" \ -fc="f77 -n32 -mips4 -r10000" \ -opt="-O2" \ -device=ch_shmem
configure -arch=IRIX32 \ -cc="cc -32" \ -fc="f77 -32" \ -opt="-O2" \ -device=ch_shmem
See the comments under SGI workstations for different 32 and 64 bit options.
For networks of workstations (can interoperate with other types of workstations),
- DEC Alpha
In order to get the full advantages of ANSI C, you may need to add
-cflags="-std". For strict ANSI C, use -cflags="-std1".
- IBM RS6000
In order to get the advantages of ANSI C, you may need to add
-cflags="-qlanglvl=ansi". The MPICH code uses __STDC__ to check
for the presence of ANSI C features; the IBM RS6000 compilers do not
define this by default.
configure -device=ch_p4Some SGI systems support both 32 and 64 bit pointers (addresses). MPICH uses the architecture IRIX to refer to 32 bit systems and IRIX64 for 64 bit systems. You can use -arch=IRIX or -arch=IRIX64 to force a particular system. If you need to generate a particular version that corresponds to the -32, -n32, or -64 compiler/linker options on SGI, use the architectures IRIX32, IRIXN32, or IRIX64 respectively instead of sgi.
- SGI multiprocessor
(such as an Onyx, Challenge, or Power Challenge),
using the shared memory for fast message-passing
configure -device=ch_p4 -comm=sharedUse -arch=IRIX to force 32 bit pointers and -arch=IRIX64 to force 64 bit pointers.
On the SGI Origin machines, it is better to let configure choose the architecture and to attach the flags to the compilers and linker as follows:
configure -cc='cc -64' -fc='f77 -64' ....
- Sun SunOS
including the mpe libraries (See
the Users Guide :
configure -device=ch_p4 -arch=sun4 -mpe
- Sun Solaris
configure -device=ch_p4 -arch=solaris -mpe
- DEC Alpha
configure -device=ch_p4 -arch=alpha
For a network of PC's running the FreeBSD version of Unix:
configure -device=ch_p4 -arch=freebsd
- Nexus device
For a machine using the Nexus device, change the device in the above
examples to ch_nexus. The Nexus device requires an ANSI
C compiler, because the Nexus header files use the function
prototypes. If the default compiler is does not support function prototypes
(assuming ansicc is your compiler), add -cc=ansicc to the
configure command line. Remember to point this at the correct compiler
for your message passing system (e.g., mpcc for the SP, or
icc for the Paragon).
You may need to set the environment variables NEXUS_INCLUDE_DIR and NEXUS_LIB_DIR to indicate the directories containing the Nexus include files and libraries respectively. Do this before running configure.
- HP HPUX
For a network of HP's, including the mpe library but leaving out of it
the MPE X graphics routines:
configure -device=ch_p4 -arch=hpux -mpe -no_mpegraphics
For a network of Fujitsu M780s running UXP/M, the following options
have been tested:
setenv FC frt configure -arch=UXPM -device=ch_p4 -fflags="-Oe,-Uep -Eml -Aabe" \ -mpe -mpedbg -prefix=/usr/local/mpi \ -tcldir=/usr/local -tkdir=/usr/local -wish=/usr/local/bin/wish
- NEC SX-4
For an NEC SX-4 shared-memory vector multiprocessor, use
configure -device-lfshmemto get the lock-free shared-memory device described in .
mpich can be run on a heterogeneous network of workstations of various kinds. For simple collections of workstations, the mpirun command can be used; more complex collections of heterogeneous machines require a p4 ``procgroup file'' (for the ch_p4 device) or a ``startup file'' (for the ch_nexus device). The format of these files is described in Section Special Considerations for Running on a Network of Workstations .
Up: Contents Next: Building a production mpich Previous: Documentation