[mvapich-discuss] Re: errors building mvapich2-0.9.8p1 on ppc64 [MVAPICH2/MVAPICH ROMIO Support]

[Shaun Rowland]

Kyle Schochenmaier wrote:
> My goal is to build mvapich2 support for both the openfabrics stack and 
> romio with pvfs2 support, and I've run into problems building it on ppc64.
> I was able to build at least the ofa support on an amd64 system, so I 
> would think this isnt related to our openfabrics release.. but no luck 
> yet on the ppc64.
> 
> I'm using the following configure line to try to build on my setup:
> OPEN_IB_HOME=/usr/local/ LIBS="-lpvfs2" LDFLAGS="-m64 
> -L/usr/src/pvfs2-4.21.07/Bppc64/lib" CFLAGS="-m64 
> -I/usr/src/pvfs2-4.21.07/Bppc64/include" ./make.mvapich2.ofa 
> --enable-romio --with-file-system=pvfs2
> 
> First off, is this the correct way to build pvfs2 support?  The README's 
> describe how to build pvfs support manually, this is of course pvfs2.

This issue was solved off of the mvapich-discuss mailing list. I am
responding on the list here specifically so that the solution is
archived in the list and in case that others might find it useful.

The make.mvapich.ofa script gives the user the ability to override many
of the environment variables it uses. The following environment
variables are the general candidates for setting before running the script:

- features (ROMIO, SHARED_LIBS, MUTLI_THREAD, ENABLE_CKPT)

- compilers (CC, CXX, F77, F90)

- locations (OPEN_IB_HOME, PREFIX)

- misc (ARCH, OPT_FLAG)

The other variables, such as CFLAGS, LIBS, etc. are more troublesome.
These contain specific CFLAGS and library settings required for
MVAPICH2. To override those, you need to know exactly what would be put
in them if the script were executed at this point, but it is possible to
override them if you know exactly what should be there. I do not suggest
doing this in the general case. The make.mvapich.ofa script also does
not take additional configure arguments on its command line.

ROMIO support can be enabled by setting ROMIO=yes in the environment
prior to running the script. As with all environment variables, this can
be directly set in the script at the proper place as well if desired.
However, this only enables the default filesystems (nfs+ufs). To build
with more filesystems at this time, it will be necessary to do the
following:

- Add the proper settings to the make.mvapich2.ofa script variables.
- Add the proper configure argument to the point in the
make.mvapich2.ofs script where configure is called.

An example of what variables might need to be changed is shown in the
top-level directory README.romio file. They depend on the filesystem
desired and how that software was installed. These settings can be added
to the corresponding variables in the make.mvapich2.ofa script. The
argument that needs to be added to configure is of the form similar to:

--with-file-system=pvfs2+nfs+ufs

The above specifies the default values plus pvfs2. The top-level
configure script does not have help output about this option, but it is
used in the configure located in the src/mpi/romio directory. Support
for the following filesystems should be there:

nfs
ufs
pfs
pvfs
pvfs2
testfs
xfs
panfs
gridftp

This configure option is mentioned in the README.romio file as well.
Much of this applies to MVAPICH and the other make scripts we provide as
well. In the MVAPICH case, the following differences apply:

- The configure argument is -file_system, using the same syntax.

- The details about ROMIO are outlined in the romio/README file.

- If you build with a Fortran 90 compiler and F90 is set, ROMIO will be
enabled by default. If not, the --with-romio option must also be added
to the configure call in the MVAPICH make script.

- Not all of the MVAPICH2 variables I outlined above apply to MVAPICH.
See the make script to determine what can be set exactly.

Any future changes to this, in order to make it easier to deal with,
will be added to the User Guide as we make new releases, as well as
being present in the script if looked at directly.
-- 
Shaun Rowland	rowland at cse.ohio-state.edu
http://www.cse.ohio-state.edu/~rowland/