[mvapich-discuss] recompile with -fPIC

Fri Aug 24 18:13:38 EDT 2012

Thanks Jonathan,

That worked .... I feel dumb for asking now. 

After re-compiling I was able to run several of the OSU benchmarks including osu_alltoall on 2 nodes of 8 processors so I can confirm that mvapich2-1.8 is functional. Also the Pstream lib in OpenFOAM linked to the .so with no issues so I was pleased to see that is working now too. 

When I tried to run a test OpenFOAM case decomposed over 20 processors, i.e. 2 nodes of 10 processors I got the following error.

=====(error)=======
[mpiexec at compute-0-9.local] Launch arguments: /usr/bin/ssh -x compute-0-10 "/users/p20558/apps/mvapich2-1.8-gcc/bin/hydra_pmi_proxy" --control-port compute-0-9:54875 --debug --rmk user --launcher ssh --demux poll --pgid 0 --retries 10 --proxy-id 1 
/users/p20558/apps/mvapich2-1.8-gcc/bin/hydra_pmi_proxy: error while loading shared libraries: libmpl.so.1: cannot open shared object file: No such file or directory
SHbuoyantPimpleFoam: symbol lookup error: /users/p20558/apps/mvapich2-1.8-gcc/lib/libmpich.so.3: undefined symbol: MPL_env2str
====================

I verified that libmpl.so.1 was in my LD_LIBRARY_PATH and I even used the -env option to specify both LD_LIBRARY_PATH & PATH variables ensuring that the path to libmpl.so.1 was included, again it didn't work. Next I tried to run the 20 processor job on one node & I received the error below:

=====(error)=======
[mpiexec at compute-0-9.local] Launch arguments: /users/p20558/apps/mvapich2-1.8-gcc/bin/hydra_pmi_proxy --control-port compute-0-9:55519 --debug --rmk user --launcher ssh --demux poll --pgid 0 --retries 10 --proxy-id 0 
SHbuoyantPimpleFoam: symbol lookup error: /users/p20558/apps/mvapich2-1.8-gcc/lib/libmpich.so.3: undefined symbol: MPL_env2str
====================

This time it skips the libmpl.so.1 part and goes straight to the "undefined symbol: MPL_env2str"

A " nm ~/apps/mvapich2-1.8-gcc/lib/libmpl.so.1 | grep MPL_env2str" confirms that MPL_env2str is in libmpl.so.1 so it seems both of these errors are related to the failure to find libmpl.so.1

I am wondering if I need to perform some additional environmental setup to make use of shared libraries, they're used all over the place in OpenFOAM.  I have attached to this email my -v log from mpiexec for the error listed above.

Please let me know if you see anything obviously wrong or if I can provide any additional information.

Thanks & Best Regards

Scott 

Scott Haynes
Senior Meteorologist
Wind Development
Iberdrola Renewables
1320 Pearl St., Suite 330, Boulder, CO, 80302
Telephone (303) 245-8156, Mobile (503) 841-8524, Fax (303) 245-8155
scott.haynes at iberdrolaren.com
  In the interests of the environment, please print only if necessary and recycle

-----Original Message-----
From: Jonathan Perkins [mailto:perkinjo at cse.ohio-state.edu] 
Sent: Friday, August 24, 2012 6:48 AM
To: Haynes, Scott
Cc: mvapich-discuss at cse.ohio-state.edu
Subject: Re: [mvapich-discuss] recompile with -fPIC

Hello Scott.

Try building mvapich2 with the following configure...
    ./configure --with-slurm=../slurm/usr/local/ --enable-shared

Then make sure OpenFOAM links against the libmpich.so instead of
libmpich.a.

On Thu, Aug 23, 2012 at 04:58:10PM -0700, Haynes, Scott wrote:
> Hello List,
> 
>  
> 
> I have been trying to build OpenFOAM 2.1 with mvapich2-1.8-r5609.  The
> error I am receiving is pasted below.  "
> 
>  
> 
> ===== (Error)======
> 
> /users/p20558/apps/binutils/bin/ld:
> /users/p20558/apps/mvapich2-1.8-gcc/lib/libmpich.a(setbotf.o):
> relocation R_X86_64_32 against `mpifcmb2_' can not be used when making a
> shared object; recompile with -fPIC
> 
> /users/p20558/apps/mvapich2-1.8-gcc/lib/libmpich.a: could not read
> symbols: Bad value
> 
> collect2: ld returned 1 exit status
> 
> =================
> 
>  
> 
> I am using gcc 4.6 to compile both mvapich2 & OpenFOAM with the -fPIC
> flag (position independent code). I think the error above is the linker
> telling me that mvapich2 is not PIC & I need to recompile libmpich.a  .
> I have tried rebuilding mvapich2 using several different configurations
> each time libmpich.a is not PIC & I have been unsuccessful changing
> that.  My best guess at a configure line is below:
> 
>  
> 
> ./configure --with-slurm=../slurm/usr/local/ --enable-shared=gcc
> CFLAGS=-fPIC CXXFLAGS=-fPIC CPPFLAGS=-fPIC
> 
>  
> 
> I assume it is possible to compile mvapich2 as PIC? Is there something
> obvious I could add or delete from the configure statement above?  Any
> other debugging information I could provide?
> 
>  
> 
> Thanks & Best Regards
> 
>  
> 
> Scott 
> 
>  
> 
>  
> 
> Scott Haynes
> Senior Meteorologist
> Wind Development
> Iberdrola Renewables
> 
> 1320 Pearl St., Suite 330, Boulder, CO, 80302
> Telephone (303) 245-8156, Mobile (503) 841-8524, Fax (303) 245-8155
> 
> scott.haynes at iberdrolaren.com
> P  In the interests of the environment, please print only if necessary
> and recycle
> 
>  
> 
>  
> 
> 
> This message is intended for the exclusive attention of the recipient(s) indicated.  Any information contained herein is strictly confidential and privileged.  If you are not the intended recipient, please notify us by return e-mail and delete this message from your computer system. Any unauthorized use, reproduction, alteration, filing or sending of this message and/or any attached files may lead to legal action being taken against the party(ies) responsible for said unauthorized use. Any opinion expressed herein is solely that of the author(s) and does not necessarily represent the opinion of the Company. The sender does not guarantee the integrity, speed or safety of this message, and does not accept responsibility for any possible damage arising from the interception, incorporation of viruses, or any other damage as a result of manipulation.
> 
> 

> _______________________________________________
> mvapich-discuss mailing list
> mvapich-discuss at cse.ohio-state.edu
> http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss

-- 
Jonathan Perkins
http://www.cse.ohio-state.edu/~perkinjo
-------------- next part --------------
A non-text attachment was scrubbed...
Name: out-1node20proc
Type: application/octet-stream
Size: 20677 bytes
Desc: out-1node20proc
Url : http://mail.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20120824/fa1cd16e/out-1node20proc-0001.obj