[mvapich-discuss] Re: install mvapich when there are two gcc in the system

[zhangyigang]

Dear Pavel Shamis: 

Thank you so much for your response. I guess I have both openibd (not sure
the name) and opensm running (I have followed closely the manual), but the
port refuse to become active if I do not turn off/on the switch. I will
have a look at the opensm log file and see what I can find there. 

I do not know what FAE means. I have asked the person who sold us the
cluster, then he asked the person who sold him the card. Finally the person
who sold the card asked someone from Mellanox in Beijing, China. Maybe they
will give a solution, but for the moment not yet. 

For the gcc, I installed them by configure and make. I am not sure if I
should keep both gcc as I was told that the operating system red hat linux
is compiled with the old gcc so that some libraries compiled with the old
gcc can still work. For the new gcc, it may be hard to remove it because
there is no uninstall and the files installed are all over many places. 

I have one more question. For installing mvapich-1.0, the make.mvapich.gen2
asks the parameter IBHOME. So I give it /usr/ofed. This the only directory
that looks like the one given by make.mvapich.gen2. The thing that puzzles
me is that when I enter the directory there is nothing in the directory
except a bunch of subdirectories (i.e., no specific files). Does this mean
I did something wrong when installing ofed? When installing ofed, I chose
install everything. What is the name of the file that mvapich need from
ofed? Otherwise what is the IBHOME parameter used for? The thing that
puzzles me is that with the old gcc, everything seems working with the
empty directory. I start to worry if this hurt the performance. Is the
empty directory normal? 

Thanks again for your response. 

yigang zhang 


Pavel Shamis (Pasha) д: 


Thanks for your response. I used ifort for fortran compiler and gcc and g++
for c and c++ compiler. I used make.mvapich.gen2, and changed the
parameters at lines starting with "export". I do not know how to specify a
particular gcc, so I just 'cp gcc gcc-old' and 'rm gcc'. When I use 'which
gcc', it gives me the one that I have not changed name and is the one that
I want to use. Before I installed the new gcc4.2, everything works just
fine. After the install of gcc 4.2,I tried with both the old and new gcc,
and none of them work now. The gcc themselves and ifort seem to work when I
type 'gcc'or 'ifort', but both mpicc and mpif90 do not work anymore and
give me the undefined reference error.

I do not see reason to change the make.mvapich.gen2.
You need:
export CC=/full_path_to_compiler/gcc
export CXX=/full_path_to_compiler/g++
export F77=/full_path_to_compiler/gfortran
export F90=/full_path_to_compiler/gfortran
./make.mvapich.gen2 

Maybe the new installment of gcc 4.2 get some libarary mixed up. I do not
know what to do.

The MIX may be a problem. If you installed it from rpm it should be easy
to fix.
rpm -qa | grep gcc
Find the version that you want to remove, and uninstall it:
rpm -e gcc_that_you_want_to_uninstall 

By the way, as you are from Mellanox, My infinibad switch (infiniscale) and
HCA cards are from your company. The problem is sometimes when I reboot my
computers the state of the cards becomes PORT_INIT(2).
The subnet manager should bring up the port to ACTIVE state.
There is no way to make them active except to switch off the infiniscale
switch and and then switch on all the computers. Do you know a better way?

I guess that you have the subnet manager running on the switches and
from some reason it fails to bring up new ports. You may run additional
subnet manager - OpenSM on one of your machines /etc/init.d/opensm
start. Or maybe we already have Opensm running on the cluster but from
some reason it fails to ACTIVATE new port. In this case you should check
the opensm log file. 

BTW who is FAE that supports you ? 

Thanks,
Pasha
 

 ----- Original Message -----
From: "Pavel Shamis (Pasha)" <pasha at dev.mellanox.co.il>
To: "zhang yigang" <zhangyg at mail.iggcas.ac.cn>
Cc: <mvapich-discuss at cse.ohio-state.edu>
Sent: Wednesday, March 26, 2008 4:09 PM
Subject: Re: [mvapich-discuss] install mvapich when there are two gcc in
the system 



I had to use ATLAS for my calculation code VASP. The new ATLAS need to
be compiled using gcc 4.2. So I install gcc4.2. Everything seems fine.
I can use the new ATLAS in VASP now using the old mvapich-1.0 that I
compiled without the gcc4.2. However, when I deleted the old mvapich
that I compiled when there was no gcc 4.2 in my system and recompiled
mvapich-1.0, the compiling seems OK, but when I run mpicc or mpif90,
it always say:
/opt/intel/fce/10.1.012/lib/for_mai.o(.text+0x26): In function
'main'::undefined reference to 'MAIN__"

Actually "/opt/intel/fce" shows that you some how compiled your stuff
with intel compiler?
Before running mvapich compilation script you should define
CC/CXX/F77/F90 env. variable that will point exactly on compiler that
you want to use. 


-- 
Pavel Shamis (Pasha)
Mellanox Technologies
 

-- 
Pavel Shamis (Pasha)
Mellanox Technologies