[mvapich-discuss] MPI_Cart_Create
rcbord at wm.edu
rcbord at wm.edu
Wed May 14 12:20:10 EDT 2008
Hi Amith,
I could not make the tcp version of mvapich-0.9.9 using gnu or pgi.
I was able to compile the tcp version of mvapich1 with gnu but
it did not build the mpif77 or mpif90. So I will skip the mvapich-0.9.9
testing for now.
Chris Bording
Application Analyst
High Performance Computing Group
Information Technology
The College of William and Mary
(757)-221-3488
rcbord at wm.edu
On Wed, 14 May 2008 rcbord at wm.edu wrote:
> Hi Amith,
> The make.mvapich.tcp script fails
> Changed the F77 value to gfortran as we don't have g77 on the system.
> I changed to the configure flags to:
> ./configure --prefix=$PREFIX --with-device=ch_p4 --with-arch=LINUX
> --without-mpe
>
> Here is the output of make.mvapich.tcp where it fails. Am I using the
> right device and arch values?
>
>
>
> make overtake
> /usr/mpi/mvapich-0.9.9/bin/mpicc -O2 -DUSE_SOCKLEN_T -DUSE_U_INT_FOR_XDR
> -DHAVE_MPICHCONF_H -DHAVE_STDLIB_H=1 -DHAVE_UNISTD_H=1 -DHAVE_STRING_H=1
> -DUSE_STDARG=1 -DHAVE_LONG_DOUBLE=1 -DHAVE_LONG_LONG_INT=1
> -DHAVE_PROTOTYPES=1 -DHAVE_SIGNAL_H=1 -DHAVE_SIGACTION=1 -DHAVE_SLEEP=1
> -DHAVE_SYSCONF=1 -c overtake.c
> /usr/mpi/mvapich-0.9.9/bin/mpicc -O2 -DUSE_SOCKLEN_T -DUSE_U_INT_FOR_XDR
> -DHAVE_MPICHCONF_H -DHAVE_STDLIB_H=1 -DHAVE_UNISTD_H=1 -DHAVE_STRING_H=1
> -DUSE_STDARG=1 -DHAVE_LONG_DOUBLE=1 -DHAVE_LONG_LONG_INT=1
> -DHAVE_PROTOTYPES=1 -DHAVE_SIGNAL_H=1 -DHAVE_SIGACTION=1 -DHAVE_SLEEP=1
> -DHAVE_SYSCONF=1 -c test.c
> /usr/mpi/mvapich-0.9.9/bin/mpicc -o overtake overtake.o test.o
> make clean
> make --no-print-directory mpi-addons
> make --no-print-directory mpelib
> make --no-print-directory mpi-utils
> gcc -O2 -DUSE_SOCKLEN_T -DUSE_U_INT_FOR_XDR -c
> /usr/mpi/mvapich-0.9.9/mpid/server/serv_p4.c -I./ -o serv_p4.o
> gcc -O2 -DUSE_SOCKLEN_T -DUSE_U_INT_FOR_XDR -c -I./ server_ssl.c
> gcc -o serv_p4 serv_p4.o server_ssl.o -lcrypt
> cp: cannot stat `/usr/mpi/mvapich-0.9.9/bin/tarch': No such file or directory
> cp: cannot stat `/usr/mpi/mvapich-0.9.9/bin/tdevice': No such file or
> directory
> make[1]: *** [mpi-utils] Error 1
> make: *** [mpi] Error 2
> Failure in Building MVAPICH
> Please file an error report to mvapich-discuss at cse.ohio-state.edu with all
> your log files.
>
>
>
>
> Chris Bording
> Application Analyst
> High Performance Computing Group Information Technology
> The College of William and Mary
> (757)-221-3488
> rcbord at wm.edu
>
> On Wed, 14 May 2008, amith rajith mamidala wrote:
>
>>
>> Hi Chris,
>>
>> For using tcp, you can try it out with gnu first.
>>
>> Thanks,
>> Amith
>>
>> On Wed, 14 May 2008 rcbord at wm.edu wrote:
>>
>>> Amith,
>>> Should I use the default gnu compilers or do you want me
>>> to build with pgi?
>>>
>>> Chris Bording
>>> Application Analyst
>>> High Performance Computing Group
>>> Information Technology
>>> The College of William and Mary
>>> (757)-221-3488
>>> rcbord at wm.edu
>>>
>>> On Wed, 14 May 2008, amith rajith mamidala wrote:
>>>
>>>> Hi Chris,
>>>>
>>>> Can you try out mvapich1 with tcp? You can use make.mvapich.tcp to
>>>> compile. This way we will narrow down the code path causing the
>>>> problem.
>>>>
>>>> Thanks,
>>>> Amith
>>>>
>>>> On Wed, 14 May 2008 rcbord at wm.edu wrote:
>>>>
>>>>> Amith,
>>>>> I was able to test mvapich1 but I get the same bad results
>>>>> MPI_Cart_Create is returning two communication values.
>>>>>
>>>>> When I add the environment variable VIADEV_USE_SHMEM_COLL=0
>>>>> and the MPI_sendrecv function changes the dest and source values
>>>>> that are set by the mpi_cart_shift function.
>>>>> Note these errors only occur with F90. I have C/C++ codes
>>>>> that work correctly.
>>>>>
>>>>> I can re-compile any or all mvapich-0.9.9, mvapich1 and mvapich2
>>>>> easily enough using the make.mvapich.gen2 scripts. What flags
>>>>> should I be using for the pgi-7.0 compiler? I can update the
>>>>> compiler too if necessary.
>>>>>
>>>>>
>>>>> Chris Bording
>>>>> Application Analyst
>>>>> High Performance Computing Group
>>>>> Information Technology
>>>>> The College of William and Mary
>>>>> (757)-221-3488
>>>>> rcbord at wm.edu
>>>>>
>>>>> On Wed, 7 May 2008, amith rajith mamidala wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Can you try these two options?
>>>>>>
>>>>>> 1. use MVAPICH-1.0 and see if you are seeing the issue.
>>>>>>
>>>>>> 2.Can you run the code by passing the environment variable:
>>>>>> VIADEV_USE_SHMEM_COLL=0 with mvapich
>>>>>>
>>>>>>
>>>>>> Thanks,
>>>>>> Amith
>>>>>>
>>>>>>
>>>>>> On Wed, 7 May 2008 rcbord at wm.edu wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>> I am having an issue with the MVAPICH-0.9.9 compiled with
>>>>>>> PGI-7.0 with
>>>>>>> ofed 1.2 for infinaband. We have be able to use it for 6 months
>>>>>>> without
>>>>>>> any problem. I tried to port a users fortran code that runs
>>>>>>> under
>>>>>>> solaris-sparc without any problems. The code is fairly vanilla
>>>>>>> fotran-90
>>>>>>> with mpi I know because I wrote it. When I tried to port it to
>>>>>>> the Linux
>>>>>>> Cluster it hangs a mpisendrecv function. I included a simple
>>>>>>> test code
>>>>>>> that shows that the new communicator (comm2d) returned by the
>>>>>>> MPI_Cart_create function has multiple values. Same code run on
>>>>>>> the
>>>>>>> sun sparc cluster returns a single value. I am guessing it
>>>>>>> could be how
>>>>>>> I compiled MVAPICH, but have tried a few more flags without any
>>>>>>> success.
>>>>>>> I tried this with MVAPICH2 also and got the same error, but I
>>>>>>> compiled it
>>>>>>> in a similar fashion. Could it be a PGI problem?
>>>>>>>
>>>>>>>
>>>>>>> Has anyone else seen this? Thanks for any help!
>>>>>>>
>>>>>>> Program mpicart
>>>>>>> ! With input files
>>>>>>> ! Conversion of efit2d.f90 to 3d JPB 11-2007
>>>>>>>
>>>>>>> Implicit none
>>>>>>>
>>>>>>> include "mpif.h"
>>>>>>>
>>>>>>> Interface
>>>>>>> Subroutine read_model(Pdim0,Pdim1)
>>>>>>> Integer, Intent(out) ::Pdim0,Pdim1
>>>>>>> end Subroutine read_model
>>>>>>> end Interface
>>>>>>>
>>>>>>> Double Precision :: t1,t2
>>>>>>> !
>>>>>>> ! define MPI variables
>>>>>>> !
>>>>>>> Integer :: Pid, N_proc, ierr
>>>>>>> Integer :: comm2d = 0
>>>>>>> Integer :: status
>>>>>>> Integer :: period(2),Pdim_size(2),coords(2)
>>>>>>> Integer :: nbrleft,nbrright,nbrtop,nbrbottom
>>>>>>> Integer :: Pdim0, Pdim1
>>>>>>> Integer :: i, t, numt
>>>>>>>
>>>>>>> call MPI_INIT(ierr)
>>>>>>> call MPI_COMM_RANK(MPI_COMM_WORLD,Pid,ierr)
>>>>>>> call MPI_COMM_SIZE(MPI_COMM_WORLD,N_proc,ierr)
>>>>>>>
>>>>>>> if (Pid .EQ. 0) then
>>>>>>> call read_model(Pdim0,Pdim1)
>>>>>>>
>>>>>>> end if
>>>>>>> !
>>>>>>> ! Broad cast input parameters values to all processors
>>>>>>> !
>>>>>>> call MPI_Bcast(Pdim0,1,MPI_INTEGER,0, &
>>>>>>> & MPI_COMM_WORLD,ierr)
>>>>>>> call MPI_Bcast(Pdim1,1,MPI_INTEGER,0, &
>>>>>>> & MPI_COMM_WORLD,ierr)
>>>>>>>
>>>>>>> call MPI_BARRIER(MPI_COMM_WORLD,ierr)
>>>>>>>
>>>>>>> Pdim_size(1) = Pdim0
>>>>>>> Pdim_size(2) = Pdim1
>>>>>>>
>>>>>>> period(1) = 0
>>>>>>> period(2) = 0
>>>>>>>
>>>>>>> call MPI_Dims_create(N_proc,2,Pdim_size,ierr)
>>>>>>>
>>>>>>> call
>>>>>>> MPI_Cart_create(MPI_COMM_WORLD,2,Pdim_size,period,.true.,comm2d,ierr)
>>>>>>> do i = 0,N_proc-1
>>>>>>> if (Pid == i) then
>>>>>>> write(*,*) 'pid ',Pid,' mpi_comm_2d ',comm2d
>>>>>>> end if
>>>>>>> end do
>>>>>>>
>>>>>>> call MPI_BARRIER(MPI_COMM_WORLD,ierr)
>>>>>>> call MPI_Comm_free(comm2d,ierr)
>>>>>>> call MPI_FINALIZE(ierr)
>>>>>>>
>>>>>>> end program mpicart
>>>>>>>
>>>>>>> ###################### OUTPUT #########################
>>>>>>>
>>>>>>> Model parameters for mpicart test
>>>>>>>
>>>>>>> Processor Topology is ( 3 by 4 )
>>>>>>>
>>>>>>> pid 0 mpi_comm_2d 140
>>>>>>> pid 11 mpi_comm_2d 138
>>>>>>> pid 2 mpi_comm_2d 140
>>>>>>> pid 4 mpi_comm_2d 140
>>>>>>> pid 6 mpi_comm_2d 140
>>>>>>> pid 8 mpi_comm_2d 140
>>>>>>> pid 3 mpi_comm_2d 138
>>>>>>> pid 10 mpi_comm_2d 140
>>>>>>> pid 1 mpi_comm_2d 138
>>>>>>> pid 5 mpi_comm_2d 138
>>>>>>> pid 7 mpi_comm_2d 138
>>>>>>> pid 9 mpi_comm_2d 138
>>>>>>>
>>>>>>>
>>>>>>> Chris Bording
>>>>>>> Application Analyst
>>>>>>> High Performance Computing Group
>>>>>>> Information Technology
>>>>>>> The College of William and Mary
>>>>>>> (757)-221-3488
>>>>>>> rcbord at wm.edu
>>>>>>> _______________________________________________
>>>>>>> mvapich-discuss mailing list
>>>>>>> mvapich-discuss at cse.ohio-state.edu
>>>>>>> http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>
>>
>>
> _______________________________________________
> mvapich-discuss mailing list
> mvapich-discuss at cse.ohio-state.edu
> http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
>
More information about the mvapich-discuss
mailing list