[mvapich-discuss] How to compile install MV2-gdr from MV2 source code?

[Jonathan Perkins]

Hello Dun.  It is not possible to build with GDR support from the MV2
tarball at this time.

Instead, I suggest that you use a tool like alien to convert the rpm to a
deb and install that way.

If you do not have root permission you can use rpm2cpio to extract the
package:
    rpm2cpio mvapich2-gdr-cuda6.5-gnu-2.1a-8.el6.x86_64.rpm | cpio -id

When using the rpm2cpio method, you will need to update the mpi compiler
scripts, such as mpicc, in order to point to the correct path of where you
place the package.

On Fri, Apr 24, 2015 at 3:03 AM Dun Liang <randonlang at gmail.com> wrote:

> Hi, developers
>
> I use debian, so I have to compile install MV2-gdr
> I compile install MV2 with below config:
>
> Configuration
> --prefix=/home/liangdun/mvapich/build --enable-cuda --disable-mcast --w
> ith-cuda=/usr/local/cuda --with-device=ch3:mrail --enable-rdma-cm
>
> and Plugin module to enable GPUDirect RDMA , gdrcopy are installed also,
> but I do not achieve the prformance showed in
> http://mvapich.cse.ohio-state.edu/performance/gdr-pt_to_pt/
>
> So, my question is:
> How to compile install MV2-gdr from MV2 source code?
>
> here is my test result, Looks like problematic
>
> # OSU MPI-CUDA Latency Test
> # Send Buffer on DEVICE (D) and Receive Buffer on DEVICE (D)
> # Size            Latency (us)
> 0                         1.29
> 1                      1290.74
> 2                      1207.41
> 4                      1212.52
> 8                      1216.27
>
> here is the env config:
>
> export MV2_PATH=/home/liangdun/mvapich/build export
> MV2_GPUDIRECT_GDRCOPY_LIB=/home/liangdun/mvapich/tool/gdrcopy_build/ export
> MV2_USE_GPUDIRECT_GDRCOPY=1 export MV2_USE_CUDA=1 $MV2_PATH/bin/mpirun_rsh
> -export -np 2 debian83 debian81 \ $MV2_PATH/libexec/mvapich2/get_local_rank
> \ $MV2_PATH/libexec/mvapich2/osu_latency D D
>
> _______________________________________________
> mvapich-discuss mailing list
> mvapich-discuss at cse.ohio-state.edu
> http://mailman.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20150424/be6937da/attachment.html>