[mvapich-discuss] [cuda_stage_free] cudaMemcpy failed with 11 at 1564

[sreeram potluri]

Hi,

This issue has been resolved and is available as part of our nightly
tarballs for MVAPICH2 1.9 trunk. Let us know if this helps.

http://mvapich.cse.ohio-state.edu/nightly/mvapich2/trunk/

Best
Sreeram Potluri

On Tue, Nov 6, 2012 at 7:32 AM, Maxim Milakov <mmilakov at nvidia.com> wrote:

> Hi all,****
>
> ** **
>
> I am building and running code with MVAPICH 1.9a. The code is built with
> no CUDA support – it is not using CUDA at all.****
>
> ** **
>
> Still MVAPICH2 is configured with CUDA support:****
>
> ** **
>
> CPPFLAGS="-D x86_64 -D__align__\(n\)=__attribute__\(\(aligned\(n\)\)\)
> -D__location__\(a\)=__annotate__\(a\) -DCUDARTAPI=" F77=pgf77 FC=pgf90
> CC=pgcc CXX=pgcpp ./configure --prefix=/usr/local/pgi_specific/mvapich2
> --enable-f77 --enable-fc --enable-cxx --enable-cuda
> --with-cuda=$CUDA_INSTALL_PATH --disable-mcast****
>
> ** **
>
> Whenever I run the code with no specific options I get no errors, the code
> runs fine, showing significant performance improvement on configurations
> with IB (in comparison with MPICH).****
>
> ** **
>
> If I set MV2_USE_CUDA=1 (I tried it as the next step would be to run the
> version of the code ported to GPU) then the code aborts with the following
> error message:****
>
> ** **
>
> [wm040:mpi_rank_0][cuda_stage_free] cudaMemcpy failed with 11 at 1564****
>
> [wm040:mpi_rank_1][cuda_stage_free] cudaMemcpy failed with 11 at 1564****
>
> ** **
>
> It seems that the MVAPICH2 falsely detects that the pointer passed to
> mpi_send is referencing GPU memory.****
>
> ** **
>
> I am using CUDA 5.0, still I experience the error when I use CUDA 4.2
> (with new driver from installed with CUDA 5.0).****
>
> The error occurs even if the code runs just a single rank on each node.***
> *
>
> ** **
>
> I found similar issue here, with no resolution:
> http://mail.cse.ohio-state.edu/pipermail/mvapich-discuss/2012-August/004014.html
> ****
>
> ** **
>
> Thank you.****
>
> ** **
>
> ** **
>
> Best regards,****
>
> Maxim Milakov****
>
> ** **
>  ------------------------------
>  This email message is for the sole use of the intended recipient(s) and
> may contain confidential information.  Any unauthorized review, use,
> disclosure or distribution is prohibited.  If you are not the intended
> recipient, please contact the sender by reply email and destroy all copies
> of the original message.
>  ------------------------------
>
>
> _______________________________________________
> mvapich-discuss mailing list
> mvapich-discuss at cse.ohio-state.edu
> http://mail.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20121106/6201ef4b/attachment-0001.html