[mvapich-discuss] Lot of errors when running with cuda-memcheck

[Nuno Cardoso]

Hi,

I'm getting a lot of errors:
========= Program hit CUDA_ERROR_INVALID_VALUE (error 1) due to "invalid
argument" on CUDA API call to cuPointerGetAttribute.
=========     Saved host backtrace up to driver entry point at error
=========     Host Frame:/usr/lib64/nvidia/libcuda.so.1
(cuPointerGetAttribute + 0x16c) [0x148a0c]
=========     Host Frame:/home/ncardoso/.mpi/lib/libmpi.so.12
(is_device_buffer + 0x6f) [0x38c70f]
=========     Host Frame:/home/ncardoso/.mpi/lib/libmpi.so.12
(MPIR_Localcopy + 0x8b8) [0x2d9c58]
=========     Host Frame:/home/ncardoso/.mpi/lib/libmpi.so.12
(MPIR_Allreduce_reduce_shmem_MV2 + 0x35c) [0xcb4ac]
=========     Host Frame:/home/ncardoso/.mpi/lib/libmpi.so.12
(MPIR_Allreduce_two_level_MV2 + 0x2d8) [0xce808]
=========     Host Frame:/home/ncardoso/.mpi/lib/libmpi.so.12
(MPIR_Allreduce_index_tuned_intra_MV2 + 0x916) [0xcf316]
=========     Host Frame:/home/ncardoso/.mpi/lib/libmpi.so.12
(MPIR_Allreduce_impl + 0x24) [0x6c704]
=========     Host Frame:/home/ncardoso/.mpi/lib/libmpi.so.12
(PMPI_Allreduce + 0x54e) [0x6ccfe]

This happens if the env MV2_USE_CUDA is active and without using device
pointers in MPI functions.
This is bad in the sense that I cannot use cuda-memcheck if a make my
program  mpi cuda-aware.

There is a solution to avoid this?


Best regards,
Nuno Cardoso
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20150422/c0d38dd7/attachment.html>