[mvapich-discuss] Segmentation fault while running application

[Chaitra Kumar]

Hi Hari,

Thanks for the quick reply.
 I am using tuned MPI implementation available on Graph500 site (
http://www.graph500.org/referencecode).  I haven't modified this code.

Only for some experiments it throws segmentation fault, other experiments
complete without any errors. For eg.,

The following command fails with segmentation fault:

mpiexec -f hostfile -np 50  -env MV2_DEBUG_CORESIZE=unlimited -env
MV2_DEBUG_SHOW_BACKTRACE=1  -env MV2_ENABLE_AFFINITY=0
./graph500_mpi_custom_50 *29*

whereas if i change the scale to 28 or 30 the same code works without any
error:

mpiexec -f hostfile -np 50  -env MV2_DEBUG_CORESIZE=unlimited -env
MV2_DEBUG_SHOW_BACKTRACE=1  -env MV2_ENABLE_AFFINITY=0
./graph500_mpi_custom_50 *30* .

Please let me know if you want me to run with some other options.

Regards,
Chaitra


On Wed, Aug 6, 2014 at 12:20 AM, Hari Subramoni <subramoni.1 at osu.edu> wrote:

> Hello Chaitra,
>
> From the backtrace it seems that the failure (possibly a double free) is
> happening in the application code. You mentioned that the Graph500 is a
> tuned version. Does this mean that you have made local code changes to it?
> If there are, could you try using an unmodified version of Graph500 and see
> if the same failure happens?
>
> Regards,
> Hari.
>
>
> On Tue, Aug 5, 2014 at 1:57 PM, Chaitra Kumar <chaitragkumar at gmail.com>
> wrote:
>
>> Hi Team,
>>
>> I am trying to run Graph500 (tuned mpi version) on MVAPICH2-2.0. The
>> machine has infiniband.
>>
>> I am using following configuration to build MVAPICH2 (have enabled
>> debugging options):
>>
>> ./configure --prefix=/home/padmanac/mvapich2 --enable-cxx
>> --enable-threads=multiple --with-device=ch3:mrail --with-rdma=gen2
>> --disable-fast --enable-g=all --enable-error-messages=all
>>
>> The command which I am using is to launch Graph500 is:
>> mpiexec -f hostfile -np 50  -env MV2_DEBUG_CORESIZE=unlimited -env
>> MV2_DEBUG_SHOW_BACKTRACE=1  -env MV2_ENABLE_AFFINITY=0
>> ./graph500_mpi_custom_50 29
>>
>> This command always result in segmentation fault. From the coredump i
>> have got the backtrace.
>>
>> Please find the trace below:
>>
>> (gdb) bt
>> #0  0x00007f38f84b52dc in _int_free ()
>>    from /home/padmanac/mvapich2/lib/libmpich.so.12
>> #1  0x00007f38f84b3a96 in free ()
>>    from /home/padmanac/mvapich2/lib/libmpich.so.12
>> #2  0x00000000004054e5 in xfree ()
>> #3  0x000000000040ca10 in bitmap::clear() () at bitmap.cpp:54
>> #4  0x0000000000415cda in run_bfs(long, long*, bfs_settings const&) ()
>>     at bfs_custom.cpp:2036
>> #5  0x0000000000403a5d in main () at main.cpp:370
>>
>>
>> Please let me know how to solve this? Am I missing some configuration
>> flag?
>>
>> Thanks in advance.
>>
>> Regards,
>> Chaitra
>>
>> _______________________________________________
>> mvapich-discuss mailing list
>> mvapich-discuss at cse.ohio-state.edu
>> http://mailman.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20140806/2a72c888/attachment.html>