[mvapich-discuss] CMA fails if executable's read permission bit is disabled

[Adam T. Moody]

Thanks, Hari.  It does run if we set MV2_SMP_USE_CMA=0.
-Adam

On 11/10/2016 01:46 PM, Hari Subramoni wrote:
> Hi Adam,
>
> Thanks for the report. We will look into it and get back to you. You can
> disable CMA at runtime by setting MV2_SMP_USE_CMA=0. Can you please try
> this while we work on a patch?
>
> Regards,
> Hari.
>
> On Thu, Nov 10, 2016 at 4:42 PM, Adam T. Moody <moody20 at llnl.gov> wrote:
>
>> Hello MVAPICH team,
>> We have an application that disables its read permission bit (i.e., chmod
>> -r).  If we run this app with MV2-2.2 on gen2 using at least two procs on
>> one node, we get the following error:
>>
>> [MPIDI_CH3I_SMP_init] CMA is not available. Set MV2_SMP_USE_CMA=0 to
>> disable CMA.
>> Fatal error in MPI_Init:
>> Other MPI error, error stack:
>> MPIR_Init_thread(514)....:
>> MPID_Init(365)...........: channel initialization failed
>> MPIDI_CH3_Init(404)......:
>> MPIDI_CH3I_SMP_Init(2132): process_vm_readv: Operation not permitted
>>
>> If we enable the read bit (chmod +r) it works.
>>
>> It seems that something goes wrong when read permission is disabled when
>> using process_vm_writev() from check_cma_usability() in
>> src/mpid/ch3/channels/mrail/src/rdma/ch3_smp_progress.c.
>>
>> This should reproduce with any simple MPI job.
>>
>> Can we get CMA to work when disabling the read bit like this? That's an
>> important requirement for this application.
>>
>> Otherwise, could we patch MV2 so that it dynamically disables CMA instead
>> of failing with the error?
>> Thanks,
>> -Adam
>>
>> _______________________________________________
>> mvapich-discuss mailing list
>> mvapich-discuss at cse.ohio-state.edu
>> http://mailman.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
>>