[mvapich-discuss] Profiling of osu_mbw_mr test

[Maya Khaliullina]

Hello,

We develop model of concurrent communications for Infiniband network of our
HPC-cluster:
Node: 2xQuad Core Intel Xeon 2.33 GHz
O/S: RHEL4.5
File System: GPFS

To investigate behaviour of infiniband  interconnect we used profiling of
osu_mbw_mr test from OMB with Allinea Optimization & Profiling Tool (OPT).

Version of MVAPICH is 2-1.2.

We reduced number of iterations in osu_mbw_mr test to 5 and used only 2 MB
messages.

We found that there are 2 main variants of communication behavior resulting
in different summary bandwidth:

1) (pic.1) In the first case we see that one pair of communicating processes
(2 and 6) works faster than others and finishes earlier. Corresponding
bandwidth ~ 950 MB/sec.

2) (pic.2) In the second case all pairs work similarly with summary
bandwidth ~ 960 MB/sec.

Could you please explain the reason why we have such cases?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20090416/e94d90ce/attachment.html