[mvapich-discuss] Why Latency test reports lower bandwidth than Bandwidth test in the OSU benchmark?

[Ibai Gurrutxaga]

Dear All,

I am trying to understand the differences between the Latency and
Bandwidth tests from the OSU benchmark. I read their description and
their source code and I understand how they work.

I expected that translating Latency results from seconds to MBps would
show slightly lower results than results from Bandwidth tests, at least
for medium and large message sizes. However, the bandwidth values
derived from the latency tests are 3-5 times lower than the values from
the Bandwidth test in my Infiniband network. These are much lower than I
expected.

I found similar results in the MVAPICH website's Performance section
[1]. For example, for a message of 1K the latency is about 2.25 us
(equivalent to 455 MBps) while the bandwidth test reports more than 2000
MBps for the same message size.

It is obvious that I am missing something, but I cannot guess what it
is. It would be very helpful for me if anyone could point out why the
Bandwidth test reports 3-5 times higher bandwidth than the Latency test.

Thanks,
Ibai Gurrutxaga.

[1] http://mvapich.cse.ohio-state.edu/performance/pt_to_pt/#sandy-connectib