[mvapich-discuss] Why Latency test reports lower bandwidth than Bandwidth test in the OSU benchmark?

[Hari Subramoni]

Hello,

The aim of the latency test (osu_latency) is to identify the "one-way"
latency, which is half of the round trip time. Thus, the latency tests are
carried out in a ping-pong fashion. The sender sends a message with a
certain data size to the receiver and waits for a reply from the receiver.
The receiver receives the message from the sender and sends back a reply
with the same data size. Many iterations of this ping-pong test are carried
out and average one-way latency numbers are obtained.

The objective of this bandwidth test is to determine the maximum sustained
date rate that can be achieved at the network level. Thus, the bandwidth
tests are carried out by having the sender sending out a fixed number of
back-to-back messages to the receiver so that the network pipe is fully
occupied and then waiting for a reply from the receiver. The receiver sends
the reply only after receiving all these messages. This process is repeated
for several iterations and the bandwidth is calculated based on the elapsed
time (from the time sender sends the first message until the time it
receives the reply back from the receiver) and the number of bytes sent by
the sender.

This is the primary reason for the difference seen in the bandwidth
obtained using the osu_bw and osu_latency. I hope this clarifies things.

Please refer to our benchmarks page for a complete description of all OSU
microbenchmarks.

http://mvapich.cse.ohio-state.edu/benchmarks/

Regards,
Hari.

On Tue, Oct 28, 2014 at 11:43 AM, Ibai Gurrutxaga <i.gurrutxaga at ehu.es>
wrote:

> Dear All,
>
> I am trying to understand the differences between the Latency and
> Bandwidth tests from the OSU benchmark. I read their description and
> their source code and I understand how they work.
>
> I expected that translating Latency results from seconds to MBps would
> show slightly lower results than results from Bandwidth tests, at least
> for medium and large message sizes. However, the bandwidth values
> derived from the latency tests are 3-5 times lower than the values from
> the Bandwidth test in my Infiniband network. These are much lower than I
> expected.
>
> I found similar results in the MVAPICH website's Performance section
> [1]. For example, for a message of 1K the latency is about 2.25 us
> (equivalent to 455 MBps) while the bandwidth test reports more than 2000
> MBps for the same message size.
>
> It is obvious that I am missing something, but I cannot guess what it
> is. It would be very helpful for me if anyone could point out why the
> Bandwidth test reports 3-5 times higher bandwidth than the Latency test.
>
> Thanks,
> Ibai Gurrutxaga.
>
> [1]
> http://mvapich.cse.ohio-state.edu/performance/pt_to_pt/#sandy-connectib
> _______________________________________________
> mvapich-discuss mailing list
> mvapich-discuss at cse.ohio-state.edu
> http://mailman.cse.ohio-state.edu/mailman/listinfo/mvapich-discuss
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cse.ohio-state.edu/pipermail/mvapich-discuss/attachments/20141031/1359b00b/attachment-0001.html>