InfiniBand tests (version 2.0.6)

Test

The goal of the tests is to give the users overall impression of the performance characteristics of ØMQ/2.0.6 in terms of latency and throughput. Also, it can be thought of as a check to ensure that the new version of the software haven't lost the performance levels offered by the preceding versions.

The results were contributed by Michael Santy.

Environment

  • ØMQ/2.0.6
  • CPU: Intel Core i7 940 @ 2.93 GHz
  • Memory: 12GB DDR3 @ 1333 MHz
  • IB HCA: Mellanox MT25204 — InfiniHost III Lx HCA
  • IB Switch: Mellanox MT47396 Infiniscale III
  • OS: Centos 5.4 x86_64
  • OFED: 1.4.1
  • libSDP: 1.1.99

Results

message size latency [us] throughput [msgs/s] throughput [Mb/s]
1 B 33.566 7305826 58.447
2 B 33.903 6401719 102.428
4 B 33.877 6745770 215.865
8 B 34.573 6884214 440.590
16 B 34.064 6295711 805.851
32 B 35.218 4677759 1197.506
64 B 35.736 4767554 2440.988
128 B 35.775 3885802 3979.061
256 B 35.994 2689235 5507.553
512 B 35.932 1598083 6545.748
1 kB 36.836 867274 7104.709
2 kB 47.187 407486 6676.251
4 kB 44.569 221717 7265.223
8 kB 54.324 110846 7264.403
16 kB 79.018 54030 7081.820
32 kB 93.768 33698 8833.729
64 kB 153.736 16934 8878.293
128 kB 194.159 8611 9029.288
256 kB 312.330 4377 9179.234
512 kB 518.899 2184 9160.360
1 MB 1130.965 1100 9227.469
2 MB 2083.748 544 9126.806
4 MB 3747.207 269 9026.142
8 MB 7212.642 135 9059.697
16 MB 13607.344 67 8992.588
32 MB 27424.148 33 8858.370
64 MB 55334.758 17 9126.806
128 MB 113366.031 8 8589.935
256 MB 222521.812 4 8589.935
512 MB 444433.406 2 8589.935

Latency

Latency depending on message length:

ib1.png

The same graph with in log-log represntation so that latencies for small messages are more visible:

ib2.png

Throughput

Throughput depending on message length in terms of messages per second:

ib3.png

Throughput depending on message length in terms of bits per second:

ib4.png