[HiBD] Announcing the release of RDMA for Apache Hadoop-2.x 0.9.5

[Panda, Dhabaleswar]

The High-Performance Big Data (HiBD) team is pleased to announce the
release of Hadoop-2.x 0.9.5 package (for Hadoop 2.x series) with the
following features.

* RDMA for Apache Hadoop-2.x 0.9.5 Features

    - Based on Apache Hadoop 2.5.0

    - High performance design with native InfiniBand and RoCE
      support at the verbs level for the HDFS, MapReduce,
      and RPC components

    - Compliant with Apache Hadoop 2.5.0 APIs and applications

    - Easily configurable for native InfiniBand, RoCE, and the
      traditional sockets based support (Ethernet and InfiniBand
      with IPoIB)

    - On-demand connection setup

    - HDFS over native InfiniBand and RoCE
      - RDMA-based write
      - RDMA-based replication
      - Parallel replication support
      - Optimizations for SSD

    - MapReduce over native InfiniBand and RoCE
      - RDMA-based shuffle
      - Pre-fetching and caching of map outputs
      - In-memory merge
      - Advanced optimization in overlapping
        - map, shuffle, and merge
        - shuffle, merge, and reduce
      - Optional disk-assisted shuffle

    - RPC over native InfiniBand and RoCE
      - JVM-bypassed buffer management
      - RDMA or send/recv based adaptive communication
      - Intelligent buffer allocation and adjustment for serialization

    - Tested with
      - Mellanox InfiniBand adapters (DDR, QDR and FDR)
      - RoCE support with Mellanox adapters
      - Various multi-core platforms
      - Different file systems with disks and SSDs

For downloading RDMA for Apache Hadoop-2.x 0.9.5 package and the
associated user guide, please visit the following URL:

http://hibd.cse.ohio-state.edu

Sample performance numbers for benchmarks using RDMA for Apache
Hadoop-2.x 0.9.5 version can be viewed by visiting the `Performance'
tab of the above website.

All questions, feedback and bug reports are welcome. Please post it
to the rdma-hadoop-discuss mailing list (rdma-hadoop-discuss at
cse.ohio-state.edu).

Thanks,

The High-Performance Big Data (HiBD) Team

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cse.ohio-state.edu/pipermail/hibd-announce/attachments/20141127/e3dc3419/attachment.html>