[Hadoop-RDMA] Announcing the Release of Hadoop-RDMA 0.9 package
Dhabaleswar Panda
panda at cse.ohio-state.edu
Fri Jun 14 01:26:04 EDT 2013
The Hadoop-RDMA team is pleased to announce the release of Hadoop-RDMA 0.9
package with the following features.
* Hadoop-RDMA 0.9 Features
- Based on Apache Hadoop 0.20.2
- High performance design with native InfiniBand
support at the verbs-level for HDFS, MapReduce
and RPC components
- Compliant with Apache Hadoop 0.20.2 APIs and applications
- Easily configurable for both native InfiniBand and the traditional
sockets-based support (Ethernet and InfiniBand with IPoIB)
- HDFS over native InfiniBand
- RDMA-based write
- RDMA-based replication
- Tuned packet size and optimized memory footprint
- MapReduce over native InfiniBand
- RDMA-based shuffle
- Prefetching and caching of map output
- In-memory merge
- Overlapping of merge and reduce
- RPC over native InfiniBand
- JVM-bypassed buffer management
- On-demand InfiniBand connection setup
- RDMA or send/recv based adaptive communication
- Tested with
- Mellanox InfiniBand adapters (DDR, QDR and FDR)
- Various multi-core platforms
- Different file systems with Hard-disks and SSDs
For downloading Hadoop-RDMA 0.9 package and the associated user guide,
please visit the following URL:
http://hadoop-rdma.cse.ohio-state.edu
Sample performance numbers for benchmarks using Hadoop-RDMA 0.9
version can be viewed by visiting the `Performance' tab of the above
website.
All questions, feedbacks and bug reports are welcome. Please post it
to the hadoop-rdma-discuss mailing list
(hadoop-rdma-discuss at cse.ohio-state.edu).
Thanks,
The Hadoop-RDMA Team
More information about the hibd-announce
mailing list