[Hadoop-RDMA-discuss] INFO mapred.LocalJobRunner: reduce > copy

[Suri Shelvapille]

I forgot to attach the output:

[root at lgfs10-ib rdma-hadoop-2.x-0.9.5]#  bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.5.0.jar wordcount  /home/hadoop/2.x-0.9.5/data01/tmp3 /home/hadoop/2.x-0.9.5/data01/tmp3-output
14/12/14 08:48:50 INFO Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id
14/12/14 08:48:50 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
14/12/14 08:48:50 INFO input.FileInputFormat: Total input paths to process : 1
14/12/14 08:48:50 INFO mapreduce.JobSubmitter: number of splits:1
14/12/14 08:48:50 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_local2061256206_0001
14/12/14 08:48:51 WARN conf.Configuration: file:/tmp/hadoop-root/mapred/staging/root2061256206/.staging/job_local2061256206_0001/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
14/12/14 08:48:51 WARN conf.Configuration: file:/tmp/hadoop-root/mapred/staging/root2061256206/.staging/job_local2061256206_0001/job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
14/12/14 08:48:51 WARN conf.Configuration: file:/tmp/hadoop-root/mapred/local/localRunner/root/job_local2061256206_0001/job_local2061256206_0001.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval;  Ignoring.
14/12/14 08:48:51 WARN conf.Configuration: file:/tmp/hadoop-root/mapred/local/localRunner/root/job_local2061256206_0001/job_local2061256206_0001.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts;  Ignoring.
14/12/14 08:48:51 INFO mapreduce.Job: The url to track the job: http://localhost:8080/
14/12/14 08:48:51 INFO mapreduce.Job: Running job: job_local2061256206_0001
14/12/14 08:48:51 INFO mapred.LocalJobRunner: OutputCommitter set in config null
14/12/14 08:48:51 INFO mapred.LocalJobRunner: OutputCommitter is org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter
14/12/14 08:48:51 INFO mapred.LocalJobRunner: Waiting for map tasks
14/12/14 08:48:51 INFO mapred.LocalJobRunner: Starting task: attempt_local2061256206_0001_m_000000_0
14/12/14 08:48:51 INFO mapred.Task:  Using ResourceCalculatorProcessTree : [ ]
14/12/14 08:48:51 INFO mapred.MapTask: Processing split: hdfs://10.10.16.122:9000/home/hadoop/2.x-0.9.5/data01/tmp3/pg20417.txt:0+674570
14/12/14 08:48:51 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer
14/12/14 08:48:51 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)
14/12/14 08:48:51 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100
14/12/14 08:48:51 INFO mapred.MapTask: soft limit at 83886080
14/12/14 08:48:51 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600
14/12/14 08:48:51 INFO mapred.MapTask: kvstart = 26214396; length = 6553600
14/12/14 08:48:51 INFO input.LineRecordReader: Found UTF-8 BOM and skipped it
14/12/14 08:48:51 INFO mapred.LocalJobRunner:
14/12/14 08:48:51 INFO mapred.MapTask: Starting flush of map output
14/12/14 08:48:51 INFO mapred.MapTask: Spilling map output
14/12/14 08:48:51 INFO mapred.MapTask: bufstart = 0; bufend = 1086544; bufvoid = 104857600
14/12/14 08:48:51 INFO mapred.MapTask: kvstart = 26214396(104857584); kvend = 25775024(103100096); length = 439373/6553600
14/12/14 08:48:51 INFO mapred.MapTask: Finished spill 0
14/12/14 08:48:51 INFO mapred.Task: Task:attempt_local2061256206_0001_m_000000_0 is done. And is in the process of committing
14/12/14 08:48:51 INFO mapred.LocalJobRunner: map
14/12/14 08:48:51 INFO mapred.Task: Task 'attempt_local2061256206_0001_m_000000_0' done.
14/12/14 08:48:51 INFO mapred.LocalJobRunner: Finishing task: attempt_local2061256206_0001_m_000000_0
14/12/14 08:48:51 INFO mapred.LocalJobRunner: map task executor complete.
14/12/14 08:48:51 INFO mapred.LocalJobRunner: Waiting for reduce tasks
14/12/14 08:48:51 INFO mapred.LocalJobRunner: Starting task: attempt_local2061256206_0001_r_000000_0
14/12/14 08:48:51 INFO mapred.Task:  Using ResourceCalculatorProcessTree : [ ]
14/12/14 08:48:51 INFO mapred.ReduceTask: Reducer thread started.
14/12/14 08:48:51 INFO reduce.HOMRMergeManagerImpl: MergerManager: memoryLimit=334338464, maxSingleShuffleLimit=83584616, mergeThreshold=220663392, ioSortFactor=10, memToMemMergeOutputsThreshold=10
14/12/14 08:48:51 INFO reduce.This: attempt_local2061256206_0001_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events
14/12/14 08:48:52 INFO mapreduce.Job: Job job_local2061256206_0001 running in uber mode : false
14/12/14 08:48:52 INFO mapreduce.Job:  map 100% reduce 0%
14/12/14 08:48:57 INFO mapred.LocalJobRunner: reduce > copy
14/12/14 08:49:00 INFO mapred.LocalJobRunner: reduce > copy
14/12/14 08:49:03 INFO mapred.LocalJobRunner: reduce > copy
14/12/14 08:49:06 INFO mapred.LocalJobRunner: reduce > copy
14/12/14 08:49:09 INFO mapred.LocalJobRunner: reduce > copy
14/12/14 08:49:12 INFO mapred.LocalJobRunner: reduce > copy
14/12/14 08:49:15 INFO mapred.LocalJobRunner: reduce > copy
14/12/14 08:49:18 INFO mapred.LocalJobRunner: reduce > copy
14/12/14 08:49:21 INFO mapred.LocalJobRunner: reduce > copy
14/12/14 08:49:24 INFO mapred.LocalJobRunner: reduce > copy

From: RDMA-Hadoop-discuss [mailto:rdma-hadoop-discuss-bounces at cse.ohio-state.edu] On Behalf Of Suri Shelvapille
Sent: Sunday, December 14, 2014 2:58 PM
To: rdma-hadoop-discuss at cse.ohio-state.edu
Subject: [Hadoop-RDMA-discuss] INFO mapred.LocalJobRunner: reduce > copy

Dear Folks:

I have installed rdma-hadoop version 2.x-0.9.5 on two nodes. Just to test the installation itself, I am trying to run some of the map-reduce examples, like TestDFSIO, wordcount, terasort. When any of these programs are run, I see no errors but in the end it prints "INFO mapred.LocalJobRunner: reduce > copy" in a loop and it never stops.
Is some task not running? Can you please provide some guidance,
Many thanks,
Suri
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.cse.ohio-state.edu/pipermail/rdma-hadoop-discuss/attachments/20141214/34baef0b/attachment-0001.html>