◢ <[wiki:NTUOSS160412/Lab4 實作四]> | <[wiki:NTUOSS160412 回課程大綱]> ▲ | <[wiki:NTUOSS160412/Lab6 實作六]> ◣

= 實作五 Lab5 =

{{{
#!html
<p style="text-align: center;"><big style="font-weight: bold;"><big>MapReduce 基本指令操作<br/>Basic Commands of Hadoop MapReduce</big></big></p>
}}}

[[PageOutline]]

== Sample 1 : WordCount ==
 
 * 如名稱，WordCount會對所有的字作字數統計，並且從a-z作排列[[BR]]WordCount example will count each word shown in documents and sorting from a to z.
{{{
~$ hadoop fs -put ${HOME}/hadoop/conf lab11_input
~$ hadoop fs -rmr lab11_out2
~$ hadoop jar ${HOME}/hadoop/hadoop-examples-*.jar wordcount lab11_input lab11_out2
}}}
 * 檢查輸出結果的方法同之前方法[[BR]]Let's check the computed result of '''wordcount''' from HDFS : 
{{{
~$ hadoop fs -ls lab11_out2
~$ hadoop fs -cat lab11_out2/part-r-00000 
}}}
 * 結果如下[[BR]]You should see results like this: 
{{{
"".	4
"*"	9
"127.0.0.1"	3
"AS	2
"License");	2
"_logs/history/"	1
"alice,bob	9

( ... skip ... )
}}}

== Sample 2: grep ==
 
 * grep 這個命令是擷取文件裡面特定的字元，在Hadoop example中此指令可以擷取文件中有此指定文字的字串，並作計數統計[[BR]]grep is a command to extract specific characters in documents. In hadoop examples, you can use this command to extract strings match the regular expression and count for matched strings.
{{{
~$ hadoop fs -ls lab11_input
~$ hadoop jar ${HOME}/hadoop/hadoop-examples-*.jar grep lab11_input lab11_out3 'dfs[a-z.]+' 
}}}
 * 運作的畫面如下：[[BR]]You should see procedure like this:  
{{{
11/04/19 10:00:20 INFO mapred.FileInputFormat: Total input paths to process : 25
11/04/19 10:00:20 INFO mapred.JobClient: Running job: job_201104120101_0645
11/04/19 10:00:21 INFO mapred.JobClient:  map 0% reduce 0%
( ... skip ... )
}}}
 * 接著查看結果[[BR]]Let's check the computed result of '''grep''' from HDFS :
{{{
~$ hadoop fs -ls lab11_out3
~$ hadoop fs -cat lab11_out3/part-00000 
}}}
 * 結果如下[[BR]]You should see results like this: 
{{{
4	dfs.permissions
4	dfs.replication
4	dfs.name.dir
3	dfs.namenode.decommission.interval.
3	dfs.namenode.decommission.nodes.per.interval
3	dfs.
( ... skip ... )
}}}

== More Examples ==
 
 可執行的指令一覽表：[[BR]]Here is a list of hadoop examples :

 || aggregatewordcount ||  An Aggregate based map/reduce program that counts the words in the input files. || 
 || aggregatewordhist || An Aggregate based map/reduce program that computes the histogram of the words in the input files. || 
 || grep ||  A map/reduce program that counts the matches of a regex in the input. || 
 || join || A job that effects a join over sorted, equally partitioned datasets || 
 || multifilewc ||  A job that counts words from several files. || 
 || pentomino  || A map/reduce tile laying program to find solutions to pentomino problems. || 
 || pi ||  A map/reduce program that estimates Pi using monte-carlo method. || 
 || randomtextwriter ||  A map/reduce program that writes 10GB of random textual data per node. || 
 || randomwriter || A map/reduce program that writes 10GB of random data per node. || 
 || sleep ||  A job that sleeps at each map and reduce task. || 
 || sort || A map/reduce program that sorts the data written by the random writer. || 
 || sudoku ||  A sudoku solver. || 
 || wordcount || A map/reduce program that counts the words in the input files. || 

You could find more detail at [http://hadoop.apache.org/core/docs/current/api/org/apache/hadoop/examples/package-summary.html org.apache.hadoop.examples]