{{{ #!html
實做四: Hadoop 程式編譯
}}} [[PageOutline]] = 練習 1 : Word Count 初級版 = * 上傳內容到hdfs內 {{{ $ cd /opt/hadoop $ mkdir lab4_input $ echo "I like NCHC Cloud Course." > lab4_input/input1 $ echo "I like nchc Cloud Course, and we enjoy this course." > lab4_input/input2 $ bin/hadoop fs -put lab4_input lab4_input $ bin/hadoop fs -ls lab4_input }}} * 下載 [raw-attachment:wiki:Hadoop_Lab4:WordCount.java WordCount.java] 並存到/opt/hadoop; {{{ $ wget http://trac.nchc.org.tw/cloud/raw-attachment/wiki/Hadoop_Lab4/WordCount.java }}} * 運作程式 {{{ $ mkdir MyJava $ javac -classpath hadoop-*-core.jar -d MyJava WordCount.java $ jar -cvf wordcount.jar -C MyJava . $ bin/hadoop jar wordcount.jar WordCount lab4_input/ lab4_out1/ $ bin/hadoop fs -cat lab4_out1/part-00000 }}} * lab4_out1 執行結果 {{{ #!text Cloud 2 Course, 1 Course. 1 I 2 NCHC 1 and 1 course. 1 enjoy 1 like 2 nchc 1 this 1 we 1 }}} ----- = 練習 2 : Word Count 進階版 = {{{ $ echo "\." >pattern.txt && echo "\," >>pattern.txt $ bin/hadoop fs -put pattern.txt ./ $ mkdir MyJava2 }}} * 下載 [raw-attachment:wiki:Hadoop_Lab4:WordCount2.java WordCount2.java] 並存到/opt/hadoop; {{{ $ wget http://trac.nchc.org.tw/cloud/raw-attachment/wiki/Hadoop_Lab4/WordCount2.java }}} {{{ $ javac -classpath hadoop-*-core.jar -d MyJava2 WordCount2.java $ jar -cvf wordcount2.jar -C MyJava2 . $ bin/hadoop jar wordcount2.jar WordCount2 lab4_input lab4_out2 -skip pattern.txt $ bin/hadoop fs -cat lab4_out2/part-00000 }}} * lab4_out2 執行結果 {{{ #!text Cloud 2 Course 2 I 2 NCHC 1 and 1 course 1 enjoy 1 like 2 nchc 1 this 1 we 1 }}} {{{ $ bin/hadoop jar wordcount2.jar WordCount2 -Dwordcount.case.sensitive=false lab4_input lab4_out3 -skip pattern.txt $ bin/hadoop fs -cat lab4_out3/part-00000 }}} * lab4_out3 執行結果 {{{ #!text and 1 cloud 2 course 3 enjoy 1 i 2 like 2 nchc 2 this 1 we 1 }}}