Changes between Initial Version and Version 1 of Hadoop_Lab4

Timestamp:

Aug 28, 2009, 6:19:09 PM (15 years ago)

Author:

waue

Comment:

--

Hadoop_Lab4

@@ +1 @@
+{{{
+#!html
+<div style="text-align: center;"><big
+ style="font-weight: bold;"><big><big>實做四: Hadoop 程式編譯</big></big></big></div>
+}}}
+[[PageOutline]]
+
+
+== 前言：啟動Hadoop環境 == 
+ 
+ * 重新啟動昨天的環境
+   * 在 node1 上操作
+{{{
+$ cd ~
+$ wget http://hadoop.nchc.org.tw/~waue/clean.sh
+$ chmod 755 clean.sh
+$ ./clean.sh
+$ 
+}}}
+   * 請檢查 hadoop 是否正確運作.
+
+ == 練習 1 : Word Count 初級版 == 
+
+ * 上傳內容到hdfs內
+
+{{{
+$ cd /opt/hadoop
+$ bin/hadoop fs -mkdir input
+$ echo "I like NCHC Cloud Course." > input1
+$ echo "I like nchc Cloud Course, and we enjoy this course." > input2
+$ bin/hadoop fs -put input1 input
+$ bin/hadoop fs -put input2 input
+$ bin/hadoop fs -ls input
+}}}
+
+ * 點此連結 [attachment:wiki:jazz/Hadoop_Lab6:WordCount.java?format=raw WordCount.java] 並將他存到 /opt/hadoop; 
+
+
+ * 運作程式
+
+{{{
+$ mkdir MyJava
+$ javac -classpath hadoop-*-core.jar -d MyJava WordCount.java
+$ jar -cvf wordcount.jar -C MyJava .
+$ bin/hadoop jar wordcount.jar WordCount input/ output/
+$ bin/hadoop fs -cat output/part-00000
+}}}
+
+-----
+
+ == 練習 2 : Word Count 進階版 ==
+
+{{{
+$ echo "\." >pattern.txt && echo "\," >>pattern.txt
+$ bin/hadoop fs -put pattern.txt ./
+$ mkdir MyJava2
+}}}
+
+ * 點此連結 [attachment:wiki:jazz/Hadoop_Lab6:WordCount2.java?format=raw WordCount2.java] 並將他存到 /opt/hadoop; 
+
+{{{
+$ javac -classpath hadoop-*-core.jar -d MyJava2 WordCount2.java
+$ jar -cvf wordcount2.jar -C MyJava2 .
+$ bin/hadoop jar wordcount2.jar WordCount2 input output2 -skip pattern.txt
+$ bin/hadoop fs -cat output2/part-00000
+$ bin/hadoop jar wordcount2.jar WordCount2 -Dwordcount.case.sensitive=false input output3 -skip pattern.txt
+$ bin/hadoop fs -cat output3/part-00000
+}}}