Changes between Initial Version and Version 1 of III140705/Lab21

Timestamp:

Jul 5, 2014, 8:34:57 AM (10 years ago)

Author:

jazz

Comment:

--

III140705/Lab21

@@ +1 @@
+◢ <[wiki:III140705/Lab20 實作二十]> | <[wiki:III140705 回課程大綱]> ▲ | <[wiki:III140705/Lab22 實作二十二] > ◣
+
+= 實作二十一 Lab21 =
+
+{{{
+#!html
+<p style="text-align: center;"><big style="font-weight: bold;"><big> 修改檔案輸入格式<br/> KeyValueTextInputFormat </big></big></p>
+}}}
+
+[[PageOutline]]
+
+{{{
+#!text
+請先連線至 nodeN.3du.me , N 為您的報名編號
+}}}
+
+{{{
+cd ~/hadoop_labs/lab012
+ant
+mkdir -p kv_input
+printf "A\t1\n" >  kv_input/input1
+printf "B\t2\n" >> kv_input/input1
+printf "C\t3\n" >> kv_input/input1
+printf "A\t1\n" >  kv_input/input2
+printf "C\t2\n" >> kv_input/input2
+printf "B\t1\n" >> kv_input/input2
+hadoop fs -put kv_input kv_input
+hadoop jar WordCount.jar kv_input kv_output
+hadoop fs -ls kv_output
+hadoop fs -cat kv_output/part-*
+
+export HADOOP_CONF_DIR=~/hadoop/conf.local/
+hadoop jar WordCount.jar kv_input kv_output
+ls -al kv_output
+cat kv_output/part-*
+unset HADOOP_CONF_DIR
+}}}
+
+ * Reference:
+   * http://hadoop.apache.org/docs/r1.0.4/api/org/apache/hadoop/mapreduce/lib/input/FileInputFormat.html
+   * http://hadoop.apache.org/docs/r1.0.4/api/org/apache/hadoop/mapreduce/lib/input/TextInputFormat.html
+{{{
+#!text
+public class TextInputFormat
+extends FileInputFormat<LongWritable,Text>
+}}}
+   * http://hadoop.apache.org/docs/r1.0.4/api/org/apache/hadoop/mapreduce/lib/input/KeyValueTextInputFormat.html
+{{{
+#!text
+public class KeyValueTextInputFormat
+extends FileInputFormat<Text,Text>
+}}}
+   * http://hadoop.apache.org/docs/r1.0.4/api/org/apache/hadoop/mapreduce/lib/input/NLineInputFormat.html
+{{{
+#!text
+public class NLineInputFormat
+extends FileInputFormat<LongWritable,Text>
+
+NLineInputFormat which splits N lines of input as one split.
+}}}
+
+== 實作習題 ==
+
+ <問題 1>