close
Warning:
Can't synchronize with repository "(default)" (Unsupported version control system "svn": libldap_r-2.4.so.2: failed to map segment from shared object: Cannot allocate memory). Look in the Trac log for more information.
- Timestamp:
-
Aug 6, 2010, 11:36:04 AM (15 years ago)
- Author:
-
jazz
- Comment:
-
--
Legend:
- Unmodified
- Added
- Removed
- Modified
-
v1
|
v2
|
|
2 | 2 | #!html |
3 | 3 | <div style="text-align: center;"><big |
4 | | style="font-weight: bold;"><big><big>實做四: Hadoop 程式編譯</big></big></big></div> |
| 4 | style="font-weight: bold;"><big><big>實作四: MapReduce 程式編譯</big></big></big><br/><big><big>Lab4: Compiling Hadoop MapReduce Java Program</big></big></div> |
5 | 5 | }}} |
6 | 6 | [[PageOutline]] |
7 | 7 | |
8 | | = 練習 1 : Word Count 初級版 = |
| 8 | = Practice 1 : Word Count #1 (Basic) = |
9 | 9 | |
10 | | * 上傳內容到hdfs內 |
| 10 | * 上傳內容到 HDFS 內[[BR]]upload data to HDFS |
11 | 11 | |
12 | 12 | {{{ |
… |
… |
|
19 | 19 | }}} |
20 | 20 | |
21 | | * 下載 [http://secuse.nchc.org.tw/class/WordCount.java WordCount.java] 並存到/opt/hadoop; |
| 21 | * 下載 [http://secuse.nchc.org.tw/class/WordCount.java WordCount.java] 並存到/opt/hadoop;[[BR]]Download [http://secuse.nchc.org.tw/class/WordCount.java WordCount.java] and save to /opt/hadoop |
22 | 22 | {{{ |
23 | | $ wget http://secuse.nchc.org.tw/class/WordCount.java |
| 23 | ~$ cd /opt/hadoop |
| 24 | /opt/hadoop$ wget http://secuse.nchc.org.tw/class/WordCount.java |
24 | 25 | }}} |
25 | 26 | |
26 | | * 運作程式 |
| 27 | * 運作程式[[BR]]Compile WordCount.java and run it by '''hadoop jar''' command |
27 | 28 | |
28 | 29 | {{{ |
… |
… |
|
34 | 35 | }}} |
35 | 36 | |
36 | | * lab4_out1 執行結果 |
| 37 | * lab4_out1 執行結果 [[BR]]You should see results like this : |
37 | 38 | {{{ |
38 | 39 | #!text |
… |
… |
|
52 | 53 | ----- |
53 | 54 | |
54 | | = 練習 2 : Word Count 進階版 = |
| 55 | = Practice 2 : Word Count #2 (Advanced) = |
55 | 56 | |
56 | 57 | {{{ |
… |
… |
|
61 | 62 | |
62 | 63 | |
63 | | * 下載 [raw-attachment:wiki:Hadoop_Lab4:WordCount2.java WordCount2.java] 並存到/opt/hadoop; |
| 64 | * 下載 [raw-attachment:wiki:Hadoop_Lab4:WordCount2.java WordCount2.java] 並存到/opt/hadoop;[[BR]]Download [raw-attachment:wiki:Hadoop_Lab4:WordCount2.java WordCount2.java] to /opt/hadoop |
64 | 65 | {{{ |
65 | | $ wget http://trac.nchc.org.tw/cloud/raw-attachment/wiki/Hadoop_Lab4/WordCount2.java |
| 66 | ~$ cd /opt/hadoop |
| 67 | /opt/hadoop$ wget http://trac.nchc.org.tw/cloud/raw-attachment/wiki/Hadoop_Lab4/WordCount2.java |
66 | 68 | }}} |
67 | 69 | |
… |
… |
|
73 | 75 | }}} |
74 | 76 | |
75 | | * lab4_out2 執行結果 |
| 77 | * lab4_out2 執行結果[[BR]]You should see results like this: |
76 | 78 | {{{ |
77 | 79 | #!text |
… |
… |
|
89 | 91 | }}} |
90 | 92 | |
| 93 | * Let's given case insensitive and ignore pattern for this example |
91 | 94 | {{{ |
92 | | $ bin/hadoop jar wordcount2.jar WordCount2 -Dwordcount.case.sensitive=false lab4_input lab4_out3 -skip pattern.txt |
93 | | $ bin/hadoop fs -cat lab4_out3/part-00000 |
| 95 | /opt/hadoop$ echo "\," > pattern.txt && echo "\." >> pattern.txt |
| 96 | /opt/hadoop$ bin/hadoop jar wordcount2.jar WordCount2 -Dwordcount.case.sensitive=false lab4_input lab4_out3 -skip pattern.txt |
| 97 | /opt/hadoop$ bin/hadoop fs -cat lab4_out3/part-00000 |
94 | 98 | }}} |
95 | 99 | |
96 | | * lab4_out3 執行結果 |
| 100 | * lab4_out3 執行結果[[BR]]You should see results like this: |
97 | 101 | {{{ |
98 | 102 | #!text |