close Warning: Can't synchronize with repository "(default)" (Unsupported version control system "svn": /usr/lib/python2.7/dist-packages/libsvn/_fs.so: failed to map segment from shared object: Cannot allocate memory). Look in the Trac log for more information.

Changes between Initial Version and Version 1 of TCCA140822/Lab9


Ignore:
Timestamp:
Aug 23, 2014, 12:11:47 AM (11 years ago)
Author:
jazz
Comment:

--

Legend:

Unmodified
Added
Removed
Modified

  • TCCA140822/Lab9

    v1 v1  
     1[[PageOutline]]
     2
     3◢ <[wiki:TCCA140822/Lab8 實作八]> | <[wiki:TCCA140822 回課程大綱]> ▲ | <[wiki:TCCA140822/Lab10 實作十]> ◣
     4
     5= 實作九 Lab 9 =
     6
     7{{{
     8#!html
     9<div style="text-align: center;"><big style="font-weight: bold;"><big>Hadoop Streaming 搭配不同程式語言練習<br/>Hadoop Streaming in different Language</big></big></div>
     10}}}
     11
     12{{{
     13#!text
     14以下練習，請連線至 hadoop.3du.me 操作。底下的 userXX 等於您的用戶名稱。
     15}}}
     16
     17== 搭配現存二進位執行檔 ==
     18== Existing Binary ==
     19
     20{{{
     21~$ hadoop fs -put /opt/hadoop/conf lab9_input
     22~$ hadoop jar hadoop-streaming.jar -input lab9_input -output lab9_out1 -mapper /bin/cat -reducer /usr/bin/wc
     23~$ hadoop fs -cat lab9_out1/part-00000
     24}}}
     25
     26== 搭配 Bash Shell Script ==
     27
     28{{{
     29~$ echo "sed -e \"s/ /\n/g\" | grep ." > streamingMapper.sh
     30~$ echo "uniq -c | awk '{print \$2 \"\t\" \$1}'" > streamingReducer.sh
     31~$ chmod a+x streamingMapper.sh
     32~$ chmod a+x streamingReducer.sh
     33~$ hadoop jar hadoop-streaming.jar -input lab9_input -output lab9_out2 -mapper streamingMapper.sh -reducer streamingReducer.sh -file streamingMapper.sh -file streamingReducer.sh
     34~$ hadoop fs -cat lab9_out2/part-00000
     35}}}