Changes between Initial Version and Version 1 of THU120907/Lab8

Timestamp:

Sep 6, 2012, 11:47:09 PM (13 years ago)

Author:

jazz

Comment:

--

THU120907/Lab8

@@ +1 @@
+◢ <[wiki:THU120907/Lab7 實作七]> | <[wiki:THU120907 回課程大綱]> ▲ | ◣
+
+= 實作八 Lab 8 =
+[[PageOutline]]
+{{{
+#!html
+<div style="text-align: center;"><big style="font-weight: bold;"><big>練習豬的拉丁語<br/>Pig Latin in Practice</big></big></div>
+}}}
+
+{{{
+#!text
+以下練習，請連線至 hadoop.nchc.org.tw 操作。底下的 hXXXX 等於您的用戶名稱。
+}}}
+
+== Aggregation (Local Mode) ==
+
+{{{
+~$ wget http://hadoop.nchc.org.tw/excite-small.log
+~$ pig -x local
+grunt> log = LOAD 'excite-small.log' AS (user, timestamp, query);
+grunt> grpd = GROUP log BY user;
+grunt> cntd = FOREACH grpd GENERATE group, COUNT(log);
+grunt> STORE cntd INTO 'lab8_out1';
+grunt> quit
+~$ head lab8_out1
+}}}
+
+== Filter (Local Mode) ==
+
+{{{
+~$ pig -x local
+grunt> log = LOAD 'excite-small.log' AS (user, timestamp, query);
+grunt> grpd = GROUP log BY user;
+grunt> cntd = FOREACH grpd GENERATE group, COUNT(log) AS cnt;
+grunt> fltrd = FILTER cntd BY cnt > 50;
+grunt> STORE fltrd INTO 'lab8_out2';
+grunt> quit
+~$ head lab8_out2
+}}}
+
+== Sorting (Local Mode) ==
+
+{{{
+~$ pig -x local
+grunt> log = LOAD 'excite-small.log' AS (user, timestamp, query);
+grunt> grpd = GROUP log BY user;
+grunt> cntd = FOREACH grpd GENERATE group, COUNT(log) AS cnt;
+grunt> fltrd = FILTER cntd BY cnt > 50;
+grunt> srtd = ORDER fltrd BY cnt;
+grunt> STORE srtd INTO 'lab8_out3';
+grunt> quit
+~$ head lab8_out3
+}}}
+
+== Connect Pig to Hadoop (Full Distributed Mode) ==
+
+{{{
+~$ hadoop fs -put excite-small.log .
+~$ pig
+grunt> log = LOAD 'excite-small.log' AS (user, timestamp, query);
+grunt> grpd = GROUP log BY user;
+grunt> cntd = FOREACH grpd GENERATE group, COUNT(log);
+grunt> STORE cntd INTO 'lab8_out1';
+grunt> quit
+~$ hadoop fs -cat lab8_out1/part-00000
+}}}