Changes between Version 18 and Version 19 of Hadoop_Lab5
- Timestamp:
- Apr 26, 2010, 6:28:55 PM (15 years ago)
Legend:
- Unmodified
- Added
- Removed
- Modified
-
Hadoop_Lab5
v18 v19 37 37 1 將javadoc (jdk-6u10-docs.zip) 下載下來 38 38 [https://cds.sun.com/is-bin/INTERSHOP.enfinity/WFS/CDS-CDS_Developer-Site/en_US/-/USD/ViewProductDetail-Start?ProductRef=jdk-6u10-docs-oth-JPR@CDS-CDS_Developer 下載點] 39 [[Image(wiki: waue:2009:0617:1-1.png)]]40 [[Image(wiki:0428Hadoop_Lab1:hadoop_administration.png)]] 39 [[Image(wiki:Hadoop_Lab5:1-1.png)]] 40 41 41 42 42 2 下載完後將檔案放在 /tmp/ 下 … … 196 196 一開始會出現問你要將工作目錄放在哪裡:在這我們用預設值 197 197 198 [[Image(wiki: waue:2009:0617:2-1.png)]]198 [[Image(wiki:Hadoop_Lab5:2-1.png)]] 199 199 200 200 ------- … … 208 208 || window -> || open pers.. -> || other.. -> || map/reduce|| 209 209 210 [[Image(wiki: waue:2009:0617:win-open-other.png)]]210 [[Image(wiki:Hadoop_Lab5:win-open-other.png)]] 211 211 212 212 ------- 213 213 214 214 設定要用 Map/Reduce 的視野 215 [[Image(wiki: waue:2009:0617:2-2.png)]]215 [[Image(wiki:Hadoop_Lab5:2-2.png)]] 216 216 217 217 --------- 218 218 219 219 使用 Map/Reduce 的視野後的介面呈現 220 [[Image(wiki: waue:2009:0617:2-3.png)]]220 [[Image(wiki:Hadoop_Lab5:2-3.png)]] 221 221 222 222 -------- … … 225 225 226 226 || file -> || new -> || project -> || Map/Reduce -> || Map/Reduce Project -> || next || 227 [[Image(wiki: waue:2009:0617:file-new-project.png)]]227 [[Image(wiki:Hadoop_Lab5:file-new-project.png)]] 228 228 229 229 -------- … … 231 231 建立mapreduce專案(1) 232 232 233 [[Image(wiki: waue:2009:0617:2-4.png)]]233 [[Image(wiki:Hadoop_Lab5:2-4.png)]] 234 234 235 235 ----------- … … 243 243 }}} 244 244 245 [[Image(wiki: waue:2009:0617:2-4-2.png)]]245 [[Image(wiki:Hadoop_Lab5:2-4-2.png)]] 246 246 247 247 … … 256 256 Step1. 右鍵點選project的properties做細部設定 257 257 258 [[Image(wiki: waue:2009:0617:2-5.png)]]258 [[Image(wiki:Hadoop_Lab5:2-5.png)]] 259 259 260 260 ---------- … … 263 263 264 264 hadoop的javadoc的設定(1) 265 [[Image(wiki: waue:2009:0617:Image(2-5-1.png)]]265 [[Image(wiki:Hadoop_Lab5:Image(2-5-1.png)]] 266 266 267 267 * java Build Path -> Libraries -> hadoop-0.20.2-ant.jar … … 278 278 ------------ 279 279 Step3. hadoop的javadoc的設定完後(2) 280 [[Image(wiki: waue:2009:0617:Image(2-5-2.png)]]280 [[Image(wiki:Hadoop_Lab5:Image(2-5-2.png)]] 281 281 282 282 ------------ … … 285 285 * javadoc location -> 輸入:file:/usr/lib/jvm/java-6-sun/docs/api/ 286 286 287 [[Image(wiki: waue:2009:0617:2-5-3.png)]]287 [[Image(wiki:Hadoop_Lab5:2-5-3.png)]] 288 288 289 289 ----- … … 295 295 -------- 296 296 Step1. 視窗右下角黃色大象圖示"Map/Reduce Locations tag" -> 點選齒輪右邊的藍色大象圖示: 297 [[Image(wiki: waue:2009:0617:2-6.png)]]297 [[Image(wiki:Hadoop_Lab5:2-6.png)]] 298 298 299 299 ------------- 300 300 Step2. 進行eclipse 與 hadoop 間的設定(2) 301 [[Image(wiki: waue:2009:0617:2-6-1.png)]]301 [[Image(wiki:Hadoop_Lab5:2-6-1.png)]] 302 302 303 303 {{{ … … 312 312 313 313 設定完後,可以看到下方多了一隻藍色大象,左方展開資料夾也可以秀出在hdfs內的檔案結構 314 [[Image(wiki: waue:2009:0617:2-6-2.png)]]314 [[Image(wiki:Hadoop_Lab5:2-6-2.png)]] 315 315 ------------- 316 316 … … 330 330 331 331 || File -> || new -> || mapper || 332 [[Image(wiki: waue:2009:0617:file-new-mapper.png)]]332 [[Image(wiki:Hadoop_Lab5:file-new-mapper.png)]] 333 333 334 334 ----------- … … 336 336 2. create 337 337 338 [[Image(wiki: waue:2009:0617:3-1.png)]]338 [[Image(wiki:Hadoop_Lab5:3-1.png)]] 339 339 {{{ 340 340 #!sh … … 375 375 376 376 建立mapper.java後,貼入程式碼 377 [[Image(wiki: waue:2009:0617:3-2.png)]]377 [[Image(wiki:Hadoop_Lab5:3-2.png)]] 378 378 379 379 ------------ … … 384 384 385 385 * File -> new -> reducer 386 [[Image(wiki: waue:2009:0617:file-new-reducer.png)]]386 [[Image(wiki:Hadoop_Lab5:file-new-reducer.png)]] 387 387 388 388 ------- 389 389 2. create 390 [[Image(wiki: waue:2009:0617:3-3.png)]]390 [[Image(wiki:Hadoop_Lab5:3-3.png)]] 391 391 392 392 {{{ … … 427 427 428 428 * File -> new -> Map/Reduce Driver 429 [[Image(wiki: waue:2009:0617:file-new-mr-driver.png)]]429 [[Image(wiki:Hadoop_Lab5:file-new-mr-driver.png)]] 430 430 ---------- 431 431 … … 435 435 436 436 建立WordCount.java,此檔用來驅動mapper 與 reducer,因此選擇 Map/Reduce Driver 437 [[Image(wiki: waue:2009:0617:3-4.png)]]437 [[Image(wiki:Hadoop_Lab5:3-4.png)]] 438 438 ------------ 439 439 … … 489 489 490 490 三個檔完成後並存檔後,整個程式建立完成 491 [[Image(wiki: waue:2009:0617:3-5.png)]]491 [[Image(wiki:Hadoop_Lab5:3-5.png)]] 492 492 493 493 ------- … … 507 507 * 右鍵點選WordCount.java -> run as -> run on Hadoop 508 508 509 [[Image(wiki:waue:2009:0617:run-on-hadoop.png)]]510 511 512 509 513 510 = 五、結論 =