Changes between Version 43 and Version 44 of waue/2009/0617
- Timestamp:
- Sep 24, 2010, 5:14:17 PM (15 years ago)
Legend:
- Unmodified
- Added
- Removed
- Modified
-
waue/2009/0617
v43 v44 209 209 一開始會出現問你要將工作目錄放在哪裡:在這我們用預設值 210 210 211 [[Image(wiki:waue :2009:0617:2-1.png)]]211 [[Image(wiki:waue/2009/0617:2-1.png)]] 212 212 213 213 ------- … … 221 221 || window -> || open pers.. -> || other.. -> || map/reduce|| 222 222 223 [[Image(wi n-open-other.png)]]223 [[Image(wiki:waue/2009/0617:win-open-other.png)]] 224 224 225 225 ------- 226 226 227 227 設定要用 Map/Reduce 的視野 228 [[Image(wiki:waue :2009:0617:2-2.png)]]228 [[Image(wiki:waue/2009/0617:2-2.png)]] 229 229 230 230 --------- 231 231 232 232 使用 Map/Reduce 的視野後的介面呈現 233 [[Image(wiki:waue :2009:0617:2-3.png)]]233 [[Image(wiki:waue/2009/0617:2-3.png)]] 234 234 235 235 -------- … … 238 238 239 239 || file -> || new -> || project -> || Map/Reduce -> || Map/Reduce Project -> || next || 240 [[Image(wiki:waue :2009:0617:file-new-project.png)]]240 [[Image(wiki:waue/2009/0617:file-new-project.png)]] 241 241 242 242 -------- … … 244 244 建立mapreduce專案(1) 245 245 246 [[Image(wiki:waue :2009:0617:2-4.png)]]246 [[Image(wiki:waue/2009/0617:2-4.png)]] 247 247 248 248 ----------- … … 256 256 }}} 257 257 258 [[Image(wiki:waue :2009:0617:2-4-2.png)]]258 [[Image(wiki:waue/2009/0617:2-4-2.png)]] 259 259 260 260 … … 269 269 Step1. 右鍵點選project的properties做細部設定 270 270 271 [[Image(wiki:waue :2009:0617:2-5.png)]]271 [[Image(wiki:waue/2009/0617:2-5.png)]] 272 272 273 273 ---------- … … 276 276 277 277 hadoop的javadoc的設定(1) 278 [[Image(wiki:waue :2009:0617:2-5-1.png)]]278 [[Image(wiki:waue/2009/0617:2-5-1.png)]] 279 279 280 280 * java Build Path -> Libraries -> hadoop-0.20.0-ant.jar … … 291 291 ------------ 292 292 Step3. hadoop的javadoc的設定完後(2) 293 [[Image(wiki:waue :2009:0617:2-5-2.png)]]293 [[Image(wiki:waue/2009/0617:2-5-2.png)]] 294 294 295 295 ------------ … … 298 298 * javadoc location -> 輸入:file:/usr/lib/jvm/java-6-sun/docs/api/ 299 299 300 [[Image(wiki:waue :2009:0617:2-5-3.png)]]300 [[Image(wiki:waue/2009/0617:2-5-3.png)]] 301 301 302 302 ----- … … 308 308 -------- 309 309 Step1. 視窗右下角黃色大象圖示"Map/Reduce Locations tag" -> 點選齒輪右邊的藍色大象圖示: 310 [[Image(wiki:waue :2009:0617:2-6.png)]]310 [[Image(wiki:waue/2009/0617:2-6.png)]] 311 311 312 312 ------------- 313 313 Step2. 進行eclipse 與 hadoop 間的設定(2) 314 [[Image(wiki:waue :2009:0617:2-6-1.png)]]314 [[Image(wiki:waue/2009/0617:2-6-1.png)]] 315 315 316 316 {{{ … … 325 325 326 326 設定完後,可以看到下方多了一隻藍色大象,左方展開資料夾也可以秀出在hdfs內的檔案結構 327 [[Image(wiki:waue :2009:0617:2-6-2.png)]]327 [[Image(wiki:waue/2009/0617:2-6-2.png)]] 328 328 ------------- 329 329 … … 343 343 344 344 || File -> || new -> || mapper || 345 [[Image(wiki:waue :2009:0617:file-new-mapper.png)]]345 [[Image(wiki:waue/2009/0617:file-new-mapper.png)]] 346 346 347 347 ----------- … … 349 349 2. create 350 350 351 [[Image(wiki:waue :2009:0617:3-1.png)]]351 [[Image(wiki:waue/2009/0617:3-1.png)]] 352 352 {{{ 353 353 #!sh … … 388 388 389 389 建立mapper.java後,貼入程式碼 390 [[Image(wiki:waue :2009:0617:3-2.png)]]390 [[Image(wiki:waue/2009/0617:3-2.png)]] 391 391 392 392 ------------ … … 397 397 398 398 * File -> new -> reducer 399 [[Image(wiki:waue :2009:0617:file-new-reducer.png)]]399 [[Image(wiki:waue/2009/0617:file-new-reducer.png)]] 400 400 401 401 ------- 402 402 2. create 403 [[Image(wiki:waue :2009:0617:3-3.png)]]403 [[Image(wiki:waue/2009/0617:3-3.png)]] 404 404 405 405 {{{ … … 440 440 441 441 * File -> new -> Map/Reduce Driver 442 [[Image(wiki:waue :2009:0617:file-new-mr-driver.png)]]442 [[Image(wiki:waue/2009/0617:file-new-mr-driver.png)]] 443 443 ---------- 444 444 … … 448 448 449 449 建立WordCount.java,此檔用來驅動mapper 與 reducer,因此選擇 Map/Reduce Driver 450 [[Image(wiki:waue :2009:0617:3-4.png)]]450 [[Image(wiki:waue/2009/0617:3-4.png)]] 451 451 ------------ 452 452 … … 502 502 503 503 三個檔完成後並存檔後,整個程式建立完成 504 [[Image(wiki:waue :2009:0617:3-5.png)]]504 [[Image(wiki:waue/2009/0617:3-5.png)]] 505 505 506 506 ------- … … 521 521 * 右鍵點選WordCount.java -> run as -> run on Hadoop :沒有效果 522 522 523 [[Image(wiki:waue :2009:0617:run-on-hadoop.png)]]523 [[Image(wiki:waue/2009/0617:run-on-hadoop.png)]] 524 524 525 525 * 因此,4.1 提供一個eclipse 上解除 run-on-hadoop 封印的方法。而4.2 則是避開run-on-hadoop 這個功能,用command mode端指令的方法執行。 … … 619 619 * 而回到eclipse視窗,我們可以看到下方視窗run完的job會呈現出來;左方視窗也多出output資料夾,part-r-00000就是我們的結果檔 620 620 621 [[Image(wiki:waue :2009:0617:4-1.png)]]621 [[Image(wiki:waue/2009/0617:4-1.png)]] 622 622 ------ 623 623 * 因為有設定完整的javadoc, 因此可以得到詳細的解說與輔助 624 [[Image(wiki:waue :2009:0617:4-2.png)]]624 [[Image(wiki:waue/2009/0617:4-2.png)]] 625 625 626 626 === make output ===