Changes between Version 38 and Version 39 of waue/2009/0617
- Timestamp:
- Apr 26, 2010, 4:31:08 PM (15 years ago)
Legend:
- Unmodified
- Added
- Removed
- Modified
-
waue/2009/0617
v38 v39 207 207 208 208 一開始會出現問你要將工作目錄放在哪裡:在這我們用預設值 209 [[Image(2-1.png)]] 209 210 [[Image(wiki:waue:2009:0617:2-1.png)]] 211 210 212 ------- 211 213 … … 218 220 || window -> || open pers.. -> || other.. -> || map/reduce|| 219 221 220 [[Image(wi n-open-other.png)]]222 [[Image(wiki:waue:2009:0617:win-open-other.png)]] 221 223 222 224 ------- 223 225 224 226 設定要用 Map/Reduce 的視野 225 [[Image( 2-2.png)]]227 [[Image(wiki:waue:2009:0617:2-2.png)]] 226 228 227 229 --------- 228 230 229 231 使用 Map/Reduce 的視野後的介面呈現 230 [[Image( 2-3.png)]]232 [[Image(wiki:waue:2009:0617:2-3.png)]] 231 233 232 234 -------- … … 235 237 236 238 || file -> || new -> || project -> || Map/Reduce -> || Map/Reduce Project -> || next || 237 [[Image( file-new-project.png)]]239 [[Image(wiki:waue:2009:0617:file-new-project.png)]] 238 240 239 241 -------- … … 241 243 建立mapreduce專案(1) 242 244 243 [[Image( 2-4.png)]]245 [[Image(wiki:waue:2009:0617:2-4.png)]] 244 246 245 247 ----------- … … 253 255 }}} 254 256 255 [[Image( 2-4-2.png)]]257 [[Image(wiki:waue:2009:0617:2-4-2.png)]] 256 258 257 259 … … 266 268 Step1. 右鍵點選project的properties做細部設定 267 269 268 [[Image( 2-5.png)]]270 [[Image(wiki:waue:2009:0617:2-5.png)]] 269 271 270 272 ---------- … … 273 275 274 276 hadoop的javadoc的設定(1) 275 [[Image( 2-5-1.png)]]277 [[Image(wiki:waue:2009:0617:Image(2-5-1.png)]] 276 278 277 279 * java Build Path -> Libraries -> hadoop-0.20.0-ant.jar … … 288 290 ------------ 289 291 Step3. hadoop的javadoc的設定完後(2) 290 [[Image( 2-5-2.png)]]292 [[Image(wiki:waue:2009:0617:Image(2-5-2.png)]] 291 293 292 294 ------------ … … 295 297 * javadoc location -> 輸入:file:/usr/lib/jvm/java-6-sun/docs/api/ 296 298 297 [[Image( 2-5-3.png)]]299 [[Image(wiki:waue:2009:0617:2-5-3.png)]] 298 300 299 301 ----- … … 305 307 -------- 306 308 Step1. 視窗右下角黃色大象圖示"Map/Reduce Locations tag" -> 點選齒輪右邊的藍色大象圖示: 307 [[Image( 2-6.png)]]309 [[Image(wiki:waue:2009:0617:2-6.png)]] 308 310 309 311 ------------- 310 312 Step2. 進行eclipse 與 hadoop 間的設定(2) 311 [[Image( 2-6-1.png)]]313 [[Image(wiki:waue:2009:0617:2-6-1.png)]] 312 314 313 315 {{{ … … 322 324 323 325 設定完後,可以看到下方多了一隻藍色大象,左方展開資料夾也可以秀出在hdfs內的檔案結構 324 [[Image( 2-6-2.png)]]326 [[Image(wiki:waue:2009:0617:2-6-2.png)]] 325 327 ------------- 326 328 … … 340 342 341 343 || File -> || new -> || mapper || 342 [[Image( file-new-mapper.png)]]344 [[Image(wiki:waue:2009:0617:file-new-mapper.png)]] 343 345 344 346 ----------- … … 346 348 2. create 347 349 348 [[Image( 3-1.png)]]350 [[Image(wiki:waue:2009:0617:3-1.png)]] 349 351 {{{ 350 352 #!sh … … 385 387 386 388 建立mapper.java後,貼入程式碼 387 [[Image( 3-2.png)]]389 [[Image(wiki:waue:2009:0617:3-2.png)]] 388 390 389 391 ------------ … … 394 396 395 397 * File -> new -> reducer 396 [[Image( file-new-reducer.png)]]398 [[Image(wiki:waue:2009:0617:file-new-reducer.png)]] 397 399 398 400 ------- 399 401 2. create 400 [[Image( 3-3.png)]]402 [[Image(wiki:waue:2009:0617:3-3.png)]] 401 403 402 404 {{{ … … 437 439 438 440 * File -> new -> Map/Reduce Driver 439 [[Image( file-new-mr-driver.png)]]441 [[Image(wiki:waue:2009:0617:file-new-mr-driver.png)]] 440 442 ---------- 441 443 … … 445 447 446 448 建立WordCount.java,此檔用來驅動mapper 與 reducer,因此選擇 Map/Reduce Driver 447 [[Image( 3-4.png)]]449 [[Image(wiki:waue:2009:0617:3-4.png)]] 448 450 ------------ 449 451 … … 499 501 500 502 三個檔完成後並存檔後,整個程式建立完成 501 [[Image( 3-5.png)]]503 [[Image(wiki:waue:2009:0617:3-5.png)]] 502 504 503 505 ------- … … 518 520 * 右鍵點選WordCount.java -> run as -> run on Hadoop :沒有效果 519 521 520 [[Image( run-on-hadoop.png)]]522 [[Image(wiki:waue:2009:0617:run-on-hadoop.png)]] 521 523 522 524 * 因此,4.1 提供一個eclipse 上解除 run-on-hadoop 封印的方法。而4.2 則是避開run-on-hadoop 這個功能,用command mode端指令的方法執行。 … … 616 618 * 而回到eclipse視窗,我們可以看到下方視窗run完的job會呈現出來;左方視窗也多出output資料夾,part-r-00000就是我們的結果檔 617 619 618 [[Image( 4-1.png)]]620 [[Image(wiki:waue:2009:0617:4-1.png)]] 619 621 ------ 620 622 * 因為有設定完整的javadoc, 因此可以得到詳細的解說與輔助 621 [[Image( 4-2.png)]]623 [[Image(wiki:waue:2009:0617:4-2.png)]] 622 624 623 625 === make output ===