close
Warning:
Can't synchronize with repository "(default)" (Unsupported version control system "svn": /usr/lib/python2.7/dist-packages/libsvn/_fs.so: failed to map segment from shared object: Cannot allocate memory). Look in the Trac log for more information.
- Timestamp:
-
Apr 26, 2010, 4:31:08 PM (16 years ago)
- Author:
-
waue
- Comment:
-
--
Legend:
- Unmodified
- Added
- Removed
- Modified
-
|
v38
|
v39
|
|
| 207 | 207 | |
| 208 | 208 | 一開始會出現問你要將工作目錄放在哪裡:在這我們用預設值 |
| 209 | | [[Image(2-1.png)]] |
| | 209 | |
| | 210 | [[Image(wiki:waue:2009:0617:2-1.png)]] |
| | 211 | |
| 210 | 212 | ------- |
| 211 | 213 | |
| … |
… |
|
| 218 | 220 | || window -> || open pers.. -> || other.. -> || map/reduce|| |
| 219 | 221 | |
| 220 | | [[Image(win-open-other.png)]] |
| | 222 | [[Image(wiki:waue:2009:0617:win-open-other.png)]] |
| 221 | 223 | |
| 222 | 224 | ------- |
| 223 | 225 | |
| 224 | 226 | 設定要用 Map/Reduce 的視野 |
| 225 | | [[Image(2-2.png)]] |
| | 227 | [[Image(wiki:waue:2009:0617:2-2.png)]] |
| 226 | 228 | |
| 227 | 229 | --------- |
| 228 | 230 | |
| 229 | 231 | 使用 Map/Reduce 的視野後的介面呈現 |
| 230 | | [[Image(2-3.png)]] |
| | 232 | [[Image(wiki:waue:2009:0617:2-3.png)]] |
| 231 | 233 | |
| 232 | 234 | -------- |
| … |
… |
|
| 235 | 237 | |
| 236 | 238 | || file -> || new -> || project -> || Map/Reduce -> || Map/Reduce Project -> || next || |
| 237 | | [[Image(file-new-project.png)]] |
| | 239 | [[Image(wiki:waue:2009:0617:file-new-project.png)]] |
| 238 | 240 | |
| 239 | 241 | -------- |
| … |
… |
|
| 241 | 243 | 建立mapreduce專案(1) |
| 242 | 244 | |
| 243 | | [[Image(2-4.png)]] |
| | 245 | [[Image(wiki:waue:2009:0617:2-4.png)]] |
| 244 | 246 | |
| 245 | 247 | ----------- |
| … |
… |
|
| 253 | 255 | }}} |
| 254 | 256 | |
| 255 | | [[Image(2-4-2.png)]] |
| | 257 | [[Image(wiki:waue:2009:0617:2-4-2.png)]] |
| 256 | 258 | |
| 257 | 259 | |
| … |
… |
|
| 266 | 268 | Step1. 右鍵點選project的properties做細部設定 |
| 267 | 269 | |
| 268 | | [[Image(2-5.png)]] |
| | 270 | [[Image(wiki:waue:2009:0617:2-5.png)]] |
| 269 | 271 | |
| 270 | 272 | ---------- |
| … |
… |
|
| 273 | 275 | |
| 274 | 276 | hadoop的javadoc的設定(1) |
| 275 | | [[Image(2-5-1.png)]] |
| | 277 | [[Image(wiki:waue:2009:0617:Image(2-5-1.png)]] |
| 276 | 278 | |
| 277 | 279 | * java Build Path -> Libraries -> hadoop-0.20.0-ant.jar |
| … |
… |
|
| 288 | 290 | ------------ |
| 289 | 291 | Step3. hadoop的javadoc的設定完後(2) |
| 290 | | [[Image(2-5-2.png)]] |
| | 292 | [[Image(wiki:waue:2009:0617:Image(2-5-2.png)]] |
| 291 | 293 | |
| 292 | 294 | ------------ |
| … |
… |
|
| 295 | 297 | * javadoc location -> 輸入:file:/usr/lib/jvm/java-6-sun/docs/api/ |
| 296 | 298 | |
| 297 | | [[Image(2-5-3.png)]] |
| | 299 | [[Image(wiki:waue:2009:0617:2-5-3.png)]] |
| 298 | 300 | |
| 299 | 301 | ----- |
| … |
… |
|
| 305 | 307 | -------- |
| 306 | 308 | Step1. 視窗右下角黃色大象圖示"Map/Reduce Locations tag" -> 點選齒輪右邊的藍色大象圖示: |
| 307 | | [[Image(2-6.png)]] |
| | 309 | [[Image(wiki:waue:2009:0617:2-6.png)]] |
| 308 | 310 | |
| 309 | 311 | ------------- |
| 310 | 312 | Step2. 進行eclipse 與 hadoop 間的設定(2) |
| 311 | | [[Image(2-6-1.png)]] |
| | 313 | [[Image(wiki:waue:2009:0617:2-6-1.png)]] |
| 312 | 314 | |
| 313 | 315 | {{{ |
| … |
… |
|
| 322 | 324 | |
| 323 | 325 | 設定完後,可以看到下方多了一隻藍色大象,左方展開資料夾也可以秀出在hdfs內的檔案結構 |
| 324 | | [[Image(2-6-2.png)]] |
| | 326 | [[Image(wiki:waue:2009:0617:2-6-2.png)]] |
| 325 | 327 | ------------- |
| 326 | 328 | |
| … |
… |
|
| 340 | 342 | |
| 341 | 343 | || File -> || new -> || mapper || |
| 342 | | [[Image(file-new-mapper.png)]] |
| | 344 | [[Image(wiki:waue:2009:0617:file-new-mapper.png)]] |
| 343 | 345 | |
| 344 | 346 | ----------- |
| … |
… |
|
| 346 | 348 | 2. create |
| 347 | 349 | |
| 348 | | [[Image(3-1.png)]] |
| | 350 | [[Image(wiki:waue:2009:0617:3-1.png)]] |
| 349 | 351 | {{{ |
| 350 | 352 | #!sh |
| … |
… |
|
| 385 | 387 | |
| 386 | 388 | 建立mapper.java後,貼入程式碼 |
| 387 | | [[Image(3-2.png)]] |
| | 389 | [[Image(wiki:waue:2009:0617:3-2.png)]] |
| 388 | 390 | |
| 389 | 391 | ------------ |
| … |
… |
|
| 394 | 396 | |
| 395 | 397 | * File -> new -> reducer |
| 396 | | [[Image(file-new-reducer.png)]] |
| | 398 | [[Image(wiki:waue:2009:0617:file-new-reducer.png)]] |
| 397 | 399 | |
| 398 | 400 | ------- |
| 399 | 401 | 2. create |
| 400 | | [[Image(3-3.png)]] |
| | 402 | [[Image(wiki:waue:2009:0617:3-3.png)]] |
| 401 | 403 | |
| 402 | 404 | {{{ |
| … |
… |
|
| 437 | 439 | |
| 438 | 440 | * File -> new -> Map/Reduce Driver |
| 439 | | [[Image(file-new-mr-driver.png)]] |
| | 441 | [[Image(wiki:waue:2009:0617:file-new-mr-driver.png)]] |
| 440 | 442 | ---------- |
| 441 | 443 | |
| … |
… |
|
| 445 | 447 | |
| 446 | 448 | 建立WordCount.java,此檔用來驅動mapper 與 reducer,因此選擇 Map/Reduce Driver |
| 447 | | [[Image(3-4.png)]] |
| | 449 | [[Image(wiki:waue:2009:0617:3-4.png)]] |
| 448 | 450 | ------------ |
| 449 | 451 | |
| … |
… |
|
| 499 | 501 | |
| 500 | 502 | 三個檔完成後並存檔後,整個程式建立完成 |
| 501 | | [[Image(3-5.png)]] |
| | 503 | [[Image(wiki:waue:2009:0617:3-5.png)]] |
| 502 | 504 | |
| 503 | 505 | ------- |
| … |
… |
|
| 518 | 520 | * 右鍵點選WordCount.java -> run as -> run on Hadoop :沒有效果 |
| 519 | 521 | |
| 520 | | [[Image(run-on-hadoop.png)]] |
| | 522 | [[Image(wiki:waue:2009:0617:run-on-hadoop.png)]] |
| 521 | 523 | |
| 522 | 524 | * 因此,4.1 提供一個eclipse 上解除 run-on-hadoop 封印的方法。而4.2 則是避開run-on-hadoop 這個功能,用command mode端指令的方法執行。 |
| … |
… |
|
| 616 | 618 | * 而回到eclipse視窗,我們可以看到下方視窗run完的job會呈現出來;左方視窗也多出output資料夾,part-r-00000就是我們的結果檔 |
| 617 | 619 | |
| 618 | | [[Image(4-1.png)]] |
| | 620 | [[Image(wiki:waue:2009:0617:4-1.png)]] |
| 619 | 621 | ------ |
| 620 | 622 | * 因為有設定完整的javadoc, 因此可以得到詳細的解說與輔助 |
| 621 | | [[Image(4-2.png)]] |
| | 623 | [[Image(wiki:waue:2009:0617:4-2.png)]] |
| 622 | 624 | |
| 623 | 625 | === make output === |