Changes between Version 5 and Version 6 of crawlzilla/stress_testing

Timestamp:

Sep 9, 2010, 1:50:32 PM (15 years ago)

Author:

rock

Comment:

--

crawlzilla/stress_testing

@@ -17 +17 @@
  * Ubuntu 10.04 (CPU1, RAM 512M) ('''''hadoop Heap 256M''''') in VM
    * 同時執行以上3個爬取任務時，一樣會產生 out of memory 問題
-   * error message (syslog)
+   * error message ('''syslog''')
 {{{
-#!sh
 600     Sep  9 09:59:28 ubuntu-186 kernel: [ 3708.133724] Out of memory: kill process 3843 (go.sh) score 1775788 or a child
 601     Sep  9 09:59:28 ubuntu-186 kernel: [ 3708.133791] Killed process 4205 (counter.sh)
@@ -39 +38 @@
 617     Sep  9 09:59:28 ubuntu-186 kernel: [ 3709.201865]  [<c058d0b3>] error_code+0x73/0x80
 }}}
+   * error message ('''hadoop-crawler-jobtracker-ubuntu-186.log''')
+{{{
+869     2010-09-09 10:04:16,052 INFO org.apache.hadoop.mapred.TaskInProgress: Error from attempt_201009090900_0026_m_000000_0: java.io.IOException: Cannot run program "bash": java.io.IOException: error=12, Cannot allocate memory
+870       at java.lang.ProcessBuilder.start(ProcessBuilder.java:460)
+871       at org.apache.hadoop.util.Shell.runCommand(Shell.java:149)
+872       at org.apache.hadoop.util.Shell.run(Shell.java:134)
+873       at org.apache.hadoop.fs.DF.getAvailable(DF.java:73)
+874       at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathForWrite(LocalDirAllocator.java:321)
+875       at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:124)
+876       at org.apache.hadoop.mapred.MapOutputFile.getSpillFileForWrite(MapOutputFile.java:107)
+877       at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:930)
+878       at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:842)
+879       at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
+880       at org.apache.hadoop.mapred.Child.main(Child.java:158)
+881     Caused by: java.io.IOException: java.io.IOException: error=12, Cannot allocate memory
+882       at java.lang.UNIXProcess.<init>(UNIXProcess.java:148)
+883       at java.lang.ProcessImpl.start(ProcessImpl.java:65)
+884       at java.lang.ProcessBuilder.start(ProcessBuilder.java:453)
+885       ... 10 more
+}}}