wiki:crawlzilla/crawling_speed
close Warning: Can't synchronize with repository "(default)" (Unsupported version control system "svn": /usr/lib/python2.7/dist-packages/libsvn/_delta.so: failed to map segment from shared object: Cannot allocate memory). Look in the Trac log for more information.

Version 1 (modified by shunfa, 15 years ago) (diff)

--

Crawl Speed

環境

  • 虛擬機器：VBox
  • CPU數量：1
  • RAM：1G
  • Disk：8G

爬取網站

http://www.nchc.org.tw/tw/
http://www.nchc.org.tw/en/

執行速度

Node數量爬取深度執行時間
131h:31m:00s
42h:48m:22s
23
4