[[PageOutline]] = Hadoop、Hbase、Zookeeper安裝 = == 前言 == 由於Nutch2.0開始,已經可以將索引結果存入Hbase,但由於Hbase內建的Zookeeper似乎沒有太穩定,因此自己手動整合才是王道... == 安裝環境 == * OS: Ubuntu12.04 * Java: OpenJDK 6 == 套件版本 == * Hadoop 1.0.3 * Hbase 0.90.6(沒有選用最新的原因是因為Nutch官網上有提到0.90.x為他們的測試環境) * Zookeper 3.3.6 == 安裝Hadoop == === 下載&解壓縮Hadoop1.0.3 === === 修改$HAHOOP_HOME/conf/hadoop-env.sh === * 設定JAVA_HOME {{{ #!text export JAVA_HOME=/usr/lib/jvm/java-6-openjdk }}} === 修改$HAHOOP_HOME/conf/core-site.xml === {{{ #!text fs.default.name hdfs://localhost:9000 }}} === 修改$HAHOOP_HOME/conf/mapred-site.xml === {{{ #!text mapred.job.tracker localhost:9001 }}} === 修改$HAHOOP_HOME/conf/hdfs-site.xml === {{{ #!text dfs.replication 1 dfs.support.append true }}} === 格式化HDFS === {{{ shunfa@shunfa-vmmVBox:/opt/hadoop-1.0.3$ bin/hadoop namenode -format }}} === 啟動Hadoop === {{{ shunfa@shunfa-vmmVBox:/opt/hadoop-1.0.3$ bin/start-all.sh }}} == 安裝Zookeeper == === 下載並解壓縮zookeeper到/opt/ === === 建立zookeeper運作時需要的目錄並複製預設設定檔 === {{{ ~$ cd /opt/zookeeper-3.3.6/ ~$ sudo mkdir /var/zookeeper ~$ sudo chown -R shunfa:shunfa /var/zookeeper ~$ sudo chown -R shunfa:shunfa /opt/zookeeper-3.3.6/ ~$ cp conf/zoo_sample.cfg conf/zoo.cfg }}} === 修改zoo.cfg === {{{ #!text tickTime=2000 initLimit=10 syncLimit=5 dataDir=/var/zookeeper clientPort=2181 }}} === 啟動 === {{{ ~$ /opt/zookeeper-3.3.6/bin/zkServer.sh start }}} == 安裝Hbase == === 下載並解壓縮Hbase到/opt/ === === 修改 /opt/hbase-0.90.6/conf/hbase-env.sh === {{{ #!text export HBASE_MANAGES_ZK=false export JAVA_HOME=/usr/lib/jvm/java-6-openjdk export HBASE_HEAPSIZE=256 export HBASE_CLASSPATH=/opt/hadoop-1.0.3/conf }}} === 修改 /opt/hbase-0.90.6/conf/hbase-site.xml === {{{ #!text hbase.rootdir hdfs://localhost:54310/hbase hbase.zookeeper.property.dataDir /var/zookeeper dfs.support.append true hbase.cluster.distributed true hbase.zookeeper.property.clientPort 2181 }}} === 複製相關lib === {{{ ~$ cp $HADOOP_HOME/lib/commons-configuration-1.6.jar $HBASE_HOME/lib/ ~$ cp $HADOOP_HOME/hadoop-core-1.0.3.jar $HBASE_HOME/lib/ }}} === 啟動 === {{{ ~$ /opt/hbase-0.90.6/bin/start-hbase.sh }}} == 完成 == * 執行結果如下: {{{ shunfa@shunfa-vmmVBox:~$ jps 23659 HRegionServer 21788 NameNode 22230 JobTracker 22148 SecondaryNameNode 5585 Jps 21972 DataNode 22410 TaskTracker 23069 QuorumPeerMain }}}