Changes between Version 2 and Version 3 of jazz/Hadoop_Lab3

Timestamp:

Mar 24, 2009, 5:16:50 PM (17 years ago)

Author:

waue

Comment:

--

jazz/Hadoop_Lab3

@@ -5 +5 @@
 == 前言 ==
 
- * 您手邊有兩台電腦，假設剛剛操作的電腦為node1,另一台則為node2。稍後的環境我們假設node1 為server, node2 為slaves。
+ * 您手邊有兩台電腦，假設剛剛操作的電腦為node1,另一台則為node2。則稍後的環境如下
+ 
+ || || 管理Data的身份 || 管理Job的身份 ||
+ || node1 || namenode(master)& datanode || tasktracker(slave)||
+ || node2 || datanode(slave) || jobtracker(master)& tasktracker||
+ 
  * 這個實做會架設運作在叢集環境上的Hadoop，因此若是你的電腦還存在著之前的實做一的環境，請先作step 0，以移除掉之前的設定。
 
-
+ * 以下node1_ip代表你node1的ip位址，node2_ip為你node2的ip位址，請查清楚之後作設定
+ 
 === 清除所有在實做一作過的環境 ===
 
@@ -70 +76 @@
 }}}
 
- 編輯以下資訊
-
-{{{
-#!diff
---- hadoop-0.18.3/conf/hadoop-env.sh.org
-+++ hadoop-0.18.3/conf/hadoop-env.sh
-@@ -6,7 +6,10 @@
- # remote nodes.
- # The java implementation to use.  Required.
--# export JAVA_HOME=/usr/lib/j2sdk1.5-sun
-+export JAVA_HOME=/usr/lib/jvm/java-6-sun
-+export HADOOP_HOME=/opt/hadoop
-+export HADOOP_CONF_DIR=/opt/hadoop/conf
-+export HADOOP_LOG_DIR=/home/hadooper/logs
-+export HADOOP_PID_DIR=/home/hadooper/pids
-
- # Extra Java CLASSPATH elements.  Optional.
- # export HADOOP_CLASSPATH=
+ 將以下資訊貼入conf/hadoop-env.sh檔內
+
+{{{
+#!sh
+export JAVA_HOME=/usr/lib/jvm/java-6-sun
+export HADOOP_HOME=/opt/hadoop
+export HADOOP_CONF_DIR=/opt/hadoop/conf
+export HADOOP_LOG_DIR=/home/hadooper/logs
+export HADOOP_PID_DIR=/home/hadooper/pids
 }}}
 
@@ -101 +98 @@
 
 {{{
-#!diff
---- hadoop-0.18.3/conf/hadoop-site.xml.org
-+++ hadoop-0.18.3/conf/hadoop-site.xml
-@@ -4,5 +4,31 @@
- <!-- Put site-specific property overrides in this file. -->
- <configuration>
--
-+  <property>
-+    <name>fs.default.name</name>
-+    <value>hdfs://node1_ip:9000/</value>
-+    <description>
-+      The name of the default file system. Either the literal string
-+      "local" or a host:port for NDFS.
-+    </description>
-+  </property>
-+  <property>
-+    <name>mapred.job.tracker</name>
-+    <value>hdfs://node1_ip:9001</value>
-+    <description>
-+      The host and port that the MapReduce job tracker runs at. If
-+      "local", then jobs are run in-process as a single map and
-+      reduce task.
-+    </description>
-+  </property>
-+  <property>
-+    <name>hadoop.tmp.dir</name>
-+    <value>/tmp/hadoop/hadoop-${user.name}</value>
-+    <description>A base for other temporary directories.</description>
-+  </property>
- </configuration>
+#!sh
+<!-- Put site-specific property overrides in this file. -->
+<configuration>
+  <property>
+    <name>fs.default.name</name>
+    <value>hdfs://node1_ip:9000/</value>
+    <description> </description>
+  </property>
+  <property>
+    <name>mapred.job.tracker</name>
+    <value>hdfs://node2_ip:9001</value>
+    <description>  </description>
+  </property>
+  <property>
+    <name>hadoop.tmp.dir</name>
+    <value>/tmp/hadoop/hadoop-${user.name}</value>
+    <description>A base for other temporary directories.</description>
+  </property>
+</configuration>
 }}}
 
@@ -235 +221 @@
    * bin/stop-mapred.sh腳本會參照JobTracker上${HADOOP_CONF_DIR}/slaves文件的內容，在所有列出的slave上停止TaskTracker。
 
+== 練習 ==
+
+ * 和別人組隊，組成4台node的cluster，其中node1 只當 namenode ，node2 只當 Jobtracker，而node3, node4 兩台電腦則身兼 datanode及tasktracker的工作。