hadoop偽分布式的安裝步驟-創(chuàng)新互聯(lián)

本篇內(nèi)容主要講解“hadoop偽分布式的安裝步驟”，感興趣的朋友不妨來(lái)看看。本文介紹的方法操作簡(jiǎn)單快捷，實(shí)用性強(qiáng)。下面就讓小編來(lái)帶大家學(xué)習(xí)“hadoop偽分布式的安裝步驟”吧!

創(chuàng)新互聯(lián)公司擁有網(wǎng)站維護(hù)技術(shù)和項(xiàng)目管理團(tuán)隊(duì)，建立的售前、實(shí)施和售后服務(wù)體系，為客戶(hù)提供定制化的網(wǎng)站設(shè)計(jì)、網(wǎng)站制作、網(wǎng)站維護(hù)、成都服務(wù)器托管解決方案。為客戶(hù)網(wǎng)站安全和日常運(yùn)維提供整體管家式外包優(yōu)質(zhì)服務(wù)。我們的網(wǎng)站維護(hù)服務(wù)覆蓋集團(tuán)企業(yè)、上市公司、外企網(wǎng)站、購(gòu)物商城網(wǎng)站建設(shè)、政府網(wǎng)站等各類(lèi)型客戶(hù)群體，為全球成百上千企業(yè)提供全方位網(wǎng)站維護(hù)、服務(wù)器維護(hù)解決方案。

1. 解壓縮 /opt/software/hadoop-2.8.1.tar.gz 文件 
    [root@hadoop002 software]$ cd   /opt/software/
    [root@hadoop002 software]$ pwd
    /opt/software
    [root@hadoop002 software]$ tar -xzvf  hadoop-2.8.1.tar.gz
    [root@hadoop002 software]$ ll
    total 208784
    drwxr-xr-x.  6 root   root        4096 Nov 10  2015 apache-maven-3.3.9
    -rw-r--r--.  1 root   root     8617253 Apr 23 15:14 apache-maven-3.3.9-bin.zip
    drwxr-xr-x.  7 root   root        4096 Aug 21  2009 findbugs-1.3.9
    -rw-r--r--.  1 root   root     7546219 Apr 23 15:14 findbugs-1.3.9.zip
    drwxr-xr-x. 10 root   root   4096 Apr 23 16:36 hadoop-2.8.1
    -rw-r--r--.  1 root   root   194976866 Apr 23 15:40 hadoop-2.8.1.tar.gz
    drwxr-xr-x. 10 109965   5000      4096 Apr 23 15:26 protobuf-2.5.0
    -rw-r--r--.  1 root   root     2401901 Apr 23 15:15 protobuf-2.5.0.tar.gz

2. 添加軟連接  
    [root@hadoop002 software]$ ln -s /opt/software/hadoop-2.8.1 /opt/software/hadoop
    [root@hadoop002 software]$ ll
    total 208784
    drwxr-xr-x.  6 root   root        4096 Nov 10  2015 apache-maven-3.3.9
    -rw-r--r--.  1 root   root     8617253 Apr 23 15:14 apache-maven-3.3.9-bin.zip
    drwxr-xr-x.  7 root   root        4096 Aug 21  2009 findbugs-1.3.9
    -rw-r--r--.  1 root   root     7546219 Apr 23 15:14 findbugs-1.3.9.zip
    lrwxrwxrwx.  1 root    root   26 Apr 23 15:41 hadoop -> /opt/software/hadoop-2.8.1
    drwxr-xr-x. 10 root    root   4096 Apr 23 16:36 hadoop-2.8.1
    -rw-r--r--.  1 root   root   194976866 Apr 23 15:40 hadoop-2.8.1.tar.gz
    drwxr-xr-x. 10 109965   5000      4096 Apr 23 15:26 protobuf-2.5.0
    -rw-r--r--.  1 root   root     2401901 Apr 23 15:15 protobuf-2.5.0.tar.gz


3.  設(shè)置環(huán)境變量
    [root@hadoop002 software]$  vi /etc/profile
    export HADOOP_HOME=/opt/software/hadoop
    export PATH=$HADOOP_HOME/bin:$FINDBUGS_HOME/bin:$PROTOC_HOME/bin:$MAVEN_HOME/bin:$MYSQL_HOME/bin:$JAVA_HOME/bin:$PATH

    --  環(huán)境變量生效
    [root@hadoop002 software]$ source /etc/profile


4. 修改軟連接所屬的用戶(hù)和用戶(hù)組
    [root@hadoop002 software]$ chown -R hadoop:hadoop hadoop/*

5. 將hadoop-2.8.1文件的所屬的用戶(hù)和用戶(hù)組 修改為hadoop
    [root@hadoop002 software]$ chown -R  hadoop:hadoop hadoop-2.8.1
    [root@hadoop002 software]$ chown -R  hadoop:hadoop hadoop-2.8.1/*

6.  切換hadoop用戶(hù)
    [root@hadoop002 hadoop]# su - hadoop
    [hadoop@hadoop002 ~]$ cd /opt/software/hadoop
    [hadoop@hadoop002 hadoop]$ ll
    total 148
    drwxr-xr-x. 2 hadoop hadoop  4096 Apr 18 14:11 bin
    drwxr-xr-x. 3 hadoop hadoop  4096 Apr 18 14:10 etc
    drwxr-xr-x. 2 hadoop hadoop  4096 Apr 18 14:11 include
    drwxr-xr-x. 3 hadoop hadoop  4096 Apr 18 14:10 lib
    drwxr-xr-x. 2 hadoop hadoop  4096 Apr 18 14:11 libexec
    drwxr-xr-x. 2 hadoop hadoop  4096 Apr 18 14:11 sbin
    drwxr-xr-x. 3 hadoop hadoop  4096 Apr 18 14:10 share

    bin：可執(zhí)行文件
    etc：配置文件
    sbin： shell腳本  啟動(dòng)，關(guān)閉hdfs，yarn服務(wù)

7. # 刪除txt文件
    [hadoop@hadoop002 hadoop]$ rm -f *.txt

8. 配置hadoop用戶(hù)的ssh信任關(guān)系
    [hadoop@hadoop002 hadoop]$ ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa
    Generating public/private rsa key pair.
    Created directory '/home/hadoop/.ssh'.
    Your identification has been saved in /home/hadoop/.ssh/id_rsa.
    Your public key has been saved in /home/hadoop/.ssh/id_rsa.pub.
    The key fingerprint is:
    29:06:4c:1a:a4:74:b4:f4:c2:43:8a:07:68:00:fa:e4 hadoop@hadoop002 
    The key's randomart image is:
    +--[ RSA 2048]----+
    |Bo+=.         |
    |=+*=o         |
    |=.+=o.        |
    | =  o.   .      |
    |  E   o S      |
    |     . .         |
    |                 |
    |                 |
    |                 |
    +-----------------+

    [hadoop@hadoop002 hadoop]$ cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
    [hadoop@hadoop002 hadoop]$ chmod 0600 ~/.ssh/authorized_keys

    [hadoop@hadoop002 hadoop]$ ssh hadoop002 date  # 第一次回車(chē)需要輸入yes
    The authenticity of host 'hadoop002 (192.168.90.164)' can't be established.
    RSA key fingerprint is 3a:51:6d:9b:94:d3:91:bf:fd:ab:da:0a:5b:8c:f2:6c.
    Are you sure you want to continue connecting (yes/no)? yes
    Warning: Permanently added 'hadoop002 ,192.168.90.164' (RSA) to the list of known hosts.
    Tue May 22 10:46:32 CST 2018

[hadoop@hadoop002 hadoop]$ ssh hadoop002 date # 不需要回車(chē)輸入yes，即OK
    Tue May 22 10:46:37 CST 2018

9. 配置文件
    [hadoop@hadoop002 hadoop]$ pwd
    /opt/software/hadoop
    [hadoop@hadoop002 hadoop]$  vi etc/hadoop/core-site.xml
    
        
            fs.defaultFS
            hdfs://localhost:9000
        
    


    [hadoop@hadoop002 hadoop]$  etc/hadoop/hdfs-site.xml

   
        dfs.replication
        1
   

10.   格式化 和 啟動(dòng)
    # 添加環(huán)境變量
    [hadoop@hadoop002 hadoop]$ vi etc/hadoop/hadoop-env.sh
    export JAVA_HOME=/usr/java/jdk1.8.0_11

    #查看hdfs命令的路徑
    [hadoop@hadoop002 hadoop]$ which hdfs
    /opt/sofeware/hadoop/bin/hdfs

    #格式化
    [hadoop@hadoop002 hadoop]$  bin/hdfs namenode -format
    ........
    18/04/19 12:53:42 INFO common.Storage: Storage directory /tmp/hadoop-hadoop/dfs/name has been successfully formatted.

    # 啟動(dòng)hdfs
    [hadoop@hadoop002 hadoop]$ sbin/start-dfs.sh
    Starting namenodes on [192.168.90.164]
    192.168.90.164: namenode running as process 15421. Stop it first.
    localhost: datanode running as process 15523. Stop it first.
    Starting secondary namenodes [0.0.0.0]
    0.0.0.0: secondarynamenode running as process 15717. Stop it first.

    #通過(guò) jps查看進(jìn)程
    [hadoop@hadoop002 hadoop]$ jps
    15523 DataNode
    15717 SecondaryNameNode
    15421 NameNode
    16877 Jps

11  修改dfs啟動(dòng)的進(jìn)程以hadoop002啟動(dòng)
    #修改前的啟動(dòng)
    [hadoop@hadoop002 hadoop]$ sbin/start-dfs.sh 
    Starting namenodes on [192.168.90.164]
    192.168.90.164: starting namenode, logging to /opt/software/hadoop-2.8.1/logs/hadoop-hadoop-namenode-localhost.localdomain.out
    localhost: starting datanode, logging to /opt/software/hadoop-2.8.1/logs/hadoop-hadoop-datanode-localhost.localdomain.out
    Starting secondary namenodes [0.0.0.0]
    0.0.0.0: starting secondarynamenode, logging to /opt/software/hadoop-2.8.1/logs/hadoop-hadoop-secondarynamenode-localhost.localdomain.out


    配置 在hadoop002上啟動(dòng)
    三個(gè)進(jìn)程：
    namenode：         192.168.90.164    bin/hdfs getconf -namenodes    etc/hadoop/core-site.xml        
    datanode：         localhost         using default slaves file     etc/hadoop/slaves  # 修改slaves文件里面的內(nèi)容為hadoop002
    secondarynamenode： 0.0.0.0


    #修改文件：etc/hadoop/core-site.xml:
    
       
            fs.defaultFS
            hdfs://hadoop002:9000
        
    

    # 修改slaves文件里面的內(nèi)容為hadoop002
    [hadoop@hadoop002 hadoop]$ vi  etc/hadoop/slaves
    #localhost
    hadoop002

    #修改文件：etc/hadoop/hdfs-site.xml 
    添加：
    
       
                dfs.namenode.secondary.http-address
                hadoop002:50090 #默認(rèn)是0.0.0.0:50090
        
        
                dfs.namenode.secondary.https-address
                hadoop002:50091#默認(rèn)是0.0.0.0:50091
        
    

    0.0.0.0 是一個(gè)非常特殊的IP，代表的是當(dāng)前機(jī)器的IP。

    # 配置好后，再次啟動(dòng)
    [hadoop@hadoop002 sbin]$  sbin/start-dfs.sh
    Starting namenodes on [hadoop002]
    hadoop002: starting namenode, logging to /opt/software/hadoop-2.8.1/logs/hadoop-hadoop-namenode-hadoop002.out
    hadoop002: starting datanode, logging to /opt/software/hadoop-2.8.1/logs/hadoop-hadoop-datanode-hadoop002.out
    Starting secondary namenodes [hadoop002]
    hadoop002: starting secondarynamenode, logging to /opt/software/hadoop-2.8.1/logs/hadoop-hadoop-secondarynamenode-hadoop002.out

    到這里就安裝完成了。

到此，相信大家對(duì)“hadoop偽分布式的安裝步驟”有了更深的了解，不妨來(lái)實(shí)際操作一番吧！這里是創(chuàng)新互聯(lián)網(wǎng)站，更多相關(guān)內(nèi)容可以進(jìn)入相關(guān)頻道進(jìn)行查詢(xún)，關(guān)注我們，繼續(xù)學(xué)習(xí)！

網(wǎng)站題目：hadoop偽分布式的安裝步驟-創(chuàng)新互聯(lián)
本文鏈接：http://bm7419.com/article30/gggso.html

成都網(wǎng)站建設(shè)公司_創(chuàng)新互聯(lián)，為您提供建站公司、軟件開(kāi)發(fā)、品牌網(wǎng)站設(shè)計(jì)、網(wǎng)站導(dǎo)航、域名注冊(cè)、品牌網(wǎng)站制作

廣告

聲明：本網(wǎng)站發(fā)布的內(nèi)容（圖片、視頻和文字）以用戶(hù)投稿、用戶(hù)轉(zhuǎn)載內(nèi)容為主，如果涉及侵權(quán)請(qǐng)盡快告知，我們將會(huì)在第一時(shí)間刪除。文章觀點(diǎn)不代表本網(wǎng)站立場(chǎng)，如需處理請(qǐng)聯(lián)系客服。電話：028-86922220；郵箱：631063699@qq.com。內(nèi)容未經(jīng)允許不得轉(zhuǎn)載，或轉(zhuǎn)載時(shí)需注明來(lái)源： 創(chuàng)新互聯(lián)

小程序開(kāi)發(fā)