這篇文章給大家分享的是有關Amabari怎樣搭建Hadoop集群的內(nèi)容。小編覺得挺實用的,因此分享給大家做個參考,一起跟隨小編過來看看吧。
創(chuàng)新互聯(lián)建站專注于大冶企業(yè)網(wǎng)站建設,成都響應式網(wǎng)站建設公司,商城網(wǎng)站制作。大冶網(wǎng)站建設公司,為大冶等地區(qū)提供建站服務。全流程按需求定制網(wǎng)站,專業(yè)設計,全程項目跟蹤,創(chuàng)新互聯(lián)建站專業(yè)和態(tài)度為您提供的服務
一、系統(tǒng)環(huán)境
1.系統(tǒng)版本
[root@manager ~]# cat /etc/centos-release CentOS Linux release 7.4.1708 (Core)
2.主機規(guī)劃
主機名 | IP地址 | 角色 |
manager | 192.168.10.131 | ambari-server |
vm1 | 192.168.10.128 | agent |
vm2 | 192.168.10.129 | agent |
vm3 | 192.168.10.130 | agent |
3.軟件版本
ambari-2.4.3.0-centos7.tar.gz HDP-2.5.3.0-centos7-rpm.tar.gz HDP-UTILS-1.1.0.21-centos7.tar.gz
二、環(huán)境準備(所有節(jié)點上都要配置)
1.修改主機名
2.防火墻和selinux
[root@manager ~]# systemctl status firewalld ● firewalld.service - firewalld - dynamic firewall daemon Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled; vendor preset: enabled) Active: inactive (dead) Docs: man:firewalld(1) [root@manager ~]# getenforce Disabled [root@manager ~]#
3.配置hosts文件
[root@manager ~]# cat /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 192.168.10.128 vm1 192.168.10.129 vm2 192.168.10.130 vm3 192.168.10.131 manager
4.配置SSH免密碼登陸
5.關閉linux的THP服務
#在GRUB_CMDLINE_LINUX這一行后面添加transparent_hugepage=never,如下 [root@manager ~]# cat /etc/default/grub GRUB_TIMEOUT=5 GRUB_DISTRIBUTOR="$(sed 's, release .*$,,g' /etc/system-release)" GRUB_DEFAULT=saved GRUB_DISABLE_SUBMENU=true GRUB_TERMINAL_OUTPUT="console" GRUB_CMDLINE_LINUX="crashkernel=auto rd.lvm.lv=centos/root rd.lvm.lv=centos/swap rhgb quiet transparent_hugepage=never" GRUB_DISABLE_RECOVERY="true" #執(zhí)行以下命令 [root@manager ~]# grub2-mkconfig -o /boot/grub2/grub.cfg [root@manager ~]# systemctl disable tuned #重啟后顯示如下 [root@manager ~]# cat /sys/kernel/mm/transparent_hugepage/enabled always madvise [never] [root@manager ~]# grep AnonHugePages /proc/meminfo AnonHugePages: 0 kB [root@manager ~]#
6.配置UMASK
[root@manager ~]# umask 0022 [root@manager ~]# umask 0022
7.配置時間同步NTP
[root@manager ~]# yum -y install ntp [root@manager ~]# systemctl start ntpd [root@manager ~]# systemctl enable ntpd
8.安裝JDK并配置JAVA_HOME環(huán)境變量
三、安裝Ambari集群(只在manager上配置)
1.安裝HTTP服務
[root@manager ~]# yum install -y httpd [root@manager ~]# systemctl start httpd [root@manager ~]# systemctl enable httpd
2.安裝MySQL
3.配置MySQL
#mysql有密碼復雜度要求,如果不需要密碼策略在/etc/my.cnf文件中添加如下配置并重啟mysql即可禁用 validate_password = off systemctl restart mysqld #查看mysql的初始默認密碼 [root@manager ~]# cat /var/log/mysqld.log | grep password 2017-12-13T01:49:55.629403Z 1 [Note] A temporary password is generated for root@localhost: 3aYp&Zsqq7Ov #使用初始密碼進入mysql修改密碼 ALTER USER 'root'@'localhost' IDENTIFIED BY '123456'; #在mysql中創(chuàng)建ambari數(shù)據(jù)庫及用戶 CREATE DATABASE ambari CHARACTER SET utf8 ; CREATE USER 'ambari'@'%'IDENTIFIED BY '123456'; CREATE USER 'ambari'@'local'IDENTIFIED BY '123456'; GRANT ALL PRIVILEGES ON *.* TO 'ambari'@'%'; FLUSH PRIVILEGES; #創(chuàng)建hive和其它數(shù)據(jù)庫也類似
4.下載Ambari和HDP
wget -nv http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.4.3.0/ambari-2.4.3.0-centos7.tar.gz wget -nv http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.5.3.0/HDP-2.5.3.0-centos7-rpm.tar.gz wget -nv http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.21/repos/centos7/HDP-UTILS-1.1.0.21-centos7.tar.gz
5.解壓至httpd目錄下
[root@manager ~]# mkdir /var/www/html/ambari [root@manager ~]# mkdir /var/www/html/ambari/HDP-UTILS tar zxvf ambari-2.4.3.0-centos7.tar.gz -C /var/www/html/ambari tar zxvf HDP-2.5.3.0-centos7-rpm.tar.gz -C /var/www/html/ambari tar zxvf HDP-UTILS-1.1.0.21-centos7.tar.gz -C /var/www/html/ambari/HDP-UTILS
6.安裝本地源制作相關工具
[root@manager ~]# yum install -y yum-utils createrepo yum-plugin-priorities [root@manager ~]# vim /etc/yum/pluginconf.d/priorities.conf [main] enabled = 1 gpgcheck=0 [root@manager data]# cd /var/www/html/ambari [root@manager ambari]# createrepo ./
7.配置本地源
[root@manager ~]# cat /etc/yum.repos.d/ambari.repo [ambari-2.4.3.0] name=ambari-2.4.3.0 baseurl=http://manager/ambari/ambari/centos7/ gpgcheck=1 gpgkey=http://manager/ambari/ambari/centos7/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins enabled=1 priority=1 [root@manager ~]# cat /etc/yum.repos.d/hdp.repo [HDP-2.5.3.0] name=HDP baseurl=http://manager/ambari/HDP/centos7/ gpgcheck=1 gpgkey=http://manager/ambari/HDP/centos7/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins enabled=1 priority=1 [HDP-UTILS-1.1.0.21] name=HDP-UTILS baseurl=http://manager/ambari/HDP-UTILS/ gpgcheck=1 gpgkey=http://manager/ambari/HDP-UTILS/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins enabled=1 priority=1
[root@manager ~]# yum clean all [root@manager ~]# yum makecache #查看ambari和hdp的資源庫 [root@manager ~]# yum repolist repo id repo name status HDP-2.5.3.0 HDP 200 HDP-UTILS-1.1.0.21 HDP-UTILS 64 ambari-2.4.3.0 ambari-2.4.3.0
8.安裝ambari和mysql驅動
[root@manager ~]# yum install -y ambari-server [root@manager ~]# yum install -y mysql-connector-java
9.配置ambari
[root@manager ~]# ambari-server setup Using python /usr/bin/python Setup ambari-server Checking SELinux... SELinux status is 'disabled' Customize user account for ambari-server daemon [y/n] (n)? y Enter user account for ambari-server daemon (root):root Adjusting ambari-server permissions and ownership... Checking firewall status... Checking JDK... Do you want to change Oracle JDK [y/n] (n)? y [1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8 [2] Oracle JDK 1.7 + Java Cryptography Extension (JCE) Policy Files 7 [3] Custom JDK ============================================================================== Enter choice (1): 3 WARNING: JDK must be installed on all hosts and JAVA_HOME must be valid on all hosts. WARNING: JCE Policy files are required for configuring Kerberos security. If you plan to use Kerberos,please make sure JCE Unlimited Strength Jurisdiction Policy Files are valid on all hosts. Path to JAVA_HOME: /usr/local/jdk Validating JDK on Ambari Server...done. Completing setup... Configuring database... Enter advanced database configuration [y/n] (n)? y Configuring database... ============================================================================== Choose one of the following options: [1] - PostgreSQL (Embedded) [2] - Oracle [3] - MySQL / MariaDB [4] - PostgreSQL [5] - Microsoft SQL Server (Tech Preview) [6] - SQL Anywhere [7] - BDB ============================================================================== Enter choice (3): 3 Hostname (localhost): Port (3306): Database name (ambari): Username (ambari): Enter Database Password (123456): Configuring ambari database... Copying JDBC drivers to server resources... Configuring remote database connection properties... WARNING: Before starting Ambari Server, you must run the following DDL against the database to create the schema: /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql Proceed with configuring remote database connection properties [y/n] (y)? y Extracting system views... ............. Adjusting ambari-server permissions and ownership... Ambari Server 'setup' completed successfully.
10.使用msyql用戶ambari在ambari數(shù)據(jù)庫中導入數(shù)據(jù)
[root@manager ~]# mysql -u ambari -p123456 mysql> use ambari Database changed mysql> source /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql
11.啟動ambari
[root@manager ~]# ambari-server start #在瀏覽器中輸入ambari地址 http://192.168.10.131:8080
感謝各位的閱讀!關于“Amabari怎樣搭建Hadoop集群”這篇文章就分享到這里了,希望以上內(nèi)容可以對大家有一定的幫助,讓大家可以學到更多知識,如果覺得文章不錯,可以把它分享出去讓更多的人看到吧!
分享名稱:Amabari怎樣搭建Hadoop集群
網(wǎng)站網(wǎng)址:http://bm7419.com/article40/jjceeo.html
成都網(wǎng)站建設公司_創(chuàng)新互聯(lián),為您提供做網(wǎng)站、網(wǎng)站設計、關鍵詞優(yōu)化、、搜索引擎優(yōu)化、網(wǎng)站制作
聲明:本網(wǎng)站發(fā)布的內(nèi)容(圖片、視頻和文字)以用戶投稿、用戶轉載內(nèi)容為主,如果涉及侵權請盡快告知,我們將會在第一時間刪除。文章觀點不代表本網(wǎng)站立場,如需處理請聯(lián)系客服。電話:028-86922220;郵箱:631063699@qq.com。內(nèi)容未經(jīng)允許不得轉載,或轉載時需注明來源: 創(chuàng)新互聯(lián)