大數據

Ambari搭建

Ambari 跟 Hadoop 等開源軟件一樣,也是 Apache Software Foundation 中的一個項目,並且是頂級項目。就 Ambari 的作用來說,就是創建、管理、監視 Hadoop 的集群,但是這裡的 Hadoop 是廣義,指的是 Hadoop 整個生態圈(例如 Hive,Hbase,Sqoop,Zookeeper 等等),而並不僅是特指 Hadoop。用一句話來說,Ambari 就是為了讓 Hadoop 以及相關的大數據軟件更容易使用的一個工具。

安裝說明
安裝ambari前先安裝好 Centos 7並更新。要確保之前沒有安裝過Ambari,並且沒有搭建過hadoop環境,均是新安裝的系統,未做任何設置。
配置SSH免密登陸
不做詳細介紹,運維必會技能
查看selinux狀態,並關閉

vi /etc/sysconfig/selinux

SELINUX=disabled
製作本地源
由於GFW問題,需在本機**單獨下載安裝

關閉防火牆

yum install httpd
systemctl start httpd.service
systemctl enable httpd.service

安裝製作本地源工具

yum install yum-utils createrepo

下載安裝資源

ambari下載資源

http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.6.1.0/ambari.repo
http://public-repo-1.hortonworks.com/ambari/centos7/2.x/updates/2.6.1.0/ambari-2.6.1.0-centos7.tar.gz

hdp下載資源

http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.6.5.0/hdp.repo
http://public-repo-1.hortonworks.com/HDP/centos7/2.x/updates/2.6.5.0/HDP-2.6.5.0-centos7-rpm.tar.gz
http://public-repo-1.hortonworks.com/HDP-UTILS-1.1.0.22/repos/centos7/HDP-UTILS-1.1.0.22-centos7.tar.gz
在httpd網站根目錄,默認是即/var/www/html/,創建目錄repository(自定義名字,自己根據情況命名),並且將下載的壓縮包解壓到/var/www/html/ambari目錄。

tar -zxvf ambari-2.6.1.0-centos7.tar.gz -C /var/www/html/repository/
tar -zxvf HDP-2.6.5.0-centos7-rpm.tar.gz -C /var/www/html/repository/
tar -zxvf HDP-UTILS-1.1.0.22-centos7.tar.gz -C /var/www/html/repository/

配置ambari、HDP、HDP-UTILS的本地源

vi ambari.repo

[ambari-2.6.1.0]
name=ambari Version - ambari-2.6.1.0
baseurl=http://prod-pinpoint-01/repository/ambari/centos7/2.6.1.0-143/
gpgcheck=1
gpgkey=http://prod-pinpoint-01/repository/ambari/centos7/2.6.1.0-143//RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1
vi hdp.repo

VERSION_NUMBER=2.6.5.0-292

[HDP-2.6.5.0]
name=HDP Version - HDP-2.6.5.0
baseurl=http://prod-pinpoint-01/repository/HDP/centos7/2.6.5.0-292/
gpgcheck=1
gpgkey=http://prod-pinpoint-01/repository/HDP/centos7/2.6.5.0-292/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1

[HDP-UTILS-1.1.0.22]
name=HDP-UTILS Version - HDP-UTILS-1.1.0.22
baseurl=http://prod-pinpoint-01/repository/HDP-UTILS/centos7/1.1.0.22/
gpgcheck=1
gpgkey=http://prod-pinpoint-01/repository/HDP-UTILS/centos7/1.1.0.22//RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
enabled=1
priority=1

將上面的修改過的源放到/etc/yum.repos.d/下面

mv ambari.repo /etc/yum.repos.d/
mv hdp.repo /etc/yum.repos.d/

更新yum

yum clean all
yum list update
yum makecache
yum repolist

安裝Mysql數據庫

下載repo源

wget http://repo.mysql.com/mysql57-community-release-el7-8.noarch.rpm

安裝rmp包

rpm -ivh mysql57-community-release-el7-8.noarch.rpm --nodeps --force

安裝Mysql

yum install mysql-server

啟動Mysql

systemctl start mysqld

重置root密碼

grep 'temporary password' /var/log/mysqld.log

修改root密碼

mysql -u root -p
mysql> Enter password: (輸入剛才查詢到的隨機密碼)
mysql> SET PASSWORD FOR 'root'@'localhost'= "xxxxxx";
mysql> exit

開放3306

mysql>GRANT ALL PRIVILEGES ON . TO 'root'@'%' IDENTIFIED BY 'xxxxx' WITH GRANT OPTION;
mysql>FLUSH PRIVILEGES;
mysql>exit;

創建ambari數據庫及用戶

create database ambari character set utf8 ;
CREATE USER 'ambari'@'%'IDENTIFIED BY 'xxxx';
GRANT ALL PRIVILEGES ON . TO 'ambari'@'%';
FLUSH PRIVILEGES;

安裝mysql jdbc驅動

yum install mysql-connector-java

安裝ambari

安裝

yum install ambari-server

配置

ambari-server setup

提示是否自定義設置。輸入:y

Customize user account for ambari-server daemon y/n?

ambari-server 賬號。輸入:ambari,默認為root

Enter user account for ambari-server daemon (root):ambariAdjusting ambari-server permissions and ownership...

檢查防火牆,如果關閉則不用操作。

Enter user account for ambari-server daemon (root):ambariAdjusting ambari-server permissions and ownership...

檢查JDK

Checking JDK...
[1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8
[2] Oracle JDK 1.7 + Java Cryptography Extension (JCE) Policy Files 7

[3] Custom JDK

Enter choice (1):3

置JAVA_HOME

WARNING: JDK must be installed on all hosts and JAVA_HOME must be valid on all hosts.
WARNING: JCE Policy files are required for configuring Kerberos security. If you plan to use Kerberos,please make sure JCE Unlimited Strength Jurisdiction Policy Files are valid on all hosts.
Path to JAVA_HOME: /opt/java

配置數據庫

Configuring database...
Enter advanced database configuration y/n? y

選擇數據庫

Configuring database...

Choose one of the following options:
[1] - PostgreSQL (Embedded)
[2] - Oracle
[3] - MySQL / MariaDB
[4] - PostgreSQL
[5] - Microsoft SQL Server (Tech Preview)
[6] - SQL Anywhere

[7] - BDB

Enter choice (1): 3

設置數據庫的具體配置信息,根據實際情況輸入,如果和括號內相同,則可以直接回車。

Enter choice (1): 3
Hostname (localhost):
Port (3306):
Database name (ambari):
Username (ambari):
Enter Database Password (bigdata):
Re-enter password:
Configuring ambari database...

設置數據庫連接屬性,完成配置。

Configuring remote database connection properties...
WARNING: Before starting Ambari Server, you must run the following DDL against the database to create the schema: /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql
Proceed with configuring remote database connection properties y/n? y
Extracting system views...
....ambari-admin-2.6.2.0.155.jar
.......
Adjusting ambari-server permissions and ownership...
Ambari Server 'setup' completed successfully.

將Ambari數據庫腳本導入到數據庫

mysql> use ambari;
Database changed
mysql> source /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql

啟動ambari

ambari-server start

配置nginx轉發到ambari的服務器上,暴露服務

出現登錄界面,默認管理員賬戶登錄, 賬戶:admin 密碼:admin,登陸並修改密碼

登錄成功後出現下面的界面,至此Ambari的安裝成功

點擊上面登錄成功頁面的Launch Install Wizard 按鈕進行集群配置

點擊Add Version添加自己下載的HDP版本

彈出的窗口中指定之前配置的HDP源中HDP版本文件,然後點擊Read Version Info按鈕,確定版本信息

因為HDP安裝包非常大,不建議採用Public Repository,本次安裝採用Local Repository,前邊已經配置完成,此處只需指定為前邊所配置的本地源地址即可,如下圖所示。

設置集群信息,並配置ambari-server ssh私鑰信息(前提是已經完成集群機器ssh的配置)

Leave a Reply

Your email address will not be published. Required fields are marked *