本篇文章給大家分享的是有關(guān)基于Centos7系統(tǒng)一鍵部署EFK服務(wù)的示例分析,小編覺(jué)得挺實(shí)用的,因此分享給大家學(xué)習(xí),希望大家閱讀完這篇文章后可以有所收獲,話不多說(shuō),跟著小編一起來(lái)看看吧。
我們提供的服務(wù)有:網(wǎng)站制作、成都做網(wǎng)站、微信公眾號(hào)開(kāi)發(fā)、網(wǎng)站優(yōu)化、網(wǎng)站認(rèn)證、秦淮ssl等。為上千企事業(yè)單位解決了網(wǎng)站和推廣的問(wèn)題。提供周到的售前咨詢和貼心的售后服務(wù),是有科學(xué)管理、有技術(shù)的秦淮網(wǎng)站制作公司最近平臺(tái)EFK版本均作了升級(jí),平臺(tái)采用EFK(ElasticSearch-7.6.2 + FileBeat-7.6.2 + Kibana-7.6.2)架構(gòu)。這里建議三個(gè)組件主次版本保持一致??紤]到服務(wù)器比較多,所以寫(xiě)成 腳本來(lái)批量部署。 |
腳本內(nèi)容大家看一下function就行了..
架構(gòu)
EFK采用集中式的日志管理架構(gòu)
elasticsearch:一個(gè)開(kāi)源分布式搜索引擎,提供搜集、分析、存儲(chǔ)數(shù)據(jù)三大功能。它的特點(diǎn)有:分布式,零配置,自動(dòng)發(fā)現(xiàn),索引自動(dòng)分片,索引副本機(jī)制,restful風(fēng)格接口,多數(shù)據(jù)源,自動(dòng)搜索負(fù)載等。
kibana:可以為L(zhǎng)ogstash 、Beats和ElasticSearch提供友好的日志分析Web 界面,可以幫助匯總、分析和搜索重要數(shù)據(jù)日志。
filebeat:輕量級(jí)日志采集器。需要在每個(gè)應(yīng)用服務(wù)器配置filebeat,來(lái)采集日志,并輸出到elasticsearch。
一鍵部署KIBANA腳本
function install_es7_el7() { echo "" echo -e "\033[33m****************************************************安裝ElasticSearch 7.6.2*****************************************************\033[0m" #action "********初始化JAVA環(huán)境********" /bin/true #install_jdk #下載包 if [ -f /opt/elasticsearch-7.6.2-x86_64.rpm ] && [ -f /opt/elasticsearch-analysis-ik-7.6.2.zip ] ;then echo "*****存在ElasticSearch 7.6.2安裝包,無(wú)需下載*****" else ping -c 4 artifacts.elastic.co >/dev/null 2>&1 if [ $? -eq 0 ];then wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.6.2-x86_64.rpm -O /opt/elasticsearch-7.6.2-x86_64.rpm wget https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v7.6.2/elasticsearch-analysis-ik-7.6.2.zip -O /opt/elasticsearch-analysis-ik-7.6.2.zip else echo "please download ES7 package manual !" exit $? fi fi #安裝es7.6 action "********安裝ElasticSearch 7.6.2服務(wù)********" /bin/true chmod u+x /opt/elasticsearch-7.6.2-x86_64.rpm && rpm -ivh /opt/elasticsearch-7.6.2-x86_64.rpm #建目錄及授權(quán) mkdir -p $ES_HOME/data && mkdir -p $ES_HOME/log chown -R elasticsearch:elasticsearch $ES_HOME && chmod -R 755 $ES_HOME #修改ES配置文件 cp /etc/elasticsearch/elasticsearch.yml /etc/elasticsearch/elasticsearch.yml_bak &>/dev/null cat > /etc/elasticsearch/elasticsearch.yml << EOF cluster.name: es-cluster # 設(shè)定本機(jī)節(jié)點(diǎn)名 node.name: es_node # 設(shè)定集群主節(jié)點(diǎn)清單 cluster.initial_master_nodes: ["es_node"] path.data: ${ES_HOME}/data path.logs: ${ES_HOME}/log bootstrap.memory_lock: false bootstrap.system_call_filter: false # 允許從其它機(jī)器訪問(wèn) network.host: 0.0.0.0 http.port: ${ES_PORT} discovery.zen.ping.unicast.hosts: ["${ES_IP}:${ES_PORT}"] EOF #安裝分詞器:ik-analyzer插件 #默認(rèn)情況下,ES使用內(nèi)置的標(biāo)準(zhǔn)分詞器,對(duì)文本進(jìn)行解析。但是對(duì)于中文,其會(huì)拆解為一個(gè)一個(gè)的漢字,最終失去了分詞的意義,所以安裝分詞器:ik-analyzer插件 action "********安裝ik-analyzer插件********" /bin/true mkdir -p /usr/share/elasticsearch/plugins/ik unzip /opt/elasticsearch-analysis-ik-7.6.2.zip -d /usr/share/elasticsearch/plugins/ik/ &>/dev/null chown -R elasticsearch:elasticsearch /usr/share/elasticsearch/plugins/ && chmod -R 755 /usr/share/elasticsearch/plugins/ sleep 2 #在filebeat配置文件中為索引模板添加默認(rèn)分詞器屬性。未來(lái)新創(chuàng)建的索引均引用此屬性: #setup.template.settings: # index.analysis.analyzer.default.type: "ik_max_word" # index.analysis.analyzer.default_search.type: "ik_max_word" #setup.template.overwrite: true #在Linux終端中執(zhí)行如下命令,為現(xiàn)有所有索引,設(shè)置默認(rèn)分詞器屬性: #curl -X POST "172.16.20.143:9200/_all/_close" #curl -X PUT -H'Content-Type: application/json' 'http://172.16.20.143:9200/_all/_settings?preserve_existing=true' -d '{ # "index.analysis.analyzer.default.type" : "ik_max_word", # "index.analysis.analyzer.default_search.type" : "ik_max_word" #}' #curl -X POST "172.16.20.143:9200/_all/_open" #啟動(dòng)ES并初始化數(shù)據(jù) action "********啟動(dòng)es并初始化數(shù)據(jù)********" /bin/true systemctl daemon-reload && systemctl enable elasticsearch.service systemctl restart elasticsearch.service es_version=`curl -XGET ${ES_IP}:${ES_PORT}` echo -e "\033[33m**************************************************完成ElasticSearch 7.6.2安裝***************************************************\033[0m" cat > /tmp/es7.log << EOF ES服務(wù)器IP: ${ES_IP} ES服務(wù)器端口:${ES_PORT} ES數(shù)據(jù)目錄: ${ES_HOME}/data ES日志目錄: ${ES_HOME}/log ES詳細(xì)信息: ${es_version} EOF cat /tmp/es7.log echo -e "\e[1;31m 以上信息10秒后消失,保存在/tmp/es7.log文件下 \e[0m" echo -e "\033[33m************************************************************************************************************************\033[0m" echo "" sleep 10 }
一鍵部署Elasticsearch腳本
function install_es7_el7() { echo "" echo -e "\033[33m****************************************************安裝ElasticSearch 7.6.2*****************************************************\033[0m" #action "********初始化JAVA環(huán)境********" /bin/true #install_jdk #下載包 if [ -f /opt/elasticsearch-7.6.2-x86_64.rpm ] && [ -f /opt/elasticsearch-analysis-ik-7.6.2.zip ] ;then echo "*****存在ElasticSearch 7.6.2安裝包,無(wú)需下載*****" else ping -c 4 artifacts.elastic.co >/dev/null 2>&1 if [ $? -eq 0 ];then wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.6.2-x86_64.rpm -O /opt/elasticsearch-7.6.2-x86_64.rpm wget https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v7.6.2/elasticsearch-analysis-ik-7.6.2.zip -O /opt/elasticsearch-analysis-ik-7.6.2.zip else echo "please download ES7 package manual !" exit $? fi fi #安裝es7.6 action "********安裝ElasticSearch 7.6.2服務(wù)********" /bin/true chmod u+x /opt/elasticsearch-7.6.2-x86_64.rpm && rpm -ivh /opt/elasticsearch-7.6.2-x86_64.rpm #建目錄及授權(quán) mkdir -p $ES_HOME/data && mkdir -p $ES_HOME/log chown -R elasticsearch:elasticsearch $ES_HOME && chmod -R 755 $ES_HOME #修改ES配置文件 cp /etc/elasticsearch/elasticsearch.yml /etc/elasticsearch/elasticsearch.yml_bak &>/dev/null cat > /etc/elasticsearch/elasticsearch.yml << EOF cluster.name: es-cluster # 設(shè)定本機(jī)節(jié)點(diǎn)名 node.name: es_node # 設(shè)定集群主節(jié)點(diǎn)清單 cluster.initial_master_nodes: ["es_node"] path.data: ${ES_HOME}/data path.logs: ${ES_HOME}/log bootstrap.memory_lock: false bootstrap.system_call_filter: false # 允許從其它機(jī)器訪問(wèn) network.host: 0.0.0.0 http.port: ${ES_PORT} discovery.zen.ping.unicast.hosts: ["${ES_IP}:${ES_PORT}"] EOF #安裝分詞器:ik-analyzer插件 #默認(rèn)情況下,ES使用內(nèi)置的標(biāo)準(zhǔn)分詞器,對(duì)文本進(jìn)行解析。但是對(duì)于中文,其會(huì)拆解為一個(gè)一個(gè)的漢字,最終失去了分詞的意義,所以安裝分詞器:ik-analyzer插件 action "********安裝ik-analyzer插件********" /bin/true mkdir -p /usr/share/elasticsearch/plugins/ik unzip /opt/elasticsearch-analysis-ik-7.6.2.zip -d /usr/share/elasticsearch/plugins/ik/ &>/dev/null chown -R elasticsearch:elasticsearch /usr/share/elasticsearch/plugins/ && chmod -R 755 /usr/share/elasticsearch/plugins/ sleep 2 #在filebeat配置文件中為索引模板添加默認(rèn)分詞器屬性。未來(lái)新創(chuàng)建的索引均引用此屬性: #setup.template.settings: # index.analysis.analyzer.default.type: "ik_max_word" # index.analysis.analyzer.default_search.type: "ik_max_word" #setup.template.overwrite: true #在Linux終端中執(zhí)行如下命令,為現(xiàn)有所有索引,設(shè)置默認(rèn)分詞器屬性: #curl -X POST "172.16.20.143:9200/_all/_close" #curl -X PUT -H'Content-Type: application/json' 'http://172.16.20.143:9200/_all/_settings?preserve_existing=true' -d '{ # "index.analysis.analyzer.default.type" : "ik_max_word", # "index.analysis.analyzer.default_search.type" : "ik_max_word" #}' #curl -X POST "172.16.20.143:9200/_all/_open" #啟動(dòng)ES并初始化數(shù)據(jù) action "********啟動(dòng)es并初始化數(shù)據(jù)********" /bin/true systemctl daemon-reload && systemctl enable elasticsearch.service systemctl restart elasticsearch.service es_version=`curl -XGET ${ES_IP}:${ES_PORT}` echo -e "\033[33m**************************************************完成ElasticSearch 7.6.2安裝***************************************************\033[0m" cat > /tmp/es7.log << EOF ES服務(wù)器IP: ${ES_IP} ES服務(wù)器端口:${ES_PORT} ES數(shù)據(jù)目錄: ${ES_HOME}/data ES日志目錄: ${ES_HOME}/log ES詳細(xì)信息: ${es_version} EOF cat /tmp/es7.log echo -e "\e[1;31m 以上信息10秒后消失,保存在/tmp/es7.log文件下 \e[0m" echo -e "\033[33m************************************************************************************************************************\033[0m" echo "" sleep 10 }
一鍵部署filebeat腳本
function install_filebeat7_el7() { echo "" echo -e "\033[33m****************************************************安裝Filebeat 7.6.2*****************************************************\033[0m" #下載包 if [ -f /opt/filebeat-7.6.2-x86_64.rpm ] ;then echo "*****存在Filebeat 7.6.2安裝包,無(wú)需下載*****" else ping -c 4 artifacts.elastic.co >/dev/null 2>&1 if [ $? -eq 0 ];then wget https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-7.6.2-x86_64.rpm -O /opt/filebeat-7.6.2-x86_64.rpm else echo "please download Filebeat7.6 package manual !" exit $? fi fi #安裝filebeat7.6 action "********安裝filebeat 7.6.2服務(wù)********" /bin/true chmod u+x /opt/filebeat-7.6.2-x86_64.rpm && rpm -ivh /opt/filebeat-7.6.2-x86_64.rpm #修改kibana配置文件 cp /etc/filebeat/filebeat.yml /etc/filebeat/filebeat.yml_bak cat > /etc/filebeat/filebeat.yml << EOF filebeat.inputs: - type: log enabled: true paths: - /lcp_logs/*.log filebeat.config.modules: path: /etc/filebeat/modules.d/*.yml reload.enabled: false setup.template.settings: #number_of_shards 是數(shù)據(jù)分片數(shù),默認(rèn)為5,有時(shí)候設(shè)置為3 index.number_of_shards: 3 index.analysis.analyzer.default.type: "ik_max_word" index.analysis.analyzer.default_search.type: "ik_max_word" setup.template.overwrite: true setup.kibana: host: "${KIBANA_IP}:${KIBANA_PORT}" output.elasticsearch: hosts: ["${ES_IP}:${ES_PORT}"] ilm.enabled: true ilm.rollover_alias: "fsl_uat.prod1" ilm.pattern: "{now/d}-000001" processors: - add_host_metadata: ~ - add_cloud_metadata: ~ EOF #啟動(dòng)filebeat并初始化數(shù)據(jù) action "********啟動(dòng)filebeat并初始化數(shù)據(jù)********" /bin/true systemctl daemon-reload && systemctl enable filebeat.service systemctl restart filebeat.service #nohup ./filebeat -e -c filebeat.yml >/dev/null 2>&1 & echo -e "\033[33m**************************************************完成Filebeat 7.6.2安裝***************************************************\033[0m" cat > /tmp/filebeat7.log << EOF filebeat版本: 7.6.2 filebeat路徑: /usr/bin/filebeat filebeat配置kibana: ${KIBANA_IP}:${KIBANA_PORT} filebeat配置elasticsearch: ${ES_IP}:${ES_PORT} EOF cat /tmp/filebeat7.log echo -e "\e[1;31m 以上信息10秒后消失,保存在/tmp/filebeat7.log文件下 \e[0m" echo -e "\033[33m************************************************************************************************************************\033[0m" echo "" sleep 10 }
以上就是基于Centos7系統(tǒng)一鍵部署EFK服務(wù)的示例分析,小編相信有部分知識(shí)點(diǎn)可能是我們?nèi)粘9ぷ鲿?huì)見(jiàn)到或用到的。希望你能通過(guò)這篇文章學(xué)到更多知識(shí)。更多詳情敬請(qǐng)關(guān)注創(chuàng)新互聯(lián)-成都網(wǎng)站建設(shè)公司行業(yè)資訊頻道。
網(wǎng)站標(biāo)題:基于Centos7系統(tǒng)一鍵部署EFK服務(wù)的示例分析-創(chuàng)新互聯(lián)
本文鏈接:http://bm7419.com/article4/disdoe.html
成都網(wǎng)站建設(shè)公司_創(chuàng)新互聯(lián),為您提供網(wǎng)站營(yíng)銷、手機(jī)網(wǎng)站建設(shè)、Google、搜索引擎優(yōu)化、域名注冊(cè)、定制網(wǎng)站
聲明:本網(wǎng)站發(fā)布的內(nèi)容(圖片、視頻和文字)以用戶投稿、用戶轉(zhuǎn)載內(nèi)容為主,如果涉及侵權(quán)請(qǐng)盡快告知,我們將會(huì)在第一時(shí)間刪除。文章觀點(diǎn)不代表本網(wǎng)站立場(chǎng),如需處理請(qǐng)聯(lián)系客服。電話:028-86922220;郵箱:631063699@qq.com。內(nèi)容未經(jīng)允許不得轉(zhuǎn)載,或轉(zhuǎn)載時(shí)需注明來(lái)源: 創(chuàng)新互聯(lián)
猜你還喜歡下面的內(nèi)容