kubernetes收集Pod⽇志
k8s⽇志收集⽅案
三种⽅案优缺点对⽐
⽅式优点缺点
⽅案⼀:Node上部署⼀个⽇志收集
程序每个Node仅需部署⼀个⽇志收集程序消耗资源少，对应
⽤⽆侵⼊
应⽤程序⽇志需要写到标准输出和标准错误输出，不⽀持
多⾏⽇志
⽅案⼆:Pod中附加专⽤⽇志收集的
容器低耦合
每个Pod启动⼀个收集代理，增加资源消耗，且增加运维
维护成本
⽅案三：应⽤程序直接推送⽇志⽆需额外收集⼯具侵⼊应⽤，增加应⽤复杂度以第⼆种⽅案收集NGINX⽇志为例
搭建Kibana和Elasticsearch
kind: List
apiVersion: v1
items:
-apiVersion: apps/v1
kind: Deployment
metadata:
name: kb-single
spec:
selector:
matchLabels:
app: kb-single
replicas:1
template:
metadata:
labels:
app: kb-single
spec:
containers:
-image: /kibana/kibana:6.4.0
name: kb
imagePullPolicy: IfNotPresent
env:
-name: ELASTICSEARCH_URL
value:"es-single:9200"
ports:
-name: http
containerPort:5601
-apiVersion: v1
kind: Service
metadata:
name: kb-single-svc
spec:
type: NodePort
ports:
-name: http
port:5601
targetPort:5601
nodePort:32601
selector:
app: kb-single
-apiVersion: apps/v1
kind: Deployment
kind: Deployment
metadata:
name: es-single
spec:
selector:
matchLabels:
app: es-single
replicas:1
template:
metadata:
labels:
app: es-single
spec:
containers:
-image: /elasticsearch/elasticsearch:6.4.0 imagePullPolicy: IfNotPresent
name: es
env:
-name: network.host
value:"_site_"
-
name: node.name
value:"${HOSTNAME}"
-name: ping.unicast.hosts
value:"${ES_SINGLE_NODEPORT_SERVICE_HOST}"
-name: cluster.name
value:"test-single"
-name: ES_JAVA_OPTS
value:"-Xms128m -Xmx128m"
volumeMounts:
-name: es-single-data
mountPath: /usr/share/elasticsearch/data
volumes:
-name: es-single-data
emptyDir:{}
-apiVersion: v1
kind: Service
metadata:
name: es-single-nodeport
spec:
type: NodePort
ports:
-name: http
nodeselectorport:9200
targetPort:9200
nodePort:31200
-name: tcp
port:9300
targetPort:9300
nodePort:31300
selector:
app: es-single
-apiVersion: v1
kind: Service
metadata:
name: es-single
spec:
clusterIP: None
ports:
-name: http
port:9200
-name: tcp
port:9300
selector:
app: es-single
访问任意节点IP+32601
收集NGINX⽇志
创建nginx的configmap
apiVersion: v1
kind: ConfigMap
metadata:
name: nginx-config
data:
server {
listen      80;
server_name  localhost;
root  /usr/share/nginx/html;
access_log  /var/log/nginx/host_access.log;          error_log  /var/log/nginx/host_error.log debug;          location / {
root  /usr/share/nginx/html;
index  index.html index.htm;
}
error_page  500 502 503 504  /50x.html;
location = /50x.html {
root  /usr/share/nginx/html;
}
}
创建filebeat-nginx的configmap
kind: List
apiVersion: v1
items:
-
apiVersion: v1
kind: ConfigMap
metadata:
name: filebeat-nginx-config
data:
processors:
- add_cloud_metadata:
- module: system
filebeat.inputs:
-
type: log
paths:
- /var/log/nginx/host_access.log
symlinks: true
tags: ["nginx-access-log"]
output.elasticsearch:
hosts: ['es-single:9200']
indices:
- index: "nginx-access-log-%{+yyyy.MM.dd}"            ains:
tags: "nginx-access-log"
创建nginx
apiVersion: v1
kind: Service
metadata:
name: nginx-service
spec:
type: NodePort
selector:
app: nginx
ports:
-protocol: TCP
port:80
targetPort:80
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx
spec:
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
-image: nginx
name: nginx
imagePullPolicy: IfNotPresent
volumeMounts:
-name: nginx-config
mountPath:"/etc/nginx/conf.d"
-
name: nginx-logs
mountPath:"/var/log/nginx"
-name: filebeat
image: /beats/filebeat:6.4.0
imagePullPolicy: IfNotPresent
args:["-c","/l","-e"]
volumeMounts:
-name: filebeat-config
mountPath:"/l"
subPath: l
-name: nginx-logs
mountPath:"/var/log/nginx"
volumes:
-name: nginx-config
configMap:
name: nginx-config
-name: filebeat-config
configMap:
name: filebeat-nginx-config
-name: nginx-logs
emptyDir:{}