4

「开源摘星计划」Loki实现Harbor日志的高效管理

 2 years ago
source link: https://blog.51cto.com/lidabai/5450757
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
neoserver,ios ssh client

「开源摘星计划」Loki实现Harbor日志的高效管理

原创

本文已参与「​ ​开源摘星计划​​​」，欢迎正在阅读的你加入。 ​​​​​ ​​​

在Harbor的

使用Loki实现Harbor日志的管理，在Grafana页面查询Harbor的日志。

已通过Helm在Kubernetes集群中安装Harbor，Harbor服务部署在Harbor命名空间中。

Helm部署Harbor的文档请阅读：

Helm部署Harbor,实现高可用的镜像仓库（超详细分享）~后附踩坑记录

​ ​https://blog.51cto.com/lidabai/5195706​​

[root@master1 harbor]# kubectl -n harbor  get  pods
NAME                                    READY   STATUS    RESTARTS   AGE
harbor-chartmuseum-68f68d88ff-nq97s     1/1     Running   0          2m52s
harbor-core-7565bfb9cc-w5k29            1/1     Running   0          2m52s
harbor-database-0                       1/1     Running   0          2m52s
harbor-jobservice-6747b486df-vm9zs      1/1     Running   0          2m52s
harbor-nginx-956fff988-r8dpx            1/1     Running   0          2m52s
harbor-notary-server-776f85f9c6-4nlsn   1/1     Running   0          2m52s
harbor-notary-signer-7f895d59d6-k5957   1/1     Running   0          2m52s
harbor-portal-5766b784c7-sndmz          1/1     Running   0          2m52s
harbor-redis-0                          1/1     Running   0          2m52s
harbor-registry-6b88cfb465-fcxvn        2/2     Running   0          2m52s
harbor-trivy-0                          1/1     Running   0          2m52s
[root@master1 harbor]# kubectl -n harbor  get  svc
NAME                   TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                       AGE
harbor                 NodePort    10.98.71.137     <none>        80:30002/TCP,4443:30004/TCP   3m5s
harbor-chartmuseum     ClusterIP   10.102.194.148   <none>        80/TCP                        3m5s
harbor-core            ClusterIP   10.100.87.174    <none>        80/TCP                        3m5s
harbor-database        ClusterIP   10.100.79.72     <none>        5432/TCP                      3m5s
harbor-jobservice      ClusterIP   10.111.33.230    <none>        80/TCP                        3m5s
harbor-notary-server   ClusterIP   10.97.144.222    <none>        4443/TCP                      3m5s
harbor-notary-signer   ClusterIP   10.110.131.62    <none>        7899/TCP                      3m5s
harbor-portal          ClusterIP   10.105.213.145   <none>        80/TCP                        3m5s
harbor-redis           ClusterIP   10.110.18.210    <none>        6379/TCP                      3m5s
harbor-registry        ClusterIP   10.110.129.16    <none>        5000/TCP,8080/TCP             3m5s
harbor-trivy           ClusterIP   10.109.0.155     <none>        8080/TCP                      3m5s
「开源摘星计划」Loki实现Harbor日志的高效管理_云原生
Loki简介
Loki是 Grafana Labs 团队最新的开源项目，是一个水平可扩展，高可用性，多租户的日志聚合系统。它的设计非常经济高效且易于操作，因为它不会为日志内容编制索引，而是为每个日志流编制一组标签。
与其他日志系统不同，Loki 是围绕仅索引有关日志的元数据的想法构建的：标签（就像 Prometheus 标签一样）。然后，日志数据本身会被压缩并以块的形式存储在对象存储（例如 S3 或 GCS）中，甚至本地存储在文件系统中。小索引和高度压缩的块简化了操作并显着降低了 Loki 的成本。
使用Helm部署Loki日志平台
1)创建命名空间
[root@lidabai-master1 ~]#  kubectl create namespace loki
2）安装Helm
    如果你已经安装好helm了，可以忽略本步骤。
[root@lidabai-master1 ~]# wget  https://get.helm.sh/helm-v3.7.2-linux-amd64.tar.gz
[root@lidabai-master1 ~]#  tar zxvf helm-v3.7.2-linux-amd64.tar.gz
[root@lidabai-master1 ~]#  cp linux-amd64/helm  /usr/local/bin/
[root@lidabai-master1 ~]#  helm versio
version.BuildInfo{Version:"v3.7.2", GitCommit:"663a896f4a815053445eec4153677ddc24a0a361", 
GitTreeState:"clean", GoVersion:"go1.16.10"}
3）添加Chart仓库
[root@lidabai-master1 ~]#  helm repo add grafana https://grafana.github.io/helm-charts
[root@lidabai-master1 ~]#  helm repo list
NAME      URL
grafana     https://grafana.github.io/helm-charts
[root@lidabai-master1 ~]#  helm repo update
4)搜索chart
[root@master1 ~]# helm search repo grafana
NAME                                  CHART VERSION  APP VERSION  DESCRIPTION                                       
grafana/grafana                       6.32.1         9.0.2        The leading tool for querying and visualizing t...
grafana/grafana-agent-operator        0.2.2          0.25.1       A Helm chart for Grafana Agent Operator           
grafana/enterprise-logs               2.2.2          v1.4.1       Grafana Enterprise Logs                           
grafana/enterprise-logs-simple        1.2.0          v1.4.0       DEPRECATED Grafana Enterprise Logs (Simple Scal...
grafana/enterprise-metrics            1.9.0          v1.7.0       DEPRECATED Grafana Enterprise Metrics             
grafana/fluent-bit                    2.3.1          v2.1.0       Uses fluent-bit Loki go plugin for gathering lo...
grafana/loki                          2.12.2         v2.5.0       Loki: like Prometheus, but for logs.              
grafana/loki-canary                   0.8.1          2.5.0        Helm chart for Grafana Loki Canary                
grafana/loki-distributed              0.50.0         2.5.0        Helm chart for Grafana Loki in microservices mode 
grafana/loki-simple-scalable          1.6.1          2.5.0        Helm chart for Grafana Loki in simple, scalable...
grafana/loki-stack                    2.6.5          v2.4.2       Loki: like Prometheus, but for logs.              
grafana/mimir-distributed             2.1.0          2.1.0        Grafana Mimir                                     
grafana/mimir-openshift-experimental  2.1.0          2.0.0        Grafana Mimir on OpenShift Experiment             
grafana/oncall                        1.0.2          v1.0.3       Developer-friendly incident response with brill...
grafana/promtail                      6.0.2          2.5.0        Promtail is an agent which ships the contents o...
grafana/rollout-operator              0.1.2          v0.1.1       Grafana rollout-operator                          
grafana/tempo                         0.15.4         1.4.1        Grafana Tempo Single Binary Mode                  
grafana/tempo-distributed             0.20.3         1.4.1        Grafana Tempo in MicroService mode                
grafana/tempo-vulture                 0.2.0          1.3.0        Grafana Tempo Vulture - A tool to monitor Tempo...
「开源摘星计划」Loki实现Harbor日志的高效管理_运维_02
 grafana/loki-stack: 将多个进程封装到一个Pod内，未做数据持久化，适用于测试环境,单体模式。
 grafana/loki-canary: 金丝雀更新模式；
 grafana/loki-distributed:  微服务模式，适合生产较大规模场景；
 grafana/loki-simple-scalable: 读写分离模式，简单可扩展；
下载并安装Loki Chart
  • 下载解压chart
[root@master1 ~]# helm pull  grafana/loki-stack  --untar
[root@master1 ~]# cd loki-stack/
[root@master1 loki-stack]# ls
charts  Chart.yaml  README.md  requirements.lock  requirements.yaml  templates  values.yaml
--untar：下载并解压Chart包；
  • 修改values.yaml配置
[root@master1 loki-stack]# vim values.yaml 
grafana:
  enabled: true    #启用Grafana组件
  sidecar:
    datasources:
      enabled: true
      maxLines: 1000
  image:
    tag: 8.3.5
[root@master1 loki-stack]# helm install loki-stack . -n loki
NAME: loki-stack
LAST DEPLOYED: Thu Jul  7 13:31:02 2022
NAMESPACE: loki
STATUS: deployed
REVISION: 1
NOTES:
The Loki stack has been deployed to your cluster.
Loki can now be added as a datasource in Grafana.
See http://docs.grafana.org/features/datasources/loki/ for more detail.
[root@master1 loki-stack]# helm -n loki ls
「开源摘星计划」Loki实现Harbor日志的高效管理_Harbor_03
查看验证服务
[root@master1 loki-stack]# kubectl -n loki  get  pods
NAME                                  READY   STATUS    RESTARTS   AGE
loki-stack-0                          1/1     Running   0          74s
loki-stack-grafana-69966bcf94-lp4z5   2/2     Running   0          74s
loki-stack-promtail-ckpsh             1/1     Running   0          74s
loki-stack-promtail-g7kft             1/1     Running   0          74s
loki-stack-promtail-gf86b             1/1     Running   0          74s
loki-stack-promtail-jnznc             1/1     Running   0          74s
nfs-provisioner-56fd47fc9c-lktrv      1/1     Running   3          23h
[root@master1 loki-stack]# kubectl -n loki  get  svc
NAME                    TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
loki-stack              ClusterIP   10.102.81.250   <none>        3100/TCP   76s
loki-stack-grafana      ClusterIP   10.103.62.31    <none>        80/TCP     76s
loki-stack-headless     ClusterIP   None            <none>        3100/TCP   76s
loki-stack-memberlist   ClusterIP   None            <none>        7946/TCP   76s
Pod都部署成功！
  • 修改Grafana服务访问方式为NodePort
[root@master1 loki-stack]# kubectl -n loki  edit  svc loki-stack-grafana
spec:
  type: NodePort
「开源摘星计划」Loki实现Harbor日志的高效管理_Harbor_04
Grafana配置
登录Grafana
[root@master1 loki-stack]# kubectl -n loki  get  svc loki-stack-grafana 
NAME                 TYPE       CLUSTER-IP     EXTERNAL-IP   PORT(S)        AGE
loki-stack-grafana   NodePort   10.103.62.31   <none>        80:32555/TCP   4m9s
查看Grafana密码
$ kubectl -n loki get secret loki-stack-grafana  -o jsonpath="{.data.admin-password}" | base64  --decode ; echo
fPPZ19cAnP0J7MYPiU4vkWIcMDlbxlgGJYe4jKYT
「开源摘星计划」Loki实现Harbor日志的高效管理_云计算_05
浏览器登录Grafana
  • 登录地址： k8s节点IP:NodePort
  • 用户名：admin
  • 密码： 刚才查看到的secret解析串
「开源摘星计划」Loki实现Harbor日志的高效管理_Harbor_06
Grafana查看Loki日志
「开源摘星计划」Loki实现Harbor日志的高效管理_云计算_07
然后在输入： {namespace="harbor"} 即可查看Harbor命名空间的所有日志。
「开源摘星计划」Loki实现Harbor日志的高效管理_云原生_08

            

		
        
        
    

    

        
  • 
                
                    
                    1赞
                
            
    • 
            
  • 
                
                    
                    1收藏
                
            
    • 
            
  • 
                
                    
                    评论
                
            
    • 
            
  • 
                
                    
                    分享
                
                
            
    • 
                        
  • 
                
                    
                    举报
                
            
    • 
                    

    

                 
上一篇：Nagios使用手册

                    

    


        

        

            

                report
            

        

    







        
Recommend

        



                

    
About Joyk

    
 





Aggregate valuable and interesting links.

Joyk means Joy of geeK