a >hT@s^ddlmZddlmZmZmZddlmZmZmZddl m Z m Z m Z Gddde e Z dS))Optional)datetime timedeltatimezone)JSONDecodeErrordumpsloads)PluginIndependentPlugin PluginOptc @seZdZdZdZdZdZdZdZdZ e dd d d e d d d d e dd dd e dde dde dde ddgZ e ee edddZe ee edddZddZddZd d!Zd"d#Zd$d%Zd&S)'Lokia  Collects logs and configuration from Loki. This plugin interacts with the Loki API to fetch logs based on specified labels and provides options for pagination and label detection. It also collects relevant configuration files and masks sensitive information. It works with both charmed and non-charmed Loki. To fetch internal Loki logs, run it from the Loki container. You can also run it from another machine and fetch only logs from Loki API, by providing the following parameters: `-k loki.collect-logs=true -k loki.endpoint=LOKI_URL` Usage: sos report -o loki -k loki.collect-logs=true -k loki.labels=severity:charm -k loki.detect-labels=true -k loki.paginate=true -k loki.endpoint=LOKI_URL z Loki serviceloki)Zservicesid)r collect-logsFzcollect logs from Loki API)defaultdesc detect-labelszTfetch logs for all available labels. May result in multiple files with the same logspaginatez'fetch all available logs from Loki API.labelsz1colon-delimited list of labels to fetch logs from)rZval_typerendpointhttp://localhost:3100zDloki endpoint to fetch logs from. Defaults to http://localhost:3100.)startendc Cs`|sttj}|s"|tdd}|d}|d}d|d|d|d|dtjd }|S) N)Zdaysz%Y-%m-%dT%H:%M:%S.%fZ curl -G -s 'z3/loki/api/v1/query_range' --data-urlencode 'query={z!=~".+"}' --data-urlencode 'start=z' --data-urlencode 'end=z' --data-urlencode 'limit=z' )rnowrZutcrstrftimer LOKI_QUERY_LIMIT)selfrlabelrrZstart_formattedZ end_formattedZcommandr!;/usr/lib/python3.9/site-packages/sos/report/plugins/loki.py query_command<s"   zLoki.query_commandcCsV||||||}zt|dWStyP|d|d|dYS0dS)Noutputz-An error was returned from Loki API on label z-. Error message stored, not querying further.)Zexec_cmdr#rrZ _log_warn)rrr rrr$r!r!r"get_logsOs  z Loki.get_logscCsR|dd}ttd}|D](}|dD]}t|d}t||}q0q$|S)Ndataresultʚ;valuesr)intrr timestampmin)rlogsZ log_streams earliest_logstreamlogr+r!r!r"get_earliest_log_timestamp[s   zLoki.get_earliest_log_timestampc Cs.|||dd}||d"}|t|ddWdn1sH0Yt|tr`dS|r*||}tt d}d}|t j kr*||kr*t |d} |||d| } ||d|"}|t| ddWdn1s0Yt| trdS|}|| }|d7}qdS)Nz.log)indentr(rz.log.r)r%Zcollection_filewriter isinstancestrr1r*rrr+r MAX_PAGINATION_ITERATIONSZ fromtimestamp) rrr rr-Zlogfiler.Zprevious_earliest_logZiterations_countZ log_timestampZnew_logsr!r!r"get_logs_for_labeles60   0 zLoki.get_logs_for_labelcCs|d}|||d}|||d|d|dr|dpRd}g|_|d}rt|tr|r|j|d |d r| d |d }t |d }|j|ddS)Nz/etc/loki/*.yamlz/etc/worker/*.yamlz/var/log/loki/*zpebble logs loki -n 10000rrrr:rrz/loki/api/v1/labels'r$r&) Z path_joinZ add_copy_specZadd_cmd_output get_optionrr5r6extendsplitZcollect_cmd_outputr)rZels_config_fileZcoordinated_workers_config_filerZ labels_optionZ labels_cmdZ labels_jsonr!r!r"setups$          z Loki.setupcCs6|dp d}|jD]}|d}||||qdS)Nrrr)r:rr8)rrr rr!r!r"collects  z Loki.collectcCs^ddg}gd}dd|d}dd|d}|D] }|||d|||dq8dS) NZ access_key_idZsecret_access_key)z /etc/loki/loki-local-config.yamlz/etc/loki/config.yamlz/etc/loki/local-config.yamlz/etc/loki/loki.yamlz/etc/worker/config.yaml(|z)\s*(:|=)(\S*\n.*?\\n)z)\s*(:|=)\s*[a-zA-Z0-9]*z \1\2*********)joinZ do_file_sub)rZ protect_keysZ loki_filesZmatch_exp_multilZ match_expfiler!r!r"postprocsz Loki.postprocN)__name__ __module__ __qualname____doc__Z short_descZ plugin_nameZprofilesrr7Zpackagesr r6Z option_listrrr#r%r1r8r=r>rCr!r!r!r"r sD   r N)typingrrrrZjsonrrrZsos.report.pluginsr r r r r!r!r!r" s