Cloud Log Service supports real-time log collection, with a collector to help you collect logs in various data source scenarios. After a log is collected, you can query and analyze the log data in real time on the Cloud Log Service console.
Configuring Log Sources
Currently, Cloud Log Service supports the ingestion of logs from Linux cloud servers and Cloud Container Service Engine (CCSE).
· Linux cloud server: Configure the path of the Elastic Cloud Server (ECS) logs to be collected into log units, and the collector will collect logs to Cloud Log Service based on the configured collection rules. For details, refer toIngesting ECS–Text Logs.
· CCSE: Collects standard output logs and file logs of CCSE. For details, refer to Ingesting CCSE–Application Logs.
Log Structuring Parsing
Structuring parsing of logs means that Cloud Log Service data is stored on the Cloud Log Service platform as key-value pairs. When the log data is structured, you can perform log retrieval, analysis, and processing based on the specified key-values on the Cloud Log Service console. Currently, the collector provides multiple parsing methods, the details are as follows:
Parsing Methods | Description |
Single-line Full Text | Single-line full text means that the log contains only one line of content. When logs are collected, a line break indicates the end of a log, that is, in log files, one log is separated from another with a line break. The log data itself will not be structured, and the log fields will not be extracted. Each log has a default field (message), and the collector stores log contents in the message. For details, see Log Structuring Parsing" for details. |
Multi-line Full Text | Multi-line full text means that a piece of complete log data may span multiple lines. You need to specify the first-line regular expression for matching. When a certain line of log matches the preset regular expression, it is considered to be the beginning of a log, and the next beginning of a line is considered the end identifier of this log. The log content is also stored in the message field. For details, see Log Structuring Parsing" for details. |
Single-line Regular | Single-line regular mode is used to process structured logs. For logs containing one line of content, you need to specify a regular expression and the collector will extract multiple values from a complete log based on the regular expression. For details, see Log Structuring Parsing" for details |
Multi-line Regular | The multi-line regular mode is used to process structured logs. For logs containing multiple lines of content, you need to specify a line beginning regular expression to match the beginning of the logs, and another regular expression to extract multiple values. For details, see Log Structuring Parsing" for details. |
Single-line Delimiter | The single-line delimiter mode supports splitting a log into multiple values using delimiters to achieve structured processing. This mode is only applicable to single-line logs, and each complete log ends with a line break. For details, see Log Structuring Parsing" for details. |
JSON | Supports parsing JSON logs of the Object type and extracts JSON log content as key-value pairs, that is, the key of the first layer of Object is used as a key and the value of the first layer of Object is used as a value. For details, see Log Structuring Parsing" for details. |