Index is a data storage structure that can reduce the query time of log data. After the index is configured for a log, the data in the index field can be queried and analyzed accordingly. Index needs to be configured with appropriate field types to return the analysis results you need.

Note:

By default, after you edit an index configuration (operations such as adding, editing, deleting fields, and modifying configuration items), it takes effect after one to two minutes and only takes effect on newly written logs instead of the existing ones. Re-indexing historical logs is currently not supported.

Different index configurations will generate different query and analysis results. Configure the index settings as required. Full-text index and field index do not affect each other.
By default, Cloud Log Service has been set with field indexes for built-in reserved fields. For details, see the reserved built-in indexes.

Index Field Types

Currently, two data type configurations are supported:

Field Type	Description
text	String, case sensitive.
double	Floating point, which can store integers and floating point numbers.

Configuring Index Fields

Log in to the Cloud Log Service console, and click on the target log project name in the Log Project module on the right.
Select the target log unit, click Retrieve Log to enter the retrieval page, and then click Index Configuration in the upper right corner to open the index configuration page.
Click the Add button and enter the field name and field type. The field name must be consistent with the splitting key in the collection rule.
If the log format is JSON, you can click the Batch Add Fields button, and the batch adding page can automatically obtain the latest log and automatically parse its JSON field names and types.
After completing the configuration, click OK . The index will take effect in one to two minutes and will only take effect for new data.

Description of index configuration parameters:

Parameter	Description
Field Name	Name of the collected log field. Can only contain numbers, letters, underlines, hyphens (-), and dots (.), and cannot start with an underline (__), within 64 characters.
Field Type	Field Data Type Currently, it supports double-floating point, and text-string types. Double can also store integer data.
Alias	Reserved field, used later to display a more readable name in scenarios such as log dashboards.

Note:

You can create up to 100 index fields.
An index field name needs to be consistent with the splitting key in the collection rule, otherwise, the index field is invalid.
After you edit the index configuration (operations such as adding, editing, deleting fields, and modifying configuration items), it takes effect after one to two minutes and only takes effect on newly written logs instead of the existing ones. Re-indexing historical logs is currently not supported.
Description
The following describes how to configure indexes using a single-line log and a JSON log as examples.
Single-line Log
The following is an example of a single-line log:

Description

The following describes how to configure indexes using a single-line log and a JSON log as examples.

Single-line Log

The following is an example of a single-line log:

36.111.30.4GET/app/user/login200

If spaces are used as delimiters for configuring collection rules, the logs will be parsed into four fields, and the corresponding keys for the four fields will be "ip", "method", "url", and "status". The collected logs will be:

ip:36.111.30.4method:GET url:/app/user/login status:200

According to this scenario, the following configuration can be used to index the corresponding fields. The index will take effect about one to two minutes after it is saved, and will only take effect on newly written data.

JSON Log

JSON log example:

{"ip":"36.111.30.4","methos":"GET","url":"/app/user/login","status":200,"other":{ "source":"app" }}

If the splitting mode of the configured collection rules is the JSON mode, you can define the batch adding fields in the index configuration as follows:

Note: The maximum number of layers that can be configured for a JSON index is five (similar to k1.k2.k3.k4.k5 ). Index fields exceeding five layers will be discarded.

Reserved Built-in Indexes

All logs come with reserved built-in index fields by default. The prefix of the built-in field declaration starts with a double underline (i.e. __). You cannot create an index field that conflicts with the built-in index field name, and the field name cannot start with a double underline (i.e. __). Description of built-in index fields:

Reserved Field	Field Type	Description
ts	Integer.	Log collection time, used to filter the time range for query.
message	String.	Original log text.
__tag__fileName	String.	Log file path of the collector. The collector automatically fills in this field for the log.
__tag__topic	String.	Log topic, mainly used to distinguish data from multiple scenarios in the same log unit. Currently filled with the name of the collection rule.
__tag__XXX	-	In other scenarios, such as containers, fields such as "__tag__podIp" are included by default.

Cloud Log Service

Index Configuration