This section describes how to migrate local logs to Cloud Log Service.

Scenario

Usually, applications output running logs directly to a local file. When O&M personnel query logs, they need to log in to the server and use tools in the operating system to view files, such as more, vim, and grep, and query the log content they need to attend to. In this scenario, since logs are distributed on multiple servers, limitations such as error-prone CLI operations, and server permissions result in low log search efficiency.

With Cloud Log Service, you can gain the following advantages:

Data is stored centrally, eliminating the need to log in to multiple servers for queries, which is particularly important under a microservice architecture;
Quick retrieval of logs, freeing you from cumbersome CLI operations and improving troubleshooting efficiency;
Detect abnormal logs in real time, with alarms available to accelerate fault response.

Before You Begin

You have activated Cloud Log Service;
Your application service has been deployed on an eSurfing Cloud server.

Procedure

Log in to the Cloud Log Service console.
In the Log Project module on the Console Overview page, click on the target log project name.
Under the target log unit, click Data Ingestion . If the log unit is configured for log collection for the first time, the collection configuration wizard will be provided on the page.
Select cloud servers.
If you don't have a host group available yet, perform the following operations:

o Enter the host group name.

o In the list of activated ECSs, select the target ECSs that you need to collect from logs to build a host group for central collection and management.

o Click Next.

If you already have an available host group, click Select from Existing Host Groups , select the target host group, and click Next :
To create a collection configuration, configure the parameters following the instructions below.
When the collection rules are configured, click Save and Enable to create the collection configuration and start collecting logs. Click Save and Do Not Enable and the collection configuration will only be created but will not be applied. You can enable it later on the Data Ingestion Management page.
On the completion page, click Retrieve Log to switch to the log retrieval and analysis page. For details, see Querying and Analyzing Logs.

Description of collection configuration parameters:

Parameter	Description
Collection Rule Name	Name of a collection rule, which is unique within the associated log project.
Collection Path	Set the log directory and file name based on the position of the log on the server. A log path must start with a forward slash (/), where a full name and wildcard characters are supported for directory and file names. For example, ●/var/log/auth.log: indicates the auth.log log file in the /var/log directory; ●/var/log/.log: indicates the log files suffixed with .log in the /var/log directory; ●/var/log/app_/*/.log: indicates the log files suffixed with .log in subdirectories in the app_*format in the /var/log directory.
Collection Policy	● Full: Collect data from the first line of the target file. ● Incremental: Collect data from the target file after the collection configuration is applied. If the target file is not updated at the time when the collection configuration is applied, logs in the file will not be collected.
Split Mode	For the delimiter mode of raw logs, selectSingle-line Full Text.

Cloud Log Service

Migrating Local Logs to Cloud Log Service