Purpose

Setup CloudWatch agent on a Linux box to collect application log so that we can query the logs in CloudWatch web console or cli.

Steps

The official doc is long and tedious as usual, so let me note down my simple steps to setup CloudWatch agent to collect logs.

Download and install the agent

1
2

wget https://s3.amazonaws.com/amazoncloudwatch-agent/ubuntu/amd64/latest/amazon-cloudwatch-agent.deb
sudo dpkg -i -E ./amazon-cloudwatch-agent.deb

This will install the agent to /opt/aws/amazon-cloudwatch-agent/bin

Setup AWS common credentials

I usually run the agent as root, so I put the credentials in root’s home directory.

Create/Edit /root/.aws/credentials and /root/.aws/config

You can refer to your local configs but take note, the profile name should be AmazonCloudWatchAgent

Sample:

1
2
3

[AmazonCloudWatchAgent]
aws_access_key_id=
aws_secret_access_key=

Agent config

Create/Edit /opt/aws/amazon-cloudwatch-agent/bin/config.json

My use case is dead simple, so the config itself should be self-explanatory.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

{
	"agent": {
		"run_as_user": "root"
	},
	"logs": {
		"logs_collected": {
			"files": {
				"collect_list": [
					{
						"file_path": "/var/log/xxx/api/*.log",
						"log_group_name": "xxx-api",
						"log_stream_name": "{hostname}",
						"retention_in_days": -1
					},
					{
						"file_path": "/var/log/xxx/worker/*.log",
						"log_group_name": "xxx-worker",
						"log_stream_name": "{hostname}",
						"retention_in_days": -1
					},
					{
						"file_path": "/var/log/xxx/potato-api/*.log",
						"log_group_name": "xxx-potato-api",
						"log_stream_name": "{hostname}",
						"retention_in_days": -1
					}
				]
			}
		}
	}
}

Start agent

1

sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m onPremise -s -c file:/opt/aws/amazon-cloudwatch-agent/bin/config.json

to check if the service is running:

1

sudo systemctl status amazon-cloudwatch-agent