I love AWS, but when I’m debugging issues I prefer the Linux command line over Cloudwatch Logs Insights. Numerous AWS services store their log configuration inside cloudwatch, which presents a small challenge since my tooling (ripgrep, awk, jq, sed, etc.) can’t directly access cloudwatch logs. The aws command line has a nifty get-log-events option which can solve this problem. It allows you to export logs from a log stream, and has several options to control what gets exported.
The following example shows how to export logs from the stream named LOG_STREAM_NAME, but only events that occurred in the past five minutes:
$ aws --profile la --region us-east-1 logs get-log-events --log-group-name LOG_GROUP_NAME --log-stream-name LOG_STREAM_NAME --start-time $(date "+%s%N" -d "5 minutes ago" | cut -b1-13) > /tmp/log
The export will contain one or more JSON objects similar to the following:
{
"events": [
{
"timestamp": 1580312457000,
"message": "I0129 15:40:57.094248 1 controller.go:107] OpenAPI AggregationController: Processing item v1beta1.metrics.k8s.io",
"ingestionTime": 1580312464749
}
}
Get-log-events also has a “create-export-task” option, which can be used to export logs to an S3 bucket. This can be useful for archiving logs for offline debugging. The aws CLI is super useful, and the more I work with it the more I like it!