When working with Python applications running on AWS Fargate, it is often necessary to stream logs to CloudWatch for monitoring and troubleshooting purposes. In this article, we will explore three different ways to add a Python logger to stream logs to CloudWatch within Fargate.

Option 1: Using the AWS SDK

The first option is to use the AWS SDK for Python (Boto3) to interact with CloudWatch. Boto3 provides a high-level API for various AWS services, including CloudWatch. To get started, make sure you have the Boto3 library installed:

pip install boto3

Once installed, you can use the following code to configure the logger and stream logs to CloudWatch:

import logging
import boto3

# Configure the logger
logger = logging.getLogger(__name__)
logger.setLevel(logging.INFO)

# Create a CloudWatch Logs client
client = boto3.client('logs')

# Create a log group
log_group_name = '/ecs/my-app'
client.create_log_group(logGroupName=log_group_name)

# Create a log stream
log_stream_name = 'my-app-stream'
client.create_log_stream(logGroupName=log_group_name, logStreamName=log_stream_name)

# Stream logs to CloudWatch
handler = logging.StreamHandler()
handler.setFormatter(logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s'))
logger.addHandler(handler)

# Log a sample message
logger.info('This is a sample log message')

This code sets up a logger with the desired log level (in this case, INFO). It then creates a CloudWatch Logs client using Boto3 and creates a log group and log stream. Finally, it adds a StreamHandler to the logger and logs a sample message.

Option 2: Using the AWS CLI

If you prefer using the AWS Command Line Interface (CLI), you can achieve the same result by executing the following commands:

import subprocess

# Create a log group
subprocess.run(['aws', 'logs', 'create-log-group', '--log-group-name', '/ecs/my-app'])

# Create a log stream
subprocess.run(['aws', 'logs', 'create-log-stream', '--log-group-name', '/ecs/my-app', '--log-stream-name', 'my-app-stream'])

# Stream logs to CloudWatch
subprocess.run(['aws', 'logs', 'put-log-events', '--log-group-name', '/ecs/my-app', '--log-stream-name', 'my-app-stream', '--log-events', '[{"timestamp": 1234567890, "message": "This is a sample log message"}]'])

This code uses the subprocess module to execute AWS CLI commands. It creates a log group, a log stream, and streams a sample log message to CloudWatch.

Option 3: Using a Third-Party Library

If you prefer a more high-level approach, you can use a third-party library like awslogs to simplify the process. First, install the library:

pip install awslogs

Once installed, you can use the following code to configure the logger and stream logs to CloudWatch:

import logging
from awslogs import AWSLogsHandler

# Configure the logger
logger = logging.getLogger(__name__)
logger.setLevel(logging.INFO)

# Add the AWSLogsHandler to the logger
handler = AWSLogsHandler(log_group='/ecs/my-app', log_stream='my-app-stream')
logger.addHandler(handler)

# Log a sample message
logger.info('This is a sample log message')

This code sets up a logger with the desired log level and adds an AWSLogsHandler to the logger. The AWSLogsHandler automatically creates the log group and log stream if they don’t exist and streams logs to CloudWatch.

After exploring these three options, it is clear that using the AWS SDK (Option 1) provides the most flexibility and control over the logging process. It allows you to configure the logger, create log groups and streams, and stream logs to CloudWatch programmatically. While the AWS CLI (Option 2) and third-party libraries (Option 3) offer simpler alternatives, they may not provide the same level of customization.

In conclusion, if you need fine-grained control over the logging process, using the AWS SDK is the recommended option. However, if simplicity and ease of use are more important, the AWS CLI or a third-party library can be viable alternatives.