CloudFront CDN speeds up distribution of static and dynamic web content through a worldwide network of data centers. For detailed information refer to the AWS CloudFront's official documentation.
Integration Method: S3
Tables: HTTP Activity (4002)
This integration supports the following events.
Event | Description |
|---|---|
Access logs | It contains detailed information about every user (viewer) request that CloudFront receives. |
Note:
AWS CloudFront doesn’t follow a traditional versioning system. Instead, it is a continuously updated cloud service. As of this document preparation, latest release was on Apr 9, 2024. For detailed information refer to the AWS CloudFront’s update history documentation.
Prerequisites
The user should have access to the AWS portal.
The user should have access to the CloudFront dashboard and access to create an S3 bucket.
The user should have access to the DataBee console.
Configuration Overview
Create S3 Bucket for AWS access logs.
Configure standard logging by updating distributions in AWS CloudFront.
Add AWS CloudFront data feed in the DataBee console with the below parameters.
DataBee Parameter
AWS CloudFront Parameter
S3 Bucket Name
Bucket Name
AWS Region
Bucket Region
Apply the generated Bucket policy from DataBee to S3 Bucket policy.
Create an event notification for bucket with the AWS SQS ARN from DataBee.
AWS CloudFront Configuration
Create S3 Bucket
Sign in to the AWS Management Console and open the Amazon S3 console.
In the navigation bar on the top of the page, choose the name of the currently displayed AWS Region. Next, choose the Region in which you want to create a bucket.
Create Bucket
In the left navigation pane, choose General purpose buckets.
Choose Create bucket. The “Create bucket” page opens.
For Bucket name, enter a name for your bucket. The bucket name must:
Be unique within a partition. A partition is a grouping of regions. AWS currently has three partitions: aws (commercial Regions), aws-cn (China Regions), and aws-us-gov (AWS GovCloud (US) Regions).
Be between 3 and 63 characters long.
Consist only of lowercase letters, numbers, periods (.), and hyphens (-). For best compatibility, we recommend that you avoid using periods (.) in bucket names, except for buckets that are used only for static website hosting.
Begin and end with a letter or number.
For a complete list of bucket-naming rules, see General purpose bucket naming rules.
Configure the Object Ownership, Block Public Access settings, Bucket Versioning, and Tags according to your preferences.
Select the encryption type for Default encryption of object. If you want to encrypt the data with AWS KMS key, then follow Create a KMS key.
Click Create Bucket.
Navigate to Properties of the bucket. Copy the ARN value of bucket for future use.
Navigate to Permissions > Bucket policy of the bucket. Click on the Edit button.
Add the policy below to the destination Amazon S3 bucket policy and replace the following values:
account_ID: replace with the account_id. Click on the top-right profile menu to get the account id.
arn:aws:logs:region:account_ID: replace with the aws region and account_ID.
bucket_name: replace with the bucket name.
{ "Version": "2012-10-17", "Statement": [ { "Sid": "AWSLogDeliveryWrite", "Effect": "Allow", "Principal": { "Service": "delivery.logs.amazonaws.com" }, "Action": "s3:PutObject", "Resource": "arn:aws:s3:::bucket_name/*", "Condition": { "StringEquals": { "s3:x-amz-acl": "bucket-owner-full-control", "aws:SourceAccount": "account-ID" }, "ArnLike": { "aws:SourceArn": "arn:aws:logs:region:account_ID:delivery-source:*" } } } ] }
Click on Save changes.
Configure Standard logging
Login to your AWS account and search for CloudFront service as shown below.
In “Distributions” page you will see the list of distributions. Click on the distribution to enable standard logging.
Click on Logging tab and click on Add button to add standard log destination.
Select Amazon S3 as log destination.
Enter the following details to create standard log destination.
Destination S3 bucket: Select the S3 bucket which is created to forward log.
Field selection: We can select the fields which we want to be present in the access logs.
Partitioning (Optional): You can use partitioning to organize your access logs when CloudFront sends them to your S3 bucket. By default, CloudFront will send data to the AWSLogs/{account-id}/CloudFront path.
Output format: Select JSON.
Click on Submit.
DataBee Configuration
Login to the DataBee UI, navigate to Data > Data Feeds and click the Add New Data Feed button.
Search for AWS CloudFront and click on it as shown below.
Click on the AWS S3 collection method.
In the configuration page, enter feed contact information and confirm the following and click Next.
S3 Bucket Name: Paste the bucket name.
AWS Region: Select the AWS Region in which bucket is created.
Compression: Select gzip.
Content Type: Select JSON Lines.
KMS Encryption Key (optional): Paste the KMS Key if setup the KMS key for bucket.
Copy the IAM policy and KMS policy to apply to your AWS S3 bucket
Copy the policy statement to clipboard.
Go to the Permissions tab in the bucket. Navigate to Bucket policy and click the Edit button and add the bucket policy statement copied from DataBee.
Save the bucket policy.
Click Next.
Setup AWS S3 bucket notification.
Copy the SQS ARN to clipboard.
Go to the Properties tab of the bucket.
Navigate to Event notifications and click Create event notification.
Enter the General configuration details and select the Event type for object creation.
Set the Destination.
Select SQS Queue as Destination.
Select option to Enter SQS queue ARN.
Paste the SQS queue ARN.
Click Save changes.
Click on Test Connection.
Click Submit.
Troubleshooting Tips
If you are facing an error while testing the connection, then make sure the bucket policy is applied properly and check the SQS event notification configuration as described in the steps.