You will need permission to update your bucket's Access Policy and Event notification settings.
Amazon Web Services (AWS) makes sharing data through their simple storage service (S3) buckets easy. You will be setting up DataBee to read from your S3 Bucket.
Configure an Amazon S3 Ingest Data Feed
Start by clicking on the Data button and select Data Feeds from the left-hand navigation menu.
Next, click on the +Add New Data Feed button.
Choose your preferred data feed from the list of supported data feeds.
You will now be directed to choose your ingestion method. To fetch data via AWS S3, click on AWS S3.
.png?sv=2022-11-02&spr=https&st=2026-04-30T11%3A17%3A14Z&se=2026-04-30T11%3A31%3A14Z&sr=c&sp=r&sig=Yscuf8b9vUrZIecZVxz7Hv%2BZ5t%2FJcvj8S0texlx1LPs%3D)
You will now be redirected to the "Configure Data Feed" page.
Enter data in the fields below.
Data Feed Name: a user-friendly name for the data feed.
Owner Name: enter a point of contact for the data feed.
Owner E-mail: email address for the contact.
S3 Bucket Name: name of the bucket to ingest data from.
AWS Region: AWS region of the bucket.
Delete objects in the S3 bucket after read: if this field is checked, then the file will be deleted after ingesting the data.
Compression: indicates what type of decompression (if any) should be applied to the objects before reading.
Content Type: indicates how to parse the uncompressed content.
KMS Encryption Key: provide an optional KMS encryption key if the S3 bucket is encrypted.
SNS Topic ARN: provide an optional SNS topic ARN for S3 event notifications.
Include Feed in Entity Resolution: Enable this option to include the feed in the entity resolution process. A checkbox for this field is auto-enabled for all new feeds. You can deselect it to exclude the feed from entity resolution.
Full Entity Resolution: Automatically creates and updates entities when they are discovered in the data.
Associate Only: Links new events to existing entities without modifying their attributes.
A checkbox is available for each entity type (user, device, application) to select the desired level of resolution (Full Entity Resolution, Associate Only).
There are two options for Environment: Production or Staging. This is refers to different areas in the datalake where the feed should store data to.
The Staging environment is a temporary area in the datalake where the feed output and mapping can be evaluated and tested. It will not affect dashboard data.
By selecting Production, the feed data will be available for use within the dashboards and other derivative tables.
Once you have entered the required information, scroll down to the next section.
To generate the S3 bucket/ KMS/ SNS policy click on Get S3 Bucket Policy/ Get KMS Policy/ Get SNS Policy respectively.
Copy the IAM policy and save the policy that you want to apply to your AWS S3 bucket.
If KMS is enabled, copy the KMS policy and save the policy you want to apply to your KMS key.
Apply the recommended KMS policy to the KMS key for your bucket.
Once it’s done, scroll down to the next section..
Click Get SQS ARN to generate the SQS ARN, and then copy it for later use.
Configure your source bucket to emit object-create notifications to the DataBee SQS queue, as described in the AWS Documentation link provided at the bottom of the page.
Click the Test Connection button to initiate a connection test. A Connection successful! pop up appears if the test passes.
If it fails, a pop up displays the reason for the failure. To know more about the error scenarios click here.
You can submit the data feed only after a successful connection.
Click Save Draft to save your current changes.
Once complete, click Submit to finalize and complete the configuration process.
Test Connection Error Scenarios
This section outlines common error scenarios that you can prevent during S3 ingestion onboarding.
Case 1: Missing S3 Event Notification Configuration
Failing to configure S3 bucket event notifications will cause the test connection process to fail. Users must ensure that their S3 bucket is properly configured to emit event notifications before proceeding with the ingestion setup.
The following image shows the error message when the user fails to configure SQS event notification.
.png?sv=2022-11-02&spr=https&st=2026-04-30T11%3A17%3A14Z&se=2026-04-30T11%3A31%3A14Z&sr=c&sp=r&sig=Yscuf8b9vUrZIecZVxz7Hv%2BZ5t%2FJcvj8S0texlx1LPs%3D)
Solution: Navigate to the bucket properties and configure your bucket to emit object-create notifications to DataBee SQS queue.
Case 2: Incorrect S3 Bucket Name
When users enter an incorrect bucket name during feed onboarding, the test connection button will display an error message indicating that the specified bucket cannot be found.
The following image shows the error message:
.png?sv=2022-11-02&spr=https&st=2026-04-30T11%3A17%3A14Z&se=2026-04-30T11%3A31%3A14Z&sr=c&sp=r&sig=Yscuf8b9vUrZIecZVxz7Hv%2BZ5t%2FJcvj8S0texlx1LPs%3D)
Solution: Provide the correct source bucket name.
Case 3: Missing S3 Bucket Policy
When S3 bucket policy is not properly configured, DataBee will be unable to access the bucket contents. This will cause the test connection process to fail with an "access denied on S3 operations" error message.
Solution: Navigate to bucket permissions and apply the IAM policy that was generated through Get S3 Bucket Policy button in UI.
Case 4: Missing or Invalid KMS key
When an S3 bucket is encrypted with a KMS key but no key or an incorrect key is provided during feed onboarding, the test connection will fail with the error message: "Bucket encryption enabled but KMS key not found or invalid key."
Solution: Navigate to the source bucket's Properties → Default Encryption section, copy the Encryption Key ARN, and enter it in the appropriate field on the feed onboarding page.