System Configurations

This section plays a critical role in ensuring that the information system is properly configured, secure, and efficient. This section covers several areas, including Snowflake configuration, Databricks configuration, and retention policies. DataBee provides the flexibility to select the data lake that suits your preferences and requirements. Snowflake and Databricks are used for storing and analyzing large volumes of data. This section provides a detailed guide on configuring your data lake of choice, Snowflake, or Databricks in DataBee. The retention policy section involves defining the retention policies for search history stored in the system.

Click on the settings icon at the top right corner of the portal and from the dropdown menu select System.

On the left panel, click the Data Lakes button. A list of the available data lakes will be displayed. Choose your preferred data lake from the list. A red crossed icon indicates the data lake is disabled, while a green check mark signifies it's enabled. Ensure to disable an existing data lake (if enabled) before switching to a different one.

Snowflake Configuration

You can load and search your security data into a cloud-based data warehousing platform like Snowflake using DataBee. Click on the settings icon at the top right corner of the portal and from the dropdown menu select System. Select the Snowflake button which takes you to the "Snowflake Configuration" page. You can create a new Snowflake connection in DataBee by entering the details for the following connection parameters.

• Account Identifier: Your account identifier e.g., companyname.us-east-1

• Username: Your Snowflake username for authentication (The database name is case sensitive)

• Database: Database to use for backend storage

• Warehouse: Compute resource cluster to use for this application

• Role: User account role to use when interacting with the Snowflake cluster

• Private Key: The private key for key pair authentication

• Private Key Password: Snowflake password for the key pair authentication

Snowflake Case Sensitivity

Many of these fields are required to be uppercase in Snowflake. If you run into issues connecting, try with an uppercase Username, Database, Warehouse, and Role.

Broken Connection

If any of these fields are changed in Snowflake (ex. a user is renamed) then the DataBee connection will break. Any changes to the Snowflake identifiers also requires updating this connection in the DataBee UI.

Click on the Test Connectivity button to ensure that DataBee can successfully connect to your Snowflake account. If the test is successful, click on the Submit button to save the Snowflake connection details in DataBee.

Databricks

Using DataBee, you can load and search your security data into a cloud-based data warehousing platform like Databricks. Click on the settings icon at the top right corner of the portal and from the dropdown menu select System. On the left panel, click the Data Lakes button. Select the Databricks button which takes you to the "Databricks Configuration" page. You can create a new Databricks connection in DataBee by entering the details for the following connection parameters.

• Server Hostname: Databricks server hostname

• Client ID: Databricks OAuth Client ID

• Client Secret: Databricks OAuth Client Secret

• Catalog: Unity Catalog name

• HTTP Path: Databricks compute resources URL

Click the Test Connectivity button to ensure that DataBee can successfully connect to your Databricks account. If the test is successful, click the Submit button to save the Databricks connection details in DataBee.

If you wish to change the Client ID, refer to the steps to change the service principal in Step 4: Create a Unity Catalog.

If you wish to switch to a different data lake, locate and click on the Disable button. 

Retention Policy

The system configuration section of DataBee allows you to set the retention policy for your search history. This policy determines how long your search history will be stored in the system before it is automatically deleted. You can configure the retention policy based on a specific number of days.

Click on the settings icon at the top right corner of the portal and from the dropdown menu select System. Select the Retention button which takes you to the “Retention Policies” page. Here you can enter the number of days you want to retain user Search History. Click on the Submit button to configure your retention policy.

Entity Resolution

The system configuration section of DataBee allows you to set configuration options to customize Entity Resolution to your organization if you have the Security Threats or the Security Hygiene package. Entity Resolution should be enabled and configured after completion of feed configuration to ensure accuracy. Entity Resolution allows you to exclude feeds from its learnings with the “Provider Exclusion List”. Feeds can be prioritized for Entity Resolution in the “Provider Priority” by Selecting a provider from the list, then dragging into the desired order. These inputs are leveraged directly by Entity Resolution when collisions or conflicting information is present. Entity Resolution allows for configuration of age out by 3 primary types of entities: Virtual Devices, Physical Devices, and Users. Virtual Devices take their definition from the Device Type ID in the OCSF framework, which includes for example cloud assets and virtual machines. All other device types are considered Physical Devices for consistency. Internal CIDRs and Internal Hostnames can be provided. CIDRs should be inputted in the format of X.X.X.X/Y in a comma separated list. Providing the CIDR Blocks assigned to your organization allows Entity Resolution to track external IPs as internal assets from a variety of sources. Providing Internal Hostnames allows for more accurate identification of assets.

Click on the settings icon at the top right corner of the portal and from the dropdown menu select System. Select the Entity Resolution button which takes you to the “Entity Resolution” page. Here you can configure the “Provider Exclusion List”, “Provider Priority”, Age Out by entity type, and provide internal CIRD Blocks. Click on the Submit button to configure your entity resolution.

Refer Entity Views for more details.