API Ingest

Prev Next

DataBee supports a wide range of data feeds and enables effortless data ingestion. API ingestion enables access to real-time, structured, and semi-structured data. The first step is to register DataBee with the external platform. This can be as simple as creating a service user with basic credentials or setting up an API client with policies and permissions on the target platform. Refer to the instructions on the specific data feed you are trying to ingest.

This article walks through the general steps when using API ingestion.

Configure an API Ingest Data Feed

Start by clicking on the Data button and select Data Feeds from the left-hand navigation menu.

Next, click on the +Add New Data Feed button.

Choose your preferred data feed from the list of supported data feeds.

You will now be directed to choose your ingestion method. To fetch data via API’s, click on API Ingest.

You will now be redirected to the "Configure Data Feed" page. Enter the data feed details in the fields provided.

  • Data Feed Name: a user-friendly name for the data feed.

  • Owner Name: the name of the point of contact for the data feed.

  • Owner E-mail: email address of the owner.

  • Include Feed in Entity Resolution: Enable this option to include the feed in the entity resolution process. A checkbox for this field is auto-enabled for all new feeds. You can deselect it to exclude the feed from entity resolution.

    • Full Entity Resolution: Automatically creates and updates entities when they are discovered in the data.

    • Associate Only: Links new events to existing entities without modifying their attributes.

    A checkbox is available for each entity type (user, device, application) to select the desired level of resolution (Full Entity Resolution, Associate Only).

Once you have entered the information, scroll further down to the next section.

Fill in the configuration details in the fields provided, to connect to your API endpoints.

This dialog box may look different depending on the data feed. In the example below, we are configuring ServiceNow CMDB, which uses OAuth2 as the authorization method. The best authorization method will be determined by the feed.

A summary of the fields is described below:

  • API Base URL: this is the base API endpoint that DataBee will call.

  • Event Types: the data feed can provide one or more types of data or events. This field will be prefilled will ALL the events supported.

  • Authorization Method: prefilled

  • Username: enter the username required for API authentication.

  • Password: enter the password associated with the provided username.

  • Client Key / Client Secret: credentials

  • Token URL: the URL to retrieve the access token authorization.

Feed Specific Fields:

  • CMDB Tables: this option is specific for the ServiceNow CMDB feed. It provides an option to select the tables of interest.

  • Filter: Select Include or Exclude to filter specific CMDB tables. Choose None to fetch all classes without any filtering.

  • Set API Query Start and End Time: check this box to define a specific time range for API calls. Configure the Start Time, End Time, and Timezone in the ‘Schedule Daily API Query’ section.

    Note:

    Currently, this feature is supported only for ServiceNow feeds and appears only for feeds that allow time window configuration.

Each data source will have specific instructions on filling in these fields including any <placeholders> in the URLs.

Click Save Draft to save your current changes.


The example shown below displays the Azure AD data feed configuration page. Click the Test Connection button to initiate a connection test. A Connection successful! pop up appears if the test passes. If it fails, a pop up displays the reason for the failure. You can submit the data feed only after a successful connection.

Note

The Test Connection button for the API ingest will be introduced for additional feeds in future updates.

Once complete, click Submit to finalize and complete the configuration process.