Skip to main content
Skip table of contents

Viewing Raw Data

After ingesting raw data from multiple data sources, Lucidum stores the raw data in the data sources database tables.

Use Cases

You might find it helpful to use the data sources tables to:

  • get an overview of the data available for queries, reports, and dashboards

  • make sure that data you need is in the data sources tables

  • determine if you need to configure additional connectors to get additional data

Asset Data Source

The following sections describe the pages about raw data sources for assets.

Page Elements (Assets)

When you click on Data Sources in the left navigation pane, the Assets ingested in the last 3 days page appears.

  1. Data Type. You can view the raw data for assets (Asset Data Sources) or users (User Data Sources).

  2. Asset Data Source. Clicking on a link for an asset data source leads to the Assets in Data Source page, where you can view details about all asset data ingested from the data source.

  3. Sort by. The up arrow and down arrow icons allow you to sort the table, by the selected column. Up arrow sorts in ascending order (smallest to largest or a-z). Down arrow sorts in descending order (largest to smallest or z-a). You can sort the table only by a single column.

  4. Ingested Fields. Clicking on a link for the number of ingested fields leads to the Fields in Data Source page, where you can view details about each field ingested from the selected data source.

  5. Export CSV. You can export the table, saving the filters and sorting, to a .CSV file.

  6. Total from Data Sources. Clicking this link leads to the Assets page for the master tables (select Asset from the left navigation bar). After ingesting raw data from multiple data sources, Lucidum deduplicates, triangulates, and aggregates the data to create enriched records for each asset and user. Lucidum then stores the enriched data in the Lucidum master database tables.

  7. Lucidum # of Fields. Clicking on this link leads to the Fields in Data Source page, where you can view details about each field ingested from all data sources for assets.

The Assets ingested in the last 3 days page displays the following:

  • Asset Data Source. The data source for asset data.

  • Range of Records per Ingestion. The number of records Lucidum ingests from the data source per ingestion.

  • Duplicated. Number of instances of more than one asset with a duplicate value in this field. Lucidum machine-learning resolves this duplicate.

  • Unique Asset. Number of unique assets from the data source.

  • Ingested Fields. Number of fields ingested from the data source.

Assets in Data Source page

If you click on a value in the Asset Data Source column in the Assets ingested in the last 3 days page, the Assets in Data Source page appears. This page displays all the assets that use the selected Data Source. In our example, the data source is Netskope.

The Assets in Data Source page includes the following UI elements:

  1. Sort by. The up arrow and down arrow icons allow you to sort the table, by the selected column. Up arrow sorts in ascending order (smallest to largest or a-z). Down arrow sorts in descending order (largest to smallest or z-a). You can sort the table only by a single column.

  2. Export CSV. You can export the table, saving the filters and sorting, to a .CSV file.

  3. > icon. Leads to the Details page, where you can view all the data, both Lucidum data and all raw data, for a single entry in the table.

The Assets in Data Source page displays the following about each asset:

  • Lucidum Asset Name. The name of the asset as it appears in Lucidum. Lucidum uses machine learning to deduplicate, triangulate, and aggregate records. The Lucidum Asset Name is a result of these machine learning tasks.

  • Data Sources. The connectors associated with an asset record. As more connectors are added to Lucidum, the list of connectors in this field can change.

  • Data Source Details. The raw data collected by the connectors for an asset.

  • Public IP Address. The public facing IP address(es), if any, for the asset.

  • Full Domain Name. Full domain name of the asset.

  • First Time Seen. First time asset found and included in Lucidum.

  • Source Asset Name. Name of the asset assigned by the data source.

  • IP Address. One or more IP addresses assigned to the asset.

  • Last Time Seen. Latest ingestion by Lucidum that updated data for the asset.

  • Lucidum User Name. The user name as it appears in Lucidum. Lucidum uses machine learning to deduplicate, triangulate, and aggregate records. The Lucidum User Name is a result of these machine learning tasks.

  • Source User Name. User name assigned by the data source.

Details page (Assets)

The Details page displays details about all the data for an asset. If an asset is associated with multiple data sources, the Details page displays information about each data source.

  • If an asset is associated with multiple data sources, the Data Source tab of the Details page displays a link for each data source. You can click the link for a data source to view its raw data for the selected asset.

  • When you select the > icon from the Assets in Data Source page, the Details page defaults to the Data Source tab, with the selected data source loaded.

  • The Field and Content table might be different for each data source because each data source supplies different data to Lucidum.

  • In the Details page, you can also view the Lucidum Data Group tab to see the cleaned up and enriched data for an asset.

Fields in Data Source page (Assets)

If you click on a value on a value in the Ingested Fields column in the Data Source page, the Fields in Data Source page appears. This page displays all the fields collected by the selected data source. In our example, the data source is Active Directory.

The Fields in Data Source page displays the following about the selected data source:

  • Sort by. The up arrow and down arrow icons allow you to sort the table, by the selected column. Up arrow sorts in ascending order (smallest to largest or a-z). Down arrow sorts in descending order (largest to smallest or z-a). You can sort the table only by a single column.

  • Field. This column displays the list of fields that Lucidum ingests from the data source.

  • Non-Missing Count. Number of times that Lucidum ingested this field from the selected data source. In the example above, Lucidum ingested the Asset Category from Active Directory 1845 times.

  • Missing Count. Number of times that Lucidum was unable to ingest this field from the selected data source. In the example above, Lucidum was unable to ingest Public IP Address 1511 times.

  • Missing Percentage ( % ). Percentage of Missing Count compared to the total number of times Lucidum has ingested data from the data source.

Note that Non-Missing Count, Missing Count, and Missing Percentage do not directly correlate to the number of Lucidum assets. Lucidum ingests data at least once a day and then updates data for existing assets.

  • Unique Count. Number of times the field was ingested only from this data source.

  • Unique Percentage. Percentage of times the field was ingested only from this data source.

  • Minimum. For numeric fields, the maximum value ingested from the data source.

  • Maximum. For numeric fields, the minimum value ingested from the data source.

User Data Source

The following sections describe the pages about raw data sources for users.

Page Elements (Users)

  1. Data Type. You can view the raw data for assets (Asset Data Sources) or users (User Data Sources).

  2. User Data Source. Clicking on a link for a user data source leads to the users in Data Source page, where you can view details about all user data ingested from the data source.

  3. Sort by. The up arrow and down arrow icons allow you to sort the table, by the selected column. Up arrow sorts in ascending order (smallest to largest or a-z). Down arrow sorts in descending order (largest to smallest or z-a). You can sort the table only by a single column.

  4. Ingested Fields. Clicking on a link for the number of ingested fields leads to the Fields in Data Source page, where you can view details about each field ingested from the selected data source.

  5. Export CSV. You can export the table, saving filters and sorting, to a .CSV file.

  6. Total from Data Sources. Clicking this link leads to the Users page for the master tables (select User from the left navigation bar). After ingesting raw data from multiple data sources, Lucidum deduplicates, triangulates, and aggregates the data to create enriched records for each asset and user. Lucidum then stores the enriched data in the Lucidum master database tables.

  7. Lucidum # of Fields. Clicking on this link leads to the Fields in Data Source page, where you can view details about each field ingested from all data sources for users.

The Users ingested in the last 3 days page displays the following:

  • User Data Source. The data source for user data.

  • Range of Records per Ingestion. The number of records Lucidum ingests from the data source per ingestion.

  • Duplicated. Number of instances of more than one user with a duplicate value in this field. Lucidum machine-learning resolves this duplicate.

  • Unique Asset. Number of unique users from the data source.

  • Ingested Fields. Number of fields ingested from the data source.

Users in Data Source page

If you click on a value in the Users Data Source column in the Users ingested in the last 3 days page, the Users in Data Source page appears. This page displays all the users that use the selected Data Source. In our example, the data source is Active Directory.

The Users in Data Source page includes the following UI elements:

  1. Sort by. The up arrow and down arrow icons allow you to sort the table, by the selected column. Up arrow sorts in ascending order (smallest to largest or a-z). Down arrow sorts in descending order (largest to smallest or z-a). You can sort the table only by a single column.

  2. Export CSV. You can export the table, saving filters and sorting, to a .CSV file..

  3. > icon. Leads to the Details page, where you can view all the data, both Lucidum data and all raw data, for a single entry in the table.

The Users in Data Source page displays the following about each user:

  • Lucidum User Name. The name of the user as it appears in Lucidum. Lucidum uses machine learning to deduplicate, triangulate, and aggregate records. The Lucidum User Name is a result of these machine learning tasks.

  • Data Sources. The connectors associated with a user record. As more connectors are added to Lucidum, the list of connectors in this field can change.

  • Lucidum Asset Name. Asset(s) associated with the user.

  • Data Source Details. The raw data collected by the connectors for a user.

  • First Time Seen. First time user found and included in Lucidum.

  • Last Time Seen. Latest ingestion by Lucidum that updated data for the user.

  • Last Password Set Time. Date and time the current password was defined.

  • Country Code. Country for the user.

  • MAC Address. MAC address for the asset(s) associated with the user.

  • Department. Department associated with the user.

  • Email. The user’s email address.

Details page (Users)

The Details page displays details about all the data for a user. If a user is associated with multiple data sources, the Details page displays information about each data source.

  • If a user is associated with multiple data sources, the Data Source tab of the Details page displays a link for each data source. You can click the link for a data source to view its raw data for the selected user.

  • When you select the > icon from the users in Data Source page, the Details page defaults to the Data Source tab, with the selected data source loaded.

  • The Field and Content table might be different for each data source because each data source supplies different data to Lucidum.

  • In the Details page, you can also view the Lucidum Data Group tab to see the cleaned up and enriched data for a user.

Fields in Data Source page (Users)

If you click on a value on a value in the Ingested Fields column in the Data Source page, the Fields in Data Source page appears. This page displays all the fields collected by the selected data source. In our example, the data source is Active Directory.

The Fields in Data Source page displays the following about the selected data source:

  • Sort by. The up arrow and down arrow icons allow you to sort the table, by the selected column. Up arrow sorts in ascending order (smallest to largest or a-z). Down arrow sorts in descending order (largest to smallest or z-a). You can sort the table only by a single column.

  • Field. This column displays the list of fields that Lucidum ingests from the data source.

  • Non-Missing Count. Number of times that Lucidum ingested this field from the selected data source. In the example above, Lucidum ingested the MAC Address from Active Directory 769 times.

  • Missing Count. Number of times that Lucidum was unable to ingest this field from the selected data source. In the example above, Lucidum was unable to ingest the MAC Address 199 times.

  • Missing Percentage ( % ). Percentage of Missing Count compared to the total number of times Lucidum has ingested data from the data source.

Note that Non-Missing Count, Missing Count, and Missing Percentage do not directly correlate to the number of Lucidum users. Lucidum ingests data at least once a day and then updates data for existing users.

  • Unique Count. Number of times the field was ingested only from this data source.

  • Unique Percentage. Percentage of times the field was ingested only from this data source.

  • Minimum. For numeric fields, the maximum value ingested from the data source.

  • Maximum. For numeric fields, the minimum value ingested from the data source.

JavaScript errors detected

Please note, these errors can depend on your browser setup.

If this problem persists, please contact our support.