Data Flow Visualization

Explore a visual representation of how data flows from your source files to Visier's analytic objects.

Who can use this feature?

Users with this profile:

  • Data Engineer or Data: Read (access level), Simple (view level)

Not sure if you have this feature or capability? Reach out to your administrator.

Overview

Data Flow is an interactive visualization that shows how your data connects to objects in Visier. You can use this visualization to quickly see how your data reaches its final state in a data version, including the mappings that provide instructions for how to process the data and the event stream stages and rules that transform the data.

Example: Let's say you want to change your Employee source, but you're not sure what will be impacted by the change. You can use the Data Flow visualization to find the Employee source, see every mapping that uses the source, every event stream stage and rule that Employee data goes through, and the objects in Visier that are populated by the source. In this example, the Employee source is used in 5 mappings, 19 rules, and populates the Employee subject, the Assignments property, and the Supervisory Hierarchy dimension.

You can access Data Flow in the global workspace or in a project.

  • In the global workspace, on the navigation bar, click Data > Data Flow.
  • In a project, on the navigation bar, click Data > Data Flow.

Explore the visualization

By default, the Data Flow visualization shows all connected sources, mappings, rules, and targets within a selected data category. The visualization doesn't show sources that aren't used in the data category and targets that don't have any mappings.

In the visualization, you can:

  • Search for specific sources, mappings, and targets. To focus on a specific source, mapping, or target, click on the node.
    • If focused on a source, the Data Flow shows the source's columns, related mappings, and related targets.

    • If focused on a mapping or lookup mapping, the Data Flow shows the mapping's source, how the source columns map to properties, rules that use the mapping, and related targets.

      Tip: By default, the Data Flow only shows source columns that are used in mappings and mapped properties. To see unused columns and unmapped properties, click + unused columns or + unmapped properties, as shown in the following screenshot.

    • If focused on a target, the Data Flow shows the target's properties, related sources, and related mappings.
  • Filter by specific criteria for sources, mappings, and targets, such as sources that have files loaded.
  • Set the data category to view the Data Flow for.

From left to right, the Data Flow visualization lists your sources, the mappings that use the source, tables in the normalizer and correction event stream stage, rules that transform data for the target, and targets that use the source.

  1. Toolbar: Adjust the Data Flow visualization.
    • Search: Find sources, mappings, and targets to focus on.
    • Filters: Filter the sources, mappings, and targets by specific criteria, such as sources that have files loaded.
    • Project: The project that the Data Flow is based on (production or draft).
    • Data category: The data category that the Data Flow is based on.
    • Data version: The data version that the targets in the Data Flow are based on.
  2. Sources: The sources that are used in the selected data category. For more information, see Sources.
  3. Mappings: The mappings that use the source. For more information, see Mappings.
  4. Tables in normalizer and correction stage: The object tables from a mapping that go through the Normalizer and Correction stages in the event stream. For more information, see Event stream stages.
  5. Rules stage: The rules that transform the object table data after the Normalizer and Correction stages. For more information, see Rules.
  6. Targets: The targets that have mappings in the selected data category. For more information, see Analytic Objects.

Tip: The Data Flow visualization can contains a lot of information if you have many sources, mappings, and targets. To make the visualization larger, you can zoom in and scroll in any direction as needed.

In addition to investigating your data, you can use Data Flow to easily access different parts of the data. You can change mappings and rules in a project to adjust how your data is processed in Visier.

To quickly open a mapping or rule from the Data Flow visualization:

  1. Select a source, mapping, or rule, and then click the Info button .
  2. Optional: Click the Go to source/mapping/rule button . If you're not already in a project, this opens the production project to Data > Mappings or Data > Rules to the selected mapping or rule.
  3. Optional: Click the Open in new tab button . This opens a new tab in your browser, and opens Data > Mappings or Data > Rules in a project to the selected mapping or rule.

View or download a data version table

While investigating your data, you might want to view a data version table representing all the data in a specific data version. A data version is the result of the data flow; source data is processed, transformed, and in its final state for users to interact with.

Who can use this feature?

Users with this profile:

  • Data Engineer or Data: Read (access level), Simple (view level)

Additionally, users must have the Internal Debugging additional capability and full data access, such as Super Admin Permission. For more information, see Super Admin Permission.

Not sure if you have this feature or capability? Reach out to your administrator.

To view a data version table for a specific target in the Data Flow visualization:

  1. Select a target and then click the More actions button .
  2. Do one of the following:
    1. To view the table in Visier, click View data version table.
    2. To download the table as in CSV format, click Download data version table.
  3. Optional: Set the columns to view or download. If blank, the the table contains all columns.
  4. Optional: Set the subject member IDs to filter by or download. If blank, the table contains all subject members.
  5. Optional: If downloading, define a range of rows to download. If blank, the table contains all rows.
  6. Optional: If downloading, set the maximum number of rows to download. If blank, the table contains all rows.
  7. Optional: Limit the dates to filter by or download.
  8. Optional: Include the actual values for dimension members. If not selected, dimension member values are represented as index numbers to minimize processing time (such as "1" or "2" for Hair Color). If selected, the actual values are displayed (such as "black" or "gray" for Hair Color).
  9. When finished, click Apply or Download.

Use source query

If a source has files loaded, you can access the source query tool from Data Flow. For more information about source query, see Query Source Data.

Who can use this feature?

Users with this profile:

  • Data Engineer or Data: Read (access level), Detailed (view level)

Additionally, users must have full data access to query sources, such as Super Admin Permission. For more information, see Super Admin Permission.

Not sure if you have this feature or capability? Reach out to your administrator.

To open source query:

  1. In Data > Data Flow, find a source that has files loaded.

    Tip: Filter the visualization:

    1. Click Filters.
    2. Click Advanced.
    3. In Sources, select Has files loaded.
    4. Click Apply.
  2. On the source, click the More actions button > Query source.
  3. In the Query dialog, write a query to run against the source.