Validate Data

Understand how to validate your data, including the issues that may appear when new data is added to the solution.

Overview

Validation is crucial to confirming the correctness of the data being transmitted to Visier and ensuring that transformations, like business rules, provide accurate insights. For example, when uploading updated employee data, validating those updates ensures that the changes are appearing as intended.

You can validate your data using different approaches:

Proactive validation: Verify data immediately after loading new data or updating metric formulas to catch anomalies before they reach production.
Reactive validation: Investigate data after noticing incorrect metric values or missing data in an analysis. For guidance on how to troubleshoot different data issue scenarios, see Troubleshoot Data Discrepancies.

Use this guide to understand the tools available to validate your data.

Validation notification

To receive notifications about failed data versions or unexpected data, use the following options in Visier.

Mailing list

If there's something wrong with a new data version, users on the mailing list receive an email about the data version. Users receive an email in the following scenarios:

Visier receives a file that fails file validation.
A scheduled data load attempts to process but files are missing.
The data load is not successful for any other reason.

Administrators can specify who should receive data version emails in the mailing list. To add or remove individuals from the email notification list, open Studio and click Settings > Mailing List on the global navigation bar. For more information, see Manage the Mailing List.

Alerts and issues

Note: To enable this feature, contact Visier Technical Support. Administrating tenants cannot use data alerts or issues to validate analytic tenants because issues aren't visible at the analytic tenant level.

To help with validation, administrators can define alerts for new data versions—the data in the solution—that check the data correctness of subject members and metrics.

An alert may be critical or non-critical. You as an administrator can choose the notifications you want to send and whether a data version should fail when an alert occurs. For more information, see Data Alerts Settings. A failed data version isn't released to production unless an administrator makes it visible. For more information, see Make a Failed Data Version Visible. Using alerts to fail a data version allows you to investigate major data concerns in the data version and prevent bad data from releasing to production. If a non-critical alert is triggered, issues are generated in the data version and the recipients listed in the alerts settings are notified.

For more information, see Data Alerts and Data Issues.

Metric validation

Use the following tools to validate metrics in Visier.

Metric lineage

If you run into an unexpected value while analyzing metrics, use the metric lineage tool to validate the data that is used in the context of the visualization. You can inspect the metric to understand how the value is calculated. For more information, see Validate Metric Values Using Metric Lineage.

Validation analyses

Visier provides out-of-the-box validation analyses to help you confirm that your data is correct in Visier. After loading data, you can preview the solution and explore any analysis with "Validation" in the title to validate your data. For more information about analyses, see Analyses. For more information about how to find validation analyses using the search panel, see Search.

Source validation

Comparing your source files or source systems directly against the data processed in Visier is the most straightforward validation method. Use the following tools to validate your data files and sources in Visier.

Source query

Retrieve specific records from your sources to validate the data in Visier, such as finding duplicate records or finding which files contain a specific record. For more information, see Query Source Data.

Mappings preview

Preview the result of the mappings from your source data to objects in Visier. For more information, see Preview records in a mapping.

File validation

Note: Users must have access to all data, such as the Super Admin permission. Full data access is crucial for efficient and accurate validation. To identify where the transformation has gone wrong, you must have visibility to all events passing through the system.

You can investigate file issues with the following tools:

In the global workspace, in Data > Data Transfers, you can review each uploaded file and, if there are issues with a file, download the file with validation details to see the issues for each row.
In a project, in Validation > Debugging > File Validation, you can review the severity of each file issue, the issue category, the issue's RCI code, and more. You can export the file validation table as a CSV file. This file validation tool shows all the file issues that were identified after running a receiving job. For a more complete list of file issues, review your files in the global workspace in Data > Data Transfers described above.

Data version validation

Use the following tools to validate data versions in Visier.

Tip: To learn more, take the Visier University eLearning course: Generating and troubleshooting data versions.

Data version download

To validate data records or troubleshoot data issues, download a data version table for available analytic objects, properties, and dimensions in your current data version as a TSV. For more information, see Data Version Download.

Business rules preview

Business rule preview is a tool used to check the output of a new or modified business rule without having to run a new data version, which greatly reduces time spent during business rule authoring. If there's an issue with one of your business rules, you can preview the business rule to check its output without generating a new data version. For more information, see Preview the business rule.

Debug inspector

The debug inspector allows you to inspect your data records and transformations made to your data throughout the event stream. If you identified an incorrect record in a metric but the value is correct in the source data you sent to Visier, you can use the debug inspector to find out how the record was transformed from your source data into Visier. For more information, see Debug Inspector.

Processing issues viewer

The processing issues viewer is a tool that shows all the event stream loader (ESL) issues generated from running a job. In the Processing Issues room, you can review the severity of each issue, at which stage of the event stream the issue occurred, the issue's RCI code, and more. You can export the processing issues table as a CSV file. For more information, see Processing Issues Viewer.

Tip: Use this tool after an initial data load, particularly for new topics or subjects as it can uncover silent issues that were not intended, such as replacement maps missing values, large numbers appearing in integer columns, or actions on non-existent events, and more.

Data Flow Visualization

Explore a visual representation of how data flows from your source files to Visier's analytic objects. Use this tool during the troubleshooting process to identify which mappings and rules are being applied to the data you’re debugging. This can help you locate where an issue might be occurring in the data transformation process. For more information, see Data Flow Visualization.

Next steps

If you corrected the issues that you found during validation in a project, you can publish the project to production. For more information, see Publish Project Changes.

In this section