Validate a Source

Learn how to troubleshoot errors when working with sources.

At any time, you can manually choose to include or exclude data transfers from a source. You might choose to exclude transfers due to bad data. Data transfers that fail record validation are automatically excluded from the source.

To exclude records from a source:

  1. On the global navigation bar, click Data > Sources.
  2. Navigate to Uploaded Files.
  3. In the file list, toggle the files that you want to include or exclude from the source.
  4. Click Save.

If you encounter errors while loading data and using sources, you can troubleshoot to find and correct the issue. The following table describes the error types and their suggested troubleshooting methods.

Error type Troubleshooting method
Failed source validation Investigate the source settings.

Investigate the schema of the source's columns.
Failed record validation Download the data transfer to review its record validation and descriptions.

Failed source validation

If a data transfer failed source validation, you can investigate the issue through the data transfer itself and the source its meant to connect to.

Option One: Validate a data transfer via source settings

If a data transfer encounters a source validation error, you can inspect the data transfer to understand the error. You may need to modify the intended source's settings to ensure the data transfer can connect to the source as intended.

  1. On the global navigation bar, click Data > Data Transfers.
  2. Click the data transfer that has an error.
  3. On the right, in the Info panel, review the details of the error.
  4. If a recent data transfer doesn't connect to the source automatically, navigate to Sources and select the source that the transfer should connect to.
    1. Click Settings and investigate the cause of the error, such as the regex. For more information, see Source Settings.
  5. After the error is corrected, either reassign or re-initiate the data transfer, such as via upload, to connect the transfer to the source.

Example: Data transfer fails source validation

You upload a new CSV file for your Employee source called Employee_20210131_new, but the file does not automatically connect to the established Employee source that you’ve connected data to in the past. To investigate the error, you do the following:

  1. On the global navigation bar, click Data > Sources.
  2. Select the Employee source.
  3. Navigate to Settings.

In Settings, you realize that the regex is Employee_20210131_original\.csv. This regex is too strict to allow the newly uploaded file to be recognized by the source. To fix it, you change the regex to Employee_20210131.*\.csv. This will allow the source to recognize your data transfer. You must click the Reassign button in Data Transfers to connect it to the corrected source.

Option Two: Validate the source schema

You can view the settings for each column for your source and make changes. For example, your data transfer may fail source validation if a column expects a certain data type, but the transfer includes that column with a different data type. By investigating the source settings, you'll more easily understand where the error came from. For more information about the settings, see Source Column Settings.

  1. On the global navigation bar, click Data > Sources.
  2. Navigate to Columns.
  3. Ensure that each column setting is correct, including Data Type, Mandatory, and Allow Empty.

Failed record validation

If a data transfer failed record validation, you can troubleshoot the issue by finding the problematic records and correcting them.

Option One: Download a file for record validation

  1. On the global navigation bar, click Data > Data Transfers.
  2. Do one of the following:
    1. Hover over a data transfer and click the Download button .
      1. In the dialog, select Include record validation status and descriptions in download.
    2. Select a data transfer that contains an error.
      1. In the data transfer page, in the Info panel , click Download File with Validation Details.
  3. Open the downloaded CSV file.
  4. Review the Record Validation Status column for ERROR values and investigate the Status Description column to understand the issue.