Add File Sets to a Job

Learn how to add file sets to a job.

File sets define the expected numbers of files per source. By adding file sets to your data version job, you enable an additional level of validation support to your data loads.

For example, if you configure the file set validation to expect placement events, hire events, and turnover events, but only send one of those files, the job will fail file validation due to missing files. This provides more confidence that the data version contains a complete data load.

The following screenshot shows the file sets interface in Visier, found in a project at Data > Data Categories.

1. File set. The collection of sources and their expected number of files.
2. Source. A source in the job that you can set an expected number of files.
3. Definition of expected files number. Whether the expected number of files is an exact number, range, minimum number, or maximum number.
4. Number of files. The number of files that Visier expects to receive for the source.
5. Source Optional? toggle. Whether or not the source is optional for data loads. If the source is optional, the data load may still succeed if new data for this source is missing.

Create a new file set for a job

1. Open a project.
2. Navigate to Data > Data Categories.
3. On the job that you want to add file set validation, click the More button > Edit > File Sets Validation.
4. Click Add Fileset.
5. Type a name and description.
6. Click Save.

   Result: A new file set is created and pre-populated with the available mapped sources for the job.

   Note: Validity start date and validity end date are legacy features that are no longer required for file set validation.

7. Define the file set by doing the following:
   1. Select whether the file set is an Exact number, Range, Min, or Max. The default is an exact number based on the files used in the current production version.

      Tip: If your tenant is not auto-releasing data versions, such as if you have daily data loads but do not publish data versions to production daily, Visier may double count files. Use a minimum count to avoid double counting.

   2. Type a number or range of numbers for each source to define the expected files to receive for that source.
   3. Optional: Make the source optional by enabling the Source Optional? toggle.
8. When finished, click Save.