Create a Source

Each source contains records that conform to a schema and define data for one or more targets, such as analytic objects.

Who can use this feature?

Users with this profile:

  • Data Engineer

Not sure if you have this feature or capability? Reach out to your administrator.

Overview

Data that is sent to Visier must be stored in a source, for example, Employee data is stored in the Employee source. Each time a new Employee data file is sent to Visier, it is added to the Employee source. This article describes the different methods for creating a source.

Create a source through file upload

You can create a source by uploading a new data file for the first time. The source uses the schema defined in the file. If you have previously uploaded the same type of data, for example, Employee data, your data transfer automatically connects to the existing Employee source if the file matches the existing Employee source regex. For more information, see Add or Remove Records from a Source.

  1. Upload a file. For more information, see File Upload Using the Solution.

    Note: New files initially fail source validation because they are not connected to a source.

  2. To create a source, hover over an uploaded file and click the Generate source button in the Actions column.

    Result: A source is created with default source settings. For more information, see Source Settings.

  3. Optional: Repeat the process for additional sources.

Tip: After the source is created, data files for additional time periods may be uploaded. If the file matches the existing source regex, the files that correspond to later periods are automatically associated with each file and its respective source. For more information, see Add or Remove Records from a Source.

Next steps

After your source has been created, do the following based on your use case:

  • If you are conducting an initial data load, the next step is to create a data category for the data you are uploading. A data category represents a dataset loaded into Visier that runs on a unique data load frequency. For more information, see Data Categories.

  • If you are uploading data files that have been previously uploaded, for example, Employee data, and you have added columns, the next step is to configure the existing Employee mapping. For more information, see Add a Mapping.

  • If you are uploading data files that have been previously uploaded without any additional columns, you have the option to run a job to generate a data version in the project. You can then preview your changes in the solution. For more information, see Run a Job.

    Alternatively, if your data loads are stable for this type of data, you may want to schedule your jobs to automate the data version generation process. For more information, see Schedule a Job.

Clone an existing source

You can create a new source by cloning an existing source. The new source will be identical to the original source, including its settings and included or excluded data transfers.

Cloning a source allows you to test changes to the source itself. Source cloning supports source-related changes without impacting a live production environment. For example, you might want to test the impact of changing the data types associated with your Employee source. By creating a clone of the Employee source, you can safely configure changes to the source itself in a project with no impact to your end users.

To clone a source, do the following:

  1. On the global navigation bar, click Data > Sources.
  2. Select the source that you want to clone.
  3. In the upper-right corner, click Clone.
  4. In the Clone Source dialog, type a display name and description.
  5. Optional: If you want to connect the source's mappings from the original source to the cloned source, select Create new project with updated mappings for the cloned source. This creates a project that contains the updated mappings.
  6. Optional: If you want to clone the source's connected source files, select Clone the connected source files. Depending on the number of files, this may take a while. This runs a job to clone the files. You can check the progress of the job in the Jobs room in a project. After the job completes, the cloned files are connected to the cloned source.
  7. Click Clone.

Create a source through data connectors

You can automatically generate a source associated with a data connector. For more information, see Data In Connectors.

Manually create a source

You can create a source and define its columns and settings manually.

  1. On the global navigation bar, click Data > Sources.
  2. Click Create Source.

  3. Type a display name and description.

  4. Select the new source in the list.
  5. To add columns for the source, click Columns > Create Column.
    1. In the dialog, enter a display name and select a data type.
    2. To make this column mandatory in data files, enable the Mandatory toggle.
    3. To allow empty values for this column in data files, enable the Allow Empty toggle.
    4. Click Create.
  6. To change the source settings, click Settings and make change to the available settings. For more information, see Source Settings.

Note: Don't forget to Save your changes!