Data Categories

Learn about data categories and how to create a new data category.

Overview

A data category represents a dataset loaded into Visier that runs on a unique data load frequency. Each data category produces its own data version and, if there are multiple data categories, each is joined to the primary data version at runtime. If there are multiple data categories, one is the primary data category and the others are supplementary data categories. A data category is a set of instructions to generate a data version. These instructions within a data category bring together the mappings from source data to objects in Visier's data model. For example, a mapping connects your organization's source data about your employees to Visier's Employee subject.

A data category defines how to release data versions to production, such as auto-release latest or manual release. To change a data category's release behavior, see Release Behavior.

Multiple data categories

Tenants commonly have at least two data categories: a default tenant data category for HR data and a usage data category that runs data versions about user actions in Visier.

Your tenant may have additional data categories, called supplementary data categories, to load other data alongside your primary data category, such as loading work productivity data alongside your HR data. With supplementary data categories, you can generate data versions at different frequencies, such as monthly updates for HR data and weekly updates for work productivity data. With this approach, your users will see the latest data version for each of the data categories in the solution experience.

Supplementary data categories can be useful in situations where you're adding new data, but you don't have access to the source files for the primary data category, such as if you have access to work productivity data, but not HR data. This approach lets you generate a data version that includes multiple data sources without requiring that you gain access to sensitive data.

You can designate whether a data category is primary or supplementary with the Supplementary data setting. For more information, see Create a data category.

If you're adding new data to your tenant, you must decide which data category the data mappings belong to. To make that decision, consider the following questions:

  • How often will the data be processed in Visier? If your data will be processed at the same frequency as an existing data category, use the existing data category.
  • Is there a reason to separate the data into a different data category? If you don't have a reason to separate your data, keep the data together in the same data category.

Example: Applicant data

Let's say that you already have Employee data in Visier, and now you're adding Applicant data. The Employee data is part of the Tenant data category and has a scheduled job every day. You know that you want to process your Applicant data daily in Visier, and you can't think of a reason to separate Applicant and Employee data into different categories for processing. In this scenario, you will assign your Applicant mappings to the Tenant data category.

Example: Engagement Survey data

Alternatively, let's say you already have Employee data in Visier, and now you're adding Engagement Survey data. Like the previous example, the Employee data is part of the Tenant data category and has a scheduled job every day. You don't get new survey data daily, so there's no reason to process survey data in Visier daily. If you added Engagement Survey mappings to the Tenant data category, your old survey data would be processed daily, causing your scheduled jobs to take longer. Instead, you want to process survey data quarterly to align with the frequency at which you receive new survey data. In this scenario, you can create a new data category to process survey data separately from Employee data. When a user views employee and survey data in Visier, the different data versions are both available to provide the correct data in visualizations and analyses.

To check a data category's current version, in a project, navigate to Data > Data Categories. In the list of data categories, check the Data Version column.

Create a data category

  1. In a project, on the navigation bar, click Data > Data Categories.

  2. Click Create Data Category.
  3. Type a display name and description.
  4. Click Create.

After creating a data category, configure the following:

  • Release behavior. For more information, see Release Behavior.
  • Settings. Navigate to the Settings tab.
    • Auxiliary temporal valid range: If enabled, the loader generates valid ranges (time ranges in which changes may have occurred) for auxiliary mappings based on temporal data. A valid range is a time range that a change could have occurred during.
    • Supplementary data: If enabled, allows data versions created from this data category to be loaded alongside a primary data version. If disabled, the set data category is the primary data category. A tenant can have only one data version from a primary data category active.
    • Tenant setting: Select the name of the setting as specified under Data > Tenant Settings. Visier automatically generates default tenant settings for the first data category created in your project. You must configure these auto-generated settings before proceeding. For more information, see Tenant Settings.
  • Optional: File sets validation. For more information, see Add File Sets to a Job.
  • Analytic object category. If an analytic object should be loaded from a different data category, you can change the Category in an analytic object's Settings. This is necessary for supplementary data sources, otherwise the default Tenant data category will be set. For more information, see Analytic Object Settings.

Next steps

After creating a data category and configuring its settings, do the following based on your use case:

  • If you are conducting an initial data load, the next step is to create and configure mappings to specify how the data in your source is loaded into Visier. Mappings connect the source files' columns to the properties in Visier's analytic objects. For more information, see Add a Mapping.
  • If you are uploading data files that have been previously uploaded, for example, Employee data, and you have added columns, the next step is to configure the existing Employee mapping. For more information, see Add a Mapping.
  • If you are uploading data files that have been previously uploaded without any additional columns, you have the option to run a job to generate a data version in the project. You can then preview your changes in the solution. For more information, see Run a Job.

    Alternatively, if your data loads are stable for this type of data, you may want to schedule your jobs to automate the data version generation process. For more information, see Schedule a Job.

Separate subject properties into different data categories

When you have a data source that offers additional properties or dimensions for an existing subject, rather than including all of it in a single data category, you can improve performance by separating these into distinct supplementary data categories. This allows for asynchronous loading of some columns, significantly reducing overall load times.

The columns required in the supplementary data are:

  • A property to link to your primary data category. For example, EmployeeID.
  • EventDate.
  • Properties to be added to the primary data category.

Note: Only use this method to introduce properties that don't already exist in your primary data category. Aside from a linking property like EmployeeID, including the same property in your supplementary data category may interfere with the data in your primary data category.

Enable supplementary data

To begin, create a new data category. Assuming you've already loaded the source containing the necessary properties for your new data category, you can then integrate these properties into your existing data model.

  1. Navigate to your new data category and click under Actions > Edit.
  2. Navigate to the Settings tab.
  3. Turn on the Supplementary data toggle.

Your new data category will include properties not included in the original data category. Now you must map your new data category to the appropriate subject.

Mapping your supplementary data category

  1. In a project, on the navigation bar, click Data > Mappings > Create Mapping.
  2. Select your newly created data category and enter a display name.
  3. Set the Mapping type to Regular.
  4. Select your source containing the extended properties to include in your data category.
  5. Select your target subject. Note that this supports Employee, Requisition, and Applicant subjects.
  6. Click Create.

Your properties from your supplementary data source will now be included in your primary data category.

Set the release behavior of supplementary data categories

In the event of multiple data versions of your supplementary data categories, you can configure which data version you will use in a job. The most recent data version will be used by default. For more information, see Release Behavior. To set the supplementary data category to use a data version that is not the most recent:

  1. In a project, click Data > Data Categories.
  2. Navigate to your supplementary data category and click under Actions > Edit.
  3. Navigate to the Settings tab.
  4. Turn on the Supplementary Data toggle.
  5. Navigate to the Release tab.
  6. Set the correct release behavior by expanding Release behavior and select Manual release.
  7. In Base project, set the correct data version by expanding Data version.
  8. Click Update.

Your data version has now been specified and will remain set until changed to a different data version, or the release behavior is reset to Auto-release latest.

Load an event across data categories

You can create new events on the Employee subject and load data from a supplementary data category. This is particularly useful when:

  • Event data comes from a different source system.
  • You require more frequent data updates. For example, if your primary data category updates monthly, you can configure the event to load daily. This ensures timely insights without skewing core metrics between your main data loads.
  • You need to achieve faster data loading and processing. This optimizes load times, which can significantly benefit your system's performance, especially if your primary data loads take a considerable amount of time.

The following assumes the main Employee subject already has data loaded in the primary data category.

Upload and configure a file

The following is an example of correct formatting of a data file:

EmployeeID

EventDate

Movement Type

1234

2020-01-01

Promotion

1235

2020-01-05

Demotion

1236

2020-01-15

Promotion

Only the EmployeeID and EventDate properties are needed for the event to be properly added to the subject. Any additional properties can be included provided they do not duplicate existing properties in the primary data category.

After formatting your existing data file:

  1. Upload the file with EmployeeID and EventDate columns that match the existing Employee file schema.
  2. Create a new source from this new data file. For more information, see Create a Source.

As some file name formatting will change with future uploads, checking the correct regex for your file name will reduce the chances of your files improperly loading in the future. To check and configure your regex:

  1. On the global navigation bar, click Data > Sources.
  2. Search for your new source and click your new file.
  3. In Settings > File regex, update the regex to account for potential changes in the file name formatting in future uploads. For example changing Employee_Movement_Type.csv to Employee_Movement_Type*.csv would account for any future version numbers being added to the original file name formatting. For more information, see File regex.
  4. Click Save Changes.

Create a new project and analytic object

Create a new project that includes your new event. When choosing a subject to link to, choose the subject appropriate for your use case. For more information, see Create a Project.

  1. In your new project, on the navigation bar, click Model > Analytic Objects > Create Analytic Object.
  2. In Analytic Object type, select Event.
  3. Enter a display name.
  4. Expand Subject and select the relevant subject, for example Employee.
  5. Click Create.

Create and configure a new data category

After creating a project, you must create a new supplementary data category for your event:

  1. In a project, on the navigation bar click Data > Data Categories.
  2. Open your new data category.
  3. In Settings, turn on the Supplementary data toggle.
    1. Optional: In Tenant settings the default Tenant value will use the current primary data category settings. If you want the new event to start and end on another date, you can create a different Tenant setting specifically for the new data category. For more see Tenant Settings.
  4. In the navigation bar click Model > Analytic objects.
  5. Click your new analytic object.
  6. In Settings, expand Data category and select your newly created data category.

Now your event will be pointed to the new data category, and not the same default data category as the Employee subject.

Map your new supplementary data category

To be sure the data is mapped correctly and will not result in a failed data version, you must create two separate mappings for your new source:

  1. A mapping that will map your source to the new event.
  2. A mapping that will map your source to the Employee target.

This is to ensure Visier recognizes the correct overlay event from the source file. To create your first mapping from your created event to your new event target:

  1. In a project, on the navigation bar, click Data > Mappings > Create Mapping.

  2. Set the data category to the newly created data category.

  3. Expand Source and click the newly created source.

  4. Expand Target and select your newly created target.

  5. Complete other fields in Create Mapping as needed. For more information see Add a Mapping.

  6. Click Create.

After creating your mapping, be sure to set your EmployeeID and EventDate properties to those found in your file. To create a mapping from your source to the employee target:

  1. In a project, on the navigation bar, click Data > Mappings > Create Mapping.
  2. Set the data category to the newly created data category.
  3. Expand Source and click the newly created source.
  4. Expand Target and select Employee.
  5. Complete other fields in Create Mapping as needed. For more information see Add a Mapping.
  6. Click Create.

Run a job to create new data version

To verify that the changes made will add an event to your new data category you must generate a new data version. For more information, see Run a Job.

After your job has completed you can check that your data is available in your new analytic object. Click Model > Analytic Objects > Changes. Your new analytic object will have data available.