Mapping Settings

Learn how to configure your mappings.

Overview

Ensure users see the correct data when using Visier by correctly configuring your mappings. Choosing the right mapping configuration depends on how your data is structured and how you want to load your data on an ongoing basis.

Data file type

Data file type determines how Visier treats your file uploads. For example, you can configure your mapping to treat each file upload as a snapshot of all your data at a point-in-time or an incremental update that only includes new and updated data.

Temporal

Choose temporal when taking the snapshot approach and uploading files that contain all your data at a point-in-time.

After loading the above data into a temporal mapping, the records would reflect the following:

• April headcount metric is three (EID101, EID102, and EID103).
• The May headcount is three (EID101, EID102, and EID103).
• The June headcount is two (EID101, EID102).

In the above example, the removal of EID103 is implicit in the data, and not a result of explicit removal from the source. Temporal data is snapshot-style data comprising a set of records at a point in time. This data is wide and includes all the columns in the schema for each record.

Typically, temporal data is loaded period-to-period. Visier compares the incoming data and infers if members of the subject need to be removed for periods of time.

Transactional profile

Choose transactional profile when you plan to upload files that only contain new or changed records.

A transactional profile combines elements of both temporal and transactional log data. It excludes unchanged records while fully capturing each modified record. The data structure is wide, meaning a row is included only if at least one attribute has changed, with the corresponding change timestamp recorded.

Transactional log

Choose transactional log when you plan to upload files that are structured to be tall rather than wide and where each row represents a single property value change.

A transactional log is a journal-style record that captures data changes over a specified interval. It follows a tall format, listing only the attributes that have changed along with their new values. Each entry includes a timestamp indicating when the change occurred.

This file type simplifies tracking modifications by directly recording changed values, reducing the number of records that need to be processed. Since changes are explicitly logged rather than inferred, additional extraction rule formulas are required when importing data in mappings.

The keepSuccessOnly and skipNone rules are used when importing data for transactional logs to keep source data rows distinct when assigning them to different attributes. For more information, see Visier Extraction Language (VEL). The extraction formulas needed to correctly import data in mappings differ according to which data type is being extracted. The following examples use the keepSuccessOnly rule, but the column data type designations match those used with skipNone:

1
keepSuccessOnly(if (column("Attribute") = "Contract_Type") column("Value"))

1
keepSuccessOnly(if (column("Attribute") = "FTE_Factor") doubleColumn("Value")

1
keepSuccessOnly(if (column("Attribute") = "Birth_Date") date(column("Birth_Date"), dateFormat("MM/dd/yyyy")))

Validity range

Choose validity range for data that has a known start and end date. The end date on a validate range mapping is considered exclusive. For example, in a record with an end-date of December 31, 2023, you will need to shift it to January 1, 2024 in your mapping for a valid record on December 31, 2023.

Mapping type

Mapping type is how you load and combine data from multiple sources. For example, you can populate an employee object in Visier with data that is stored in one or more files. These types are specific to subject and overlay analytic objects and do not exist for events.

Regular

Choose a regular mapping type when loading primary data that you consider to be the source of truth for a selected target. Regular mapping types create and remove objects in Visier based on the records that exist in the selected data source. When creating a mapping, regular is the default data type.

Auxiliary

Choose an auxiliary mapping type when you want to augment existing data with data loaded from a different source.

When deciding whether to use a regular or auxiliary mapping setting, it’s important to evaluate the relationship between the new data being added and whether any related data already exists within the system.

Regular mapping is used for the primary data source that determines an object's existence within the application. It is appropriate if the presence of a record in the source indicates that the object exists, while the absence of a record means it does not exist. For example, a source containing general employee data, serving as the main source of employee information, would typically use a regular mapping.

Auxiliary mapping is used to provide supplementary details about entities that are already established through regular mapping. It adds additional information without affecting the existence of an entity. For example, a table storing employee performance data would use auxiliary mapping because it supplements existing employee records but does not introduce a new employee into the system.

Lookup

Choose a lookup mapping type when you want to augment existing data with data loaded from a different source.

You can use this approach to populate the employee objects in Visier with job name and job code data stored in separate files.

Static mapping

A static map is a special key-value pair lookup mapping that is used in mapping extraction formulas. For more information, see lookup.

Static mapping is a special use case to consolidate multiple IDs in different data sources who are the same individual. For example, an HRS migration from an old system to new systems may have different IDs between them. A static map is used to replace the ID at extraction time, as you cannot change ID after the extraction, nor merge separate streams.

Correction

Choose a correction mapping type when you want to overwrite the current data with updates or corrections. To learn more about the correction mapping type, contact your Customer Success Manager.

Data file types by mapping type

The available data file types change with the selected mapping type. The following table shows the data file types that can be used per mapping type.

There are two distinct types of regular events to highlight:

• Conception events: These are journal-style records marking the beginning of a subject's validity interval. For example, an employment start event signifies the creation of an employee profile. While they are loaded like other events, Visier treats them differently as conception events should align with the employee profiling data.
• Termination events: These journal-style records indicate the end of a subject's validity interval. For example, an employee exit event marks the conclusion of an employee profile (a profile termination event). Termination events must align with the employee's removal from the profiling data.

Override behavior

Override behavior is how you specify which records to load when loading data on an ongoing basis. For example, if you load incorrect employee data for January 2024, you can re-upload the correct employee data for January 2024 and configure the override behavior to load the correct employee data and ignore the incorrect employee data.

Visier determines whether a data file is newer or older using snapshot time. By default, snapshot time is the file's upload time, but you can set a different snapshot time in the global workspace in Data > Sources > Uploaded Files. For more information about snapshot time, see How Time Is Represented in a Source.

In Visier, override behavior serves the following purposes:

• Restate data at any point in time without manually excluding previously-uploaded files.
• Group data flexibly.
• Configure override behavior once and never have to configure it again.

You can change a mapping's override behavior at any time without losing files or data.

In each mapping, choose one of the following override behaviors:

• Event date
• Event date and subject member ID
• Record period date
• Record period date and subject member ID

Event date or record period date?

Put simply, you have a choice between grouping data records for overriding by their event date or by a record period date that you assign. Event date override uses the EventDate property as both the record date and the override ID. Record period date override groups records to a specific date, such as the snapshot time, where EventDate is the record date and record period date is the override ID.

After you decide whether to override by event date or record period date, you can then decide whether to override entire files or specific IDs in a file. Overriding by file removes old data of the same event date or record period date and replaces it with the latest file's data, known as remove-and-replace logic. Overriding by subject member ID replaces old data of the same event date or record period date if there is also a matching subject member ID in the old and new files, know as upsert (update or insert) logic. Subject member ID overrides cannot remove previously-sent records, it can only update them.

The following table describes the differences between the override behaviors.

As an override behavior, record period date can do everything that event date does and more. We recommend using record period date because it's more flexible than event date. Event date is limited by its use of EventDate as both record date and override ID.

There are several details about your data files and how you plan to correct data that can help you determine which override behavior is correct for the data.

First, decide if you plan to correct data by sending a complete restatement that contains all records for the period or by only sending records that changed or were created since your last data load.

• If your data is a complete restatement that contains all records for the period, use event date or record period date. For complete restatements, you want Visier to ignore previously-sent files for the same period, which can be achieved by either event date or record period date. We recommend record period date because it can do everything that event date does and more; it's more flexible than event date!
• If your data only corrects records that changed since your last data load, use record period date. For restatements that only include new or changed records, you want Visier to keep previously-sent records for the same period and update or insert records from the latest file.

If you chose record period date, skip this next decision. If you aren't sure whether to use event date or record period date, you can narrow down your options by deciding if you'll need to correct event dates in the future.

• If all records in the file will have the same event date, use event date or record period date. You can override using the event date itself by giving all records in a period the same event date. However, we recommend record period date because it can do everything that event date does and more; it's more flexible than event date!
• If you will correct event dates in your data use record period date. Because each event date in the file will be different, you cannot use event date to identify one group of records as part of the same period, so record period date is your only option.

Finally, decide whether to drop entire files or replace records based on their subject member IDs.

• To drop previously-sent files for the same event date or record period date, use per file. In this scenario, per file removes all previously-sent records for the event date or record period date, allowing you to fully replace the period's data. You don't want to choose a per subject member ID override because it would keep some old records from previously-sent files that you want to remove.
• To replace or add new records for a previously-sent event date or record period date, use per subject instance. In this scenario, per subject member ID updates previously-sent records and inserts any new records from the latest file. You don't want to choose a per file override because it would remove all records from the restated dates that aren't present in the latest file.

If using record period date, you must also decide how to configure the value for record period date.

• If your data is a complete restatement that contains all records for a period of time, such as a month, use lastDayInMonth(dateColumn("EventDate")). This ensures that every file that contains event dates in the same month has the same record period date so that previous files for that month are replaced or dropped.
• If your data is a complete restatement that contains all records for all history, use a static date, such as tenantStartDate. This ensures that every file has the same record period date so that all previous files are dropped.
• If your data only adds new records since your last data load, use the file's upload time: sourceTimestamp. This ensures that every file has a unique record period date so that no previous files are dropped and records are not missed.

Use the following diagram to decide which override behavior to use.

Event date

Ignore records from previous file uploads that have an event date that matches the event date of a record in a newer file upload. Be aware that if you send a new data file with the same event date as a previously-loaded file, only the latest data persists.

Event date and subject member ID

Ignore records from previous file uploads that have an event date and subject member ID that matches the event date and subject member ID of a record in a newer file upload. Be aware that if you send a new data file with the same event date as a previously-loaded file that contains a record with the same subject member ID as a previously-loaded subject member ID, the latest record overrides the older record.

Record period date

Ignore records from previous file uploads that have a record period date that matches the record period date of a record in a newer file upload.

Record Period Date does not have to be a named column in your source file. Instead of a record period column, you can alternatively use the Visier Formula Language to group events based on the event date column; for example, lastDateInMonth(dateColumn("Event Date")). This is useful to load the accuracy of EventDate while also grouping events to represent the period they exist in.

Record period date and subject member ID

Ignore records from previous file uploads that have a record period date and subject member ID that matches the record period date and subject member ID of a record in a newer file upload.

Target group

Use a target group to include supplemental data in a mapping that will be considered separate from the main source. This setting in mappings allows you to incorporate additional data sources alongside existing mappings and avoid override behavior clashes, or to distinguish auxiliary mappings from one another.

A target group serves as an optional label for grouping mappings in regular and lookup scenarios but is required for auxiliary mappings. Use target groups to isolate a specific mapping, or several mappings, for a target. The job compares changes within the same target group, ensuring that data from other mappings is not treated as missing or deleted. If you have completely separate populations between two sources, use different target groups.

If the same subject ID appears in both the new and existing sources, the data will be merged, potentially causing conflicts. To prevent this, ensure that subject IDs are unique across sources. This ensures a seamless transition while maintaining data integrity and system continuity.

Frequently asked questions

How can I do a full historical restatement every data load?

With a full historical restatement, you don't want to load data from previous uploads. For example, use this setup for organizational hierarchies where you only want the current hierarchy or if you send a full history every time you send data, which is common for transactional data.

• Override behavior: Record period date

• EventDate: Set as normal

• Record period date: A static value like tenantStartDate

How can I use only the latest data for each subject instance?

To only take the latest data for each subject instance, you don't want to remove data for other subject instances that were previously sent. For example, use this setup to upload full restatements only for impacted subject instances if your source system contains all history and you frequently make corrections to older records in addition to adding new ones.

• Override behavior: Record period date and subject member ID

• EventDate: Set as normal

• Record period date: A static value like tenantStartDate

How do I set up rolling restatements?

Rolling restatements are when you send the last few periods of data with each regular load. You can send rolling restatements in a single file or multiple files.

• Override behavior: One of:

  • If your data is snapshots: Event date

  • If your data is events: Record period date

• EventDate: Set as normal

• Record period date: The first or last date of the period to which the event belongs

How do I insert data without losing previous files?

Insert-only configuration keeps all records that were ever sent. To keep old records and insert new records, use a unique record period date for each file so that no files ever overlap with the same record period date. For example, use this setup for pay events like Compensation_Payout where you only send new events and fix mistakes by sending a deletion record. For more information about deletions, see Data Load Deletions and Corrections.

• Override behavior: Record period date and subject member ID

• EventDate: Set as normal

• Record period date: A unique value like sourceTimestamp