Data Load Types, Frequency, and Granularity

Learn about the different types of data loads, data load frequency, and time granularity.

Overview

Before you start compiling data to send to Visier, you must determine the data load type, how often you're sending data, and the granularity of the data.

This article helps you answer the questions:

1. Does Visier recommend temporal or transactional data?
2. How often should I send data to Visier?
3. What time granularity is best for my data?

Recommended approach

For most scenarios, we recommend:

1. The first time you load data for an object, send the full history of the data. The "full history" is as far back in your data as you want to be available in Visier. Some organizations start with one year of historical data while others prefer as many as five years of historical data. The full history data load contains a lot of data and typically has longer data load run times.

   Tip: To test your data load process, only use a couple months of data or a subset of the population to save time on processing data in Visier. If something is wrong, at least you didn't wait for all your data to process! After testing the data load process, you can then send the full history that you want in Visier.

2. Automate your data extraction and upload process to send Visier data on a regular schedule. This minimizes the time and labor spent on compiling and sending data to Visier.
3. Send data every day that captures all your data as of the current date, plus a rolling period of three or six months that captures changes to previously-loaded data as of the end of the month. This approach offers easier validation and we recommend it for employee data. With this load approach, you can configure these mapping settings:
   • Data file type: Temporal
   • Override behavior: Record period date. Map the Record_Period_ID property to lastDateInMonth(dateColumn("DateColumnName")), such as lastDateInMonth(dateColumn("EventDate")).
4. If you can't send temporal data, as described in step 3, send data every day that captures the entire history of any record that changed since the last data load. This approach offers higher precision and we recommend it for requisition and applicant data. With this load approach, you can configure these mapping settings:
   • Data file type: Transactional profile
   • Override behavior: Record period date and subject member ID. Map the Record_Period_ID property to a static date, such as tenantStartDate().

This cadence allows you to create analyses that show the most recent data for the current month and, for historical months, show data as of the end of each historical month. For the current month, the most recent daily file replaces the daily file from the previous day. If you're unable to automate the extraction and upload process, we recommend that you send us data once a month at a monthly granularity.

Data file types

In this section, learn the different data file types: temporal, transactional, and correction.

Temporal

Send temporal data to capture all subject members for each time period. If a subject member is present in the January data but missing from the February data, Visier infers that the employee exited in February. For more information, see Data file type.

When sending temporal data, we recommend weekly or monthly time granularity. This ensures your data in Visier reflects the state of your data as of the end of the last week or month, or the current day in the current week or month. You can send data daily with rolling periods to update or restate data to keep your data correct and up-to-date.

We do not recommend daily time granularity for temporal data. To correct the data, you have to restate every single day of data since the data became incorrect. This is time-consuming and expensive. If you want to send your data at a daily time granularity, send transactional data instead.

Transactional

Send transactional data to capture changes that occurred since the last data load. Because transactional data is only a subset of records, Visier cannot infer whether subject members are starting or exiting, like we can with temporal data. In transactional data, it's important to send data that identifies when a record became inactive, such as an exit event or an inactive profile. Transactional data loads are more efficient than temporal because they only include changes since the previous load. For more information, see Data file type.

When sending transactional data, we recommend daily time granularity. You can provide transactional data in different ways, but we recommend that you send the full history for any record that changed since last data load. This ensures your data in Visier reflects the most recent state of your day every day.

Correction

A file that corrects a specific data record. This file is only sent if you cannot correct your data at the source or through rolling periods. It must have a correction mapping associated with it. For more information, see Data Load Deletions and Corrections.

Data load frequency

Data load frequency is how often you send data to Visier. You can send data to Visier as often as you want, such as daily, weekly, monthly, quarterly, or any other schedule that suits you and your data.

• Daily: Keeps your data up-to-date on a daily basis. This is the most common data load frequency.
• Multi-daily: Useful for data that changes many times a day, such as talent acquisition data.
• Weekly: Useful for updating data that isn't time sensitive, such as employee data.
• Monthly: Useful for data that isn't time sensitive or doesn't change regularly, such as employee data or learning data.
• Quarterly: Useful for data that changes very infrequently, such as performance data.

When you send data to Visier, you can optionally send updates to previously-loaded data called rolling periods. We recommend including rolling periods in your regular data loads for temporal data if your business process includes retroactive corrections to data.

To decide how many months of rolling periods are right for your organization, consider how far back a change could be backdated in your systems, then add a month or two to that value. If your organization can backdate data as much as five months ago, consider sending six or seven months of rolling periods to Visier in your regular loads.

Time granularity

Time granularity is the level of detail of the data. In your data, how often you record timestamps for your data determines the time granularity. If you record multiple timestamps per day, the time granularity is Millisecond. If you record one timestamp per month, the time granularity is Month.

Time granularity impacts the time context options when viewing subjects and events in the Explore room and analyses. You can select a time context equal to or larger than the object's defined time granularity. If the granularity for Employee is Month, you can select Months, Quarters, and Years in employee analyses, but not Weeks or Days, which are smaller than Month.

The following screenshot shows the time context options for an analytic object with Day time granularity. Because Day is the smallest time context option, all time context options are available.

Additionally, processing jobs use time granularity to build exclusive dates for analytic objects and to run system rules. If set incorrectly, jobs may fail.

You can set the time granularity per analytic object. If you don't set granularity, the default is Day. Events inherit time granularity from their associated subject. If you change a subject's time granularity, all events that use the subject inherit the new time granularity.

To set data granularity for an analytic object:

1. In a project, on the navigation bar, click Model > Analytic Objects.
2. Select the analytic object for which you want to set granularity; for example, Employee.
3. Click Default Values.
4. In Time granularity, select a value from the list. If the finest granularity in your data is time records at the day level, set the time granularity to Day.

The following table describes the available time granularity values.

Time granularity	Description	Example: Applicant applied on June 4 at 10 AM. The recruiter screened the applicant on June 4 at 1 PM. The applicant qualified on June 4 at 4 PM.
Millisecond	Records multiple timestamps per day. With millisecond granularity, the data represents the state of a record at any point throughout the day. Useful for data that updates many times per day, such as applicant data.	Millisecond granularity records all three states ("Applied", "Screened", "Qualified") for the applicant on June 4.
Second	Records multiple timestamps per day. With second granularity, the data represents the state of a record at any second throughout the day.	Second granularity records all three states ("Applied", "Screened", "Qualified") for the applicant on June 4.
Minute	Records multiple timestamps per day. With minute granularity, the data represents the state of a record at any minute throughout the day.	Minute granularity records all three states ("Applied", "Screened", "Qualified") for the applicant on June 4.
Hour	Records multiple timestamps per day. With hourly granularity, the data represents the state of a record at any hour throughout the day.	Hour granularity records all three states ("Applied", "Screened", "Qualified") for the applicant on June 4.
Day	Records one timestamp per day. With daily granularity, the data represents the state of a record as of the end of the day.	Day granularity records that the applicant's state was "Qualified" on June 4.
Week	Records one timestamp per week. With weekly granularity, the data represents the state of a record as of the end of the week.	Week granularity records that the applicant's state was "Qualified" as of the end of the week.
Week and Month	Records one timestamp per week and one timestamp per month. With week and month granularity, the data represents the state of a record as of the end of each week and the end of the month.	Week and month granularity records that the applicant's state was "Qualified" as of the end of the week and month.
Month	Records one timestamp per month. With monthly granularity, the data represents the state of a record as of the end of the month or the most recent day in the current month. Analyses that use data with monthly granularity show the most recent data for the current month and the data as of the end the month for historical months. Useful for data that updates once per month, such as employee or organization hierarchy data.	Month granularity records that the applicant's state was "Qualified" in June.
Year	Records one timestamp per year. With yearly granularity, the data represents the state of a record as of the end of the year or the most recent day in the current year. Analyses that use data with yearly granularity show the most recent data for the current year and the data as of the end the year for historical years. Useful for data that updates once per year, such as employee performance data.	Year granularity records that the applicant's state was "Qualified" as of the end of the year.

Frequently asked questions

My organization's talent acquisition data is refreshed in real time. Can Visier handle that?

Yes. For real-time data, we recommend that you set the time granularity to millisecond in the analytic object and run a new data load every day. At most, your data will only be up to 24 hours behind. However, this amount of data may cause your data load jobs to run for longer. To improve performance, we recommend that you assign the data to a separate data category from the rest of your data. This ensures that only talent acquisition data is processed every day, not all of your data. For more information, see Multiple data categories.

What's the difference between daily or monthly time granularity? Which should I use?

When data is recorded at a daily time granularity, a record has an updated state every single day. This means that the data you're working with in Visier will change and update every day if you are analyzing current month data. It is more difficult to restate data with daily time granularity because you must restate data for every single day that the data was incorrect.

When data is recorded at a monthly time granularity, a record only changes once a month. This means that mid-month changes will not be seen in your data, such as the Headcount metric, prior to the end of the month in question. This approach supports easy restatements of past data because you only have to restate data for the end of each month that the data was incorrect. When loading monthly data, Visier attempts to infer when a change might have happened; for example, if a record says the employee is inactive as of March 31 and the employee has a termination event on March 20, Visier pulls the "inactive" profile back to align with March 20.

In Visier, you should set an analytic object's time granularity to the finest granularity in your extracted data. If the finest granularity is time records at the day level, set the time granularity in Visier to Day.