Prompt Tuning Vee

Improve Vee's responses by clarifying your organization's context and terminology.

Tip: To learn more, take the Visier University eLearning course: Tuning Vee.

Overview

As an administrator, there are methods you can take to improve Vee's responses when it returns an unsatisfactory response or no response to a question. This process is called prompt tuning. Prompt tuning teaches Vee your organization's context and terminology. This helps Vee understand your questions and give more relevant answers over time.

Note: Prompt tuning is only recommended when Vee returns an unsatisfactory answer to your question. Visier will support you through the prompt tuning process and help you identify the methods that work best for you. This process is separate from setting boundaries around what Vee should decline to answer. For sensitive or off-limits topics, see Customize safeguards.

Prerequisites

Vee's functionality varies based on your organization’s specific configuration. Before asking Vee a question, ensure you've identified which Vee experience you have. For more information, see Determine your Vee experience.

The table below lists the prompt tuning methods available in each experience:

Prompt Tuning Method	Vee	Vee 2.0
Add synonyms	Yes	Yes
Exclude objects	Yes	Yes
Reset the thread	Yes	Yes
Data gathering instructions	Not applicable	Yes
Usage-based tuning	Yes	Not applicable
Create an analysis tagged with Vee Examples	Yes	Not applicable

Before you begin

To ensure a smooth prompt tuning process, adhere to these best practices before you begin:

Load the sufficient data. If the question cannot be answered from your organization’s current people data, you will need to load that data before Vee can answer your question.
Turn on the Primary Key Dimension setting in the Employee analytic object, and set its caption to Full Name, so Vee can identify individual employees. For more information, see Primary key dimension.
Populate the Employee ID field in every user's profile, or automate it with dynamic lookup, so Vee can answer self-referential questions. For more information, see Set Up Dynamic Lookup.
Commit your project changes, then ask Vee a question to initiate re-indexing. Allow a few minutes for the change to take effect. For more information on how to commit project changes, see Manage and Commit Project Changes.

Figure out what needs to be tuned

Use the Vee Logs to find questions that returned unsatisfactory answers. The logs capture user questions, along with thumbs up, thumbs down, and comment feedback on Vee's responses. You can identify the questions that got a thumbs down response, read any related comments, and begin to figure out which prompt tuning methods you might need. For more information, see Vee Logs.

When troubleshooting why Vee returned an unsatisfactory response or couldn't recognize a specific term, it helps to understand what parts of your data model are visible to Vee in Studio.

Vee can see an object in Studio, such as a metric, concept, or dimension, when both of the following are true:

Objects are visible in analytics and Include with Vee is selected.
Objects have data available.

Object visibility differs depending on your Vee experience. The differences are summarized in the table below.

Studio Element	Vee	Vee 2.0
Object Descriptions	No	Yes
Vee Example Analyses	Yes	Not applicable
Vee Instructions and Safeguards	Not applicable	Yes

Draft validation questions

In addition to using the Vee Logs to surface questions that need prompt tuning, you can also proactively validate Vee against the business questions that matter most to your organization.

Write down a short list of core business questions, and define what a good answer looks like for each. For example, a good answer to a retention question might need to reference Compa Ratio and recent exit reasons, not just the retention rate.

Ask Vee these questions to see how it currently performs. After you make a prompt tuning change, ask the same questions again and check the answers against your criteria. This helps confirm that your change improved one question without affecting others.

Decide what needs to change

An unsatisfactory response can have several different causes, and the fix depends on which objects need to be changed. Use the following decision framework to diagnose the cause and decide where to make a change:

Start in the Explore room, and build the visual you'd use to answer the question yourself. This shows you which metrics, dimensions, or concepts a good answer needs.
Check that those objects have clear, consistent names and descriptions. Confusing metadata is a common cause of unsatisfactory answers.
If the question relies on a specific group, such as a population of employees, group those members with a leveled dimension or a selection concept. For more information, see Create a Selection Concept and Create a Leveled Dimension.
If your organization uses different terms for the same object, add those terms to the object as synonyms.
If Vee still picks the wrong object in its response, try updating the object's description to note what it's not used for.
If none of the above improves Vee's response, continue to the next section and select the prompt tuning method that applies to you.

Prompt tuning methods

The following methods enhance Vee's understanding when it returns an unsatisfactory response or no response.

Add synonyms to objects

Try adding synonyms when the data exists, but Vee uses the wrong object in the response because your question uses different terminology than the object's display name. Add your organization's terms as synonyms so Vee recognizes them.

Synonyms can be single words or phrases. They also apply to global search and filter dimension searches, since a matching search term ranks the associated object higher in the results. For example, if your organization refers to managers as leaders, add Leader as a synonym to the Manager concept.

If you need to add the same synonym across multiple objects, such as adding the synonym Associates to all objects that reference Employees, we recommend that you create a concept instead of adding a synonym to the Employee subject. Adding a synonym to the Employee subject will address only that object but not all the metrics (e.g., Headcount, Turnover, Budgeted Compensation) that reference the Employee subject. Creating an Associate concept, which includes all employees who are considered associates, applies consistently as a filter across all of the related objects. Avoid using the same synonym for multiple objects that are not related.

Tip: Select the Synonyms column in the Columns menu to view synonyms for all objects at a glance within Studio.

To add a synonym, do one of the following:

For analytic objects, concepts, dimensions, metrics, and properties:
1. In a project, select the object that you want to add a synonym to.
2. Click the Basic Information tab.
3. In the Synonyms field, click Add.
4. In the Edit synonyms dialog, add the synonym.
5. Optional: To add more synonyms, click Create synonym.
6. Click Done.

For hierarchy dimension members:
1. In a project, click Model > Dimensions.
2. Select the leveled dimension that you want to add a synonym to.
3. Click the Configure tab.
4. In the Synonyms column, click the Edit button .
5. In the Edit synonyms dialog, add the synonym.
6. Optional: To add more synonyms, click Create synonym.
7. Click Done.

Example: Clarify terminology with synonyms

Vee might not recognize acronyms or the terms your organization uses for objects. Adding synonyms helps Vee match the terms in your questions to the right objects. The following scenarios show when to add synonyms.

Scenario 1: Use synonyms to define acronyms

If a dimension member uses an acronym, such as APAC for the Location dimension, Vee may not recognize it. Add an Asia Pacific synonym to the APAC dimension member so Vee understands the term.

Scenario 2: Use synonyms to add terms and phrases that refer to the same thing

Let's say your organization has a dimension member named City for the Location dimension. If your organization commonly refers to this dimension member as a site, add a Site synonym to that dimension member. You can also add phrases as synonyms. For example, add Direct Report Count Range as a synonym to the Direct Span of Control Range dimension if you want to relate it to the phrase "direct reports".

Exclude objects

Try excluding objects when the data exists, but similarly named objects, or dimensions with overlapping member values, create ambiguity about which one to use. Vee responds better when it has a smaller, clearer set of objects to choose from.

By default, every object that is visible in analytics is included with Vee. Excluding objects hides them from Vee, which helps it pull the right objects in a response. Hiding an analytic object doesn't automatically hide its related metrics and dimensions, you will need to hide those separately.

Before you exclude an object, open its Dependencies tab in Studio and look the Descendants to see if any analyses use it. Replace the object in any analysis charts if necessary.

Tip: In some instances, you may just want to change the display name of an object to remove ambiguity. For more information, see Change a display name.

You can edit object visibility individually or in bulk.

Exclude objects individually

In a project, click the object that you want to exclude.
Click the Basic Information tab.
Deselect Include with Vee.

Edit objects in bulk

Note: Analytic objects cannot be excluded in bulk.

In a project, select the objects that you want to exclude.

Note: Some object types do not support visibility editing and cannot be selected for exclusion. For more information on these objects, please contact your Customer/Partner Success Manager.
Deselect Include with Vee.

Note: Objects must be visible in analytics to be included with Vee. If Visible in Analytics is toggled off, then Include with Vee cannot be selected or deselected.
Click Apply.

Example: Dealing with ambiguity

Dimensions or metrics with similar names can cause ambiguity. Excluding the less relevant objects helps Vee give clearer responses. The following scenarios show when to exclude objects, and what to do when you cannot.

Scenario 1: Multiple similarly named dimensions

If you have three job name dimensions, such as Job Title, Job Name, and Function, decide which one your organization uses most and exclude the rest from Vee. Otherwise, Vee might answer the same question with a different dimension each time.

Scenario 2: Overlapping member values across dimensions

Ambiguity can also occur when two dimensions contain the same member values. For example, asking a question about Account Executives could pull values from the Job Function or Job Name dimension. Decide which dimension you want Vee to use, and exclude the other.

Scenario 3: Filtered variations of the same metric

If you have created multiple custom metrics that filter a base metric, such as several Turnover variants, this can confuse Vee. Exclude the variants and keep the base metric visible. Vee can apply the right filter dynamically if your concepts and dimensions are clearly labeled and available.

Scenario 4: Similar objects that must stay included

If two similarly named objects, such as the Promotion Events Count and Employees with Promotions metrics, must stay included and cannot be renamed, users need to use the exact display name in their questions. If most users only need one metric, you can hide the other through their content package instead so they can get consistent results without needing to be so precise in their wording.

Scenario 5: Overlapping values across job titles, departments, and dimensions

A value such as Nurse might appear in job titles, department names, and a Registered Nurse dimension. If you cannot rename or exclude these, create a concept, such as Nurse, to define the population directly. Vee prioritizes concepts over dimensions, so this ensures Vee consistently filters as intended.

Reset the thread

Try resetting the thread when the data exists and the objects are correct, but Vee still gives an unsatisfactory response.

Vee retains context in a conversation to better understand follow-up questions, maintain continuity, and provide more natural, relevant responses. Start a new conversation with Vee when you switch to a completely new and unrelated topic, if the current thread has become too long, or when you want a clean slate.

To start a new conversation, click the New topic button in the upper-right corner of the screen.

Data gathering instructions

With Vee 2.0, data gathering instructions are a more direct alternative to Vee Example analyses. Instead of building an analysis, you write instructions that tell Vee exactly how to turn a natural language question into the right visual and data. For more information about instructions, see Customize Vee Instructions and Safeguards.

Create a data gathering instruction when one of the following is true:

A metric needs a filter that isn't stated in the question itself, and that filter should always apply when the metric is used.
Multiple objects could answer the same question, and you want Vee to prefer one over the others.
Questions on a certain topic should always bring in related information, such as pairing retention questions with Compa Ratio and recent exit reasons.

Tip: If a filter or metric should apply to every visualization, set a default context instead. For more information, see Set the Default Context.

For example, let's say your organization defines Official Headcount as the Full-time Equivalents (FTE) metric filtered by the Official Headcount concept and you need to tell Vee how to define Official Headcount.

Create a data gathering instruction to prompt tune Vee:

In a project, on the navigation bar, click Model > Vee.
Click the Instructions tab.
Click Create Instruction.
Select the Data gathering category.
Add a Title. For example, Official Headcount Definition.
Add the Content. For example, When the user asks for official headcount, you must apply the Official Headcount concept to the FTE metric.
Optional: Select the user groups to apply the instruction to. If left blank, the instruction is applied to all users.
Click Create Instruction.

Usage-based tuning

Vee learns from the clarification selections users make. Hovering over a highlighted object in Vee's response shows a menu of alternatives, letting users correct the chosen object. Vee remembers each user's choices and prioritizes them in future responses. This personalized memory automates continual prompt tuning and results in more accurate responses tailored to each user's unique needs.

You can clear your clarification memory in your user preferences:

On the navigation bar, click your User icon > Preferences > Vee.
In the Vee settings section, click Clear user rules.

As an administrator, you can prioritize objects for all users by making a clarification selection. Once a user makes an individual selection, that will override your selection.

Vee applies usage-based tuning according to the following logic:

A user's individual preference always overrides an administrator's preference.
If there are conflicting administrator preferences, such as two rules for the same context, the more recent rule takes precedence.

Create an analysis tagged with Vee Examples

If none of the above methods work, create analyses tagged with Vee Examples to show Vee how to answer specific business questions. When creating an analysis for Vee, provide examples showcasing your common analytical questions and language, and incorporate charts with relevant data.

Vee also learns visualization preferences from your examples. If you pair a visualization type with certain metrics or dimensions, Vee favors that visualization type for similar questions later, unless you ask for something different. For example, a Vee Example analysis using a Breakdown visualization with Headcount will lead Vee to favor Breakdown visualizations for future Headcount questions.

Note:

Too many analyses tagged with Vee Examples can degrade performance, so always try to address feedback using the other methods first.
With Vee 2.0, you can migrate existing examples to instructions. For more information, see Migrate examples.

For example, let's say you asked the following question: Where are we spending the most on employees?

Questions starting with "where" can pose a challenge, since Vee might interpret it as location or department.

Let's say Vee answered based on department, but you wanted location instead. Create the following analysis and tag it with Vee Examples to guide Vee going forward:

In a project, on the navigation bar, click Model > Analyses.
In the Analyses room, click Create Analysis.
In the Create analysis dialog, type a display name and description for the analysis, and then click Create.
In the analysis, click Insert button > Add a chart button..
Click Create new chart.
In the visual title, select Breakdown of Actual Direct Compensation per FTE by Location.
In the chart title, name the chart Where are we spending the most on employees?
Click Save and Close.
Once the analysis is complete, click Done editing.
Assign the Vee Examples tag to the analysis so it becomes visible to Vee. For more information, see Assign the Vee Examples tag to an analysis.

Now, when you ask the same question in the future, Vee can associate the word "where" in the chart title to the Location group by in the chart.

Tip: Here are a few best practices to keep in mind when creating your analyses:

Start fresh: Create brand new analyses instead of using existing ones that are shared within your organization.
Use charts: Vee pulls content directly from the charts in analyses, so focus on creating a series of charts rather than incorporating text blocks.
Keep it simple: Each chart should answer one business question.
Use a question-based approach: Pose questions in your chart titles in the manner that you would typically ask Vee.

Assign the Vee Examples tag to an analysis

After you create an analysis, you must tag it with the Vee Examples tag to indicate its significance in the prompt tuning process. By default, analyses are excluded from Vee.

To assign the Vee Examples tag to an analysis:

In a project, click Model > Analyses.
Select the analysis that you want to assign the tag to.
Click the Basic Information tab.
In the Tags field, click Add or Edit.
Select Vee Examples.
Click Confirm.