Amazon S3

Visier can retrieve data through this connector if the following requirements are met.

Prerequisites

• Create an integration user in Amazon S3.
• Decide whether to use your own S3 bucket or send your files to Visier’s S3 bucket.
  • S3: When creating your Amazon S3 connector credentials in Visier, select S3 if you want to use your own S3 bucket.
  • CopyS3: When creating your Amazon S3 connector credentials in Visier, select CopyS3 if you want to send files to Visier’s S3 bucket. You can find Visier’s S3 access point during credential creation in Visier. For more information about providing your credentials, see Set Up Data Connectors.

Integration user

Create an integration user in your source system and assign it the appropriate security access so that Visier can retrieve your source data through this user.

Access	Description
If using your own S3 bucket, provide READ and LIST access	Assign Read Objects access to your files. Assign List Objects access to the S3 bucket you want Visier to retrieve.
If pushing data to Visier’s S3 bucket, provide READ and WRITE access	Assign Read Objects access to your files. Assign Write Objects access to Visier’s S3 bucket. You can find Visier’s S3 access point during credential creation in Visier.

Connector credentials

You need to provide the following credentials to Visier so that we can retrieve data from your source system. To provide your credentials in Visier, see Set Up Data Connectors.

If using your own S3 bucket, provide the following credentials.

Credential	Description	Example
Bucket Name	The name of the S3 bucket to retrieve.	visier-data-bucket
Bucket Region	The region in which the bucket is stored. For more information, see Managing AWS Regions.	us-west-2
Access Key	The integration user’s access key.	SPIA4BSBSDSNGG644POL
Secret Key	The integration user’s secret access key.	xWSOf40JjoqRs9M7sRNe0lFNi3ilxge/Sbnx7J25
Path (Optional)	The path to the S3 bucket. If unspecified, Visier will retrieve data from the top level of the bucket.	s3-data/

If sending your files to Visier’s S3 bucket, provide the following credentials.

Credential	Description	Example
IAM Role	The integration user’s AWS Identity and Access Management (IAM) role.	arn:aws:iam::007816025659:role/myrole

Data files

The Amazon S3 data connector only retrieves data stored in the accessible S3 bucket for use in Visier. The data in your S3 bucket must match Visier’s schema. The folder structure must be either path/date/filename.txt or path/filename_date.txt. For more information, see Data File Guidelines.

For other data, you must provide that data through alternative methods such as SFTP or Visier’s other data connectors. For more information about sending data to Visier, see File Upload.

How the connector loads data

Initially, data connectors retrieve a full history for each subject to generate an initial history of events. In each subsequent data retrieval, each connector handles the data load differently.

• Full load: All data from the source system is loaded into Visier.
• Delta load: Only changed data is loaded into Visier. A delta load may take either the entire history or a specific number of years of history for each changed record.
• Snapshot load: All data at a particular point in time. A snapshot load does not contain historical records or show changes over time.
• Partial load: A subset of data is loaded into Visier.

After the initial load, we recommend that you only provide data files for records that changed. However, you can send a full restatement if that's your preferred method.

Frequently asked questions

Do I need to authorize the integration user to access Visier’s S3 bucket?

If using your own S3 bucket, the integration user needs to be able to download data. If you’re pushing data to Visier’s S3 bucket, the integration user needs to be able to write data to Visier’s access point. For more information, see Integration user.

Does Visier remove my data files from Visier’s S3 bucket after the data is pulled into my Visier solution?

Yes. After a set amount of time, the data is vacuumed.

What time of user maintenance is required for the integration user?

None! If your organization has internal maintenance processes such as rotating keys on users, you will need to update the keys in the connector credentials you provided to Visier. You can update your credentials in the global workspace in the Data > Connector Credentials room.

Are Visier’s S3 buckets available in different regional Amazon Data Centers?

Yes.

If my AWS is in EMEA but I want to push data into Visier’s AWS Canada bucket, will I run into any issues?

Potentially. Because you’re pushing data across regions, your firewall may prevent you from pushing the data. If so, you might need to open a new pathway in your AWS instance.

Does Visier’s Amazon S3 data connector use S3’s webhook functionality?

No.