Add or Remove Records from a Source
Learn how to upload new data to add records to a source or exclude data files to remove records from a source.
Who can use this feature?
Users with this profile:
-
Data Engineer
Not sure if you have this feature or capability? Reach out to your administrator.
Add records to a source
To add new records to a source, upload new data files.
- On the global navigation bar, click Data > Data Transfers.
- Click Upload File.
- Select a file.
- Click Upload.
Result: If the file matches an existing source regex, the records are loaded into the existing source. For more information, see Source Settings. If your file has additional columns compared to the previously-defined source, the new columns are added to the source.
- Optional: If the file doesn't match or isn't connected to a source regex, do one of the following:
- To erase the file, click the Delete button .
- To associate the file with a different source, click the Reassign button .
- To connect the file to a source, click the Reconnect via Regex button .
Note: Records are also populated through SFTP and appear in the Data Transfers room. For more information, see File Upload Using SFTP
Exclude files from a source
To remove records from a source, exclude uploaded files from the source.
- On the global navigation bar, click Data > Sources.
- Select a source.
- In Uploaded Files, in the Exclude/Include column, turn off the toggle so it says Excluded. The file and all its records will not be included in the source anymore.
- Click Save.
Result: Future jobs won't process excluded files when generating data versions. This impacts all processing jobs in production or in projects that load data from the selected source.
Tip:
- Excluding files doesn't immediately impact your production application because your production version uses a data version that previously processed the now-excluded files.
- Do not include or exclude uploaded files in a production source while a processing job is running. To find out if a processing job is running, open a project and navigate to Data > Jobs. For more information, see Jobs.