How to upload custom data to the SoundCommerce data models

SoundCommerce recognizes that customers may want to upload supplemental data to their data warehouse from time to time, so we have enabled a process for our clients to be able to upload ad hoc files to Google Cloud Storage, which SoundCommerce can then map to our standard data models and ingest to the data warehouse we maintain for you. An overview of the steps required to add a custom dataset are below. When you are ready to begin the process of onboarding supplemental data to your data warehouse, simply submit a support request and the SoundCommerce team will respond with next steps.

Table of Contents

Flat File Requirements

Data Discovery & Preparing Files for Upload

Testing

Source Connector Deployment & Mapping

Ongoing Updates

Flat File Requirements

• Formats accepted: CSV, JSON
• Formatting: UTF-8 only, no BOM (byte order mark)
• CSV file requirements: 
  • Must have a header row
  • Comma-delimiter is required
  • Quoted fields are optional, but recommended
  • Column headers must not use spaces (replace any space with an underscore or remove spaces entirely)

Additional steps can be found in How to Prepare for an SFTP transfer under Prepare Your Data.

Data Discovery & Preparing Files for Upload

Data Discovery sessions help support the efficient mapping of these custom datasets. SoundCommerce will work with you to:

• Understand what data needs to be uploaded
• Determine which SoundCommerce data models apply to that data
• Develop a plan for mapping the data to the SoundCommerce data model
• Review the file formatting and provide feedback if necessary

Testing

Once SoundCommerce receives a sample flat file, testing will be performed to ensure that data from a file can be ingested to our data pipeline. SoundCommerce will deploy a file connector in the Sandbox environment so that any issues encountered during testing do not interfere with the client’s production data. SoundCommerce will perform all of the actions in this step.

Source Connector Deployment & Mapping

In this step, SoundCommerce deploys the Flat File connector to the production environment, then completes the remainder of the data onboarding process in the data pipeline. When completed, data from the file will land in the Data Warehouse, where it will be modeled and materialized to Direct Access and made available for use in the dashboards. 

Ongoing Updates

When this process is completed by SoundCommerce, you may proceed to uploading files in your SoundCommerce GCS bucket on an ongoing basis, and SoundCommerce will ingest, transform, and land the data on an automated basis. 

To get started on adding custom data to the SoundCommerce data models, submit a support request and the SoundCommerce team will respond with next steps.