Handling Essential Data Input

Before uploading contact data to SAP Emarsys, managing and defining contact data fields is essential. Each value links to a specific field. To store data correctly, verify that a proper field matches during the CSV import. This process helps avoid errors or misclassification. This step is critical for keeping data integrity. It also enables effective segmentation and personalization. Always verify that all necessary fields exist in SAP Emarsys before importing data.

The Field Editor shows details, such as IDs, for all account fields, including both system and custom fields, and allows users to create new fields and manage existing ones.

Each SAP Emarsys account encompasses standard system fields in its database. These fields include details of its data type and field IDs. These elements are important for automating data transfers between SAP Emarsys and other applications. They play a critical role in core marketing functions to improve targeting. Marketers commonly use system fields such as first name, gender, country, and date of birth. These fields help tailor content and campaigns to individual contacts.

SAP Emarsys defines system fields that users are unable to create or change. However, users can make a copy of these fields, which they can customize.

Let's explore some specific system fields.

Special system fields to highlight:

• Opt-in field (ID = 31): It stores permissions for email communications. SAP Emarsys allows updating this field only through a confirmed API process, not through forms or imports. Under specific conditions, campaigns proceed without requiring an explicit opt-in.​​

• Do not track me field (ID=89705): It prevents Web Extend from using external contact IDs to track browsing and purchasing events. It also gives contacts the option to opt out of website data processing. Web Extend tracks behavior and sends data to Predict servers without external contact IDs. This data helps build statistical models for Predict recommendations.​

• Date of birth field (ID=4): It supports birthday campaigns, but entries often lack reliability or are empty because people sign up with incorrect data. The system ignores birthdays over 120 years and invalidates empty fields. Contacts offering a correct month and day but a 19th-century year to hide their age don't qualify for birthday campaign benefits.​

• IETF language tag field (ID=89707): This field uses client-loaded language codes per contact to enable language-specific communication. It doesn't include set values, but it needs to follow one of these formats based on the context:

  • "en" for English, "de" for German.

  • "en-US" for English (United States), "en-GB" for English (Great Britain).

VCE multi-language campaigns demand the latter format. To verify the language tags, open a VCE email, navigate to the Language Versions tab, and copy the master version.

SAP Emarsys users create custom fields through the application interface or by using the API to improve segmentation, personalization, and contact data management across forms, segments, and campaigns. Choosing the correct field type is important for accurate data handling and functionality. Make sure date formats are consistent.

Supported field types are:

• Numeric

• Short Text

• Long Text

• Unlimited Text

• Date

• Single Choice

• Multiple Choice

• URL

Best practice: SAP Emarsys recommends creating custom fields for the following purposes:

• Contact identification: If using a custom external contact ID as an identifier, create a custom text-entry alphanumeric field to store this data.

• Newsletters: Create a custom field to capture newsletter preferences, especially when offering various newsletter options.

• Original registration date: Employ a custom field to store the original registration date, as SAP Emarsys only displays the date a contact entered its system, which differs from their actual sign-up date with the client’s brand.​

SAP Emarsys adds special custom fields (admin fields) to an account when activating specific SAP Emarsys products.

• Web behavior fields: The Web Extend data collection scripts update custom fields prefixed with ‘predict’ based on website visitor activity every few hours. These fields support use cases such as cart abandonment campaigns and cross-category upsell strategies.

• SAP CX integration fields: SAP CX connectors include a predefined set of mapped fields between the SAP Emarsys and SAP CX systems, such as the SAP Customer Data Platform. Prefixes such as CD_id, CD_C_, CD_S_, CD_P_, and CD_D_ organize consent, subscription, and profile data.

• Channel-specific fields: Enabling certain channels adds corresponding fields to the account. For example, activating the SMS channel introduces the SMS Opt-in field, while Mobile Engage includes the App-Installed field to support multi-channel engagement.

• Interests field (ID = 0): This field is a default multi-choice option. Set its values under Management > Form Settings before activation. Once configured, the field shows automatically in SAP Emarsys forms to capture registration preferences. Remove the field from the form if it doesn't serve the wanted purpose.

• Forms: SAP Emarsys offers built-in forms for collecting and updating contact data. Users can create sign-up or targeted newsletter forms to deploy on websites or messages such as Web Channel and Mobile In-app. Available types include General Registration, Contact Us, and Change Profile.

• Voucher fields: These fields belong to the legacy voucher system. Replace this approach with the new Omnichannel Voucher Management, as SAP Emarsys now promotes and transitions to the newer solution.

Once the data files are ready, upload them to SAP Emarsys. The system then allows checking whether they meet the format requirements. After that, schedule regular updates. SAP Emarsys scans for contact duplication whenever it imports a new contact. The system automatically merges contacts with the same unique external contact identifier upon import.

Preparation stage:

Before starting the data import to SAP Emarsys, keep these considerations in mind:

• Identify which field serves as the unique external contact identifier for identification and deduplication. SAP Emarsys technical representatives can index custom fields, enabling the selection at the import.

• Determine which contact properties to include.

• Verify that all needed custom fields exist in the SAP Emarsys account before importing data.

Use consistent identifiers:

• Align contact imports with a consistent, unique external contact identifier (for example, a generated, persistent alphanumeric string that is non-guessable and non-sequential).​

• When importing from third-party systems, map external contact identifiers by creating a matching external contact ID field in SAP Emarsys.

• Use the email address only if necessary, as using emails or guessable identifiers can increase security risks.​

Keep data quality:

Remove duplicate contacts before importing, ensuring accurate profile unification and avoiding engagement issues.

Keep contact data up to date:

Keep accuracy by updating contact data regularly, using:

• The SAP Emarsys contact data API endpoints

• Auto-Import feature

Extend functionality through an integration:

Use SAP Emarsys custom integrations to connect with leading e-commerce platforms, such as Shopify.

​Customers usually upload contact data manually during the initial account setup. However, manual import is also an option for occasional contact data updates.

Follow these steps to upload contact data to the SAP Emarsys account:

• Open the SAP Emarsys account.

• Navigate to Contacts > Data Import.

• Click on Import to start the Import Wizard.

Data importing process:

• Make sure that the file meets the CSV guidelines and SAP Emarsys standards.

• In the import wizard, choose a unique external contact identifier. Upon creation, the system generates an Internal contact ID for each record, which serves as a primary identifier in SAP Emarsys internal processes.​

• Confirm that the file's data correctly maps to the fields and values in SAP Emarsys.

  

• The import duration depends on the number of contacts and the complexity of the database field.

Regular contact data update: Contact data through API (Recommended):

Use the SAP Emarsys REST API to keep up-to-date contact data consistently.

• Account Owners can create API users and save the API secret, as it shows only once.

• Follow the guide to get a JSON Web Token (JWT) through OAuth 2.0 (See the note).

• Include the JWT in the HTTP authorization header when making API calls to SAP Emarsys.

• When using the SAP Emarsys API, key_id refers to the chosen unique external contact identifier.

• SAP Emarsys experts recommend using the PUT request to update contacts. If the contact doesn’t yet exist in the SAP Emarsys account, it creates a new contact record.​

• Test the connection with a staging website and/or test contacts.

Regular contact data Update: Contact data through auto-import - SFTP or WebDav:

• Go to Contacts > Data Import.

• To create an auto-import configuration, click Import File.

• Then complete the following steps:

  1. Define the name of the SAP Emarsys contact list. After each successful auto-import, the system saves the imported contacts as a contact list using the given name followed by a timestamp.

  2. Select the file source: SFTP or WebDAV. Use an end–user–hosted server for SFTP. Use the SAP Emarsys WebDAV folder for WebDAV—an SAP Emarsys expert can configure access if needed.

  3. Set the import file name. SAP Emarsys processes only files that match the specified naming convention. Use wildcards to handle dynamic elements such as timestamps.

  4. To map the fields, upload a sample file. During the match values step, pay attention to single- and multi-choice fields and their values.

     

• Select the correct unique external contact identifier to guarantee consistent removal of duplication during each import.

• Review the summary and complete the setup.

SAP Emarsys allows contact deletion across all contact lists. It also removes related data such as Loyalty rewards and Smart Insight purchase history. This action erases historical behavioral and revenue data, affects contact-level reporting and potentially changes metrics across dashboards, analytics, and Smart Insight. To ensure full GDPR compliance, the platform completes deletion within up to one month.

General guidelines:

• Use a simple .csv table format, with each row containing all the attributes of a product, including:

  • The item ID

  • The URL of the product page

  • A link to a product image, and so on.

• Use a UTF-8 encoded CSV.

• Include the field names in the first row (header) and make sure that all the following rows list the values corresponding to the fields in the header.

• Do not use more than 255 columns.

• Make sure the file list includes only letters, numbers, underscores, dashes, and slash characters. SAP Emarsys uses the slash solely for the g/prefix in the Parent/Child feature. ​

File size:

• Make sure the file has fewer than a million items. If there are more, contact an SAP Emarsys expert.

• If the size of the product is large (for example, several hundred megabytes in size), compress the data file with GZIP and update the URL so that the file has the .gz suffix.

How to upload product data:

Get the product data file ready and upload it to SAP Emarsys. Make sure it meets format requirements, then schedule regular updates.

Steps to upload product data for the first time:

• Navigate to Management > Predict Data Sources.

• To manage all product data tasks, access the product data page by clicking 'Manage product data' in the product data box.

The initial upload validates that the data format is correct before continuing with the first upload.

Choose one of the two upload methods.

1. Upload from a URL:

   • Choose from Google Product Feed (GPF) or CSV file format.

   • Select the file format in the settings box.

   • Google Product Feed:

     Enter URLs and format settings for each location (various countries/regions supported).

     
   Note

   For more information about the Parent/Child Catalog Feature, visit the Help Portal.

2. SAP Emarsys CSV:

   SAP Emarsys end users host the CSV file using an accessible URL on their server. SAP Emarsys supports safe connections using a username and password and offers formatting in the SAP Emarsys guidance.

   • Paste the full URL of the product data file in the URL field to schedule updates.

   • Set the frequency for updating the product data.

   • Click 'Save' and 'Update from URL now.'

   • Insert the URL in SAP Emarsys to trigger updates that run based on the frequency specified during setup.​

   • Make sure that the product file is regularly updated with recent data, so SAP Emarsys can collect it and deliver recommendations correctly.​

     Note

     For more information about how to upload product data files from the URL hosting it, visit the SAP Emarsys guidance in the Help Portal.

3. Upload the CSV product data file from a computer:

   • Compress CSV files that are larger than 100 MB, while handling smaller ones separately.

   • Click 'Upload file to validate' in the 'Manual update and validation' section.

   • Browse for the product data CSV file.

   

   Confirm and publish product data:

   • Validation checks happen post-upload.

   • The system lists warnings with descriptions and highlights the relevant fields. To fix problems and resubmit, click 'discard this update.'

   • Click 'Activate update' to publish data if there are no warnings. Status updates appear in the current product data box.

   

   Check the published product data:

   • Use the product search box to search for products in the client’s SAP Emarsys database.​

   • Verify product data import success on the product info page.

   • Confirm which GPF fields the system holds during conversion and how they appear in the SAP Emarsys product database after submitting a GPF product data file.

   

   Force product data updates:

   • Manually update product data if a scheduled update fails or rejects it.

   • Click 'Update from URL now' and 'Activate update' to update the product data.

   • A manual update doesn't change the upload schedule. SAP Emarsys continues scheduled updates.

   Hint

   • Use protocols such as SFTP or FTPS for hosting CSV files:
     • SFTP (SSH File Transfer Protocol)
     • FTPS (FTP over SSL/TLS). These URLs must start with ftps://.

   • Allowlist ScarabBot/1.0 user agent for URL scheduling.

   • Format URLs for password-protected storage as follows:
     • https://username:password@mywebshop.com/catalog.csv
     • sftp://username:password@mywebshop.com/catalog.csv
   • When importing product data files into SAP Emarsys, the system automatically removes redundant white spaces unless you quote them.

   Maximize the GPF product data image size:

   • Image enhancement guarantees efficient viewing and prevents large images from slowing webpage loading.

   • The system enables image enhancement by default.

   • Enabled: Automatically maximizes images for show, possibly causing distortion.

   • Disabled: Images wider than 800 px don't appear. Reduce the size using the SAP Emarsys image proxy.

   Note

   SAP Emarsys enables three out-of-the-box Tactics that use Product Data and Web Extend triggers:

   • Price drop

   • Back in stock*

   • Low in stock*

   *For 'Back in stock' and 'Low in stock,' include a stock amount as a custom field in the product file. Set thresholds to identify when the system considers an item 'low in stock.'

   Visit the Help Portal for more information about:

   • Product catalog updates to verify that all criteria meet the requirements for using this functionality.

   • How long does it take for Predict to process the product catalog?

Web Extend data collection scripts actively track visitors' interactions on the website. The data processing confirms the information. Then it gives this confirmed data to SAP Emarsys applications such as Smart Insight and Predict.

SAP Emarsys supports the maintenance of a clean, compliant database by encouraging adherence to data protection rules.

Consider the following steps before starting the implementation:

• Web Extend scripts operate asynchronously, guaranteeing they don't affect the website's speed and performance.

• Web Extend scripts handle contact data anonymously or pseudonymously, linking web behavior with actual contacts only after importing the data into the SAP Emarsys database.

• Web Extend scripts collect data on all current and future web pages, including new products and landing pages for marketing campaigns. Add the relevant commands to the templates used for creating different page types.

• Web Extend scripts integrate into dedicated mobile websites or mobile apps.

• Implementation steps for data collection focus on identifying and tracking the contact's website behavior.

1. Add the account-specific tracking code to the website:

   Integrating the Web Extend tracking code enables the tracking of visitor behavior using JavaScript API commands. These commands are important for identifying contacts across devices and tracking the success of campaigns. Go to Management > Predict Data Sources  > Web Behavior and click Show tracking code. 

2. Identify contacts:

   SAP Emarsys employs cookies by default to monitor website visitors. These cookies identify known visitors, such as logged-in contacts or visitors who offer their email addresses and link them with contacts in the database.

   setCustomerId OR setEmail: Use this command on pages that identify contacts. The type matches the unique contact identifier (setCustomerId for an external contact ID, setEmail for email).

3. Implement e-commerce JavaScript commands :

   The prerequisites include:

   • Select a base currency for the account.

   • Upload the product data file.

   • Discuss the data collection and contact identification policy with an SAP Emarsys expert before implementing Web Extend scripts.

4. E-commerce commands:

   These scripts send website data to the correct database locations for reporting, recommendations, predictions, and lifecycle management.​

   Add the following commands to ALL pages on the website:

   • Cart: Reports visitors' cart content.

   • Go: Executes the commands in the queue.

   Add the following commands to specific pages on the website:

   • View: Add to product pages. Reports the ID of the product that the website visitor views on the product page.​​

   • Category: Add to category results pages. Reports the category path that the website visitor browses.​

   • Purchase: Add to the purchase confirmation page. Reports customers' orders, offering a basis for earning statistics in SAP Emarsys, or identifying whether a contact completed a purchase.​

   Use JSFiddle to see these commands in action. Verify the implementation by using the Live Events viewer in the Web Behavior box on the Predict Data Sources page.

   

5. Complete the steps and confirm the integration:

   The system constantly collects data and enriches contact information through behavior fields.

   Visit the Predict Data Sources page regularly to check integration health, especially after front-end updates, to guarantee that data collection scripts keep intact.

   

6. Other Web Extend commands:

   Several other types of Web Extend commands exist. Implement these commands based on the use case, although they aren't required for standard SAP Emarsys functionality.

   • SearchTerm: Add to search results pages. Reports visitors' search terms entered in the website search box.​

   • Tag: This function adds a tag to the current event with optional attributes (defined by the SAP Emarsys user). The user later accesses this tag from other SAP Emarsys products. There are three different types, depending on the use case.

   • TestMode: Allows testing of commands without recording the results in SAP Emarsys.

General requirements:

• Use a CSV file with a header row; each row represents one line item.

• The file name starts with sales_items and end with .csv (For example, sales_items_202505.csv).

• Use UTF-8 encoding and avoid newline characters within fields.

• To guarantee processing of all data in each upload, end the file with an empty line.

• Keep the column order exactly as defined during onboarding—incorrect ordering causes upload failure.

• Use lowercase for standard field names (For example, price, not Price).

Required fields:

• Standard: item, price, order, timestamp, customer, email, and quantity.

• For retail clients: Also include s_sales_channel and s_store_id.

• Use customer ID (external contact ID – same as in the SAP Emarsys contact database) and, if necessary, email, but be consistent and don't mix both.​

Upload frequency and format:

• The initial file covers at least two years of sales history.

• Upload daily updates after initial load (Smart Insight processes data once daily).

• Uploads are incremental—they add to existing data and don't overwrite it.

• Avoid duplicate orders—SAP Emarsys doesn't remove duplicates.

File validation rules:

• Use no more than 255 columns; keep each field under 64 KB in length.

• Acceptable delimiters for SFTP upload: comma (,), semicolon (;), tab (\t), or pipe (|).

• Use a standard-compliant CSV export tool/library.

• Maximum file sizes:

  • Daily: 10 GB (uncompressed)

  • Hourly: 1 GB (uncompressed)

Custom fields:

• Permitted data types and their header prefixes:

  • i_ (Integer), f_ (Float), t_ (Timestamp), s_ (String).

  • For example, s_book_author, i_customer_age.

• Add new custom fields only at the end of the existing column list.

• Don't insert or reorder existing columns—this risks data loss.​

• For numeric fields, avoid empty strings, such as double quotes with no values in them (" "); use double commas (,,) for null values.

Different sales data requirements apply to using the new Sales Data Loads solution.

File structure and format:

• Use a .csv table with one row per line item.

• Encode the file in UTF-8.

• Ensure there is a header row and that the column order of the data matches this configuration.

• Customize field names, but align them with the configuration.

• Follow the configured naming pattern for the file name.

• Max file size: 100 GB

• Use any standard delimiter: comma (,), semicolon (;), tab (\t), space ( ), or pipe (|).

• Properly quote newline characters when they appear; allow single and double quotes.

• End the file with a full order image—include all items in the same file.

• Don't upload deltas (partial updates); upload the complete transaction.

Order upload rules:

• Do not update orders; upload each order only once.

• Avoid re-using the same Transaction ID + Transaction Type combination more than once.

• Duplicate handling (within 30 days):

  • If a duplicate happens within 30 days, SAP Emarsys drops the new transaction.

  • If a duplicate happens after 30 days, SAP Emarsys drops the old transaction.

Timestamp requirements:

Use ISO 8601 format with:

• Second precision

• Up to 6-digit fractional seconds

• Timezone info (UTC or offset)

• For example, 2022-06-16T12:18:51Z

Required fields:

• transactionType

• transactionId

• referencedOrderId

• account_currency

• transaction_currency

• transactionCreatedDateTime

• transactionLastChangedDateTime

• productid

• quantity

• amount_in_account_currency

• amount_in_transaction_currency

• externalContactIdType

• externalContactId

Optional fields:

• storeId

• salesChannel

Custom fields:

• SAP Emarsys allows up to 20 custom fields.

• While optional, ensure that the custom fields adhere to the correct data types.

• Supported data types:

  • String

  • Date

  • Float

  • Integer

Upload and validate sales data for the first time (through Predict):

• To upload the properly formatted sales data file, go to Management > Predict Data Sources.

• Access the Sales Data page by clicking ‘Set up sales data uploads.’​

• To avoid duplicate data, use the same file name format with repeated uploads. Change the timestamp, for example: sales_items_20250101.CSV.

• Changing the file name or uploading different files with the same orders results in duplicated data.

• Use manual upload for the initial upload of historical sales data during onboarding. It allows immediate validation and error correction.

• Click 'Validate/upload' file on the sales data page.

• Browse and select the sales data file.

  

• If warnings or errors appear, fix them and retry the upload.

  

• Click 'Activate upload' to complete the upload when there are no warnings.

  

Set up regular sales data updates through Predict:

• After the initial upload of historical sales data, set up regular updates.

• Send historical data in one upload, then schedule daily uploads for new purchases.

• Regularly update sales data to keep Smart Insight-based reports correct.

• Automate sales data imports using the current sales data API solution through Predict.

• Access the Sales Data page through Management > Predict Data Sources and click 'Manage sales data.'

• Click 'Show API upload details' to get setup information.

Duplicate data:

• If a file with the same content uploads through the API, the duplicate skipped icon shows next to the 'Timestamp' in the upload history.

• Split the file content into two files and upload them again.

• Duplicate one of the lines in the file and upload it again.

Check the status and monitor the quality of the Smart Insight data on the Smart Insight Data Quality page.

The Overview tab shows reports of Contact, Product, and Purchase data uploaded to Smart Insight and their quality.

The Data Loads  tab gives information on the status of the scheduled automatic loads and recent contact, product, and purchase file uploads from the last 30 days, highlighting successful and unsuccessful tries.

• Contact data field management: This section explains the importance of defining and managing system and custom contact fields before the data import. It also highlights the role of proper field mapping in supporting segmentation, personalization, and platform functionality.

• System and special field usage: It introduces essential system fields such as opt-in status and language tags, along with special fields, such as admin and channel-specific fields. These fields support compliance, targeting, and customized campaigns across SAP Emarsys channels.

• Contact data upload methods: Find various methods for importing and updating contact data, including manual CSV import, auto-import through SFTP/WebDAV, and real-time API integration. It also outlines the steps for guaranteeing data accuracy and deduplication.

• Product and sales data integration: Upload product and sales data using CSV files or feeds, offering file format guidelines. It emphasizes the need for consistent identifiers and field structures to avoid upload errors and data mismatches.

• Data validation and quality control: Verify the field mapping, remove duplicates, and check upload results. Check how to monitor the Smart Insight Data Quality page, and update schedules for ongoing data quality assurance.