Transformer Configuration for Master Data Imports

This guide describes how to view and adjust the import configuration (Transformer Configuration) of a master data list in the Master Data Hub. It determines which file is read from the SFTP server, how it is read (encoding, delimiter, …), and how the columns of your file are mapped to the target schema expected by Workist. The same configuration applies to files you upload in the Workbench or deliver via the API.

Each master data list (articles, customers, customer contacts, addresses, unit conversions, framework contracts, orders, offers) has its own configuration.

---

Prerequisites

• A Workbench account with the Manager, IT or Admin role (see Users and roles).
• A master data list in the Master Data Hub. Creating a list describes how to create one.
• At least one sample file whose column names you know — otherwise you cannot map them to the target schema.

Custom import scripts

For some lists, Workist sets up a custom import script, e.g. when several files are merged into one list. In that case, only the file name patterns and the file reader parameters take effect. Do not save column mappings in that case: the import fails otherwise.

---

Step 1 — Open the configuration

Open the list in the Master Data Hub and switch to the Configuration tab. You can also get there from a channel:

• via the channel menu → Manage master data and the gear (Manage lookup) in the list's row, or
• via the channel configuration → Master Data and the gear next to the list's field.

Both gears open the list in a new tab.

---

Step 2 — Page layout

The tab consists of three parts:

1. The Import configuration card with the sections File Name Patterns, File Reader Parameters and Column Mappings.
2. The Dry Run card, which tests the configuration without importing anything (see Step 7).
3. A bar at the bottom that shows whether there are Unsaved changes. It contains the Discard and Save buttons.

---

Step 3 — File name patterns

In the File Name Patterns section, you define which file in your SFTP folder /export counts as input for this list.

• Fixed file name, e.g. customer_article_list.csv.
• Wildcard *, e.g. articles_*_full.csv.
• Date and time placeholders %Y, %m, %d, %H, %M and %S, e.g. %Y%m%d_Artikel.csv for 20240315_Artikel.csv. Wildcards and date placeholders cannot be combined in one pattern.
• Regular expression with the prefix re:, e.g. re:Artikel_[0-9]+\.csv.

If several files match the pattern, Workist imports the newest one. If a file matches the patterns of several lists, Workist imports it into each of these lists.

Enter one pattern per list. Add pattern lets you add further patterns, but the standard processing imports only one file per list; with several patterns, the import fails. Only a custom import script from Workist merges several files.

For files you upload in the Workbench or deliver via the API, the file name does not matter.

tip

Use patterns that are as restrictive as possible. A pattern that is too broad, such as *.csv, may cause non-article files to be mistakenly interpreted as an article list.

---

Step 4 — File reader parameters

In the File Reader Parameters section, you configure how Workist reads your file. Each parameter is initially Not set; the default value then applies.

Parameter	Meaning	Example
Delimiter	Column separator in CSV files. The default is the comma. Workist does not detect the delimiter automatically during the import: if the file uses a different delimiter than set, the import fails with Unexpected delimiter.	; or ,
Encoding	Character set of the file. The default is UTF-8. For CSV files from Excel saved as "UTF-8 with BOM", select utf-8-sig.	utf-8, utf-8-sig, utf-16, latin-1, cp1252, ascii
Quote Character	Character used to enclose values in the CSV file. The default is ".	" or '
Escape Character	Character used to escape the quote character within a value.	\

The dry run suggests which values fit your file.

note

For Excel files (.xls, .xlsx), leave all parameters Not set. Workist detects the format from the file extension; parameters that are set make the import of Excel files fail.

---

Step 5 — Column mappings

In the Column Mappings section, you map the columns of your input file to the columns of the Workist target schema.

Each row consists of two fields:

• Column Name — the target column, selected from the columns of the master data type (see Target columns per master data type).

• Column Expression — where the value comes from. Five types of expression are possible:

  Expression	Meaning
  Artikelnummer	Direct column reference — the value is taken 1:1 from the column of the same name.
  {Bezeichnung1} {Bezeichnung2}	Template expression — placeholders in curly braces are replaced by column values.
  'STK'	String literal — the value is hardcoded (e.g. a constant unit).
  PASS	Keep if present — the column of the same name is passed through; if it is not there, the mapping is ignored.
  (empty)	No mapping — the row is removed on save.

Example for an article list:

Column Name	Column Expression
article_id1 *	Artikelnummer
description *	{Bezeichnung_DE} {Farbe}
gross_price	Listenpreis
order_units	Bestelleinheit

What to keep in mind:

• Only mapped columns are imported. As soon as a mapping is saved, Workist only takes over the mapped columns and ignores all other columns of the file. Without mappings, Workist takes over the file's columns unchanged. They must then already be named like the Workist columns.
• Rows marked with * (Required column) are always shown in the form for each master data type; you cannot remove them. Leave the column expression empty if you don't need a mapping for such a column. The import does not check whether these columns are filled. Which columns Workist actually needs for recognition is described on the pages of the individual master data types.
• partition_id is prefilled with 'main' in the form. main means: the row applies to all customers (see Assignment to customers). If your file contains a column with the customer number, replace 'main' with its name.
• Not mapped rows are added by the dry run for columns of your file that don't correspond to a Workist column. Select a target column for them; otherwise Workist removes the row on save.
• An orange ! (Column not part of the target schema) marks a target column that does not exist for this master data type. Such a mapping makes the import fail; remove the row.

Use Add column mapping to add further mappings until all columns of the target schema are used. Each optional mapping can be removed again via the trash icon.

---

Step 6 — "Raw JSON" tab for advanced fields

Via the Raw JSON tab, you see the complete Transformer Configuration as a JSON document and can edit it directly.

In the JSON tab, fields that are not shown in the form are available in particular:

• apply_common_functions — a list of predefined transformations that are applied to the file in the specified order, e.g.:
  • query — filter out rows based on a condition ("query": "source_unit != target_unit").
  • strip — remove leading and trailing whitespace.
  • drop_column — remove a column from the result.
  • remove_duplicated_rows — remove duplicate rows.
  • remove_leading_zeros — strip leading zeros from a column (common for article numbers).
  • set_default_order_unit / set_default_order_unit_to_sales_unit / ensure_sales_unit_in_order_unit — standardize order units.
  • eval_expression — evaluate an expression to compute a new column value (e.g. a composite customer description).
  • add_incremental_id — add a sequential ID as a column.
  • filter_by_length / detect_chars — filter rows based on the length or a pattern in a column.
• replace_nan — replace empty values of a column with a default value.
• query — global filter condition that is evaluated immediately after reading the file.

Example JSON for an article list with mandatory handling:

{
  "file_name_patterns": ["customer_article_list.csv"],
  "file_reader_kwargs": {
    "encoding": "utf-8",
    "delimiter": ";"
  },
  "columns": {
    "article_id1": "Artikelnummer",
    "description": "{Bezeichnung_DE} {Farbe}",
    "partition_id": "'main'",
    "gross_price": "Listenpreis",
    "order_units": "Bestelleinheit",
    "sales_units": "Verkaufseinheit"
  },
  "apply_common_functions": [
    { "strip": "description" },
    { "remove_leading_zeros": "article_id1" },
    { "remove_duplicated_rows": ["article_id1"] }
  ]
}

As soon as you change something in the JSON, a red notice appears at the bottom if the JSON is not syntactically valid. Use Format JSON to indent and tidy the text without changing its content.

warning

The column names on the left side of "columns" must come from the target schema of the respective lookup type — otherwise the configuration is rejected on save.

---

Step 7 — Test and save

1. Test the configuration with a dry run. It also uses unsaved changes and shows how Workist transforms your file. A mapping to an unknown target column or a column missing in the file shows up here, before a real import fails.
2. Click Save in the bar at the bottom. Workist then reloads the saved configuration. Discard removes unsaved changes.

When does the new configuration take effect?

From the next import. Workist does not automatically reprocess data that has already been imported. To apply the new configuration immediately, start an import from SFTP in the Master Data Hub, upload the file again, or deliver it again via SFTP. After a configuration change, Workist also reprocesses an unchanged file.

---

Target columns per master data type

Which columns are available in the Column Name field depends on the master data type. What each column means and what Workist needs it for is described on the linked page.

Master data type	Always shown in the form (*)	Other target columns
Articles	article_id1, description	partition_id, article_id2, article_id3, article_status, gross_price, order_units, sales_units, sales_organisation, variants, extra_data
Customers	client_id1	client_id2, client_id3, company_name, address1, street_name, street_number, address2, zip_code, city, country_subdivision, country, po_box, misc, client_type, client_contact_person1, domain, iln, vat, iban, swift, locale_numeric, locale, internal_note, partition_id, extra_data
Customer contacts	contact_id, first_name, last_name, partition_id	phone, mobile_phone, email, extra_data
Address	address_id1	partition_id, address_id2, address_id3, name, address1, street_name, street_number, address2, zip_code, city, country_subdivision, country, po_box, misc, contact_person, phone, extra_data
Unit conversions	source_unit, target_unit, conversion_quotient	partition_id, article_id, extra_data
Framework contracts	partition_id, contract_id1	client_id, client_name, contract_id2, contract_name, extra_data
Orders	order_number	partition_id, external_reference, order_date, delivery_address, receiver_number, order_total_price, iban, swift, account, payment_terms_type, payment_terms_discount, payment_terms_due_days, position_number, article_id1, article_id2, article_id3, article_description, article_price, price_unit, quantity, unit, total_price, delivery_date, extra_data
Offers	offer_number	partition_id, valid_from, valid_until, position_number, article_id1, article_id2, article_id3, article_description, quantity, unit

The form always shows the columns marked with *; they are not mandatory in the sense of an import check. For articles, the selection also contains description_boosted and the order_unit_conversion_* columns. These are internal or deprecated; for units, use the unit conversion. Only use the target columns from this table.

---

Help and support

If the import continues to fail after a change, or the desired values do not end up in the Workist schema, contact your point of contact at Workist and have the following information ready:

• the name of the list in the Master Data Hub and the master data type (e.g. "Articles"),
• the time of the failed import (column Imported at),
• the input file of the import (linked in the import history),
• the error message of the import or the result of a dry run.