Formats

Metadata can be imported or exported in different formats using the upload API or download API. References between metadata objects are implemented using their business keys. Reports can be generated in common formats.

Business keys

When uploading or downloading, metadata objects are identified by their business keys (in contrast to the technical UUID). A business key is composed of one of more labels and represents a unique, hierarchical path to a metadata object. Business keys are used in relationships (e.g. inCollection, hasDomain) and links (e.g. derivations) to reference metadata objects.

For example, the metadata object BusinessAttribute (in the format JSON) references other metadata objects by their unique business keys Invoice and /Data Domains/Date:

{
  "_type" : "BusinessAttribute",
  "hasDomain" : "Invoice",
  "label" : "Billing date",
  "hasRange" : "/Data Domains/Date"
}

Business keys have the following characteristics:

The individual labels in composed business keys are separated by /.
If the referenced metadata object is located in another model, the business key begins with / and the label of the model. Otherwise, the model is ommited from the business key.
Metadata objects, which must only be unique in their superordinate metadata object, but not in the model (e.g. subcollections, subsystems), must be preceeded by the superordinate metadata object (e.g. collection, system) separated by /.

Explanation

Following the last rule, for example:

Subcollections, which must only be unique in their superordinate collection, but not in the model, must be preceeded by the superordinate collection. Consequently, all superordinate collections must be specified.
Subsystems, which must only be unique in their superordinate system, but not in the system catalog, must be preceeded by the superordinate system.
Technical metadata objects (e.g. data objects, data types or enumerations), which must only be unique in their superordinate collection, but not in the data models, must be preceeded by the superordinate collection.
Business objects, which must be unique in their business data model, may be preceeded by the superordinate collection. If so, consequently all superordinate collections must be specified as well (see above).

Labels	Business key
`Address`	`Address`
`Business data model`, `Person`	`/Business data model/Person`
`Enterprise Model`, `Billing`, `Invoice`, `Date`	`/Enterprise Model/Billing/Invoice/Date`

Delimiters

If the character / or . appears in the label of a simple or composed business key, then it is necessary to indicate that this character is part of the label and not a separator of the business key. This is achieved by quoting the label (using ") within the business key. For example:

Labels	Business key
`dataspot.`	`"dataspot."`
`Technical objects`, `INPUT/OUTPUT`	`Technical objects."INPUT/OUTPUT"`
`File/Interface`, `TimeRecordings`	`"File/Interface".TimeRecordings`
`File/Interface`, `Employees.csv`	`"File/Interface"."Employees.csv"`

If the character " appears in the label of a composed business key, then it is necessary to indicate that this character is part of the label. This is achieved by quoting the label (using ") within the business key and duplicating every quotation mark within the label itself. For example:

Labels	Business key
`Category`, `28" City Bike`	`Category."28"" City Bike"`
`Projects`, `Project "Zeus"`	`Projects/"Project ""Zeus"""`
`Codename "Kronos"`	`Codename "Kronos"`

Note

The label Codename "Kronos" is converted into the business key Codename "Kronos" (without additional quotation marks), because labels containing " only have to be quoted within a composed business key.

Note

If the character " appears at the beginning (or at the end) of a label, then, following the rules above, at the beginning (or at the end) of the label within the composed business key there are three quotation marks ("""). For example, the labels Projects and Project "Zeus" are composed into Projects/"Project ""Zeus""".

Import/Export

Metadata can be imported or exported in the following formats:

XLSX (Microsoft Excel)
CSV (Zipped)
XML
JSON
DDL/SQL
XSD (Zipped)
JSON Schema (Zipped)

The properties are based on the model for which metadata is imported or exported and consist of all default properties defined in the metamodel, supplemented by all custom properties configured in the respective database.

Note

The properties of a model in a particular format can also be determined conveniently using the download in the user interface in the respective format.

The relationships between the metadata objects (e.g. inCollection, hasDomain) are represented as business keys. Links between metadata objects (e.g. derivations) are represented as separate objects:

Format	Links
`XLSX`	Links are represented as business keys in a separate table. Link attributes are columns in this table.
`CSV`	Links are represented as business keys in a separate CSV file. Link attributes are columns in this CSV file.
`XML`	Links are represented as business keys in a separate XML element. Link attributes are subordinate XML elements.
`JSON`	Links are represented as business keys in a separate JSON object. Link attributes are JSON properties in this JSON object.

XLSX (Microsoft Excel)

The import/export in the format XLSX (Microsoft Excel) is a spreadsheet, which contains separate tables for each metadata object type (e.g. Collections, Business objects, Business attributes, Relationships, etc.). Each table contains a list of the selected metadata objects of the respective type. The relationships between the metadata objects are represented as business keys.

Note

The tables in XLSX (Microsoft Excel) can be viewed analogously to the separate CSV files in the format CSV.

In the format XLSX (Microsoft Excel), the names of the properties correspond to the labels displayed in the user interface. For literal properties the label of the code (as shown in the user interface) is expected.

Tooltip

The table names and headings correspond to the labels displayed in the user interface and can be imported and exported in different languages (e.g. Sammlungen or Collections, Schreibgeschützt or Read-only). The language can be defined in the upload API and in the download API with the option language or with the HTTP header attribute Accept-Language.

Attention

Not only the selected language but also the configuration is applied to determine the table names and headings. Users or interfaces that use the format XLSX must therefore take into account that the labels are not fixed but may have been customized.

Note

For illustration purposes, certain columns are hidden in the following examples.

Links between metadata objects (e.g. derivations) are represented as business keys in a separate table (e.g. the table Derivations with the columns Derived to and Derived from). Link attributes are columns in this table (e.g. the column Qualifier).

XLSX

Tooltip

All properties that are mandatory fields according to the configuration are visualized with the background color "lilac". These fields may not be empty when uploading.

Note

The format XLSX (Microsoft Excel) restricts the size of a cell to 32,767 characters.

CSV (Zipped)

Note

CSV is encoded in UTF-8. If, after importing, certain characters (e.g. umlauts, ß, €, etc.) are incorrectly displayed as ?, then the CSV file should be converted to UTF-8 before importing.

The import/export in the format CSV is a ZIP file, which contains separate CSV files for each metadata object type (e.g. Collection.csv, BusinessObject.csv, BusinessAttribute.csv, Relationship.csv, etc.). Each CSV file contains a list of the selected metadata objects of the respective type. The relationships between the metadata objects (e.g. inCollection, hasDomain, hasRange) are represented as business keys.

Note

The CSV files can be viewed analogously to the separate tables in the format XLSX (Microsoft Excel).

In the format CSV, the names of the properties correspond to the (internal english) attribute names (according to the metamodel) and therefore differ from the labels displayed in the user interface. For literal properties the respective code (according to the metamodel) is expected. The type of each metadata object (Collection, BusinessObject, BusinessAttribute, etc.) corresponds to the name of the respective CSV file.

Note

For illustration purposes, certain CSV columns are hidden in the following examples.

Links between metadata objects (e.g. derivations) are represented as business keys in a separate CSV file (e.g. the CSV file Derivation.csv with the columns derivedTo and derivedFrom). Link attributes are columns in this CSV file (e.g. the column qualifier).

Collection.csv:

label,title
Billing,Invoice and settlement

BusinessObject.csv:

label,inCollection
Invoice,Billing

BusinessAttribute.csv:

hasDomain,label,hasRange
Invoice,Billing date,/Data Domains/Date
Invoice,Sum,/Data Domains/Amount

Derivation.csv:

derivedTo,derivedFrom,qualifier
Invoice,/Enterprise Model/Billing/Invoice,SPOT

Business keys in CSV

The format CSV requires that a string containing the character " must be quoted (using ") and that every quotation mark within the string must be duplicated.

Therefore, the following two-step procedure is necessary to generate syntactically correct business keys in CSV:

Convert labels to business key
- If the character / or . appears in the label of a simple or composed business key, then the label within the business key is quoted (using ").
- If the character " appears in the label of a composed business key, then the label within the business key is quoted (using ") and every quotation mark within the label is duplicated.
Convert business key to CSV
- Every business key containing the character " is quoted (using ") and every quotation mark within the business key is duplicated.

Pickung up on the previous examples, the business keys are converted to CSV as follows:

Business key	CSV
`Address`	`Address`
`Business data model/Person`	`Business data model/Person`
`Enterprise Model/Billing.Invoice/Date`	`Enterprise Model/Billing.Invoice/Date`
`/Business data model/Person`	`/Business data model/Person`
`/Enterprise Model/Billing/Invoice/Date`	`/Enterprise Model/Billing/Invoice/Date`
`"dataspot."`	`"""dataspot."""`
`Technical objects."INPUT/OUTPUT"`	`"Technical objects.""INPUT/OUTPUT"""`
`"File/Interface".TimeRecordings`	`"""File/Interface"".TimeRecordings"`
`"File/Interface"."Employees.csv"`	`"""File/Interface"".""Employees.csv"""`
`Category."28"" City Bike"`	`"Category.""28"""" City Bike"""`
`Projects/"Project ""Zeus"""`	`"Projects/""Project """"Zeus"""""""`
`Codename "Kronos"`	`"Codename ""Kronos"""`

Simplified procedure

In a business key, in principle, any label may be quoted (using "), regardless of whether the characters /, ., or " appear in the label or not. In CSV, in principle, any string may be quoted (using "), regardless of whether the character " appears in the string or not.

This allows the following simplified (less conservative) procedure for generating business keys in CSV:

Every label within a business key is quoted (using ") and every quotation mark within the label is duplicated. This step is omitted, if the business key is simple (non-composed) and contains neither / nor ..
The entire business key is quoted (using ") and every quotation mark within the business key is duplicated.

XML

The import/export in the format XML contains a flat, non-hierarchical list of all selected metadata objects. The relationships between the metadata objects (e.g. inCollection, hasDomain, hasRange) are represented as business keys.

Note

The export in the format XML contains a reference to the XML schema (XSD). Based on the metamodel, the XML schema contains a technical description of all structures, properties, and relationships.

In the format XML, the names of the properties correspond to the (internal english) attribute names (according to the metamodel) and therefore differ from the labels displayed in the user interface. For literal properties the respective code (according to the metamodel) is expected. The type of each metadata object (Collection, BusinessObject, BusinessAttribute, etc.) corresponds to the name of the respective XML element.

Note

For illustration purposes, certain XML elements are hidden in the following examples.

Links between metadata objects (e.g. derivations) are represented as business keys in a separate XML element (e.g. the XML element Derivation with the subordinate XML elements derivedTo and derivedFrom). Link attributes are subordinate XML elements (e.g. the subordinate XML element qualifier).

<?xml version="1.0" encoding="UTF-8"?>
<BusinessDataModel xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                   xsi:noNamespaceSchemaLocation="/api/test/meta/schemas/BusinessDataModel.xsd">
  <Collection>
    <label>Billing</label>
    <title>Invoice and settlement</title>
  </Collection>
  <BusinessObject>
    <label>Invoice</label>
    <inCollection>Billing</inCollection>
  </BusinessObject>
  <BusinessAttribute>
    <hasDomain>Invoice</hasDomain>
    <label>Billing date</label>
    <hasRange>/Data Domains/Date</hasRange>
  </BusinessAttribute>
  <BusinessAttribute>
    <hasDomain>Invoice</hasDomain>
    <label>Sum</label>
    <hasRange>/Data Domains/Amount</hasRange>
  </BusinessAttribute>
  <Derivation>
    <derivedTo>Invoice</derivedTo>
    <derivedFrom>/Enterprise Model/Billing/Invoice</derivedFrom>
    <qualifier>SPOT</qualifier>
  </Derivation>
</BusinessDataModel>

JSON

The import/export in the format JSON contains a flat, non-hierarchical list of all selected metadata objects. The relationships between the metadata objects (e.g. inCollection, hasDomain, hasRange) are represented as business keys.

In the format JSON, the names of the properties correspond to the (internal english) attribute names (according to the metamodel) and therefore differ from the labels displayed in the user interface. For literal properties the respective code (according to the metamodel) is expected. The type of each metadata object (Collection, BusinessObject, BusinessAttribute, etc.) is specified in the property _type.

Note

For illustration purposes, certain JSON properties are hidden in the following examples.

Links between metadata objects (e.g. derivations) are represented as business keys in a separate JSON object (e.g. the JSON object Derivation with the JSON properties derivedTo and derivedFrom). Link attributes are JSON properties in this JSON object (e.g. the JSON property qualifier).

[ {
  "_type" : "Collection",
  "label" : "Billing",
  "title" : "Invoice and settlement"
}, {
  "_type" : "BusinessObject",
  "label" : "Invoice",
  "inCollection" : "Billing"
}, {
  "_type" : "BusinessAttribute",
  "hasDomain" : "Invoice",
  "label" : "Billing date",
  "hasRange" : "/Data Domains/Date"
}, {
  "_type" : "BusinessAttribute",
  "hasDomain" : "Invoice",
  "label" : "Sum",
  "hasRange" : "/Data Domains/Amount"
}, {
  "_type" : "Derivation",
  "derivedTo" : "Invoice",
  "derivedFrom" : "/Enterprise Model/Billing/Invoice",
  "qualifier" : "SPOT"
} ]

DDL/SQL

Business data models can be exported as DDL/SQL. Data models can be exported and imported as DDL/SQL. The import/export is a .ddl or .sql file.

Example `Billing.sql`

CREATE SCHEMA "Billing"; 

CREATE TABLE "Billing"."Invoice" ("Billing date" DATE); 

ALTER TABLE "Billing"."Invoice"
ADD COLUMN "Sum" DOUBLE PRECISION;

The metadata objects (collections, data objects, business objects, etc.) are mapped to corresponding definitions (schemas, tables, columns, etc.) in the database - and vice versa.

Note

The specific SQL syntax depends on the selected database (PostgreSQL, Oracle, Microsoft SQL Server, MySQL) and is defined with the option targetDatabase.

XSD (Zipped)

Data models can be imported as XSD (XML Schema Definition). The import is a ZIP file, which contains one or multiple .xsd files.

Example `Billing.xsd`

<?xml version="1.0" encoding="UTF-8"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">
  <xs:element name="Invoice">
    <xs:complexType>
      <xs:attribute name="Billing date" type="xs:date"/>
      <xs:attribute name="Sum" type="xs:float"/>
    </xs:complexType>
  </xs:element>
</xs:schema>

The schema components (xs:element, xs:attribute, xs:complexType, etc.) are mapped to corresponding metadata objects (data objects, technical attributes, enumerations, etc.).

JSON Schema (Zipped)

Data models can be imported as JSON Schema. The import is a ZIP file, which contains one or multiple .json files.

Example `Billing.json`

{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "$id": "https://dataspot.io/test/billing.json",
  "title": "Invoice",
  "type": "object",
  "properties": {
    "Billing date": {
      "type": "string"
    },
    "Sum": {
      "type": "number"
    }
  }
}

The schema components (type, properties, etc.) are mapped to corresponding metadata objects (data objects, technical attributes, enumerations, etc.).

Reports

The built-in reports can be generated in the following formats:

Tooltip

In addition to the built-in reports, customer-specific reports can be implemented in XSLT and be uploaded and edited on the tenant page. Customer-specific reports may support other additional formats.

Note

Certain reports use proprietary formats while other reports comply to established standards or requirements (e.g. GDPR Processing Records).

Business keys​

Delimiters​

Import/Export​

XLSX (Microsoft Excel)​

CSV (Zipped)​

Business keys in CSV​

XML​

JSON​

DDL/SQL​

Example Billing.sql​

XSD (Zipped)​

Example Billing.xsd​

JSON Schema (Zipped)​

Example Billing.json​

Reports​

Business keys

Delimiters

Import/Export

XLSX (Microsoft Excel)

CSV (Zipped)

Business keys in CSV

XML

JSON

DDL/SQL

Example `Billing.sql`

XSD (Zipped)

Example `Billing.xsd`

JSON Schema (Zipped)

Example `Billing.json`

Reports