Skip to main content

Metadata connectors

For the integration or exchange of metadata with other systems, different connectors are provided. Connectors are programs that connect to data sources, extract metadata, and import this metadata into dataspot - or vice versa. Each connector is implemented for a specific data source or system. For example, one connector could support reading metadata directly from a database, while another connector could facilitate the exchange of metadata with a specific application (e.g. a common BI software). In this sense, connectors are ready, out-of-the-box solutions for the integration of dataspot with the surrounding system landscape.

Note

In the following examples, the database test and the server https://www.myserver.com are used for illustration purposes (e.g. in the path https://www.myserver.com/rest/test/schemes). This database or server must be replaced accordingly in all actual requests. Depending on the configuration, the URL may also have to be extended by the context path (e.g. https://www.myserver.com/dataspot/rest/test/schemes).

SAP Metadata API (by bluetelligence)​

SAP Metadata API connects to an SAP system, extracts the relevant metadata, and maps this metadata as data lineage in dataspot.

SAP system requirements​

Detailed information on the requirements of the SAP system can be found here.

Installation and configuration​

When installing SAP Metadata API, in addition to the hardware and software requirements, the server must have access to both the source SAP systems as well as to dataspot.

Note

Further information on the installation on Linux and Windows can be found in the comprehensive installation guide.

The connection to the SAP system is possible through an RFC connection or through an SQL connection directly to the HANA database. In addition to the settings of the SAP system, the server and the authentication of dataspot must be defined in appsettings.json.

RFC connection​

"BwErpSystemConfigurations": [
    {
      "SystemNumber": "<instance number of BW system>",
      "Client": "<client number of BW system>",
      "Prefix": "/BLTG/",
      "BwErpSid": "<SID of BW system>",
      "Server": "<host or IP of BW system>",
      "User": "<user to connect to BW system>",
      "Languages": ["E", "D"],
    }]
Note

The property Prefix is only necessary if a function module is required.

SQL connection​

"HanaSystemConfigurations": [
    {
      "HanaSchema": "<schema of BW application in HANA database>",
      "Database": "<HANA database>",
      "SqlPort": <SQL port of HANA database>,
      "UseSsl": <true or false>,
      "ConnectionString": null,
      "BwErpSid": "<SID of BW system>",
      "Server": "<host or IP of your HANA database>",
      "User": "<user to connect to HANA database>”,
      "Languages": ["E", "D"],
    }]
Note

The property ConnectionString can be used for connections that cannot be covered by specifying only Server, SqlPort, Database and User. Details can be found here.

dataspot​

Authentication is possible using basic authentication or OpenID Connect.

"Upload": {
    "Url": "<dataspot URL>",
    "Scheme": "<dataspot scheme>",
    "Tenant": "<dataspot tenant>",
    "BasicAuthentication": {
      "Username": "<username>",
    },
    "OnDelete": "<status>"
  }
"Upload": {
    "Url": "<dataspot URL>",
    "Scheme": "<dataspot scheme>",
    "Tenant": "<dataspot tenant>",
    "OpenIDConnect": {
      "ProviderUrl": "<Provider URL>",
      "ClientId": "<Client ID>",
      "UserId": "<User ID>",
    },
    "OnDelete": "<status>"
  }
Note

The property onDelete defines whether obsolete metadata objects are deleted or changed to the specified status.

Usage​

After installation and configuration, SAP Metadata API can be called with the command DataLineage and the option upload to upload the extracted data lineage directly to dataspot.

Windows​

DataLineage.cmd upload -s <SID of BW system>

Linux​

./DataLineage.sh upload -s <SID of BW system>
Note

When DataLineage is executed for the first time, the passwords for the BW system and for dataspot are prompted. It is therefore necessary to run DataLineage once manually before it is planned using a scheduler (e.g. cron or Windows Scheduler). Details on the configuration as a regular job can be found here.