Beyond data connectors, this report describes emerging technologies and data space concepts, as well as the technologies they are based on. See also the Data Space Radar¹ for an overview of data spaces.
Why do we need data connectors?
We need data connectors to share data because they enable secure and efficient communication and exchange within data spaces. They are a tool for connecting many data endpoints to increase the available dataset and accelerate the data economy. By connecting data connectors, data spaces become secure environments where participants can freely share data. Data sovereignty, transparency, and fairness are ensured through adherence to a set of standards. Data connectors act as nodes within a data space and provide data sovereignty by design.
Sharing and exchanging data is not new, but the requirements for doing so are evolving.
A data connector essentially performs two key functions: It provides data exchange services that are (1) the application programming interface (API) to other participants in a data space to achieve interoperability, and (2) the trust component for handling data by implementing policy enforcement mechanisms and a common baseline for cybersecurity. However, because data can differ, and the requirements for sharing it also vary, different connector variants are needed (for more information, see the RAM IDS, sections 2.23 and 2.44).
1 https://internationaldataspaces.org/adopt/data-space-radar/
2 https://forms.office.com/r/gbu1yv9K25
The following figure illustrates the diverse requirements across industrial ecosystems. A data connector for (I)IoT devices may have substantially different requirements (in terms of resource consumption, efficiency, and cybersecurity) than a connector for a data marketplace or an industrial cloud platform. Simultaneously, these services must seamlessly integrate open data.
A data connector enables interoperability and puts data to use, linking it with other data and supporting modern concepts such as digital twins (shared and distributed), AI, and federated learning. To achieve this, connectors implement archetypal patterns for service management, orchestrating cloud-based services, lightweight API gateways, or IoT gateways. They may utilize concepts like distributed ledgers, but they will rely on state-of-the-art data management capabilities.
What is a data connector?
Data sharing is already widespread across various sectors. This creates complexity that poses a challenge to interoperability and data sovereignty. A "made in Europe" technology is needed to facilitate data sharing between organizations and individuals across Europe and beyond, ensuring they remain sovereign over their data.
3https://docs.internationaldataspaces.org/ids-ram-4/context-of-the-international-data-spaces/2_1_data-driven-business_ecosystems/2_2_data_sovereignty_as_a_key_capability
4https://docs.internationaldataspaces.org/ids-ram-4/context-of-the-international-data-spaces/2_1_data-driven-business_ecosystems/2_4_intercambio_y_compartición_de_datos
New data spaces are often defined by different implementations and standards, so a strong push for convergence is needed to prevent them from becoming new data silos. While a variety of implementations is important, it's equally crucial to enable interoperability, data continuity, and common governance models to support data sovereignty across all of them.
IDSA is creating a global standard for data connectors and has developed a software architecture that facilitates the secure exchange of data between trusted parties across all ecosystems (edge, cloud, mobile, server-based, etc.). Certified users gain access to the data ecosystem, where they attach usage restriction policies to their data before making it available to other users. The IDS connector is a core component of the IDS standard, performing the data exchange services described in section 3.5.25 of the IDS RAM (see also the figure below). The IDS connector can be installed on a server, in the cloud, on an IoT device, and on a smartphone.
It uses a special container technology that guarantees "trusted execution," meaning that the data within the container is always protected from unauthorized access and manipulation.
By adopting the IDS standard and leveraging synergies with other pioneering initiatives such as Gaia-X, FIWARE, SITRA, MyData, and others, secure data spaces become a reality. Since data spaces combine technical, organizational, and legal complexities, the IDS standard addresses technical, operational, and legal arrangements in data spaces. The standard builds upon IDS RAM, which provides guidelines for data exchange and adds features such as identity management, communications security, usage control, and others. Details on the system architecture can be found in subsection 3.5.2 of IDS RAM. The IDS connector is defined in DIN SPEC 27070 as part of German standardization work. The concepts of data spaces and data connectors are increasingly subject to international standardization by ISO/IEC, CEN/CENELEC, IEEE, and W3C.
To demonstrate compliance with these requirements, the IDS13 Certification was launched at the 2022 Hannover Fair. This certification offers varying levels of confidence and assurance for both connectors and operating environments. Some connectors carry the "IDS-Ready" label.
5 https://docs.internationaldataspaces.org/ids-ram-4/layers-of-the-reference-architecture-model/3-layers-of-the-reference-architecture model/3_5_0_system_layer/3_5_2_ids_connector#ids-connector-functionalities
6 https://gaia-x.eu/
7 https://www.fiware.org/
8 https://www.sitra.fi/en/
9 https://www.mydata.org/
10 https://internationaldataspaces.org/rule-book-on-structures-and-processes-for-implementing-ids-in-the-real-world/11
https://docs.internationaldataspaces.org/ids-ram-4/introduction/1_1_goals_of_the_international_data_spaces
12https://docs.internationaldataspaces.org/ids-ram-4/layers-of-the-reference-architecture-model/3-layers-of-the-reference-architecture-model/3_5_0_system_layer/3_5_2_ids_connector
13 https://internationaldataspaces.org/use/certification/
This means they have successfully undergone pre-certification, a third-party assessment to prepare for certification.
Implementations of data connectors based on the IDS standard can be found as both closed-source and open-source software. Regarding the latter, the IDSA has developed an IDS Grading Scheme14 that provides a set of standards, processes, and criteria for managing these open-source implementations on the IDSA GitHub15.
Figure 2: Data exchange services performed by a data connector as described in section 3.5.216 of IDS-RAM
Relationship between solutions and data connector frameworks.
Data connectors differ along several dimensions. However, they can be grouped into four main categories: data connector frameworks, generic OSS solutions, proprietary generic solutions, and off-the-shelf or integrated data connectors within data-related products.
Data connector chassis are modular components of the data space used as a foundation for implementing a data connector. Most data connector chassis are available as free and open-source software (FOSS). Building upon this common foundation, extensions are available and under development to create custom solutions. Eclipse Data Space components, the FIWARE ecosystem (including the TRUE Connector), and the IDS Messaging Library are good examples of such frameworks. These frameworks are for developers to use in implementing their solutions.
14 https://github.com/International-Data-Spaces-Association/idsa/tree/main/graduation_scheme
15 International-Data-Spaces-Association/idsa:
16https://docs.internationaldataspaces.org/ids-ram-4/layers-of-the-reference-architecture-model/3-layers-of-the-
reference-architecture-model/3_5_0_system_layer/3_5_2_ids_connector - ids-connector-functionalities
These are not intended for end users to use directly for sharing and consuming data.
Generic open-source solutions offer data connectors that can be directly integrated into an IT landscape and connected to services. They often act as proxies or gateways to enterprise IT services. Sharing and consuming data typically requires configuring components and adding custom extensions. The Data Space Connector and the TNO Secure Gateway are good examples of this type of connector.
Some connectors mentioned later in this report are based on generic open-source solutions and extend them to offer additional generic open-source or proprietary software solutions.
Generic solutions are provided by companies and organizations as proprietary software for general use. Like the open-source solutions described above, they cannot be used directly for sharing and consuming data but require additional configuration and extension, such as nicos' GAIAboX.
Data connectors are out-of-the-box solutions offered as a service or as a directly usable connector solution that requires no development activity to consume and share data. However, configuration and adaptation to enterprise IT services remains necessary, but with minimal effort, such as Sovity's connector-as-a-service offering. In addition to these data connector offerings, this report also includes data connectors already integrated
into data-related products, such as Data Intelligence Hub and Tech2B Connector.
Data Connector Implementations
This section provides an overview of some of the available connectors with information about each one. This list is not exhaustive and will be updated periodically by IDSA to document advancements and new developments. In the next version of the report, the connectors on the list will be compared with the categories mentioned in section 1.4
.
