Making of X-Road 8 – PoC March Status Update

At the beginning of January 2024, NIIS kicked off a proof of concept (PoC) to develop a new major version, X-Road 8 “Spaceship”, which nurtures the proven ecosystem model and security while it takes X-Road to the next level by providing a solid data space infrastructure.

With the proof of concept, NIIS aims to validate the feasibility of replacing X-Road's custom protocol stack with standard data space protocols and align X-Road's trust framework with the Gaia-X trust framework. The project also tries to ensure smooth integration with previous X-Road versions for backwards compatibility, estimate the changes required for information systems when transitioning to X-Road 8, and assess potential changes to existing X-Road components.

In this blog post, we explore the current status of the PoC project.

Support for the standard dataspace protocol

Since the February update, we have continued to work with supporting SOAP services, caching contract negotiation results, defining access permissions using ODRL and message signing and logging.

Caching contract negotiation results

The PoC implementation has been improved by caching and reusing the control plane contract negotiation results. Before, every service request included a complete contract negotiation process that caused several requests back and forth between the data exchange parties. Now, contract negotiation is required only if the service consumer still needs to get a valid access token. In that case, the contract negotiation process is completed, and the consumer receives an access token that can be used for unlimited requests until the token expires. Once the token has expired, the contract has to be renegotiated, and a new access token is generated. Currently, the token is valid for 10 minutes, but the value will be configurable.

Defining access permissions using ODRL

The EDC defines access policy definitions using the Open Digital Rights Language (ODRL). In X-Road, access policies are defined using client identifiers, service codes, HTTP verbs and request paths. An X-Road-specific ODRL profile that extends the ODRL core was implemented to support the current X-Road functionalities. The current PoC implementation supports individual clients and global groups. Instead, local groups still need to be supported. Also, the access rights are maintained in the old Security Server configuration database and periodically exported to ODRL. However, the current implementation already demonstrates that ODRL can support X-Road's access policy definitions and that migration from the current storage format to ODRL can be automated.

Supporting SOAP services

After the latest changes, the PoC implementation supports X-Road's current REST and SOAP interfaces. However, not all the details have been implemented in the PoC, e.g., support for the request hash response header. Nevertheless, existing REST and SOAP service consumer and provider information systems can be connected to the PoC without any additional changes. In other words, it seems that it’s possible to keep the REST and SOAP interfaces fully backwards compatible.

Support for TLS

Initially, connections between two X-Road 8 Security Servers used plain text HTTP connection. The implementation has been updated to use TLS with some of the features already used in X-Road 7.

A provider Security Server must have an authentication certificate issued by an approved Certificate Authority (CA) and registered on the Central Server. A consumer Security Server verifies the certificate and checks its association with the provider Security Server in the global configuration. Instead, in the PoC implementation, the provider Security Server doesn’t verify the consumer Security Server’s authentication certificate. In other words, the PoC implementation doesn’t support mTLS between X-Road 8 Security Servers even though it’s enforced between X-Road 7 Security Servers.

How trust in TLS certificates is currently implemented in X-Road is very X-Road-specific. The issuer of a TLS certificate must be defined as trusted by the X-Road operator, the issuer must be included in the global configuration's trusted list, the TLS certificate must be registered on the Central Server and associated with the Security Server that's using it, the association must be included in the global configuration, and the certificate must have a valid OCSP response. The current implementation adds an extra layer of trust and security, but the downside is that it's not interoperable with other data space technologies. The available data space specifications do not say anything about TLS certificates. The default assumption is that a commonly trusted CA should issue them, but data space-specific exceptions are possible.

Nevertheless, in X-Road 8, it's necessary to reconsider the X-Road-specific additional layers. At least they should be limited on the data plane level so that the trust and control planes remain interoperable with other data space technologies. Otherwise, X-Road is not interoperable with other data space technologies, even if they're based on the same standards and specifications.

Message signing

Another X-Road-specific feature is the signing, logging and timestamping of messages. In March, support for message signatures was implemented in the PoC. The implementation uses the Digital Signature Services (DSS) Java library provided by the European Commission. In practice, signatures and sign certificate OCSP responses are passed in HTTP headers, meaning a separate transfer or wrapper message between the Security Servers is no longer required. The implementation uses the JAdES-B-LT signature, while currently, X-Road 7 uses XADeS signatures. However, the PoC implementation has some limitations compared to the signatures in X-Road 7. In the PoC, the message body and most of the HTTP headers are signed, while in X-Road 7, the signature covers HTTP verb, request path and URL parameters too. Also, batch signing is not supported, and therefore, each message is signed separately.

Message logging

The PoC implementation doesn't support message logging yet because the current timestamping implementation is incompatible with JAdES signatures. However, adding support for timestamping is possible, but it just requires additional work that may not be included in this PoC.

Before proceeding with the implementation, a more thorough analysis of different logging options is needed. For example, the International Data Space Reference Architecture Model version 4 (IDS-RAM 4) by the International Data Spaces Association (IDSA) defines a different approach to logging. According to IDS-RAM 4, the IDS Clearing House logs information relevant to clearing, billing, and usage control. The Clearing House is an intermediary that provides these services to all data space participants. In X-Road, the Clearing House would be the Central Server or a separate component operated by the X-Road Operator.

The data logged by the Clearing House should not include a complete message with payload but only message hashes, signatures and other non-sensitive information needed to identify the data exchange parties and verify the data's integrity. Instead, it's the responsibility of the data exchange participants to store the entire message and other related information required to verify the hashes and signatures. However, a common standard or technical specification defining implementation details for capabilities provided by the Clearing House does not exist.

Instead, in eDelivery, the evidence of data exchange is implemented using AS4 Receipt – an AS4 message that provides non-repudiation of origin and receipt at the level of message exchange. However, the AS4 Receipt doesn't include sensitive data, e.g., the message payload. Instead, the data must be stored separately by the data exchange parties.

X-Road 8 is expected to provide a solid data space infrastructure, so the Clearing House approach to logging should be followed. Since a common standard or technical specification defining the implementation details is currently missing, there's a lot of room for interpretation. From the X-Road members' point of view, the main difference to the current message log implementation is that the message log currently stores all the details required to resolve a dispute. In contrast, the Clearing House approach requires storing part of the evidence separately in an external system.

However, the current message log implementation and the Clearing House approach aren't mutually exclusive. X-Road could follow the Clearing House approach and continue to support the current logging method. In other words, logging could be configurable so that X-Road members can decide which logging alternative to use based on the data exchange use case - or both simultaneously. Service usage policies could be used to set requirements on the logging level that must be used when accessing a specific service. In this way, it would be up to the service provider to define the required logging level for both data exchange parties.

However, logging-related questions are more than just technical since multiple legal and organisational aspects define requirements for the technical implementation. Therefore, the topic needs to be studied more detail to understand the relevant constraints better.

Support for the Gaia-X trust framework

We have set up an internal instance of the Gaia-X Digital Clearing House (GXDCH) services to have more flexibility. The instance includes Gaia-X Registry, Gaia-X Compliance, Gaia-X Notary and Gaia-X Wizard. Also, we have configured an instance of the X-Road Test CA as a trust anchor, enabling us to issue our own certificates trusted by the GXDCH services. Now, we can issue Gaia-X credentials that we can use for testing purposes.

What’s next?

Next, the main challenge is integrating the Gaia-X trust framework with the EDC based X-Road 8 Security Server. The integration includes looking into supporting X-Road's current trust framework and Gaia-X trust framework in parallel since backwards compatibility is required.

In practice, the next step is to set up the EDC Minimum Viable Dataspace (MVD) example that uses credentials issued by the latest version of the GXDCH services. It includes using the EDC Identity Hub component to store Gaia-X credentials and using them in contract negotiation and defining access control policies. Once the MVD setup has been successfully completed, the same changes can be applied to the X-Road 8 Security Server. Finally, the Gaia-X trust framework must be connected with the EDC control and data planes so that all the planes work seamlessly together.

More blog posts about X-Road 8 and the proof of concept will follow. Stay tuned!

Making of X-Road 8 – PoC February Status Update

At the beginning of January 2024, NIIS kicked off a proof of concept (PoC) to develop a new major version, X-Road 8 “Spaceship”, which nurtures the proven ecosystem model and security while it takes X-Road to the next level by providing a solid data space infrastructure.

With the proof of concept, NIIS aims to validate the feasibility of replacing X-Road’s custom protocol stack with standard data space protocols and align X-Road’s trust framework with the Gaia-X trust framework. The project also tries to ensure smooth integration with previous X-Road versions for backwards compatibility, estimate the changes required for information systems when transitioning to X-Road 8, and assess potential changes to existing X-Road components.

In this blog post, we explore the current status of the PoC project.

Getting started

The PoC was kicked off at the beginning of January with an onboarding sprint whose aim was to look into existing data space initiatives and available open-source components and building blocks. Since the main challenge is to replace X-Road’s custom protocol stack with the data space protocol stack, we decided to start with integrating the standard dataspace protocol into X-Road and concentrate on the trust framework later.

First, we investigated available data space connectors and decided to focus on the Eclipse Dataspace Components (EDC) and the Tractus-X ConnectorKit. We were not looking for an off-the-shelf connector product but a customisable open-source framework that can be extended with additional features. Also, our focus was on Java-based building blocks since that’s the technology we’re the most familiar with. The EDC met these requirements well, while the Tractus-X ConnectorKit provided practical examples of implementing extensions on top of the EDC. Therefore, we decided to go with the EDC.

Let’s get working!

We decided to use the X-Road 7 Security Server as a starting point for the PoC and extend its current functionality by integrating the EDC into it. In this way, supporting existing functionalities together with the standard dataspace protocol is easier. The aim is to keep the interface between information systems and the Security Server as-is so that no changes to information systems are required. The interface between information systems and the Security Server is defined by the X-Road Message Protocol for REST and X-Road Message Protocol for SOAP.

In practice, the very first iteration of the X-Road 8 Security Server supports the standard dataspace protocol and X-Road's custom protocol stack. When two X-Road 8 Security Servers communicate, the standard dataspace protocol is used. Instead, when X-Road 8 and X-Road 7 Security Servers communicate, X-Road's custom protocol stack is used. Either way, information systems that exchange data over X-Road don’t know which protocol is used between Security Servers since the protocol between information systems and the Security Server has stayed the same.

Information about the protocols supported by a Security Server was added to the global configuration to enable the selection of the communication protocol between Security Servers. That way, the Security Server knows what protocols other Security Servers support. Therefore, the consumer Security Server can select a protocol the target Security Server supports. Whether the global configuration in its current form will be used in X-Road 8 remains to be seen. Still, a registry containing information about member organisations and Security Servers will be required. In the data space architecture, the component is called the Dataspace Registry.

The control plane and data plane

The standard dataspace protocol is divided into two different layers:

• The control plane manages contract negotiation and controls the data transfer process. It is generic and sharable between different data spaces. The standard data space protocol defines it.

• The data plane performs the actual data transfer. It reuses existing protocols and technologies and doesn't require defining a new data transfer protocol. Also, the data plane is typically use-case and/or data space-specific.

Since the control plane is standardised, the EDC provides the control plane implementation out of the box. However, the standard dataspace protocol specification is still a work in progress and has yet to reach a stable version. Nevertheless, we’re not going to touch the control plane implementation during the PoC but instead rely on the updates provided by the EDC. This approach enables the PoC to concentrate on implementing an X-Road-specific data plane.

X-Road data plane

The control plane discovery, contract negotiation and data transfer management processes are asynchronous and require multiple requests/responses between the data exchange parties. In the PoC, we have chosen to hide this complexity from the connected information systems, which makes it possible to keep the interface between them and the Security Server backwards compatible. Despite the contract negotiation process, a client information system needs to send a single request to the Security Server, and it will receive the requested data as a response. Diagram 1 provides a simplified version (despite the complexity, it's a streamlined version and doesn't include all the steps) of a data exchange process (e.g., a synchronous REST API call) using this approach.

Diagram 1. The first version of the X-Road data plane. Open the diagram in a new tab.

So far, we have implemented the data exchange flow illustrated in Diagram 1 in a backwards-compatible way. However, the first version of the X-Road 8 data plane implementation still needs many critical features required for a production-level implementation. Despite a good start, it's still too early to say whether a fully backwards-compatible implementation is feasible. For example, the following features are missing:

• Caching and reuse of contract negotiation results.

• Define access permissions using the Open Digital Rights Language (ODRL).

• Support for SOAP services.

• Signing of messages.

• Logging and timestamping.

• Authentication and authorisation.

• Support for operational and environmental monitoring.

• Support for federation. 

Light context

Also, we aim to look into providing support for the so-called "light context", which refers to a setup where the Security Server is required only on the service provider side. In other words, a service consumer could invoke services by connecting to the provider's Security Server directly without having a Security Server on the consumer side. Diagram 2 provides a simplified version of the scenario.

Diagram 2. Light context vs. normal request flow.

The idea is that the control plane phase would generate an API key or access token that could be utilised by a consumer information system operating in the light context. The API key or access token could be obtained from the service catalogue separately in advance so that the consumer information system could completely ignore the contract negotiation phase during a data transaction. This would make consuming X-Road services easier compared to the current situation being aligned with the standard dataspace protocol at the same time. The API key generation in the light context is ignored in Diagram 2. However, the light context requires compromises regarding the security guarantees X-Road provides.

What about the Gaia-X trust framework?

One goal of the PoC is to align X-Road’s trust framework with the Gaia-X trust framework. For now, we have used X-Road’s existing trust framework in the PoC since it enabled us to get started faster with the standard dataspace protocol-related changes. However, it doesn’t mean the Gaia-X trust framework has been forgotten.

So far, we have set up our instance of the Gaia-X Digital Clearing House (GXDCH) services. For clarification, our instance is for our internal testing and development purposes only – NIIS is not planning to become an official Gaia-X Digital Clearing House. The reason for setting up our instance instead of using the publicly available GXDCH instances is that it enables us to experiment more freely. For example, we are free to define our own trust anchors. Also, we have set up our instance of the Gaia-X Wizard.

The next step is building technical compatibility and interoperability with the Gaia-X trust framework. It means that X-Road should be aligned with the technical specifications and meet the technical requirements of the Gaia-X trust framework. This is an enormous change for X-Road since the current trust framework is based on X.509 certificates. In contrast, the Gaia-X trust framework is based on the Self-Sovereign Identity (SSI) principles and technologies like W3C Verified Credentials, Linked Data (RDF) and Decentralised Identifiers (DID).

Support for the Gaia-X trust framework can be achieved in different ways. For example, X-Road could continue to use its current trust framework and support the Gaia-X trust framework. Alternatively, X-Road's trust framework could be based on Gaia-X's technical specifications, making it technically interoperable with Gaia-X by default. A final decision hasn’t been taken yet, but at this point, making X-Road’s trust framework aligned with Gaia-X’s technical specifications is the more likely option. That’s the approach that will be investigated in the PoC.

However, making X-Road’s trust framework technically compatible and interoperable with the Gaia-X trust framework doesn’t automatically make all X-Road ecosystems and user organisations Gaia-X compliant. Instead, it makes X-Road user organisations technically compatible with Gaia-X, but being Gaia-X compliant requires that the user organisations meet the requirements of the Gaia-X Compliance rules. In other words, using X-Road 8 doesn't automatically make an organisation a Gaia-X participant. Instead, X-Road 8 will provide the technical means to become a Gaia-X participant when an organisation meets the requirements of the Gaia-X Compliance rules.

What’s the relationship between X-Road 7 and X-Road 8?

The decision to use the X-Road 7 Security Server as a basis for the X-Road 8 PoC doesn’t mean that X-Road 8 will be based on X-Road 7. It might be that some X-Road 7 components will initially be included in X-Road 8 to make the upgrade from X-Road 7 to X-Road 8 smoother for existing users. Once the upgrade has been completed, the X-Road 7 components can be dropped from X-Road 8.

I also want to highlight that new X-Road ecosystems that will be set up using X-Road 8 don't have to deal with X-Road 7. If backwards compatibility with X-Road 7 is not required, X-Road 8 can be deployed without X-Road 7-related dependencies.

What’s next?

Currently, the PoC is divided into two work streams:

• Support for the Standard Dataspace Protocol.

• Support for Gaia-X trust framework.

Support for the Standard Dataspace Protocol work stream continues to work on the X-Road data plane and extend its functionalities. Next, we will look into supporting SOAP services, caching contract negotiation results, defining access permissions using ODRL and message signing and logging.  

Instead, support for the Gaia-X trust framework work stream continues experimenting with the GXDCH services and the Gaia-X Wizard. Next, we plan to utilise existing sign keys and certificates of X-Road member organisations to create and sign Gaia-X Verifiable Credentials and Verifiable Presentations.

More blog posts about X-Road 8 and the proof of concept will follow. Stay tuned!

Unveiling the Open Data Product Specification (ODPS): A Holistic Approach to Data Products

In the ever-evolving landscape of data management, the concept of data products has gained prominence, offering a standardized approach to packaging and consuming relevant data resources. At the forefront of this movement is the Open Data Product Specification (ODPS), a vendor-neutral, open-source, and machine-readable data product metadata model. Let's delve into the unique features that set ODPS apart in the realm of data product standards.

Technical Emphasis and Current Version. ODPS places a strong emphasis on technical aspects, defining objects and attributes related to data products such as data pipelines, access, data models, and deployment. The current latest production version, 2.1, reflects continuous refinement, with the next release scheduled for early 2024. Naturally, ODPS has a governance model which provides needed structure and guidance for long-term support. 

Holistic Metadata Model: ODPS distinguishes itself by offering a full-stack metadata model that includes attributes for pricing plans, built-in support for internationalization (i18n), contract elements, Service Level Agreements (SLA), DataOps, licensing, and provider details. This comprehensive approach sets ODPS apart from other standards and enables the complete decoupling of data products from vendor systems, fostering decentralization.

Business-Focused Standards: The shift towards data product-focused standards signifies the growing need for a refreshed approach to treating data as a business-related commodity rather than a purely technical asset. ODPS addresses this need by providing an easy-to-adopt, developer-friendly standard with a distinct business focus, challenging the limitations faced by alternatives like DCAT.

Community Recognition: In Europe, the Data Spaces community acknowledges the importance of standards in creating data products and product offerings. Three standard templates stand out: DCAT, ODPS, and the TM Forum product model. ODPS, with its unique attributes, has gained recognition as a key player in this space.

Comparison with Alternatives: While ODPS differs significantly from the popular DCAT (Data Catalog Vocabulary), it's important to note that various standards, such as Data Product Descriptor Specification and Data Product Specification, share some similarities. However, ODPS stands out due to its holistic approach and inclusion of critical business-related attributes.

ODPS vs. DCAT: Understanding the Differences: A common question arises regarding the differences between ODPS and DCAT. A brief comparison reveals key distinctions:

• ODPS has a flat structure, while DCAT follows a linked structure.

• ODPS focuses solely on a single data product, whereas DCAT includes cataloging features.

• ODPS is lightweight and fast to understand, in contrast to the exhaustive nature of DCAT.

• Both share numerous attributes, but ODPS is tailored for data marketplace and sales, including data exchange.

• DCAT, maintained by W3C, leans towards cataloging and sharing, while ODPS is community-maintained and oriented towards data marketplace dynamics.

The value of ODPS in the eyes of new emerging companies is big. They consider ODPS to provide same value as DCAT, but with faster adoption as we can see from the testimonial of one architect responsible for a trusted data exchange platform: “For me as a software architect/developer the ODPS standard is a lot more approachable. I think the usage of JSON is also a lot more flexible than the RDF structure proposed by DCAT. Furthermore, the current document structure of ODPS is more clear and easier to understand with almost the same amount of properties.”

Extensibility with OpenAPI Specification: ODPS offers flexibility by easily integrating with the OpenAPI Specification (OAS), including business, legal, and SLA attributes that may be missing from the standard package. A proof-of-concept test conducted in December 2023 showcased the seamless extension of ODPS in the API description of X-Road.

Service Discovery in X-Road

When services are connected to X-Road, their technical service descriptions are published on the Security Server by the Security Server administrator. The supported formats are Open API Specification (OAS) for REST services and WSDL for SOAP services. The service descriptions can then be accessed using a service discovery mechanism provided by X-Road. However, the mechanism is quite technical and requires direct access to the Security Server's messaging interface. Also, getting a list of all services available in the ecosystem would require querying each Security Server separately. Therefore, a more user-friendly service catalogue is needed.

A service catalogue is a web portal that contains descriptions of all the services available in the ecosystem. The primary purpose of the service catalogue is to provide a user-friendly channel to search and discover available services.

Image 1. Collecting service descriptions from Security Servers to Service Catalogue.

When implementing the service catalogue, collecting the service descriptions from the Security Servers can be automated. In that way, the descriptions need to be maintained in a single place only, and all the changes in the source are automatically updated to the catalogue. Nevertheless, additional organisation and data product metadata must be manually added and maintained on the catalogue by an administrator representing the organisation owning the service. The metadata may include any information related to the service and its use, e.g., a more detailed description of the data set, terms and conditions, contact information, pricing information, SLAs, etc.

Using ODPS in X-Road

 Instead of maintaining the data product related metadata manually on the service catalogue, the metadata can be published in a machine readable format on the Security Server using ODPS. In practice, the OAS service descriptions can be extended with ODPS by including the selected parts of ODPS in them.

Image 2. Including ODS attributes in an OAS service description on the Security Server.

Starting to use the ODPS extension in the X-Road OAS service descriptions doesn’t require any code or configuration changes to the Security Server. It’s enough to update the OAS service description and the changes are immediately available over X-Road’s service discovery mechanism. This makes them accessible to all the service consumers and the X-Road Catalog extension (and other metadata harvesters) that’s used to centrally harvest service descriptions from all the service providers within an X-Road ecosystem. The approach was validated in a small PoC implemented in December 2023.

Image 3. An OAS service description with ODPS attributes. A full example is available here.

The attributes provided by ODPS can be utilised by the service catalogue which reduces the number of metadata that the X-Road Member Administrator must add to the catalogue manually. Using ODPS makes the information available in the service catalogue more unified, improves its quality, makes it machine readable, and eases its maintenance. However, utilising ODPS attributes in service catalogues provided by X-Road Operators may require changes to the catalogues so that they’re able to process and visualize the attributes. Nevertheless, at least some of the catalogues support visualizing and downloading raw OAS service descriptions which makes ODPS available to the catalogue users without additional changes. Since the Security Server already supports adding ODPS attributes to the OAS service descriptions, service providers may start utilising the extension immediately.

Conclusions

In conclusion, the Open Data Product Specification emerges as a unique and impactful player in the data product standards landscape, providing a holistic model that caters to the evolving needs of the data economy. Its focus on business-related attributes, coupled with technical precision, positions ODPS as a valuable asset for organizations navigating the complexities of data management in the modern era.

Extending X-Road OAS service descriptions with ODPS makes data product-related metadata available in the service catalogue more unified, improves its quality and enables more automation in the maintenance process. Service providers may start to use ODPS right away by updating their OAS service descriptions. Depending on the service catalogue solution, the ODPS attributes may be accessible in the catalogue without any additional changes to the catalogue software. However, visualising the ODPS-related information in a more user friendly way probably requires some changes. Overall, implementing ODPS is a low hanging fruit that improves the quality of the service metadata in X-Road ecosystems.

---

Jarkko Moilanen is the founder and strategy group chair of the Open Data Product Initiative.

Jarkko is an experienced senior data management professional with strong expertise on data monetization, platform and API economy. Currently Jarkko spearheads the Open Data Products track within the Data Enablement Program's data exchange platform under the aegis of the Abu Dhabi government. Jarkko also had a role in early phases of taking X-Road into national use in Finland and introducing REST APIs to the platform.

Open Data Product Specification (ODPS) which has matured into governed standard was ignited 2019 by Jarkko. ODPS was initially a sidetrack initiated as part of Data Product Toolkit development, but later was discovered valuable enough to be independent project as it gained traction from companies around the world.

On the international front, Jarkko was the first MIT CDOIQ Country CDO Ambassador for Finland for a few years. On top of practicing the data value creation, Jarkko has also written several business focused books on AI ("AI-Powered Data Products", 2023), API economy ("API Economy 101", 2018) and Data Economy (Deliver Value in the Data Economy, 2022).

NIIS Announces Proof of Concept for Revolutionary X-Road 8 “Spaceship” 🚀

NIIS is thrilled to announce the start of a proof of concept to develop a new major version, X-Road 8 “Spaceship”, which nurtures the proven ecosystem model and security while it takes X-Road to the next level by providing a solid data space infrastructure.

With the proof of concept, NIIS aims to validate the feasibility of replacing X-Road’s custom protocol stack with standard data space protocols and align X-Road’s trust framework with the Gaia-X trust framework.

In this blog post, we explore the X-Road 8 “Spaceship” plans and the scope of the proof of concept. Are you ready for the launch?

The current state of X-Road

X-Road provides a secure and unified way to exchange data between X-Road member organisations. X-Road enables data exchange within one X-Road ecosystem and between two independent X-Road ecosystems through federation. Members of the federated ecosystems can publish and consume services with each other as if they were members of the same ecosystem.

The interoperability within one X-Road ecosystem and between two independent X-Road ecosystems is based on the X-Road protocols. X-Road defines and implements a set of protocols that guarantee the interoperability and security of the data exchange. Multiple standards are used in X-Road's implementation, but no X-Road protocols have been standardised.

X-Road’s source code and all X-Road protocols are open, and documentation is publicly available, so anyone is free to implement X-Road’s protocol stack. However, no other open-source products implement X-Road’s protocol specifications. Instead, one commercial data exchange solution implements X-Road protocols to some extent, but estimating how compatible the implementation is with X-Road isn't easy.

X-Road is being developed as a product, and the protocol specifications are adapted according to the needs coming through the product development. Also, the specifications include some product-specific details like component version numbers. Therefore, developing and maintaining another product implementing X-Road’s protocol specifications would be challenging since the specifications include product-specific details. In practice, it’s unlikely that alternative implementations of X-Road’s protocol specifications will be seen, and the specifications will remain specific to the X-Road product.

Image 1. The current state of X-Road.

eDelivery for cross-border data exchange in Europe

eDelivery is a building block of the European Commission and is the default technology for cross-border data exchange in Europe. eDelivery is based on the AS4 specification, and multiple open-source and commercial solutions that conform to the specification are available. Therefore, users of eDelivery are free to choose any of the available solutions; alternatively, they can also decide to implement their own. Either way, interoperability between different solutions is based on the common AS4 specifications. The European Commission is coordinating the development of the specifications independently from any of the implementing products.

Since X-Road and eDelivery are based on different specifications, they’re not directly compatible with each other. Making X-Road and eDelivery compatible requires implementing a custom gateway component responsible for conversions between X-Road and eDelivery protocols. However, implementing one common gateway is not feasible because of the differences between various eDelivery ecosystems. Therefore, multiple gateway implementations are required. Even though eDelivery is based on the common AS4 specifications, the specifications leave room for ecosystem-specific differences in the implementation.

Data spaces enable data sharing and exchange within and across data ecosystems

Data space is a distributed system defined by a governance framework that enables secure and trustworthy data transactions between participants while supporting trust and data sovereignty. Data spaces enable data sharing and exchange within and across different data ecosystems. Just like eDelivery, data spaces are based on a set of common specifications and standards. Interoperability within and between data spaces requires following the same specifications and standards. Participants of different data spaces can choose the software products they use as long as the products implement the required standards.

Multiple organisations are developing specifications and standards for data spaces, e.g., the International Data Spaces Association (IDSA), Gaia-X, Fiware, the Data Spaces Support Centre (DSSC) and the Data Spaces Business Alliance (DSBA). The organisations work together to keep the specifications and standards aligned and not create multiple overlapping or competing specifications.

Currently, connecting X-Road to a data space requires implementing a custom gateway component responsible for conversions between X-Road and data space protocols. The same approach applies to eDelivery and other data exchange ecosystems, too.

The target state with X-Road

Implementing custom gateway components that enable X-Road to communicate with other data exchange ecosystems is not a feasible solution in the long term. The number of gateway components that need to be developed and maintained grows over time, and achieving full compatibility through them is highly challenging. Therefore, a better approach for X-Road is to move from X-Road-specific protocols to the data space protocol stack.

In practice, the protocols used in data spaces can be divided into three different layers:

• Trust plane manages participant identities and authentication. It can be sharable between different data spaces or data space-specific.

• Control plane manages contract negotiation and controls the data transfer process. It is generic and sharable between different data spaces. It’s defined by the standard data space protocol.

• Data plane performs the actual data transfer. It reuses existing protocols and technologies, and therefore, it doesn’t require defining a new data transfer protocol. Also, the data plane is typically use-case and/or data space specific.

Replacing the current X-Road-specific protocols with the data space protocol stack would mean that the same protocols are used within one X-Road ecosystem, between two federated X-Road ecosystems and between X-Road and other data exchange ecosystems. That way, the same set of protocols is used for internal and external data exchange.

Image 2. Target state with X-Road.

However, being interoperable with another data exchange ecosystem requires that the other ecosystem implements the same data space protocols, too. At least right now, some data exchange ecosystems call themselves data spaces even if they’re not using the data space protocol stack. Also, not all the data exchange ecosystems will start using the stack in the future either. In other words, the change doesn’t guarantee full interoperability with all different data exchange ecosystems. Still, at least it improves the current situation significantly and increases the level of standardisation in X-Road’s implementation.

The data space protocol stack covers various aspects of data exchange, e.g., usage conditions and policies, contract negotiation, and data transfer process. They do not set restrictions to the actual data transfer protocol or wire protocol – the protocol that is used to transfer the data. In other words, any existing transfer protocol can be used, which enables the use of existing solutions and supports many different transfer protocols. This is a big difference compared to X-Road and eDelivery, which both have their own transfer protocol.

Another difference is related to expressing and enforcing usage conditions and policies. In data spaces, they’re expressed in human-readable and machine-readable ways, which can be enforced in organisational or technical manners. Instead, in X-Road, they’re expressed in a human-readable way, which can be enforced only in an organisational manner.

Trust framework is another crucial factor in interoperability within and between data spaces. Different data space initiatives have their own trust frameworks, but the initiatives are currently aligning their trust frameworks to avoid those becoming a blocker for interoperability between various data space initiatives.

Many data space initiatives have trust frameworks based on technologies different from X-Road’s trust framework, e.g., Gaia-X. X-Road's current trust framework is compatible with the Gaia-X trust framework on some level. However, in practice, many additional steps are required for an X-Road member to become a member of a Gaia-X-compliant data space utilising the X-Road trust framework. Therefore, replacing X-Road’s trust framework with the trust framework of a selected data space initiative is required to maximise interoperability.

Image 3. The data space protocol stack.

Implementing the change doesn’t mean having to build everything from scratch. Several well-established and conformant open-source solutions are already available that can be taken as a basis for the implementation, e.g., Eclipse Dataspace Components (EDC). Many of the solutions support or are going to support the data space protocol stack, which means that it would be enough to implement changes that enable current X-Road users an easy migration path to the new version. The goal would be minimising the changes the current X-Road users must implement.

NIIS is currently responsible for developing and maintaining the X-Road core and extensions. Despite the worldwide X-Road community, external contributions have remained low. Migrating to the data space protocol stack by taking an existing open-source solution as a basis would change the current situation.

It is fair to assume that the chosen open-source solution already has a well-established developer community contributing to the core solution's maintenance and development. For example, the EDC project is hosted and facilitated by the Eclipse Foundation AISBL. The foundation is a Europe-based not-for-profit organisation that acts as a steward of the Eclipse open-source software development community. When NIIS isn’t alone responsible for the core development and maintenance, it enables NIIS to concentrate on areas that bring the most value to the NIIS members.

The downside of the approach is that NIIS would lose some control over the direction of X-Road’s future development because various data space initiatives lead the development of different layers of the data space protocol stack. NIIS could contribute to the development and suggest changes as needed, but it wouldn’t have the power to make decisions alone as it has now. However, it is unlikely that NIIS members’ needs would differ significantly from the needs of others. Also, the development of the X-Road core would be partly in the hands of another open-source project, and NIIS would be one of the contributors instead of being the project owner.

Data Space Infrastructure for X-Road

Implementing X-Road on top of a well-known open-source project conforming to the data space protocol stack would make X-Road interoperable with other data exchange ecosystems using the same protocols. However, connecting X-Road to data exchange ecosystems following other specifications would still require custom gateway solutions. Connecting X-Road to any other data exchange ecosystem today requires a custom gateway solution, so being interoperable with many other ecosystems would significantly improve X-Road.

Image 4. Data space infrastructure for X-Road.

Another benefit of the new approach is more flexibility in supporting different data transfer protocols. Currently, adding support for new data transfer protocols requires a lot of effort and is challenging to implement in a backwards-compatible manner. Instead, the standard data space protocols enable using various data transfer protocols which allows X-Road to support new transfer protocols more easily, e.g., data streaming, asynchronous data exchange.

Choosing the right trust framework for X-Road is important. To maximise the benefits, the best alternative is to utilise and extend the trust framework of a selected data space initiative, e.g., Gaia-X. In that way, there's no need to maintain a fully X-Road-specific trust framework and make it interoperable with the trust frameworks of different data space initiatives. Also, available data space connectors that can be used as a basis for the X-Road implementation are designed to support different trust frameworks.

Overall, the new approach makes it easier for X-Road users to exchange data with other data exchange ecosystems. Also, it helps to keep X-Road aligned with European data exchange initiatives and policies and relevant for many years to come. Most likely, it would also encourage other countries to select X-Road as a solution compatible with the European infrastructure.

The X-Road 8 Proof of Concept in 2024

The changes to X-Road presented in this blog post are significant and the plan is to implement them in the next major version, X-Road 8 “Spaceship”. The very first step is to implement a proof of concept that is divided into two phases taking place in 2024.

The first phase takes place in H1/2024 and its aim is to validate the feasibility of the selected approach – replacing X-Road's custom protocol stack with the data space protocol stack. The project also tries to ensure smooth integration with previous X-Road versions for backwards compatibility, estimate the changes required for information systems when transitioning to X-Road 8, and assess potential changes to existing X-Road components.

The proof of concept results are expected for review in May 2024. The second half of the year will witness another project to test the results within Estonia’s X-Road ecosystem.

More blog posts about X-Road 8 and the proof of concept will follow. Stay tuned!

X-Road 7.4.0 Is Here

The year 2023 is ending, and it's time to publish X-Road 7.4.0! The previous version, 7.3.0, was released in June, and it provided a new Central Server UI and management REST API. Version 7.4.0 includes several improvements for the Central Server, but the Security Server hasn’t been forgotten either.

The beta version is already out, and the official release version will be published in December 2023. The release notes are available here. Please note that the beta version doesn’t include all the changes mentioned in the release notes.

Let’s see what the highlights of the version 7.4.0 are!

Publishing global configuration over HTTPS

Until now, the Central Server and Configuration Proxy have only supported publishing global configuration over HTTP. Since the global configuration is signed, its integrity and authenticity are guaranteed despite lacking HTTPS support. Nevertheless, securing the global configuration download with HTTPS helps to maintain the privacy of the global configuration data.

A new private key and a self-signed TLS certificate are created automatically when installing a new Central Server or upgrading an existing installation from an older version. After the installation or upgrade, the Central Server Administrator must manually apply for a TLS certificate from a trusted Certificate Authority (CA) and then configure the certificate. This step is required because the Security Server does not trust the new automatically generated self-signed certificate by default. The Security Server supports disabling certificate verification, but disabling it in production environments is not recommended.

Starting from version 7.4.0, the global configuration will be available on the Central Server and Configuration Proxy ports 80 and 443. The old HTTP endpoint on port 80 guarantees that older Security Servers not supporting downloading global configuration over HTTPS continue to work normally. Also, the new Security Servers that support downloading global configuration over HTTPS will fall back to HTTP if the TLS certificate of the new HTTPS endpoint is not configured correctly.

Changes in the global configuration download ports may cause changes to firewall configuration on the Central Server, Configuration Proxy and Security Server. Inbound traffic from the Security Servers to port 443 must be allowed on the Central Server and Configuration Proxy. Similarly, outbound traffic to the Central Server and Configuration Proxy port 443 must be allowed on the Security Servers.

When upgrading the Central Server from version < 7.4.0 to version >= 7.4.0, the configuration anchor must be re-generated, distributed to all the Security Server administrators through an external channel (e.g., email) and imported to each Security Server to enable downloading global configuration over HTTPS. Without the new configuration anchor Security Servers will continue to download global configuration over HTTP.

Enforcing new key creation when generating CSR on the Security Server

A good security practice is creating a new key pair when applying for a new authentication or a sign certificate on the Security Server. Starting from version 7.4.0, the Security Server supports enforcing new key creation when generating a certificate sign request (CSR) on the Security Server. The feature is enabled by default, which means that it's not possible to generate a new CSR for a key that already has an existing certificate. The feature can be disabled by setting the “proxy-ui-api.allow-csr-for-key-with-certificate” system property to true on the Security Server.

Configuring a minimum required client Security Server version

From version 7.4.0, service providers can configure a minimum required X-Road software version for client Security Servers. It means that client Security Servers older than the configured version cannot access services anymore.

Service providers can configure the required minimum version using the “proxy.server-min-supported-client-version” system property. The property has no value by default, meaning a minimum version hasn't been set. Instead, when the value is set, all the minor and patch versions starting from the configured version are approved.

X-Road Operators may set this value in the country-specific meta packages, e.g., Estonia and Finland are setting the value to the three latest versions, which means that requests that are originating from client Security Servers older than version 7.2.0 are not accepted anymore in the Estonian and Finnish X-Road ecosystems. The value is rolling, meaning it changes for every X-Road minor version, e.g., when version 7.5.0 is released, the minimum supported version is 7.3.0.

Changes in default client information system communications ports

The default client information system inbound communication ports are changed from 80 to 8080 and from 443 to 8443 on Ubuntu-based Security Servers. In this way, the default ports are consistent regardless of the hosting operating system.

The change is only applied to new installations, and existing ones are unaffected by it. Also, if custom ports have been configured locally by the Security Server administrator, they are unaffected.

X-Road Operators may set the default port values in the country-specific meta packages. The Estonian ecosystem uses ports 80 and 443 by default for new installations after the change. In other words, the Estonian X-Road users organisations are unaffected by the change.

LDAP group mapping support

The Central Server and Security Server administrator user permissions are managed using X-Road-specific Linux groups mapped to X-Road user roles. An administrator must be a member of the X-Road-specific groups to access the management UI.

Starting from version 7.4.0, it’s possible to map additional Linux groups to X-Road user roles using the “proxy-ui-api.complementary-user-role-mappings” and “admin-service.complementary-user-role-mappings” system properties. It enables granting administrators permissions to the management UI using existing Linux groups instead of adding administrators to the X-Road-specific groups. In this way, existing Linux groups can be mapped to the X-Road user roles, and there's no need to change users' groups.

Rotating global configuration sign keys

The Central Server global configuration sign keys are included in the configuration anchor imported to the Security Server. Using the information in the anchor, the Security Server can verify the global configuration signature. Suppose the sign keys are rotated. In that case, a new configuration anchor must be generated on the Central Server, distributed to all the Security Server administrators through an external channel (e.g., email) and imported to each Security Server. Only after that, the changes are applied by each Security Server.

Starting from version 7.4.0, the Central Server global configuration sign keys are included in the global configuration. Thanks to this, rotating the sign keys is possible without importing a new version of the configuration anchor to each Security Server. Instead, the Security Server gets the updated configuration information from the global configuration directly and applies it immediately.

Importing the configuration anchor to the Security Server manually is also possible. It's required when the Security Server is initialised for the first time and when a Security Server has been offline or turned off when the global configuration sign key has been rotated.

Disabling a subsystem temporarily

Disabling a subsystem temporarily enables the Security Server administrator to disable a subsystem so that all the services added under the subsystem cannot receive requests. If the same subsystem is registered on multiple Security Servers, disabling the subsystem on one Security Server doesn’t affect the others. When a subsystem is disabled, the association between the subsystem and the Security Server where the subsystem is disabled is not visible to other Security Servers in the ecosystem. This applies to the local and federated ecosystems.

Instead, disabling a subsystem doesn't change any configuration items regarding the subsystem or its services. It makes the association between the Security Server and subsystem invisible to other Security Servers so that they cannot send requests to it. Similarly, a disabled subsystem cannot be used as a service client either. 

Disabling and enabling a subsystem has a delay of a couple of minutes when the default configuration is used. During the delay, the subsystem may still receive service requests. However, the requests are not forwarded to the service provider information systems by the Security Server. Instead, all received requests will fail and an error message is returned to the service consumer. This is because the subsystem state is maintained on the Central Server, and updating it requires the Security Server to communicate with the Central Server. The length of the delay depends on the global configuration generation interval on the Central Server and the global configuration download interval on the Security Server. Other Security Servers become aware of a change in a subsystem’s state only after they download an updated version of the global configuration. Therefore, they may try to contact services under a disabled subsystem during the delay period.

Changing the Security Server address

The Security Server address is added to the Central Server when the authentication certificate is registered. Before version 7.4.0, updating the address afterwards required the Security Server administrator to contact the Central Server administrator over some external channel, e.g., email. Instead, starting from version 7.4.0, updating the address using the Security Server administrator UI or management REST API is possible.

The Security Server administrator can update the Security Server address by using the Security Server administrator UI and management REST API. The change is approved automatically by the Central Server and doesn't require manual approval from the Central Server administrator. By default, the change becomes visible to all the Security Servers in the ecosystem within a couple of minutes. However, the length of the delay depends on the global configuration generation interval on the Central Server and the global configuration download interval on the Security Server.

Migrate from Akka to gRPC

In September 2022, Lightbend announced that Akka, one of the third-party open-source libraries that X-Road heavily depends on, would change its licensing model. The new licensing model is commercial, but open-source projects meeting specific criteria may apply for an exception. However, the conditions of the license granted to open-source projects differ from the Apache 2.0 open-source license that Akka used before the change. From X-Road's perspective, the new license includes some unfavourable conditions, so Akka has been replaced with gRPC in X-Road.

Because of the change, any Akka-related system properties are not supported anymore starting from version 7.4.0, e.g., “<component>.akka.remote.artery.advanced.maximum-frame-size”. If Akka-related system properties have been used to override Akka’s default configuration values, they should be manually removed from the configuration.

Technical changes and upgrades

Besides functional changes and new features, several more technical changes have also taken place under the hood. The UI framework has been upgraded from Vue 2 to Vue 3, and the Spring Boot backend framework has been upgraded from version 2 to 3.

Also, the supported Java version has been upgraded from Java 11 to Java 17. The new Java version is installed automatically on other supported platforms except Red Hat Enterprise Linux 7 (RHEL7). On RHEL7, the Security Server administrator must manually configure a new package repository. More information about the upgrade process will be available in the release notes.

Other improvements and updates 

Besides the already-mentioned features, version 7.4.0 includes many other minor improvements and updates. For example, a new command line tool to verify message log archive hash chains, remove deprecated rate limiting parameters on the Security Server, improve the Central Server UI, publish REST services using OpenAPI 3.1, replace the SHA-1 hashing algorithm with SHA-256, and much more. To fully understand all the changes, please review the release notes document.

What’s next?

Initially, support for automated certificate management through the Automatic Certificate Management Environment (ACME) protocol was scheduled for version 7.4.0. Unfortunately, support for ACME didn't make it to version 7.4.0, but it will be included in version 7.5.0 instead. Other changes that will be introduced in version 7.5.0 include but are not limited to, support for Ubuntu 24.04 LTS and support for Red Hat Enterprise Linux (RHEL) 9. Version 7.5.0 will be released during Q2 in 2024.

At the same time, when developing X-Road 7, NIIS is already working on the next X-Road major version – X-Road 8. If you’re interested in hearing more about the topic, please check out the talk that I delivered at the X-Road Community Event 2023 in September. More news regarding X-Road 8 will be published soon. Stay tuned!

Is X-Road a Data Space Technology?

Data spaces are a hot topic right now in the field of data exchange and interoperability. However, it's not a new concept since its roots date back to 2005. The original definition was somewhat technical, but over the years, the term has expanded to cover all four layers of interoperability defined by the European Interoperability Framework (EIF).

Today, the term data space has several different definitions depending on whom you ask. Multiple international initiatives are working on data spaces to create common governance models, specifications, reference implementations, etc. Let's look at the definitions created by some of those initiatives.

---

”An infrastructure that enables data transactions between different data ecosystem parties based on the governance framework of that data space. Data space should be generic enough to support the implementation of multiple use cases.”

Source: Data Spaces Support Centre - DSSC Glossary - Version 1.0

”Decentralised, governed and standard-based structure to enable trustworthy data sharing between the data space participants on a voluntary basis. Data spaces may be purpose- or sector-specific, or cross-sectoral. Common European data spaces are a subset of data spaces within the scope of EU policies.”

Source: Gaia-X Glossary

”A Dataspace is a set of technical services that facilitate interoperable asset sharing between entities.”

Source: IDS Dataspace Protocol v0.8

”A data space is a federated data ecosystem within a certain application domain and based on shared policies and rules.”

Source: Sitra Rulebook for a Fair Data Economy

”A common European data space brings together relevant data infrastructures and governance frameworks in order to facilitate data pooling and sharing.”

Source: Commission Staff Working Document on Common European Data Spaces

---

When looking at the definitions, it’s clear that they are defining the same subject even though they’re approaching it from different angles. Also, it’s evident that data spaces are more than a technology since governance and policies play a significant role in the definitions.

The first version of X-Road was released in Estonia in 2001 – four years before the term data space appeared in an article for the first time. The technical implementation of X-Road has evolved and changed over the years, but the main concept has remained the same since the beginning. Also, X-Road is not only about technology; it also has an organisational model and trust framework. When comparing X-Road to the data space definitions, it’s easy to find many common factors and similarities. In fact, the definitions could very well be used to describe X-Road on a high level. But does that make X-Road a data space technology?

To answer the question, digging deeper into data spaces is necessary. Let's look at more detailed documentation and specifications to see how well X-Road is aligned with them.

The International Data Spaces Association

The International Data Spaces Association (IDSA) is a not-for-profit organisation aiming to create a global standard for international data spaces (IDS) and foster the global data spaces community. IDSA provides several data spaces related assets, for example:

• IDS Rulebook

• IDS Reference Architecture Model (IDS-RAM)

• IDS G

• IDS Dataspace Protocol.

Let’s study some of the assets provided by the IDSA and see how well X-Road is aligned with them. However, this will not be a complete comparison between the IDSA assets and X-Road. Instead, only selected parts of specific assets will be reviewed to highlight similarities and differences. Studying the source materials is recommended if you're interested in the topic in more detail.

Data space fundamentals

The IDS Rulebook defines foundational concepts of data space that seem to apply to X-Road on a high level:

• Establishing trust

• Data discoverability

• Data contract negotiation

• Data sharing & usage

• Observability

• Vocabularies and semantic models.

The IDS Rulebook and X-Road have the same vision of enabling data sovereignty and creating a trusted data-sharing ecosystem. They’re both based on the idea that trust is rooted in one or more trust anchors, trusted participants must meet specific policies that can vary between different ecosystems, and data sharing consists of peer-to-peer interactions between the participants.

Policies play an essential role in any data ecosystem. The IDS Rulebook defines multiple policy groups and the relationships between them. The same policy groups apply to X-Road, too, with the difference that X-Road doesn't define the structure and relationship of the policies or offer tools to express them. For example, the IDS specifications cover defining usage and contract policies and negotiating them as a part of the data exchange process. In X-Road, it’s up to the data exchange parties to agree on those policies using some external channel, e.g., email or service catalogue. Also, X-Road supports managing access to services. Still, the access policies are not technically associated with any other policies since the other policies are created and maintained outside of X-Road.

Operating models

Image 1. Data space operating models. Source.

The IDS Rulebook defines three different operating models for data spaces:

• Centralised data space authority

• Federated / distributed data space authority

• Decentralised data space authority.

 In the context of X-Road, the data space authority is equal to the X-Road Governing Authority. X-Road supports centralised and federated authority. Centralised authority is the most typical alternative for X-Road, and it's the model that a single X-Road ecosystem uses. Instead, when two X-Road ecosystems are federated, the responsibility can be considered distributed between the authorities of the federated ecosystems. However, even in a federated setup, the authorities only have control over their ecosystem.

Conceptual model

Image 2. Data space conceptual model. Source.

The conceptual model defined by the IDS Dataspace Protocol is very similar to X-Road’s conceptual model. Doing a mapping between the models is straightforward:

•  Dataspace – X-Road Ecosystem

• Dataspace Authority – X-Road Governing Authority

• Participant – Member

• Participant Agent – Security Server

• Identity Provider – Certificate Authority

• Credential Issuer – N/A

• Dataspace Registry – Central Server, Service Catalogue

The descriptions of the organisational entities (Dataspace Authority, Participant, Identity Provider) are pretty well aligned. Instead, there are differences in the descriptions of the technical entities (Participant Agent, Dataspace Registry). The technical differences are covered in the Components section.

Components

Image 3. Interaction between data space components. Source.

Comparing the components provides a good overview of the similarities and differences between the IDS-RAM and X-Road from a technical perspective. The IDS Identity Provider is not actively involved in the data exchange process, so it's omitted from the illustration.

IDS Identity Provider

The IDS Identity Provider consists of three complementary components:

• Certificate Authorities (CAs) issue and manage technical identity claims.

• The Dynamic Attribute Provisioning Service (DAPS) issues short-lived tokens providing information about Connectors.

• The Participant Information Service (ParIS) provides business-related details on Participants.

The IDS Identity Provider combines X-Road's Certificate Authority and Central Server. In X-Road, the roles of the CA and DAPS are combined since Member and Security Server-related information is included in the certificates. At the same time, the IDS Identity Provider decouples the certificate and the associated information. In that way, it's possible to alter the attributes of a certificate, e.g., attach new attributes to a certificate, without reissuing the certificate.

In X-Road, the Central Server stores some business-related information about Members. In addition, the Service Catalogue is a complementary X-Road component that may provide additional information about Members. Therefore, the Central Server and Service Catalogue are equal to the ParIS.

IDS Metadata Broker

The IDS Metadata Broker is basically a registry of IDS Connectors, their capabilities, characteristics, and metadata of the data they offer. The IDS Metadata Broker provides endpoints for registering, publishing, maintaining, and querying the metadata. The X-Road Central Server has a similar role as a registry of Members and Security Servers, with the difference that the Central Server doesn't store any information about the data or services offered by Security Servers. Instead, the Service Catalogue is an optional X-Road component that provides information about the data and services.

IDS Connector

The IDS Connector is the point of access to a data space, providing standardised data exchange between Participants. The IDS Connector is connected to all the other data space components since it provides data and metadata to them. For example, the IDS Connector provides technical interface description, authentication mechanism, and associated data usage policies to the Metadata Broker and usage contracts to the Clearing House. Also, several other IDS components (App Sore, Metadata Broker, and Clearing House) are based on the IDS Connector architecture.

In X-Road, the component that provides access to an X-Road ecosystem is the Security Server. Like the IDS Connector, the Security Server provides secure and standardised data exchange between Members. The Security Server communicates with all the other components of an X-Road instance – either directly or indirectly.

IDS Clearing House

The IDS Clearing House provides a logging service that records relevant information for clearing, billing and usage control. For example, the information is used to provide a settlement service based on usage contracts which can help to automate payments between the data exchange parties. In addition, the logged usage control data can be used to validate access to resources. Technically, the IDS Clearing House consists of an IDS Connector.

X-Road doesn’t have a component that would be a direct match with the IDS Clearing House. However, the optional X-Road Metrics component provides some similar capabilities, e.g., collecting service usage statistics that can be used to automate payments, collecting service statistics. In addition, the Security Server provides a digitally signed and timestamped log record of each data exchange transaction that can be used to validate the transaction afterwards. However, the data exchange parties store the log records locally, and no third parties have access to them.

IDS App Store

The IDS App Store is a platform to distribute IDS Apps for IDS Connectors. Currently, X-Road doesn’t have a similar component.

IDS Vocabulary Hub

The IDS Vocabulary Hub is a vocabulary management platform for IDS use cases. It’s a platform to host, maintain, publish, and document vocabularies. It gives access to the vocabulary terms and their descriptions. The IDS Information Model is the lowest common denominator and can be extended with additional vocabularies. The IDS Vocabulary Hub communicates with IDS Connectors and infrastructure components.

The X-Road architecture doesn’t include a vocabulary component, but it goes without saying that interoperable data exchange requires semantic interoperability. Therefore, it’s up to the X-Road Governing Authority and data exchange parties to agree on where and how vocabularies are managed and stored. For example, the Finnish Digital Agency has developed an Interoperability Platform that can be used to maintain and publish common terminologies.

Conclusions 

After comparing data spaces and X-Road from various aspects, they have a lot in common on a high level. However, looking into the details shows that they also have many differences.

For example, the IDS specifications cover various aspects of data exchange, e.g., usage conditions and policies, contract negotiation, and data transfer process. They do not set restrictions to the data transfer protocol or wire protocol – the protocol used to transfer the data. In other words, any existing transfer protocol can be used, which enables using existing solutions and supporting many different transfer protocols. This is a big difference compared to X-Road, which has its own transfer protocol. Also, X-Road doesn’t technically cover the before-mentioned aspects (usage conditions and policies, contract negotiation) covered by the IDS specifications.

Let's return to the original question – can X-Road be considered a data space technology? The answer is no if X-Road is strictly compared to the IDS specifications, despite the similarities. The IDSA has a certification scheme for the IDS components, and X-Road would not meet the certification requirements. The IDS Testbed is a platform that can be used to conduct evaluations that ensure that an IDS component is implemented securely and conform to the relevant specifications. Components that pass the evaluation receive certification and are approved in the IDS ecosystems.

On the other hand, the European Commission considers data exchange ecosystems to be data spaces even if they are not based on technologies conforming to the IDS specifications, for example, the Once-Only Technical System (OOTS) and European Health Data Space (EHDS). Those ecosystems align with the common definition and characteristics for data spaces, but the technologies they use in the implementation do not conform to the IDS specifications. With the same logic, an X-Road ecosystem could also be considered a data space.

Multiple organisations and initiatives are developing specifications and standards for data spaces, e.g., International Data Spaces Association (IDSA), Gaia-X, and Fiware. The initiatives work together, aiming to keep the specifications and standards aligned and not create multiple overlapping or competing specifications. However, the work is currently in progress, and not everything is fully ready and aligned yet. Therefore, it might be too early to define what is a data space and what’s not based on the deliverables of a single actor. Nevertheless, in the long-run, common standards and specifications are the only way to achieve interoperability within and between data spaces.

New X-Road® Central Server UI and management REST API are here!

In recent years, the Security Server has experienced a total external and internal makeover. The process got started in 2019 when support for REST services was added. In 2020, the Security Server got a new user interface (UI) and a management REST API that enabled the automation of common configuration and maintenance tasks. Releasing X-Road 7 in 2021 enhanced the UI's look and feel and brought several other significant changes and improvements under the hood. While all these major changes have been implemented for the Security Server, the Central Server has received only some smaller updates. However, the Central Server has been remembered and will be the star of X-Road 7.3.0.

The beta version of X-Road 7.3.0 is already out, and the official release version will be published at the end of June 2023.

Easier administration and streamlined onboarding process

The most significant change in X-Road version 7.3.0 is the fully renewed Central Server UI. The new UI improves the usability and user experience of the Central Server. The new intuitive UI makes regular administrative tasks easier and supports streamlining the onboarding process of new X-Road members.

For example, complementary management requests for authentication certificates and client registration requests are no longer required. It's enough to send a registration request from the Security Server and approve it with two clicks on the Central Server. And like before, enabling automatic approval of registration requests makes the approval process fully automated.

Image 1. List pending management requests.

Image 2. Management request details.

Image 3. Approve management request.

Management REST API allows to automate tasks

Another significant change in X-Road version 7.3.0 is the brand-new Central Server management REST API. The API provides all the same functionalities as the UI and can be used to automate common maintenance and management tasks. Maintaining and operating the Central Server can be done more efficiently as configuration and maintenance tasks require less manual work. Also, the new UI uses the same API under the hood too.

The Central Server User Guide provides more information about the API, and the API's OpenAPI 3 description is available on GitHub. Access to the API is controlled using API keys that can be managed through the Central Server UI or through the API itself. In addition, access to the API can be restricted using IP filtering.

Changes in the architecture

The new UI and management REST API have also caused changes in the Central Server architecture and packaging. The previously existing Jetty (xroad-jetty) component has been replaced with the new UI and API (xroad-center), registration service (xroad-center-registration-service) and management service (xroad-center-management-service) components. These changes have affected Central Server’s log files, directories, software packages, and services. It’s strongly recommended that Central Server administrators study the details of these changes from the release notes before upgrading to version 7.3.0.

Image 4. Changes in the Central Server architecture - before version 7.3.0 (left) and starting from version 7.3.0 (right).

Wait, there’s more!

Even though the new Central Server UI and management REST API are the most significant and most visible changes in version 7.3.0, the new version contains many other new features, improvements, and fixes. Here’s a short overview of other changes included in the latest version.

• Security improvements on the Central Server:

  • Encrypt backup files (opt-in)

  • Verify the integrity of backup files on restore.

• Run all the X-Road components on Java 11. Remove support for Java 8.

• Create a separate security hardening guide that provides information about hardening the Central Server and Security Server host configurations.

• Implement configurable request rate and size limits for the Central Server REST API and management services.

• Changes in allowed characters in X-Road system identifiers and improved validation of the identifiers.

• Technology updates and a decrease in technical debt.  

The complete list of changes with more detailed descriptions is available in the release notes.

What’s next?

Implementing the new Central Server was a long process that required more time and effort than was initially expected. Unfortunately, it has caused postponing the implementation of some other new features. More changes to the Central Server are scheduled in the upcoming X-Road versions, but the focus will now shift to other roadmap items.

More information about the X-Road development roadmap is available here. More detailed information about the backlog items scheduled for version 7.4.0 is available here.

Third-party security experts have assessed the security of the new Central Server. However, should you have any findings, they can be reported through the newly launched X-Road Bug Bounty program.

Unravelling the Complexities of National Data Exchange Networks: A Network Science Approach

Introduction

This post is based on the findings from my research project titled "Graph Analysis of Dynamic National Data Exchange Networks."

In the age of relentless digital connectivity, understanding complex networks has become increasingly critical, spanning from social media platforms to the emerging world of blockchain technologies. X-Road, an established data exchange infrastructure, has been embraced by countries such as Estonia, Finland, Iceland, Colombia, Argentina, and Vietnam. Catering to millions of individuals, X-Road can be viewed as a complex network where government bodies, companies, non-profits, and various other organisations exchange data with one another.

In this post, we shall explore the intricacies of national data exchange networks through the lens of network science. By investigating the Estonian X-Road network (X-tee), I aimed to better understand the underlying patterns within data exchange networks and pinpoint potential areas for enhancement. Estonia has been collecting the network's transaction data (through the X-Road Metrics component, an open-source extension to X-Road) since 2016. The anonymised open data serves as a valuable starting point, and the insights derived could potentially be applicable to other nations as well.

Key Findings: A Network Science Approach

By analysing over 30 million data queries on the Estonian X-Road network, several key insights were obtained using network science analysis methods. The network shares common attributes with other real-world networks: 

1. Sparsity: an overall low number of connections compared to the maximum possible connections among its members.

2. Central giant component: a dominant connected subgraph in which a large fraction of the network's nodes/members are interconnected. 

3. Power law distribution for parts of the network: revealing a small number of highly connected nodes and a large number of less connected ones. 

These identified characteristics suggest that the network is well-suited for further modelling and analysis using network science methodologies.

Some of the key findings from the analysis:

• Public sector organisations, particularly governmental institutions, form the backbone of the data exchange infrastructure, being the most connected and active members of the network

• Nighttime is the prime time for mass data queries from government organisations on people and companies, particularly for tax authorities and bankruptcy bailiffs. During the daytime, service sectors like healthcare flourish, with the Health Insurance Fund and hospitals among the most active X-tee members.

• The network's most active members could be grouped into five distinct communities: 

• Healthcare

• IT and Infrastructure

• Social Security and Taxes

• Internal Affairs and Transport

• Education, Defence, and Environment.

Figure 1. 50 most active member clustered into 5 communities. See the full size image.

Though the community groupings may not be flawless, it's crucial to emphasise that these communities were identified solely by analysing query volumes between network members throughout the day. The content of the data queries, which is not publicly available, was not factored into the community detection process. This implies meaningful relationships between network members and groups of members could be discovered even without contextual information.

Implications and Future Directions

The findings from this analysis project have several implications. 

First, the research demonstrates the value of network science in modelling and analysing data exchange networks. This paves the way for more advanced prediction models and real-time monitoring tools. By discerning interaction patterns and activity distribution, decision-makers can enhance system performance, addressing both cybersecurity and economic concerns.

Second, the research highlights the importance of the public sector in driving data exchange, as well as the diverse range of services that rely on these networks. Understanding these interactions could help policymakers optimise resource allocation and improve the overall functioning of public services.

Lastly, the ability to accurately identify communities within the network suggests that further insights can be gained by examining the data transaction flows between these groups. This could potentially lead to a better understanding of the relationships between different sectors and the dynamics of the data economy.

Limitations and Challenges

While the findings of this research project provide valuable insights into the intricacies of national data exchange networks, it is essential to acknowledge some limitations that could impact the conclusions drawn from the analysis.

1. Lack of contextual information: The reliance on transaction data, without the content of the data queries, limits the depth of understanding of the relationships between network members. Including contextual information could provide a more comprehensive view of how different sectors interact within the network.

2. Generalizability: The analysis is based on the Estonian X-Road network, and the findings may not be directly applicable to other countries or networks with distinct characteristics or data exchange practices.

3. Possible biases: The data or methodology used in the analysis may introduce biases that could affect the outcomes and conclusions. Further investigation may be required to identify and address these biases to ensure the reliability and validity of the findings.

4. Dynamic nature of data exchange networks: As networks evolve over time, the findings from this research may be impacted by changes in the network structure or the interactions between members. Periodic re-analysis or real-time monitoring would be needed to maintain an accurate understanding of the network dynamics.

5. Need for further research: The findings presented in this blog post warrant additional investigation to validate or expand on the conclusions. Future research could explore the impact of incorporating contextual information, compare data exchange networks across countries, or investigate the relationships between different sectors and the dynamics of the data economy more thoroughly.

By acknowledging and addressing these limitations, the research can be further refined, and the understanding of national data exchange networks can be deepened, ultimately contributing to more effective decision-making and policy development.

Conclusion

In conclusion, this research project demonstrated the power of network science in shedding light on the complex world of national data exchange networks. As an increasing number of countries adopt data exchange solutions like X-Road, understanding their intricacies will be crucial in improving decision-making, reducing bureaucracy, and enhancing the overall happiness of citizens. The methodologies and insights derived from this project could serve as a valuable foundation for future work in this domain and may also encourage more countries and municipalities to adopt secure data exchange layers, ultimately benefiting millions of people around the world.

Andrius Matšenas, a recent Mathematics graduate from the University of Southampton, has a strong interest in network science, which he delved into in his BSc thesis – the basis for this blog post. With a passion for designing software products, Andrius co-founded Stardust Network, where he led a team to develop apps that empower users to take control of their personal data. He also gained valuable product development experience as a Product Analyst at NFTPort. Find out more: matsenas.ee

Database, dataset, data service, or service? Getting to know X-Road services.

Services are essential building materials of any data exchange ecosystem, and X-Road is no exception. Therefore, the number of available services is one of the key metrics when measuring the effects and benefits of an X-Road ecosystem.

For clarity, in X-Road, services are technical interfaces between information systems that are not used by end-users directly. Instead, end-users communicate with X-Road services indirectly through other systems and platforms, e.g., using the state portal that fetches information from multiple base registries over X-Road.

Besides the number of services, other important metrics are the number of member organisations and connected information systems, and the amount of data and queries exchanged between the members. However, from the ecosystem perspective, the number of available services may be the most essential metric. According to a study by Kristjan Vassil in 2016, the discrete threshold for ecosystem growth appears at 50 data repositories.

X-Road comes with tools to measure the number of available services in an ecosystem. Since X-Road is based on a decentralized architecture, the X-Road member organizations maintain information about available services locally on their Security Servers. There's no centralized list of available services by default, but the X-Road Operator may collect the data from Security Servers and publish it in a service catalog. For this purpose, the Security Server provides a set of built-in metadata services that can query metadata about the services published by different member organisations. However, interpreting the numbers requires some background knowledge of what the term service means in the context of X-Road. This blog post aims to explain the anatomy of service in X-Road.

Database, dataset, data service, or service?

Over the years, many different terms have been used to describe information systems in service providers' roles in X-Road. The term database was the most used for many years, while the term service has become the most common in recent years. Nevertheless, different terms are still used interchangeably.

Technically, X-Road doesn't distinguish between a database, data set, data service, or service. Also, there's no difference between a data service, a business service, or an aggregated service. Instead, X-Road divides services into SOAP, OpenAPI, and REST. In other words, dividing services into different categories is based on their technical characteristics rather than the type of the service. 

Identifying services

In X-Road, all services are identified using a unique service identifier string. The identifier is used to invoke services, and the number of available services in an X-Road ecosystem is counted by calculating the number of service identifiers. The service identifier includes information about the X-Road ecosystem, the organization owning the service, and the information system providing the service. However, the service identifier doesn't contain any information about the service category, but the built-in metadata services can be used to access the category information.

Connecting services

Publishing a service to X-Road requires that the organization owning the service has been registered as a member of an X-Road ecosystem and has access to a Security Server. First, the organization must complete an onboarding process to join an X-Road ecosystem. The organisation’s identity is verified by a trusted Certificate Authority (CA), and the X-Road Operator registers the organization. As a result, the organization is given an X-Road organization identifier.

Security Server is the organisation’s technical access point to the X-Road ecosystem. The organization may deploy its own Security Server, use a shared Security Server, or buy a Security Server as a service from a commercial service provider.

When an X-Road member organization has access to a Security Server, an information system providing a service must be connected to the X-Road ecosystem. In X-Road's terms, it means registering a new subsystem identifier on the Security Server(s) or utilizing an existing subsystem. A subsystem represents an information system or a logical group of information systems. The information system may be in a service consumer role, a service provider role, or both.

The service is then added under the subsystem. If the information system provides multiple services, all the services can be added under the same subsystem. Alternatively, various services published by the same information system can also be added under different subsystems. Access permissions to the services are defined on a service level, so whether the same or different subsystem is used doesn't affect them.

A service can be published on one or more Security Servers simultaneously, and one Security Server can publish multiple services owned by different organisations. For high availability, publishing services on multiple Security Servers is recommended. The Security Server supports high availability in two different ways: internal and external load balancing. The number of Security Servers where a service is published doesn’t affect the service identifier – it’s always the same.

On the other hand, it's also possible to publish the same service under two different service identifiers. For example, a free version of a service is published on a standalone Security Server with no high availability, and a paid version of the same service is published with a different service identifier on a Security Server cluster with external load balancing. That way, it's possible to provide two versions of the same service with different SLAs.

SOAP, OpenAPI, and REST services

X-Road supports three service categories: SOAP, OpenAPI, and REST. Regardless of the category, all the services are identified using a service identifier. However, the way how the services work and are managed varies between different categories.

SOAP

X-Road Message Protocol for SOAP defines how service consumers and service producers communicate with the Security Server. The protocol is based on SOAP profile 1.1. It comes with some X-Road-specific limitations and additional requirements, e.g., support for synchronous request-response operations only, some mandatory SOAP headers are required, and document/literal style SOAP body is required.

A common approach is to have an additional adapter service component between the Security Server and a SOAP client or service. The adapter service implements the X-Road Message Protocol for SOAP and converts all incoming/outgoing messages to/from the X-Road SOAP profile.

SOAP services are connected to the Security Server by providing a URL of a WSDL service description that may contain one or more SOAP service endpoints. The Security Server validates the structure of the WSDL description, but it doesn’t validate the WSDL against the service endpoint implementation.

Typically, a single SOAP endpoint represents a service that implements one action or procedure. Each endpoint has a unique service identifier. Also, endpoints can have multiple versions that have their own identifiers. For example, a WSDL description with four SOAP endpoints counts for four X-Road services since each endpoint gets its own X-Road service identifier.

Access permissions to SOAP services are managed on the service endpoint level. When a single WSDL contains multiple SOAP endpoints, access rights must be defined for each endpoint separately.

OpenAPI

Consuming and producing OpenAPI and REST services via X-Road is possible without an additional adapter service component. X-Road-specific information required by the Security Server (e.g., service client identifier, service provider identifier, message id, etc.) is transferred and processed so that existing REST-style services and service consumers can be connected to X-Road with minimal changes or no changes at all. This is achieved by transferring X-Road-specific information required by the Security Server in HTTP headers and URL parameters outside the message payload. The full details are available in the X-Road Message Protocol for REST document.

OpenAPI services are REST APIs that have an OpenAPI Specification available. The Security Server doesn't set any restrictions to the content type of the API messages, so the content type isn't limited to JSON only.

OpenAPI services are connected to the Security Server by providing a URL of an OpenAPI specification that describes a REST API with one or more API endpoints. The Security Server validates the structure of the OpenAPI specification, but it doesn’t validate the specification against the API implementation.

Typically, REST APIs are resource-centric, and endpoints are used to change the state of resources. However, REST APIs may also be RPC-style and implement actions or procedures. Either way, an OpenAPI service has a unique service identifier that covers all its API endpoints. For example, an OpenAPI service with four API endpoints counts for one X-Road service identifier.

A single OpenAPI service may support multiple API versions, or different API versions may be published as separate OpenAPI services. If the API version is included in the API base path URL (e.g., https://api.example.com/v1), a new OpenAPI service must be created for a new API version. Instead, if the API version isn't included in the base path URL (e.g., https://api.example.com), the same OpenAPI service can access different API versions. Access to the API works so that all the paths under the base path URL are accessible to service clients with sufficient access permissions. Therefore, the base path URL must be defined with caution. 

Access permissions to OpenAPI services can be managed on the API and API endpoint levels. Giving access on the API level means providing access to all the API endpoints by default. Also, if the API has endpoints not defined in the OpenAPI specification, they can be accessed too. Instead, giving access on the API endpoint level only provides access to specific endpoints. API endpoint level access permissions are defined using HTTP request method and path combination. Therefore, it is possible to define access rights for a single endpoint or alternatively for a subset of endpoints using wildcards. By default, the Security Server has a list of all the endpoints defined in the OpenAPI specification, but adding new endpoints manually is supported. Security Server’s access rights management only supports allowing access – explicitly denying access is not supported, e.g., allowing access to all endpoints on the API level and denying access to a single endpoint is not supported.

Besides access rights management, the Security Server does not use the endpoint-related information for anything else, e.g., the Security Server does not validate if an endpoint defined in a request by a client information system exists under an API or not. In other words, if a client information system has sufficient access rights to invoke an API endpoint, the Security Server forwards the request to the specified endpoint without any further validations.

REST

REST services are REST APIs that don’t have an OpenAPI specification available. REST services are connected to the Security Server by providing the API's base path URL. A REST service has a unique service identifier that covers all its API endpoints. For example, a REST API with four API endpoints counts for one X-Road service identifier.

REST services can be used to connect any HTTP-based endpoints to the Security Server. The Security Server doesn't set any restrictions to the content type of the API messages, so the content type isn't limited to JSON only. For example, a group of SOAP services could be connected to the Security Server using a REST service. It would be enough to provide the base URL of the SOAP services without a WSDL service description. In that case, the group of SOAP services would count for one X-Road service identifier.

Access permissions to REST services can be managed on the API and API endpoint levels. Giving access on the API level means providing access to all the API endpoints. Instead, giving access on the API endpoint level only allows access to specific endpoints. Since REST services don't have OpenAPI specification that defines the API endpoints, the endpoints must be added manually by the Security Server administrator if they need to be used in access rights management.

Metadata services

The Security Server provides a set of built-in methods that can be used to discover what services are available to them and download the machine-readable service descriptions. These methods are known as metadata services and are accessed using the service metadata protocol for SOAP and the service metadata protocol for REST. The metadata services have separate versions for SOAP and REST services.

Counting members, information systems, and services

The number of registered member organisations can easily be counted based on the member identifiers. Instead, the number of connected information systems can be calculated by the subsystem identifiers. However, the number of subsystem identifiers doesn't directly tell the number of connected information systems since a single information system may have multiple subsystems, and various information systems can share the same subsystem. Therefore, the number of connected information systems is only indicative.

With services, things get more complicated. The number of service identifiers doesn’t directly match the number of connected services. SOAP service endpoints are counted separately, while OpenAPI services are calculated on the API level rather than the API endpoint level. The same applies to REST services as well. However, X-Road supports counting the number of individual API endpoints, too, if they have been defined in an OpenAPI specification or manually.

Nevertheless, the service-related numbers don't say anything about the type of services. For example, they may provide simple access to data, execute a business process, provide service orchestration, etc. Therefore, additional analysis going behind the numbers is needed when comparing the available services of two X-Road ecosystems or evaluating the service coverage of a single ecosystem.

In addition to these metrics, it’s highly recommended the X-Road Operators implement the X-Road Metrics extension to get more detailed insights on the data exchange-related details in the X-Road ecosystem.