What is Data Fabric?
Gartner defines data fabric as a design concept that serves as an integrated layer (fabric) of data and connecting processes. A data fabric utilises continuous analytics over existing, discoverable and inferenced metadata assets to support the design, deployment and utilisation of integrated and reusable data across all environments, including hybrid and multi-cloud platforms.
UXP has been ahead of its time and has provided data fabric for digital societies for years. With its standardised integration, security and privacy-preserving capabilities, and metadata support, it has been a preferred integration platform for many countries, municipalities and private sector organisations.
Why is Data Fabric Useful?
Data fabric reduces time for integration design by 30%, deployment by 30% and maintenance by 70%, because the technology designs draw on the ability to use or reuse and combine different data integration styles. Plus, data fabrics can leverage existing skills and technologies from data hubs, data lakes and data warehouses while also introducing new approaches and tools for the future.
Clear economic benefits of the data fabric design pattern are cost savings and improved staff productivity. Thanks to its architectural design approach, UXP adds operational resilience and business agility on top of that.
The resilience of UXP operations is expressed in fewer security incidents, fewer legal disputes and compliance with privacy regulations.
Business agility or flexibility of business operations means that it is easy to switch from one data source to another and modify the virtual distributed data schema when new data service providers are added to UXP-facilitated data fabric.
Real-Time Network-Based Integrations Instead of Data Copies
Traditional approach related to any data processing has always been copying and transforming.
However, there are several intrinsic problems when the copied data is used. First, the data users can never be sure if a given data set is up-to-date and usable for business-critical decisions. Secondly, having several sources of truth may and ultimately will lead to a situation where none of the copies is correct.
Thus, it is best to request the data directly from the source every time it needs to be used. The next question is how to make the real-time integration convenient. Data fabric pattern provides a solution to this.
Management of point-to-point integrations goes quickly out of control when the number of integrated systems increases. Data fabric standardises the connections and adds controls to the connections such as authentication of communicating parties, access control of data services, standardised security and privacy.
This approach allows to activate a large number of integration endpoints and create collaboration between a large number of organisations while keeping complexity and costs of integration low.
Meaningful Data Ownership
In the modern digital world, access to data needs to be authorised by the organisation that is the primary holder of the data. It implies that we must be able to identify the organisation, be it a data holder or a user, in a secure manner.
The easiest way to achieve this is through a data directory. UXP Directory provides searchable descriptions of the data services and links to the organisations which have ownership or act as custodians of data.
When a data service is taken into use, not only does UXP authenticate both the data service provider and the consumer, but it also logs the data exchange in the integrity-preserving and legally binding (requiring appropriate support of legislation, though) way.
Active Metadata
Services can be enriched with metadata in the service input/output field level in the service description, and also in the service and organisation level in the service directory.
Metadata can be used in the analysis and statistics of the data services. The key metadata metrics enable AI/ML algorithms that learn over time and churn out advanced predictions regarding data management and integration.
Network Effects of Data Fabric
The more endpoint provider and data consumer organisations are added to the data fabric, the more value it creates for both parties. The "two-sided marketplace" type of network effect fosters natural expansion of UXP networks. Thus, when governed right, the data fabric ecosystem evolves and grows easily as all parties benefit from it.
Written by Taavi Toomere
Product Manager for Data Exchange Technologies