POLYGLOT AND HYBRID PERSISTENCE ARCHITECTURES FOR BIG DATA ANALYTICS
TYPHON Overview
The aim of TYPHON is to provide an industry-validated methodology and integrated technical offering for designing, developing, querying, evolving, analysing and monitoring scalable hybrid data persistence architectures that will meet the growing scalability and heterogeneity requirements of European industry, which drives the project through four use cases from the domains of automotive, earth observation, banking, and telecom.
Polystore Design
TYPHON has developed a new modeling language (TyphonML) to support the design of hybrid polystores. Using TyphonML, engineers are able to model the data that needs to be persisted in a homogeneous manner, abstracting over the specificities of the underlying technologies. The language supports modelling both open and closed datasets, i.e. datasets which conform to an open or strict schema respectively.
Polystore Deployment
TYPHON has developed an intermediate polystore deployment modelling language (TyphonDL), which provides concepts that lie at an abstraction level between that of TyphonML, and that of specific data stores and virtual machine configuration technologies. The process of producing ready-to-use polystores from TyphonML models will comprise a step where TyphonML models are transformed to TyphonDL models, and a second step where TyphonDL models are used to generate the necessary installation and configuration facilities that, when executed, can assemble the polystore in an automated manner.
Polystore Querying
Distributing data across a number of heterogeneous databases introduces new challenges in terms of expressing and efficiently delegating queries to the underlying back-ends. TYPHON has built on the knowledge encoded in querying and meta-querying languages tailored to different data organisation paradigms, such as JPQL, Gremlin, Cypher and SPARQL in order to design and develop a next-generation language (TyphonQL) for querying heterogeneous data distributed across diverse databases.
Polystore Monitoring and Analytics
To avoid tight coupling between the core business logic and monitoring and analytics functionality, TYPHON utilises a modular event-based architecture where data access and update events are published by the hybrid polystore API to a high-performance bus so that these can be consumed by distributed topologies of components which can contribute real-time monitoring and analytics functionality in a loosely-coupled manner (e.g. complex events processing engines such as Esper).
Polystore Evolution
To facilitate disciplined and automated evolution of polystores, TYPHON provides: 1) a tool-supported methodology for schema evolution, taking into account the presence of multiple, heterogeneous, interdependent and possibly overlapping data stores; 2) Data migration tools, migrating the data between stores and between paradigms (relational, NoSQL, textual); 3) Query migration tools, allowing the automated translation of queries expressed on top of the initial schema(s) into equivalent queries compliant with the target structures; and 4) Data usage monitoring tools, that aim to analyze the actual data usage in the hybrid polystores, to recommend evolutions when relevant.
