React to Effects Fast by Learning, Evaluation, and eXtracted InformatiON

The next step in utilizing high-tech systems data


The Reflexion project aims to support the emerging high-tech industry paradigm shift from selling ‘boxes’ to supporting ‘integrated solutions’ by using operational data to improve the development lifecycles, maintenance and troubleshooting of products. This leads to a faster time to market, lower costs and greater competitiveness.

Reflexion’s objectives are:

to react to unforeseen problems or emerging needs in a speedy and cost-effective way by augmenting products with an introspective layer of data sensing and data analytics;

thereby creating value out of the high-tech system’s operational data, which is (for now) still characterized by legacy choices on infrastructural and analytical approaches; and

to (automatically) propagate this knowledge back into the product development lifecycle and the service and maintenance flow.

Vision and challenges

In order to attain/maintain (global) leading positions in today’s globalized competitive markets, OEMs need to bring the latest technology products to market with a highly efficient time to market, limited costs and a controlled quality level. There are a number of challenges and opportunities for the Reflexion project:

  • A better understanding of a customer’s needs, way of working and operational context will enable a focus on R&D investments.

  • Early market introduction leads to a better competitive positioning. Note: only the top three players find success; however, this is only true when the market introduction meets adequate quality criteria.

  • The industrial costs of poor quality sometimes amount to up to 5% of total turnover. Typically, more than 20% of R&D budgets is spent on resolving quality-related problems.

  • There is a strong industrial need to employ scarce human resources to the R&D activities that generate the most business impact.

  • Access to the OEMs’ development processes directly translates into new primary business capabilities, such as by creating opportunities to develop new tool functionalities and services for an as-of-yet underexplored area of application.

  • The possibility to incubate and validate proof-of-concepts for (general purpose) tool functionality and services in an industrial context.

  • Improving the visibility of the OEM industry in terms of advancing the tool and service providers’ market penetration.

Philippe Letellier, ITEA Vice-Chairman

Unique results are emerging at the very beginning of the industry chain, at the design level. But is it more revolutionary than that. There is no more start or end.

Eureka award for the Reflexion project


To address the above challenges, high-tech systems companies must take the next step in integrating operational data into a product’s development lifecycle.

ESI is developing a generic methodology that integrates domain knowledge and data into a system-level reasoning framework. As shown in the above figure, ESI uses domain-specific languages for knowledge engineering in order to systematically model domain knowledge, which is often scattered across documents or in the heads of experts. For data-driven business applications, knowledge engineering supports the effective analysis of operational data by applying (for example) feature selection or constraints in learning algorithms. The exploitation of profiling, process and data mining techniques allow for the generation of context-specific operational models that can either support the automated testing or customization of system operations (among other things) or be used to develop new systems.

Lessons learned

The Reflexion project understands that the main challenges are valorizing the emerging operational data in high-tech systems and reacting to emerging needs in an efficient and cost-effective way by augmenting products with an introspective layer of data sensing and data analytics

The lessons learned include:

  • The exploitation of a high-tech system’s operational data is characterized to a high degree by legacy engineering choices in infrastructural and analytical approaches.

  • The high-tech industry (R&D) still has an immature view on the exploitation of data science for the valorization of operational data. Building up an awareness of how the integration of data into smart engineering processes can be a valuable and competitive business asset is currently more important than building up expertise.

  • The focus should be on bridging the world of data science with the world of high-tech systems development by exploiting data analytics expertise/techniques to extract value from existing operational system logging.

  • High-tech systems are quite diverse: there are little to no out-of-the-box generic (data science) approaches; in reality, serious domain insights/modeling are required, mixed with real system understanding and craftmanship.

Results - 1. Dissemination
  • 4 conference papers

  • 7 invited talks

  • 4 articles

  • 7 workshops

  • 7 master theses

  • 7 demos at innovation markets

Results - 2. Exploitation
Reflexion Partner Achievement Contact
TNO System-data demonstrator: industrial showcases of how operational data adds value to high-tech industry. Bas Huijbrechts
SynerScope First market releases of SynerScope's Ixiwa and Iximeer products suite: quickly navigate, search, link and improve structured and unstructured data; explore all dimensions of the data at once. Jorik Blaas
Yazzoom Extension of the Yazzoom Yanomaly big data platform for anomaly detection on machine generated data & Yasense virtual sensor tool suite. David Verstraeten
Axini Extension of the Axini TestManager user profile driven MBT engine prototype, generating testcases following the same distribution & characteristics of real users interacting with the system. Machiel Van der Bijl
Barco Developed and released Nexxis Care Plan, Barco's web portal for its distributors, resellers and system integrators to manage their installed Barco products. Wim Sandra
Philips Developed new Image Guided Therapy system verification processes using model-based testing incorperating usage models learnt from field log data by applying machine learning techniques. Rob Ekkel
Philips Developed and released a data analysis framework which collects & visualizes machine data of the fleet of operational MRI systems, supporting services incl. MRI system dashboards for commercial customers. Mark van Helvoort
Océ Developed the Optimal Diagnostic Analysis System (ODAS) data infrastructure tool suite to support data analytics using Python Jupyter Notebooks for R&D engineering community. Lou Somers
Siemens ISW Developed and validated a methodology to train machine learning algorithms with simulation-generated data for conditional monitoring of machines. Bram Cornelis
all partners 25 data science specialists (FTEs) were employed by the consortium partners to work on operational data roadmap activities as a direct result of the project activities. Bas Huijbrechts
Axini, SynerScope, Yazzoom 10 commercial tool set releases including - in the project developed - generic purpose data visualization, analytics and MBT functionality to be applied in an industrial setting directly resulted from the Reflexion project activities. Bas Huijbrechts
Results - 3. Publications
Keywords Source Date Involved partners Contact
Reflexion project success "Er is data genoeg, maar er wordt weinig mee gedaan" , ProcessControl #6 2019 TNO Bas Huijbrechts
Semantic search engine Corrado Grappiolo, Emile van Gerwen, Jack Verhoosel, Lou Somers. "The Semantic Snake Charmer Search Engine, a Tool to Facilitate Data Science in High-tech Industry Domains". Proceedings of the ACM SIGIR Conference on Human Information Interaction and Retrieval. Demonstration paper. demo March 2019 TNO Corrado Grappiolo
Probabilistic string clustering Eline Verwielen, Corrado Grappiolo, Nils Noorman, Kuno Huisman. "The Growing N-Gram Algorithm: A Novel Approach to String Clustering". Proceedings of the 8th International Conference on Pattern Recognition Applications and Methods. February 2019 TNO Corrado Grappiolo
Transition system visualization Dennis van der Werf. "Visualizing Symbolic Transition Systems". Master Thesis, University of Amsterdam, 2018. July 2018 Axini Machiel van de Bijl
Process mining for testing Aswathy George. "Process Mining in Model Based Testing", Master Thesis, University of Amsterdam, 2018. July 2018 Axini Machiel van de Bijl
Test cases grouping Martijn Willensen. "Improving Diagnosis by Grouping Test Cases to Reduce Complexity", Master Thesis, University of Twente, 2018 July 2018 Axini Machiel van de Bijl
Bearing fault Detection Paulo Sousa, "Bearing fault detection: a feature-based approach", MSc Thesis, Univ. Porto (FEUP), 2018. June 2018 Siemens Industry Software Bram Cornelis
Fleet-based fault detection Hendrickx, Kilian & Meert, Wannes & Cornelis, Bram & Janssens, Karl & Gryllias, Konstantinos & Davis, Jesse. "A fleet-wide approach for condition monitoring of similar machines using time-series clustering". International Conference on Condition Monitoring of Machinery in Non-stationary Operations (CMMNO2018), 2018. June 2018 Siemens Industry Software Bram Cornelis
Ontology evaluation Kambiz Sekandar. "A quality measure for automatic ontology evaluation and improvement". Master thesis, Utrecht University, 2018. June 2018 TNO Jan Verhoosel
Probabilistic string clustering Eline Verwielen. "Growing N-grams: a probabilistic approach to string clustering". Master thesis, Tilburg University, 2018. March 2018 TNO Corrado Grappiolo
The value of exploiting data ITEA Magazine #29 March 2018 TNO Bas Huijbrechts
Smart high-tech system-diagnostics with operational data Yonghui Li, Marina Velikova, Emile van Gerwen, Martijn Hendriks, Michael Borth, Koen Kanters. "Demonstrator of high-tech system-diagnostics with operational data". ICT.OPEN 2018. March 2018 TNO Bas Huijbrechts
Data mining for condition monitoring Sander Castermans, "Data mining techniques for machine/vehicle condition monitoring", MSc Thesis, KU Leuven, 2017. June 2017 Siemens Industry Software Bram Cornelis
Simulation-driven machine learning Cameron Sobie and Carina Freitas and Mike Nicolai. "Simulation-driven machine learning: Bearing fault classification". Mechanical Systems and Signal Processing, pp 403-419, 2018 March 2017 Siemens Industry Software Cameron Sobie
Data-science, een kwestie van goed samenwerken Joost Janse. "Data-science, een kwestie van goed samenwerken". Bits&Chips, July 2016. July 2016 Océ Joost Janse
Results - 4. Spin-offs
  • ITEA OXILATE project

  • Secured 25 data science specialist positions for industry

Results - 5. Awards
  • ITEA Award of Excellence 2019

  • EUREKA Award 2019

Additional information