Article

XAI Quality Analytics Toolkit

28 februar 2024

The Idletechs XAI Quality Analytics Toolkit is a software package designed for online monitoring of manufacturing processes. The goal of the system is to track product quality and process conformity through different production steps, to provide a holistic, always up-to-date image of quality and changes in the process.

The toolkit is based multiple components, including a near real-time (nearline) edge analytics system that performs fault detection and process analysis, and a secondary (online) system that performs higher-order analysis on fault-sets, trends, and provide underlying support to perform interactive analysis. The systems are trained based on gathered process data, rather than heuristic rules and hardcoded limits.

Users of the toolkit can have diverse backgrounds and roles in the organization, but this solution is mainly designed for operators that monitor and control the daily production. This includes engineers that plan production settings, machine configurations, and recipes, or that serve as subject experts and trouble-shooters; maintenance personnel that monitor, inspect, and repair machines and equipment; and production supervisors that plan production capacity, schedule orders, and monitor product quality.  These are collectively referred to as the Human Operator.

The system does not intend to replace human decisions and quality judgements, but to support the identification and monitoring of abnormal situations, to give the Human Operator better inspection tools, and to allow the Human Operator to work more efficiently to form fault mitigation strategies. As such, the module can be considered to be a decision support tool, but the results can of course also be used in automated pipelines.

To support our strong believe in explainable solutions, some key design limitations were employed during the development of the toolkit:

Transparent
Implications: It shall be possible to inspect why the system has judged an item to be of a certain quality, the underlying decision variables, and to trace the uncertainty in decisions. Minimal use of black-box methods.
Why: Due to stricter regulation and to ensure user trust in AI/ML-based systems, maintaining insight and tracing of decisions are a key aspect in the design of new solutions.

User-oriented
Implications: Dashboards and user experience shall specifically cusomtized for the end-application users. Shall use a co-design process where users affect the interface choices and user experience.
Why: To ensure user adoption and suitable interfaces for efficient work flows. Our aim is always to generate fit-for-purpose systems designed for and with the users to improve not only the process, but also the user.

Flexible
Implications:Must use a distributed architecture that does not tie in too tightly with proprietary platforms, and that supports common interfaces.
Why: To support adaptation into many types of infrastructure and frameworks the solution should be flexible in how the components are deployed and integrated with other systems. Avoid costly lock-ins to specific platform vendors.

Scalable
Implications: Shall use technology capable of easy, and demand-driven scaling of deployment. Shall use methods and models that can be initialized with small amount of training data.
Why: To be able to adapt to new situations, and to balance performance with sustainability it should be easy to scale to new situations and workloads and minimize resource utilization. Preserving computing resources will be a main endevour in many new system design.

Trustworthy
Implication: Shall quantify uncertainty in decisions. Shall detect unknown situations and situations where decisions are based on sparse training data.
Why: To build trust in the system we should always be open and clear on when the solution is not able to generate a good solution or when there is high uncertainty in decisions.

The Idletechs XAI Quality Analytics Toolkit has been co-funded by the European Union through the Dat4.Zero project (grant no. 958352).