Field of study: computer science, mathematics, software design, software engineering, technical computer science or comparable.
Machine Learning (ML) models are reaching a maturity level that allows their operational use in businesses. However, in some areas, this use is limited by their ”black box” nature: the decision-making logic and potential errors of a model are not transparent, making it unsuitable for safety-critical applications or those requiring trust in the model. The field of Explainable Artificial Intelligence (XAI) addresses this by providing methods to make model behavior more interpretable. Among these, concept-based and prototype-based methods show promise in offering intuitive insights into model decisions. To truly build trust and ensure safe deployment of models, however, it is not enough for XAI methods to be intuitive — they must must also meet some key requirements. For example, the methods need to be reliable and their explanations need to be faithful to the model, while having a complexity level appropriate for human users. To ensure that these properties are met, XAI methods must be rigorously validated. Furthermore, such an evaluation should be systematic, allowing to compare most methods on the same ground. A framework for this is still largely missing in current XAI pipelines.
This thesis investigates the systematic benchmarking of concept-based explanation methods for machine learning models. It adapts an existing benchmarking framework, originally developed for pro- totype methods, to support the evaluation of concept-based explanations. The project also includes the empirical testing of concept extraction methods, evaluating their effectiveness and reliability using diverse metrics and datasets. The work contributes toward standardizing the evaluation of XAI techniques to ensure that generated explanations are meaningful and faithful to the underlying model.
What you will do
The candidate will first conduct a literature review to identify desirable properties of trustworthy explanations and corresponding evaluation criteria. This includes analyzing existing benchmarks, theoretical foundations, and practical requirements of concept-based XAI methods. Based on this, suitable evaluation metrics will be selected or developed and integrated into the benchmarking pipeline. The newly implemented metrics will then be used to evaluate a concept extraction method in various scenarios.
This requires proficiency in Python and familiarity with modern ML libraries.
Scope:
* Identifying and formalizing evaluation properties for concept-based XAI methods
* Adapting an existing benchmark suite for prototype methods to accommodate concept-based explanations
* Implementing and testing relevant evaluation metrics
* Empirical benchmarking of a selected concept extraction method across multiple datasets and
models
What you bring to the table
* Solid understanding of machine learning
* Strong programming skills in Python
* Ideally, prior experience with explainability or XAI methods
* Independent, reliable, and result-oriented working style
* Good English communication skills
What you can expect
* Interesting tasks in applied research
* Intensive support during the project
* Collaboration projekt with University of Stuttgart IFF and RWTH Aachen University DSME
We value and promote the diversity of our employees' skills and therefore welcome all applications - regardless of age, gender, nationality, ethnic and social origin, religion, ideology, disability, sexual orientation and identity. Severely disabled persons are given preference in the event of equal suitability.
With its focus on developing key technologies that are vital for the future and enabling the commercial utilization of this work by business and industry, Fraunhofer plays a central role in the innovation process. As a pioneer and catalyst for groundbreaking developments and scientific excellence, Fraunhofer helps shape society now and in the future.
Interested? Apply online now. We look forward to getting to know you!
Ms. Lisa Bauer
Recruiting
Tel. +49 711 970-3681
Fraunhofer Institute for Manufacturing Engineering and Automation IPA
Requisition Number: 79958