In cooperation with Fraunhofer IML and the company Ubermetrics Technologies GmbH, the research department Speech and Language Technology (SLT) of the German Research Center for Artificial Intelligence (DFKI) in Berlin has developed a new innovation-oriented evaluation metric based on artificial intelligence methods as part of the first phase of the EVALITECH project. The project, funded by the Federal Ministry of Education and Research (BMBF), aimed to facilitate the search for and selection of candidates in application and appointment processes in the Industrie 4.0 environment in the future.
The partners developed an innovation-oriented evaluation metric that takes into account criteria relevant to Industrie 4.0. In the first phase of EVALITECH, a key task was to design the new metric, assess its automatability, and develop a demonstrator as a first prototype. For this purpose, information was collected from experts regarding this metric, implemented, and tested on selected candidates.
In addition to this empirical work, artificial intelligence methods were tested to automatically capture the criteria. Ubermetrics extended their in-house product "Delta" with regard to intelligent information retrieval in the entire publicly accessible Internet, DFKI used advanced language technologies for information extraction on the previously captured unstructured documents, Fraunhofer IML dedicated itself to visualization within a web demonstrator. In the future, the new metrics can be used in application and appointment processes and support the appointment committee in the search and selection of candidates.
In the EVALITECH project, the team were able to demonstrate the added value of the new indicator as well as the multidimensional representation and systematic weighting of the criteria adapted to the respective job advertisement.
Professor Dr. Dr. h.c. mult. Wolfgang Wahlster, who initiated the EVALITECH project with the support of the scientific advisory board of Plattform Industrie 4.0 and acatech, the German Academy of Science and Engineering, emphasizes the outstanding significance of EVALITECH for the engineering sciences:
"Due to the outstanding importance of Industrie 4.0 for Germany as a business location, a very large number of engineers with a profound technical science education will continue to be sought in the coming years. The current shortage of interdisciplinarily trained specialists for this purpose can only be countered by further expanding the relevant faculties with corresponding professorships in the medium term. In view of the tight applicant situation, EVALITECH in its planned full version can help to identify suitable candidates and to create a systematic, comparative evaluation metric for them with regard to the respective requirement profile in order to support and fundamentally improve the selection and decision-making process of appointment committees. EVALITECH can also provide valuable impetus for the selection of leaders in public and private research organizations in the future."
Publication of the EVALITECH website
With the publication of the EVALITECH website, the project’s current status is now available to all interested parties: evalitech.dfki.de. Through the website, the intent of the EVALITECH project is explained in more detail and the design of the new metric is presented. Furthermore, visitors to the website will get an insight into the most relevant functionalities of the pilot version of the web-based selection assistance system developed during the preliminary research.
Prospects for EVALITECH
In the next phase a further step of the EVALITECH project is to build a transparent and public portal based on the previous research results in which information can also be added manually and determined automatically. The system is intended to make the data basis transparent in order to enable users of the selection assistance system to assess the reliability of the information themselves.
A fundamental goal of the portal is to systematically find suitable applicants and to facilitate the work of selection committees in the future by means of a flexible, multi-criteria indicator system tailored to the respective job advertisement.
The project was funded by the German Federal Ministry of Education and Research (BMBF) (grant number 02P17D262).