InfraSTructurEs for Living LAbs II
The STELLA project, which was funded by the German Research Foundation and has now been successfully completed, aimed to establish an evaluation infrastructure to evaluate search and recommender services during live operation of web-based search systems with real users. STELLA provides an integrated e-research environment for the research areas of information retrieval and recommender systems. The experimental set-up differs significantly both from traditional TREC studies that can only be carried out offline and from other user studies that can only be performed under laboratory conditions. STELLA gives researchers access to an evaluation methodology that could previously only be utilised on large commercial platforms in the industrial arena.
In STELLA II, the partners will extend the existing framework to include the additional external partners DIPF, TIB, and ZBW and their corresponding search platforms and evaluation use-cases. Furthermore, STELLA’s current ad-hoc retrieval and recommendation setup will be extended as a continuous evaluation platform.
ZB MED’s role in the project
- Contribute to the technical consolidation of the STELLA framework including improved Docker infrastructure, a common index and a rich logging component.
- Integrate the new STELLA version into ZB MED selected portals.
- Support external partners in the integration of STELLA into their own portals.
- Creation of test collections so that experts can annotate recommendations and rankings for selected queries and topics in the test portals.
- Contribute to the development of the continuous evaluation and evolution cycle integrating expert-based test collections as well as real and simulated user interaction into ZB MED selected portals.
- Contribute to the definition of a long-term sustainability strategy for the STELLA framework.
Duration
1 August 2023 –31 July 2026
Funding bodies
German Research Foundation (DFG) - Grant No. 407518790
Partners
- TH Köln – University of Applied Sciences
- GESIS Leibniz Institute for the Social Sciences, Cologne
