Project | OpenGPT-X

Publications about the project

1. Large Language Models are Echo Chambers

   Jan Nehring; Aleksandra Gabryszak; Pascal Jürgens; Aljoscha Burchardt; Stefan Schaffer; Matthias Spielkamp; Birgit Stark

   In: Nicoletta Calzolari; Min-Yen Kan; Veronique Hoste; Alessandro Lenci; Sakriani Sakti; Nianwen Xue (Hrsg.). Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024). Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING-2024), ELRA and ICCL, 2024.
2. Symmetric Dot-Product Attention for Efficient Training of BERT Language Models

   Martin Courtois; Malte Ostendorff; Leonhard Hennig; Georg Rehm

   In: The 62nd Annual Meeting of the Association for Computational Linguistics. Annual Meeting of the Association for Computational Linguistics (ACL-2024), Findings of the Association for Computational Linguistics, August 11-16, Bangkok, Thailand, Association for Computational Linguistics, 2024.
3. Efficient Language Model Training through Cross-Lingual and Progressive Transfer Learning

   Malte Ostendorff; Georg Rehm

   PML4DC @ ICLR 2023, 2023.