D’Souza, J.; Hrou, M.; Auer, S.: Evaluating Prompt-Based Question Answering for Object Prediction in the Open Research Knowledge Graph. In: Strauss, C.; Amagasa, T.; Kotsis, G.; Tjoa, A M.; Khalil, I. (Eds.): Database and Expert Systems Applications: 34th International Conference, DEXA 2023, Penang, Malaysia, August 28–30, 2023, Proceedings, Part I. New York, NY : Springer, 2023 (Lecture Notes in Computer Science ; 14146), S. 508-515. DOI:
https://doi.org/10.1007/978-3-031-39847-6_40
Zusammenfassung: |
Recent investigations have explored prompt-based training of transformer language models for new text genres in low-resource settings. This approach has proven effective in transferring pre-trained or fine-tuned models to resource-scarce environments. This work presents the first results on applying prompt-based training to transformers for scholarly knowledge graph object prediction. Methodologically, it stands out in two main ways: 1) it deviates from previous studies that propose entity and relation extraction pipelines, and 2) it tests the method in a significantly different domain, scholarly knowledge, evaluating linguistic, probabilistic, and factual generalizability of large-scale transformer models. Our findings demonstrate that: i) out-of-the-box transformer models underperform on the new scholarly domain, ii) prompt-based training improves performance by up to 40% in relaxed evaluation, and iii) tests of the models in a distinct domain reveals a gap in capturing domain knowledge, highlighting the need for increased attention and resources in the scholarly domain for transformer models.
|
Lizenzbestimmungen: |
This document may be downloaded, read, stored and printed for your own use within the limits of § 53 UrhG but it may not be distributed on other websites via the internet or passed on to external parties. |
Publikationstyp: |
BookPart |
Publikationsstatus: |
publishedVersion |
Erstveröffentlichung: |
2023 |
Schlagwörter (englisch): |
Knowledge Graph Completion, Natural Language Processing, Open Research Knowledge Graph, Prompt-based Question Answering, Question Answering
|
Fachliche Zuordnung (DDC): |
620 | Ingenieurwissenschaften und Maschinenbau
|
Kontrollierte Schlagwörter: |
Konferenzschrift
|