Tahmasebzadeh, G.; Hakimov, S.; Ewerth, R.; Müller-Budack, E.: Multimodal Geolocation Estimation of News Photos. In: Kamps, J.; Goeuriot, L.; Crestani, F. et al. (Eds.): Advances in Information Retrieval : 45th European Conference on Information Retrieval, ECIR 2023, Dublin, Ireland, April 2–6, 2023, Proceedings, Part II. Cham : Springer, 2023 (Lecture Notes in Computer Science ; 13981), S. 204-220. DOI: https://doi.org/10.1007/978-3-031-28238-6_14
Zusammenfassung: | |
The widespread growth of multimodal news requires sophisticated approaches to interpret content and relations of different modalities. Images are of utmost importance since they represent a visual gist of the whole news article. For example, it is essential to identify the locations of natural disasters for crisis management or to analyze political or social events across the world. In some cases, verifying the location(s) claimed in a news article might help human assessors or fact-checking efforts to detect misinformation, i.e., fake news. Existing methods for geolocation estimation typically consider only a single modality, e.g., images or text. However, news images can lack sufficient geographical cues to estimate their locations, and the text can refer to various possible locations. In this paper, we propose a novel multimodal approach to predict the geolocation of news photos. To enable this approach, we introduce a novel dataset called Multimodal Geolocation Estimation of News Photos (MMG-NewsPhoto). MMG-NewsPhoto is, so far, the largest dataset for the given task and contains more than half a million news texts with the corresponding image, out of which 3000 photos were manually labeled for the photo geolocation based on information from the image-text pairs. For a fair comparison, we optimize and assess state-of-the-art methods using the new benchmark dataset. Experimental results show the superiority of the multimodal models compared to the unimodal approaches. | |
Lizenzbestimmungen: | Es gilt deutsches Urheberrecht. Das Dokument darf zum eigenen Gebrauch kostenfrei genutzt, aber nicht im Internet bereitgestellt oder an Außenstehende weitergegeben werden. |
Publikationstyp: | BookPart |
Publikationsstatus: | acceptedVersion |
Erstveröffentlichung: | 2023-03-17 |
Die Publikation erscheint in Sammlung(en): | Forschungszentren |
Pos. | Land | Downloads | ||
---|---|---|---|---|
Anzahl | Proz. | |||
1 | ![]() |
Germany | 72 | 31,17% |
2 | ![]() |
United States | 40 | 17,32% |
3 | ![]() |
Russian Federation | 15 | 6,49% |
4 | ![]() |
Spain | 13 | 5,63% |
5 | ![]() |
Portugal | 8 | 3,46% |
6 | ![]() |
United Kingdom | 6 | 2,60% |
7 | ![]() |
Vietnam | 5 | 2,16% |
8 | ![]() |
No geo information available | 5 | 2,16% |
9 | ![]() |
Netherlands | 5 | 2,16% |
10 | ![]() |
France | 4 | 1,73% |
andere | 58 | 25,11% |
Hinweis
Zur Erhebung der Downloadstatistiken kommen entsprechend dem „COUNTER Code of Practice for e-Resources“ international anerkannte Regeln und Normen zur Anwendung. COUNTER ist eine internationale Non-Profit-Organisation, in der Bibliotheksverbände, Datenbankanbieter und Verlage gemeinsam an Standards zur Erhebung, Speicherung und Verarbeitung von Nutzungsdaten elektronischer Ressourcen arbeiten, welche so Objektivität und Vergleichbarkeit gewährleisten sollen. Es werden hierbei ausschließlich Zugriffe auf die entsprechenden Volltexte ausgewertet, keine Aufrufe der Website an sich.