Faster Zero-shot Multi-modal Entity Linking via Visual#2;Linguistic Representation

• 作者： Qiushuo, Zheng ( 1 ) ; Hao, Wen ( 2 ) ; Meng, Wang ( 2,3 ) ; Guilin, Qi ( 2,3 ) ; Chaoyu, Bai ( 1 ) ;
• 作者单位： 1.School of Cyber Science and Engineering, Southeast University, Nanjing 211189, China; 2.School of Computer Science and Engineering, Southeast University, Nanjing 211189, China; 3.Key Laboratory of Computer Network and Information Integration (Southeast University), Ministry of Education, Nanjing 211189, China;
• 提交时间：2022-11-28

摘要：Multi-modal entity linking plays a crucial role in a wide range of knowledge-based modal-fusion tasks, i.e., multi-modal retrieval and multi-modal event extraction. We introduce the new ZEro-shot Multi-modal Entity Linking (ZEMEL) task, the format is similar to multi-modal entity linking, but multi-modal mentions are linked to unseen entities in the knowledge graph, and the purpose of zero-shot setting is to realize robust linking in highly specialized domains. Simultaneously, the inference efficiency of existing models is low when there are many candidate entities. On this account, we propose a novel model that leverages visual#2; linguistic representation through the co-attentional mechanism to deal with the ZEMEL task, considering the trade-off between performance and efficiency of the model. We also build a dataset named ZEMELD for the new task, which contains multi-modal data resources collected from Wikipedia, and we annotate the entities as ground truth. Extensive experimental results on the dataset show that our proposed model is effective as it significantly improves the precision from 68.93% to 82.62% comparing with baselines in the ZEMEL task.

• 期刊： DATA INTELLIGENCE
• 分类： 计算机科学 >> 计算机科学的集成理论
• 引用： ChinaXiv:202211.00418 (或此版本 ChinaXiv:202211.00418V1)
  (DOI:10.1162/dint_a_00146)
  (CSTR:32003.36.ChinaXiv.202211.00418.V1)
• 推荐引用方式：Qiushuo, Zheng,Hao, Wen,Meng, Wang,Guilin, Qi, Chaoyu, Bai.(2022).Faster Zero-shot Multi-modal Entity Linking via Visual#2;Linguistic Representation.DATA INTELLIGENCE.CSTR:32003.36.ChinaXiv.202211.00418.V1 (点此复制)