藏文紧缩格识别方法 postprint

Abstract: Word segmentation is a basic work of natural language processing, which has a great influence on the subsequent work of it, the recognition of abbreviated case-auxiliary words is one of the most difficult and important technologies of Tibetan word segmentation. Through dissecting the existing recognition methods of abbreviated case-auxiliary words, this paper analyzed the characteristics of Tibetan words, targetedly proposed recognition algorithm of Tibetan abbreviated case-auxiliary words rules, add - restore algorithm and the maximum entropy models feature template, then the methods of recognizing abbreviated case-auxiliary words based on the rules, add-restore methods and the maximum entropy model were obtained. The experimental data showed that the accuracy, recall rate and F value of the method is 99.26%, 96.47%, and 97.85% respectively , which shows a obvious progress than that of the existing methods.