Current Location:home > Browse

1. chinaXiv:201910.00076 [pdf]

Masked Sentence Model based on BERT for Move Recognition in Medical Scientific Abstracts

Yu, Gaihong; Zhang, Zhixiong; Liu, Huan ; Ding, Liangping
Subjects: Computer Science >> Natural Language Understanding and Machine Translation

Purpose: Move recognition in scientific abstracts is an NLP task of classifying sentences of the abstracts into different types of language unit. To improve the performance of move recognition in scientific abstracts, a novel model of move recognition is proposed that outperforms BERT-Base method. Design: Prevalent models based on BERT for sentence classification often classify sentences without considering the context of the sentences. In this paper, inspired by the BERT's Masked Language Model (MLM), we propose a novel model called Masked Sentence Model that integrates the content and contextual information of the sentences in move recognition. Experiments are conducted on the benchmark dataset PubMed 20K RCT in three steps. And then compare our model with HSLN-RNN, BERT-Base and SciBERT using the same dataset. Findings: Compared with BERT-Base and SciBERT model, the F1 score of our model outperforms them by 4.96% and 4.34% respectively, which shows the feasibility and effectiveness of the novel model and the result of our model comes closest to the state-of-the-art results of HSLN-RNN at present. Research Limitations: The sequential features of move labels are not considered, which might be one of the reasons why HSLN-RNN has better performance. And our model is restricted to dealing with bio-medical English literature because we use dataset from PubMed which is a typical bio-medical database to fine-tune our model. Practical implications: The proposed model is better and simpler in identifying move structure in scientific abstracts, and is worthy for text classification experiments to capture contextual features of sentences. Originality: The study proposes a Masked Sentence Model based on BERT which takes account of the contextual features of the sentences in abstracts in a new way. And the performance of this classification model is significantly improved by rebuilding the input layer without changing the structure of neural networks.

submitted time 2019-10-29 Hits41952Downloads1072 Comment 0

2. chinaXiv:201910.00073 [pdf]


Subjects: Computer Science >> Natural Language Understanding and Machine Translation


submitted time 2019-10-15 Hits7255Downloads855 Comment 0

3. chinaXiv:201905.00012 [pdf]

Transfer Learning for Scientific Data Chain Extraction in Small Chemical Corpus with BERT-CRF Model

Na Pang; Li Qian; Weimin Lyu; Jin-Dong Yang
Subjects: Computer Science >> Natural Language Understanding and Machine Translation

Abstract. Computational chemistry develops fast in recent years due to the rapid growth and breakthroughs in AI. Thanks for the progress in natural language processing, researchers can extract more fine-grained knowledge in publications to stimulate the development in computational chemistry. While the works and corpora in chemical entity extraction have been restricted in the biomedicine or life science field instead of the chemistry field, we build a new corpus in chemical bond field anno- tated for 7 types of entities: compound, solvent, method, bond, reaction, pKa and pKa value. This paper presents a novel BERT-CRF model to build scientific chemical data chains by extracting 7 chemical entities and relations from publications. And we propose a joint model to ex- tract the entities and relations simultaneously. Experimental results on our Chemical Special Corpus demonstrate that we achieve state-of-art and competitive NER performance.

submitted time 2019-05-12 Hits17929Downloads854 Comment 0

4. chinaXiv:201902.00062 [pdf]

Multimedia Short Text Classification via Deep RNN-CNN Cascade

Subjects: Computer Science >> Natural Language Understanding and Machine Translation

Abstract—With the rapid development of mobile technologies, social networking softwares such as Twitter, Weibo and WeChat are becoming ubiquitous in our every day life. These social networks generate a deluge of data that consists of not only plain texts but also images, videos, and audios. As a consequence, the traditional approaches that classify the short text by counting only the key words become inadequate. In this paper, we propose a multimedia short text classification approach by deep RNN(Recurrent neural network ) and CNN(Convolutional neural network) cascade. We first employ an LSTM(Long short-term memory) net- work to convert the information in the images into text information. Then a convolutional neural network is used to classify the multimedia texts by taking into account both the texts generated from the image as well as those contained in the initial message. It is seen through experiments using MSCOCO dataset that the proposed method exhibits significant performance improvement over the traditional methods.

submitted time 2019-02-22 Hits6818Downloads485 Comment 0

5. chinaXiv:201809.00191 [pdf]


李明; 肖培伦; 张矩; 顾心盟
Subjects: Computer Science >> Natural Language Understanding and Machine Translation

加权极限学习机对不同类别的样本赋予不同的权值,在一定程度上提高了分类准确 率,但加权极限学习机只考虑了不同类别样本之间差异,忽视了样本噪声和同类样本之间的 差异。本文提出了一种基于文本类别信息熵的极限学习机集成方法,该方法以Adaboost.M1 为算法框架,通过文本的类内分布熵和类间分布熵生成文本类别信息熵,由文本类别信息熵 构造代价敏感矩阵,把代价敏感极限学习机集成到Adaboost.M1 框架中。实验结果表明,该 方法与其他类型的极限学习机相比较有更好的准确性和泛化性。

submitted time 2018-09-27 Hits1742Downloads923 Comment 0

6. chinaXiv:201710.00001 [pdf]

Network of Recurrent Neural Networks

Wang, Chao-Ming
Subjects: Computer Science >> Natural Language Understanding and Machine Translation

We describe a class of systems theory based neural networks called "Network Of Recurrent neural networks" (NOR), which introduces a new structure level to RNN related models. In NOR, RNNs are viewed as the high-level neurons and are used to build the high-level layers. More specifically, we propose several methodologies to design different NOR topologies according to the theory of system evolution. Then we carry experiments on three different tasks to evaluate our implementations. Experimental results show our models outperform simple RNN remarkably under the same number of parameters, and sometimes achieve even better results than GRU and LSTM.

submitted time 2017-10-02 Hits4910Downloads1219 Comment 0

7. chinaXiv:201703.00230 [pdf]


孙萌; 华却才让; 姜文斌; 吕雅娟; 刘群
Subjects: Computer Science >> Natural Language Understanding and Machine Translation


submitted time 2017-03-10 Hits2416Downloads1759 Comment 0

8. chinaXiv:201703.00228 [pdf]


王志洋; 吕雅娟; 孙萌; 姜文斌; 刘群
Subjects: Computer Science >> Natural Language Understanding and Machine Translation


submitted time 2017-03-10 Hits2198Downloads1584 Comment 0

9. chinaXiv:201703.00187 [pdf]


吕雅娟; 刘群; 姜文斌
Subjects: Computer Science >> Natural Language Understanding and Machine Translation


submitted time 2017-03-09 Hits2282Downloads1714 Comment 0

10. chinaXiv:201611.00727 [pdf]


王敬东; 张智雄
Subjects: Computer Science >> Natural Language Understanding and Machine Translation


submitted time 2016-11-14 Hits1635Downloads948 Comment 0

12  Last  Go  [2 Pages/ 12 Totals]