TMIANE: Design of an efficient Transformer-Based Model for Identification & Feature Analysis of Named Entities via Ensemble Operations

Main Article Content

Pooja P. Walke, Farha Haneef

Abstract

Named Entity Recognition (NER) is a critical task in natural language processing that involves identifying and categorizing named entities in text. These entities can include people, organizations, locations, and more. Extracting features from these entities is equally important as it enables downstream tasks like sentiment analysis, text classification, and more. In this research paper, we propose an efficient Transformer-based model for identifying named entities and analyzing their features through ensemble operations. Our proposed model leverages the power of Transformer models such as BERT and XLNet for identifying named entities. We then convert the identified entities into feature vector sets using a combination of BERT and XLNet. These features are classified using the GoogLeNet convolutional neural network for model validation operations. By combining these different models through ensemble operations, we aim to improve the accuracy, precision, recall, and delay of the model for different use cases. The need for such a model arises due to the limitations of existing models for named entity recognition and feature analysis. While these models have achieved significant success, they still suffer from low accuracy, precision, recall, and high delay. Our proposed model overcomes these limitations by using ensemble operations to combine the strengths of different models. We compare the performance of our proposed model with existing models on standard datasets and show that it outperforms these models in terms of accuracy, precision, recall, and delay. Our results demonstrate the potential of ensemble operations in developing efficient models for named entity recognition and feature analysis. Overall, this research paper contributes to the development of more accurate and efficient models for natural language processing tasks.

Article Details

Section
Articles
Author Biography

Pooja P. Walke, Farha Haneef

[1]Ms. Pooja P. Walke

2Dr Farha Haneef

 

[1] Research scholar, CSE, Oriental University, Indore, & Asst. Professor at St. Vincent Pallotti College of Engg. & Technology, Nagpur, India.

pwalke@stvincentngp.edu.in

2Ph.D Research Guide, CSE, Oriental University, Indore, India.

farhahaneef2014@gmail.com

 

References

T. Al-Moslmi, M. Gallofré Ocaña, A. L. Opdahl and C. Veres, "Named Entity Extraction for Knowledge Graphs: A Literature Overview," in IEEE Access, vol. 8, pp. 32862-32881, 2020, doi: 10.1109/ACCESS.2020.2973928.

P. Ma, B. Jiang, Z. Lu, N. Li and Z. Jiang, "Cybersecurity named entity recognition using bidirectional long short-term memory with conditional random fields," in Tsinghua Science and Technology, vol. 26, no. 3, pp. 259-265, June 2021, doi: 10.26599/TST.2019.9010033.

J. Diao, Z. Zhou and G. Shi, "Leveraging Integrated Learning for Open-Domain Chinese Named Entity Recognition," in International Journal of Crowd Science, vol. 6, no. 2, pp. 74-79, June 2022, doi: 10.26599/IJCS.2022.9100015.

J. Diao, Z. Zhou and G. Shi, "Leveraging Integrated Learning for Open-Domain Chinese Named Entity Recognition," in International Journal of Crowd Science, vol. 6, no. 2, pp. 74-79, June 2022, doi: 10.26599/IJCS.2022.9100015.

F. Zhao, X. Gui, Y. Huang, H. Jin and L. T. Yang, "Dynamic Entity-Based Named Entity Recognition Under Unconstrained Tagging Schemes," in IEEE Transactions on Big Data, vol. 8, no. 4, pp. 1059-1072, 1 Aug. 2022, doi: 10.1109/TBDATA.2020.2998770.

Y. Luo, H. Zhao, Z. Zhang and B. Tang, "Open Named Entity Modeling From Embedding Distribution," in IEEE Transactions on Knowledge and Data Engineering, vol. 34, no. 11, pp. 5472-5483, 1 Nov. 2022, doi: 10.1109/TKDE.2021.3049654.

T. Qian, M. Zhang, Y. Lou and D. Hua, "A Joint Model for Named Entity Recognition With Sentence-Level Entity Type Attentions," in IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 29, pp. 1438-1448, 2021, doi: 10.1109/TASLP.2021.3069295.

E. C. Jibril and A. C. Tantuğ, "ANEC: An Amharic Named Entity Corpus and Transformer Based Recognizer," in IEEE Access, vol. 11, pp. 15799-15815, 2023, doi: 10.1109/ACCESS.2023.3243468.

C. Liu, Y. Yu, X. Li and P. Wang, "Named Entity Recognition in Equipment Support Field Using Tri-Training Algorithm and Text Information Extraction Technology," in IEEE Access, vol. 9, pp. 126728-126734, 2021, doi: 10.1109/ACCESS.2021.3109911.

S. A. A. Shah, M. Ali Masood and A. Yasin, "Dark Web: E-Commerce Information Extraction Based on Name Entity Recognition Using Bidirectional-LSTM," in IEEE Access, vol. 10, pp. 99633-99645, 2022, doi: 10.1109/ACCESS.2022.3206539.

V. Sornlertlamvanich and S. Yuenyong, "Thai Named Entity Recognition Using BiLSTM-CNN-CRF Enhanced by TCC," in IEEE Access, vol. 10, pp. 53043-53052, 2022, doi: 10.1109/ACCESS.2022.3175201.

C. Li, G. Wang, J. Cao and Y. Cai, "A Multi-Agent Communication Based Model for Nested Named Entity Recognition," in IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 29, pp. 2123-2136, 2021, doi: 10.1109/TASLP.2021.3086978.

Y. Chen et al., "A Boundary Assembling Method for Nested Biomedical Named Entity Recognition," in IEEE Access, vol. 8, pp. 214141-214152, 2020, doi: 10.1109/ACCESS.2020.3040182.

Q. H. Ngo, T. Kechadi and N. -A. Le-Khac, "Domain Specific Entity Recognition With Semantic-Based Deep Learning Approach," in IEEE Access, vol. 9, pp. 152892-152902, 2021, doi: 10.1109/ACCESS.2021.3128178.

D. Zhang et al., "Improving Distantly-Supervised Named Entity Recognition for Traditional Chinese Medicine Text via a Novel Back-Labeling Approach," in IEEE Access, vol. 8, pp. 145413-145421, 2020, doi: 10.1109/ACCESS.2020.3015056.

L. -H. Lee and Y. Lu, "Multiple Embeddings Enhanced Multi-Graph Neural Networks for Chinese Healthcare Named Entity Recognition," in IEEE Journal of Biomedical and Health Informatics, vol. 25, no. 7, pp. 2801-2810, July 2021, doi: 10.1109/JBHI.2020.3048700.

S. Dadas and J. Protasiewicz, "A Bidirectional Iterative Algorithm for Nested Named Entity Recognition," in IEEE Access, vol. 8, pp. 135091-135102, 2020, doi: 10.1109/ACCESS.2020.3011598.

P. Zhu et al., "Improving Chinese Named Entity Recognition by Large-Scale Syntactic Dependency Graph," in IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 30, pp. 979-991, 2022, doi: 10.1109/TASLP.2022.3153261.

H. Kim and J. Kang, "How Do Your Biomedical Named Entity Recognition Models Generalize to Novel Entities?," in IEEE Access, vol. 10, pp. 31513-31523, 2022, doi: 10.1109/ACCESS.2022.3157854.

C. Zheng, Z. Wu, T. Wang, Y. Cai and Q. Li, "Object-Aware Multimodal Named Entity Recognition in Social Media Posts With Adversarial Learning," in IEEE Transactions on Multimedia, vol. 23, pp. 2520-2532, 2021, doi: 10.1109/TMM.2020.3013398.

Y. Lou, T. Qian, F. Li and D. Ji, "A Graph Attention Model for Dictionary-Guided Named Entity Recognition," in IEEE Access, vol. 8, pp. 71584-71592, 2020, doi: 10.1109/ACCESS.2020.2987399.

Y. Lou, T. Qian, F. Li and D. Ji, "A Graph Attention Model for Dictionary-Guided Named Entity Recognition," in IEEE Access, vol. 8, pp. 71584-71592, 2020, doi: 10.1109/ACCESS.2020.2987399.

J. Kim, Y. Kim, S. Kang and J. Seo, "Adaptive Named Entity Recognition Using Distant Supervision for Contemporary Written Texts," in IEEE Access, vol. 9, pp. 80405-80414, 2021, doi: 10.1109/ACCESS.2021.3067315.

J. Li, S. Shang and L. Chen, "Domain Generalization for Named Entity Boundary Detection via Metalearning," in IEEE Transactions on Neural Networks and Learning Systems, vol. 32, no. 9, pp. 3819-3830, Sept. 2021, doi: 10.1109/TNNLS.2020.3015912.

S. Su, J. Qu, Y. Cao, R. Li and G. Wang, "Adversarial Training Lattice LSTM for Named Entity Recognition of Rail Fault Texts," in IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 11, pp. 21201-21215, Nov. 2022, doi: 10.1109/TITS.2022.3182371.

C. Yang, L. Sheng, Z. Wei and W. Wang, "Chinese Named Entity Recognition of Epidemiological Investigation of Information on COVID-19 Based on BERT," in IEEE Access, vol. 10, pp. 104156-104168, 2022, doi: 10.1109/ACCESS.2022.3210119.

N. Liu, Q. Hu, H. Xu, X. Xu and M. Chen, "Med-BERT: A Pretraining Framework for Medical Records Named Entity Recognition," in IEEE Transactions on Industrial Informatics, vol. 18, no. 8, pp. 5600-5608, Aug. 2022, doi: 10.1109/TII.2021.3131180.

H. Poostchi and M. Piccardi, "BiLSTM-SSVM: Training the BiLSTM with a Structured Hinge Loss for Named-Entity Recognition," in IEEE Transactions on Big Data, vol. 8, no. 1, pp. 203-212, 1 Feb. 2022, doi: 10.1109/TBDATA.2019.2938163.

R. Yin, Z. Zhou and Z. Gao, "A Joint Model for Hierarchical Nested Information Extraction," in IEEE Access, vol. 10, pp. 50985-50995, 2022, doi: 10.1109/ACCESS.2022.3172970.

J. Kim, Y. Ko and J. Seo, "Construction of Machine-Labeled Data for Improving Named Entity Recognition by Transfer Learning," in IEEE Access, vol. 8, pp. 59684-59693, 2020, doi: 10.1109/ACCESS.2020.2981361.