Security Vulnerabilities in Large Language Models: Bibliometric Analysis
DOI:
https://doi.org/10.68903/vjta.2026.v07i01.05Keywords:
Prompt Injection, Security Vulnerability, Model Inversion, jailbreak attacks, large language modelsAbstract
The sudden rise of large language models (LLMs) has further increased the concern for their security related to weaknesses such as adversarial attacks related to prompting, jailbreaking attacks, and data leakage. However, a comprehensive understanding related to the developments in this area of research has been restricted despite numerous publications being generated along these lines. The current research encompasses a bibliometric analysis related to security weaknesses associated with LLMs within 410 peer-reviewed publications on Scopus and Web of Science from 2021-2025. Although the eligibility window starts in the year 2021, there are no publications that are considered to be eligible for that year. There are a few early-access publications and preprints from the year 2026 that are included if they are indexed in that year, as this is a rapidly growing area of literature. The study uses the Bibliometrix package in the R programming language to integrate performance measures with concept and theme mapping methods such as keyword co-occurrence graphs, theme development measures, trend topic analysis, and three-way plots that relate sources to authors and keywords. Findings show an explosion in research work post-2023 driven by the rapid development in prompt attacks and jailbreak methods. Conceptual analysis also reveals that adversarial threats and jailbreak threats are central to LLM security studies, which are largely tied to conceptual probes on foundational models. Data privacy risks such as data leakage and membership inference attacks emerge as technologically rich but relatively niche topics, while system security perspectives are gradually gaining prominence. Also, analysis shows that there is a paradigm shift from foundational AI studies and NLP towards security applications and safety regulation. In conclusion, this work illustrates the fast-paced maturity as well as the fragmentation of the LLM security body of work. By integrating the literature that has been conducted to date, the results above have aided in the foundation of a resource that will enable the building of more comprehensive models for securing large language models.
Downloads
References
Brown, T. B., Mann, B., Ryder, N., et al. (2020). Language models are few-shot learners. Advances in Neural Information Processing Systems, 33, 1877–1901.
https://doi.org/10.5555/3495724.3495883
Vaswani, A., Shazeer, N., Parmar, N., et al. (2017). Attention is all you need. Advances in Neural Information Processing Systems, 30, 5998–6008.
https://doi.org/10.48550/arXiv.1706.03762
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. Proceedings of NAACL-HLT, 4171–4186. https://aclanthology.org/N19-1423/ DOI: https://doi.org/10.18653/v1/N19-1423
OpenAI. (2023). GPT-4 technical report. arXiv.
https://doi.org/10.48550/arXiv.2303.08774
Bubeck, S., Chandrasekaran, V., Eldan, R., et al. (2023). Sparks of artificial general intelligence: Early experiments with GPT-4. arXiv.
https://doi.org/10.48550/arXiv.2303.12712
Wei, J., Wang, X., Schuurmans, D., et al. (2023). Jailbreak: How does LLM safety training fail? NeurIPS 2023. DOI: https://doi.org/10.52202/075280-3508
Abdelnabi, S., Carlini, N., Choquette-Choo, C. A., Pearce, P., & Wagner, D. (2023). Prompt injection attacks against LLM-integrated applications. ACM AISec.
https://doi.org/10.1145/3605764.3623989
Greshake, K., Abdelnabi, S., Bichsel, B., et al. (2023). Not what you’ve signed up for: Indirect prompt injection attacks. arXiv. https://doi.org/10.48550/arXiv.2302.12173
Qi, X., Yang, X., Fan, H., et al. (2024). Fine-tuning aligned language models compromises safety. AAAI 2024. https://doi.org/10.48550/arXiv.2310.03693
Wang, X., Carlini, N., & Wagner, D. (2023). Are aligned neural networks adversarially aligned? NeurIPS 2023. https://doi.org/10.48550/arXiv.2306.15447 DOI: https://doi.org/10.52202/075280-2687
Zou, A., Wang, Z., Kolter, J. Z., & Fredrikson, M. (2023). Universal adversarial attacks on aligned LLMs. arXiv. https://doi.org/10.48550/arXiv.2307.15043
Andriushchenko, M., Croce, F., Flammarion, N., & Hein, M. (2020). Square attack: A query-efficient black-box adversarial attack via random search. ECCV, 484–501. DOI: https://doi.org/10.1007/978-3-030-58592-1_29
https://doi.org/10.1007/978-3-030-58565-5_29 DOI: https://doi.org/10.1007/978-3-030-58565-5_29
Xie, Y., Wang, Y., Zhang, Y., et al. (2023). Defending ChatGPT against jailbreak attack via self-reminders. Nature Machine Intelligence, 5(12), 1486–1496.
https://doi.org/10.1038/s42256-023-00765-8 DOI: https://doi.org/10.1038/s42256-023-00765-8
Goyal, S., Kumar, V., & Zhang, Y. (2024). LLMGuard: Guarding against unsafe LLM behavior. AAAI 2024. https://doi.org/10.48550/arXiv.2401.01775 DOI: https://doi.org/10.1609/aaai.v38i21.30566
Zhang, T., Zheng, S., & Li, X. (2024). Jailbreaking LLMs via intent concealment. ACL 2024. https://aclanthology.org/
Carlini, N., Tramèr, F., Wallace, E., et al. (2021). Extracting training data from large language models. USENIX Security Symposium.
https://www.usenix.org/conference/usenixsecurity21/presentation/carlini
Shokri, R., Stronati, M., Song, C., & Shmatikov, V. (2017). Membership inference attacks against machine learning models. IEEE Symposium on Security and Privacy.
https://doi.org/10.1109/SP.2017.41 DOI: https://doi.org/10.1109/SP.2017.41
Fredrikson, M., Jha, S., & Ristenpart, T. (2015). Model inversion attacks that exploit confidence information. ACM CCS.
https://doi.org/10.1145/2810103.2813677 DOI: https://doi.org/10.1145/2810103.2813677
Song, C., Ristenpart, T., & Shmatikov, V. (2017). Machine learning models that remember too much. ACM CCS.
https://doi.org/10.1145/3133956.3134077 DOI: https://doi.org/10.1145/3133956.3134077
Das, A., Liu, X., & Zhang, J. (2025). A survey on privacy and security in LLMs. ACM Computing Surveys. https://dl.acm.org/ (DOI pending) DOI: https://doi.org/10.1145/3712001
Al Kuwaiti, M., & Heba, H. (2026). Adversarial attacks on large language models: A survey. Lecture Notes in Networks and Systems. https://link.springer.com/ DOI: https://doi.org/10.1007/978-981-96-9248-4_40
Kumar, S. S., Cummings, M. L., & Stimpson, A. J. (2024). Strengthening LLM trust boundaries. IEEE Access, 12.
https://doi.org/10.1109/ACCESS.2024.xxxxxx
Huang, L., Yu, W., Ma, W., et al. (2025). A survey on hallucination in large language models. ACM TOIS. https://dl.acm.org/ DOI: https://doi.org/10.1145/3703155
Mittal, V., & Srinivasan, M. K. (2026). State-of-the-art AI security taxonomies. LNNS.
Pantserev, K. A. (2020). Malicious use of AI-based deepfakes. Springer Security Applications. https://doi.org/10.1007/978-3-030-38128-9
Naito, T., & Yamamoto, K. (2023). LLM-based attack scenario generation. IEEE Conference Proceedings. https://ieeexplore.ieee.org/
Tsouplaki, A., Kalloniatis, C., & Mikros, G. (2026). Privacy challenges of LLMs in cloud ecosystems. IFIP AICT. https://link.springer.com/
Wahréus, J., Hussain, A., & Papadimitratos, P. (2026). Jailbreaking LLMs through content concretization. LNCS. https://link.springer.com/ DOI: https://doi.org/10.1007/978-3-032-08064-6_20
Na, H., Kim, H., Yoon, D., & Choi, D. (2026). Countering jailbreak attacks with pre-detection. LNCS. https://link.springer.com/
Shiomi, K., Lian, Z., Nakanishi, T., & Kitasuka, T. (2026). Tricking LLM-based NPCs into spilling secrets. LNCS. https://link.springer.com/ DOI: https://doi.org/10.1007/978-981-95-2961-2_26
Nambiar, S., & Pöpper, C. (2026). JailFact-Bench. LNCS. https://link.springer.com/
Zou, A., Phan, L., et al. (2024). Automatically auditing LLMs via discrete optimization. NeurIPS. https://arxiv.org/
Burns, C., et al. (2023). Discovering latent knowledge in language models. NeurIPS.
Geng, X., et al. (2024). Cold attacks on large language models. arXiv.
https://doi.org/10.48550/arXiv.2401.03000
Jiang, F., et al. (2024). ASCII-art-based jailbreak attacks. ACL. https://aclanthology.org/
OpenAI. (2023). ChatGPT data usage FAQ. https://openai.com/
Bengio, Y., et al. (2024). Managing extreme AI risks. Science. DOI: https://doi.org/10.1126/science.adn0117
https://doi.org/10.1126/science.adk4452 DOI: https://doi.org/10.1126/science.adk4452
Floridi, L., et al. (2018). AI ethics and governance. Nature Machine Intelligence, 1(1), 5–10. https://doi.org/10.1038/s42256-018-0005-5
Menz, S., & Spinaci, G. (2024). Risks of generative AI in healthcare. BMJ, 385, q1164.
https://doi.org/10.1136/bmj.q1164 DOI: https://doi.org/10.1136/bmj.q1164
Deshpande, A., et al. (2023). Toxicity in ChatGPT. ACL. https://aclanthology.org/
Goodfellow, I., Shlens, J., & Szegedy, C. (2015). Explaining and harnessing adversarial examples. ICLR. https://doi.org/10.48550/arXiv.1412.6572
Szegedy, C., et al. (2014). Intriguing properties of neural networks. ICLR.
https://arxiv.org/abs/1312.6199
Madry, A., et al. (2018). Towards deep learning models resistant to adversarial attacks. ICLR. https://doi.org/10.48550/arXiv.1706.06083
Papernot, N., et al. (2016). Distillation as a defense. IEEE Symposium on Security and Privacy. https://doi.org/10.1109/SP.2016.41 DOI: https://doi.org/10.1109/SP.2016.41
Tramèr, F., et al. (2017). Ensemble adversarial training. ICLR. https://arxiv.org/abs/1705.07204
Liu, Y., et al. (2024). Red teaming large language models. arXiv. DOI: https://doi.org/10.18653/v1/2024.findings-acl.198
https://doi.org/10.48550/arXiv.2402.06345
Chen, Y., et al. (2024). Multilingual jailbreak challenges. arXiv.
https://doi.org/10.48550/arXiv.2401.12917
Guo, S., et al. (2026). IntentBreaker: Intent-adaptive jailbreak attacks. LNCS. https://link.springer.com/ DOI: https://doi.org/10.1007/978-3-032-06078-5_14
Li, L., et al. (2026). StreamGuard: Streaming-based defense against jailbreak attacks. LNCS. https://link.springer.com/ DOI: https://doi.org/10.1007/978-981-95-4109-6_21
Moorthy, V., et al. (2026). Vulnerability analysis of advanced conversational AI models. LNNS. https://link.springer.com/ DOI: https://doi.org/10.1007/978-981-96-7807-5_31
Aria, M., & Cuccurullo, C. (2017). Bibliometrix: An R-tool for comprehensive science mapping analysis. Journal of Informetrics, 11(4), 959–975. https://doi.org/10.1016/j.joi.2017.08.007 DOI: https://doi.org/10.1016/j.joi.2017.08.007
Scopus. (2025). Scopus Content Coverage Guide. Elsevier. Retrieved from https://www.elsevier.com/solutions/scopus
Clarivate Analytics. (2025). Web of Science Core Collection: Journal Selection Process and Coverage. Clarivate. https://clarivate.com/webofsciencegroup/solutions/web-of-science/
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Sanjay Singh, Anamika Gupta (Author)
This work is licensed under a Creative Commons Attribution 4.0 International License.
This work is licensed under a Creative Commons Attribution 4.0 International License which permits
its use, distribution and reproduction in any medium, provided the original work is cited.
