Awesome-LLM-Uncertainty-Reliability-Robustness
This repository, called UR2-LLMs contains a collection of resources and papers on Uncertainty, Reliability and Robustness in Large Language Models.
"Large language models have limited reliability, limited understanding, limited range, and hence need human supervision. " - Michael Osborne, Professor of Machine Learning in the Dept. of Engineering Science, University of Oxford, January 25, 2023
Welcome to share your papers, thoughts and ideas in this area!
Contents
Resources
Introductory Posts
The Determinants of Controllable AGI
Allen Schmaltz
[Link]
3 Mar 2025
GPT Is an Unreliable Information Store
Noble Ackerson
[Link]
20 Feb 2023
“Misusing” Large Language Models and the Future of MT
Arle Lommel
[Link]
20 Dec 2022
Large language models: The basics and their applications
Margo Poda
[Link]
9 Feb 2023
Prompt Engineering: Improving Responses & Reliability
Peter Foy
[Link]
19 Mar 2023
OpenAI’s Cookbook on Techniques to Improve Reliability
OpenAI
[Github]
18 Mar 2023
GPT/calibration tag
Gwern Branwen
[Link]
Prompt Engineering
Lilian Weng
[Link]
LLM Powered Autonomous Agents
Lilian Weng
[Link]
Reliability in Learning Prompting
[Link]
Building LLM applications for production
Chip Huyen
[Link]
11 Apr 2023
Practical, Real-World Neural Network Interpretability and Deployment
Allen Schmaltz
[Link]
11 Dec 2021
Technical Reports
GPT-4 Technical Report
OpenAI
arXiv 2023. [Paper][Cookbook]
16 Mar 2023
GPT-4 System Card
OpenAI
arXiv 2023. [Paper] [Github]
15 Mar 2023
Tutorial
Uncertainty Estimation for Natural Language Processing
Adam Fisch, Robin Jia, Tal Schuster
COLLING 2022. [Website]
Papers
Evaluation & Survey
AI data agent reliability tests
AI Analyst Lab, Shane Butler
Logs, prompts and synthetic data showing fresh ChatGPT and Claude Code sessions choosing different metric definitions for the same data question.
GitHub 2026. [Github]
Sep 2026
Don’t Pass@k: A Bayesian Framework for Large Language Model Evaluation
Mohsen Hariri, Amirhossein Samandar, Michael Hinczewski, Vipin Chaudhary
ICLR 2026. [Paper][Github]
5 Oct 2025
Wider and Deeper LLM Networks are Fairer LLM Evaluators
Xinghua Zhang, Bowen Yu, Haiyang Yu, Yangyu Lv, Tingwen Liu, Fei Huang, Hongbo Xu, Yongbin Li
arXiv 2023. [Paper][Github]
3 Aug 2023
A Survey on Evaluation of Large Language Models
Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Kaijie Zhu, Hao Chen, Linyi Yang, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, Wei Ye, Yue Zhang, Yi Chang, Philip S. Yu, Qiang Yang, Xing Xie
Arxiv 2023. [Paper][Github]
6 Jul 2023
DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Models
Boxin Wang, Weixin Chen, Hengzhi Pei, Chulin Xie, Mintong Kang, Chenhui Zhang, Chejian Xu, Zidi Xiong, Ritik Dutta, Rylan Schaeffer, Sang T. Truong, Simran Arora, Mantas Mazeika, Dan Hendrycks, Zinan Lin, Yu Cheng, Sanmi Koyejo, Dawn Song, Bo Li
Arxiv, 2023. [Paper] [Github] [Website]
20 Jun 2023
In ChatGPT We Trust? Measuring and Characterizing the Reliability of ChatGPT
Xinyue Shen, Zeyuan Chen, Michael Backes, Yang Zhang
arXiv, 2023. [Paper]
18 Apr 2023
Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond
Jingfeng Yang, Hongye Jin, Ruixiang Tang, Xiaotian Han, Qizhang Feng, Haoming Jiang, Bing Yin, Xia Hu
arXiv 2023. [Paper][Github]
27 Apr 2023
How Robust is GPT-3.5 to Predecessors? A Comprehensive Study on Language Understanding Tasks
Xuanting Chen, Junjie Ye, Can Zu, Nuo Xu, Rui Zheng, Minlong Peng, Jie Zhou, Tao Gui, Qi Zhang, Xuanjing Huang
arXiv 2023. [Paper][Github]
1 Mar 2023
Holistic Evaluation of Language Models
Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, Benjamin Newman, Binhang Yuan, Bobby Yan, Ce Zhang, Christian Cosgrove, Christopher D. Manning, Christopher Ré, Diana Acosta-Navas, Drew A. Hudson, Eric Zelikman, Esin Durmus, Faisal Ladhak, Frieda Rong, Hongyu Ren, Huaxiu Yao, Jue Wang, Keshav Santhanam, Laurel Orr, Lucia Zheng, Mert Yuksekgonul, Mirac Suzgun, Nathan Kim, Neel Guha, Niladri Chatterji, Omar Khattab, Peter Henderson, Qian Huang, Ryan Chi, Sang Michael Xie, Shibani Santurkar, Surya Ganguli, Tatsunori Hashimoto, Thomas Icard, Tianyi Zhang, Vishrav Chaudhary, William Wang, Xuechen Li, Yifan Mai, Yuhui Zhang, Yuta Koreeda
arXiv 2022. [Paper] [Website] [Github] [Blog]
16 Nov 2022
Prompting GPT-3 To Be Reliable
Chenglei Si, Zhe Gan, Zhengyuan Yang, Shuohang Wang, Jianfeng Wang, Jordan Boyd-Graber, Lijuan Wang
ICLR 2023. [Paper] [Github]
17 Oct 2022
Plex: Towards Reliability using Pretrained Large Model Extensions
Dustin Tran, Jeremiah Liu, Michael W. Dusenberry, Du Phan, Mark Collier, Jie Ren, Kehang Han, Zi Wang, Zelda Mariet, Huiyi Hu, Neil Band, Tim G. J. Rudner, Karan Singhal, Zachary Nado, Joost van Amersfoort, Andreas Kirsch, Rodolphe Jenatton, Nithum Thain, Honglin Yuan, Kelly Buchanan, Kevin Murphy, D. Sculley, Yarin Gal, Zoubin Ghahramani, Jasper Snoek, Balaji Lakshminarayanan
arXiv 2022. [Paper]
15 Jul 2022
Language Models (Mostly) Know What They Know
Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield-Dodds, Nova DasSarma, Eli Tran-Johnson, Scott Johnston, Sheer El-Showk, Andy Jones, Nelson Elhage, Tristan Hume, Anna Chen, Yuntao Bai, Sam Bowman, Stanislav Fort, Deep Ganguli, Danny Hernandez, Josh Jacobson, Jackson Kernion, Shauna Kravec, Liane Lovitt, Kamal Ndousse, Catherine Olsson, Sam Ringer, Dario Amodei, Tom Brown, Jack Clark, Nicholas Joseph, Ben Mann, Sam McCandlish, Chris Olah, Jared Kaplan
arXiv 2022. [Paper]
11 Jul 2022
Augmented Language Models: a Survey
Grégoire Mialon, Roberto Dessì, Maria Lomeli, Christoforos Nalmpantis, Ram Pasunuru, Roberta Raileanu, Baptiste Rozière, Timo Schick, Jane Dwivedi-Yu, Asli Celikyilmaz, Edouard Grave, Yann LeCun, Thomas Scialom
arXiv 2023. [Paper]
15 Feb 2023
A Survey of Evaluation Metrics Used for NLG Systems
Ananya B. Sai, Akash Kumar Mohankumar, Mitesh M. Khapra
ACM Computing Survey, 2022. [Paper]
18 Jan 2022
NL-Augmenter: A Framework for Task-Sensitive Natural Language Augmentation
Kaustubh D. Dhole, et al.
ACL 2021. [Paper][Github]
6 Dec 2021
TextFlint: Unified Multilingual Robustness Evaluation Toolkit for Natural Language Processing
Tao Gui et al.
arXiv 2021. [Paper][Github]
21 Mar 2021
Robustness Gym: Unifying the NLP Evaluation Landscape
Karan Goel, Nazneen Rajani, Jesse Vig, Samson Tan, Jason Wu, Stephan Zheng, Caiming Xiong, Mohit Bansal, Christopher Ré
ACL 2021. [Paper] [Github]
13 Jan 2021
Beyond Accuracy: Behavioral Testing of NLP models with CheckList
Marco Tulio Ribeiro, Tongshuang Wu, Carlos Guestrin, Sameer Singh
ACL 2020. [Paper][Github]
8 May 2020
Uncertainty
Uncertainty Estimation
Efficient Self-Evaluation for Diffusion Language Models via Sequence Regeneration
Linhao Zhong, Linyu Wu, Wen Wang, Yuling Xi, Chenchen Jing, Jiaheng Zhang, Hao Chen, Chunhua Shen
ACL 2026. [Paper]
3 Mar 2026
DRIFT: Detecting Representational Inconsistencies for Factual Truthfulness
Rohan Bhatnagar, Youran Sun, Chi Andrew Zhang, Yixin Wen, Haizhao Yang
arXiv 2026. [Paper]
24 Jan 2026
BLoB: Bayesian Low-Rank Adaptation by Backpropagation for Large Language Models
Yibin Wang, Haizhou Shi, Ligong Han, Dimitris Metaxas, Hao Wang
arXiv 2024. [Paper]
18 Jun 2024
Kernel Language Entropy: Fine-grained Uncertainty Quantification for LLMs from Semantic Similarities
Alexander Nikitin, Jannik Kossen, Yarin Gal, Pekka Marttinen
NeurIPS 2024. [Paper] [Github]
30 May 2024
Uncertainty Estimation and Quantification for LLMs: A Simple Supervised Approach
Linyu Liu, Yu Pan, Xiaocheng Li, Guanting Chen
arXiv 2024. [Paper]
24 Apr 2024
MARS: Meaning-Aware Response Scoring for Uncertainty Estimation in Generative LLMs
Bakman et al.
ACL 2024. [Paper]
19 Feb 2024
Shifting Attention to Relevance: Towards the Uncertainty Estimation of Large Language Models
Jinhao Duan, Hao Cheng, Shiqi Wang, Alex Zavalny, Chenan Wang, Renjing Xu, Bhavya Kailkhura, Kaidi Xu
arXiv 2023. [Paper]
9 Oct 2023
Look Before You Leap: An Exploratory Study of Uncertainty Measurement for Large Language Models
Yuheng Huang, Jiayang Song, Zhijie Wang, Shengming Zhao, Huaming Chen, Felix Juefei-Xu, Lei Ma
arXiv 2023. [Paper]
16 Jul 2023
Quantifying Uncertainty in Natural Language Explanations of Large Language Models
Sree Harsha Tanneru, Chirag Agarwal, Himabindu Lakkaraju
arXiv 2023. [Paper]
6 Nov 2023
Conformal Autoregressive Generation: Beam Search with Coverage Guarantees
Nicolas Deutschmann, Marvin Alberts, María Rodríguez Martínez
arXiv 2023. [Paper]
7 Sep 2023
Quantifying Uncertainty in Answers from any Language Model and Enhancing their Trustworthiness
Jiuhai Chen, Jonas Mueller
arXiv 2023. [Paper]
30 Aug 2023
Uncertainty in Natural Language Generation: From Theory to Applications
Joris Baan, Nico Daheim, Evgenia Ilia, Dennis Ulmer, Haau-Sing Li, Raquel Fernández, Barbara Plank, Rico Sennrich, Chrysoula Zerva, Wilker Aziz
arXiv 2023. [Paper]
28 July 2023
Generating with Confidence: Uncertainty Quantification for Black-box Large Language Models
Zhen Lin, Shubhendu Trivedi, Jimeng Sun
arXiv 2023. [Paper] [Github]
30 May 2023
Human Uncertainty in Concept-Based AI Systems
Katherine M. Collins, Matthew Barker, Mateo Espinosa Zarlenga, Naveen Raman, Umang Bhatt, Mateja Jamnik, Ilia Sucholutsky, Adrian Weller, Krishnamurthy Dvijotham
arXiv 2023. [Paper]
22 Mar 2023
Navigating the Grey Area: Expressions of Overconfidence and Uncertainty in Language Models
Kaitlyn Zhou, Dan Jurafsky, Tatsunori Hashimoto
arXiv 2023. [Paper]
25 Feb 2023
DEUP: Direct Epistemic Uncertainty Prediction
Salem Lahlou, Moksh Jain, Hadi Nekoei, Victor Ion Butoi, Paul Bertin, Jarrid Rector-Brooks, Maksym Korablyov, Yoshua Bengio
TMLR 2023. [Paper]
3 Feb 2023
On Compositional Uncertainty Quantification for Seq2seq Graph Parsing
Zi Lin, Du Phan, Panupong Pasupat, Jeremiah Zhe Liu, Jingbo Shang
ICLR 2023. [Paper]
1 Feb 2023
Neural-Symbolic Inference for Robust Autoregressive Graph Parsing via Compositional Uncertainty Quantification
Zi Lin, Jeremiah Liu, Jingbo Shang
EMNLP 2022. [Paper]
16 Jan 2023
Teaching Models to Express Their Uncertainty in Words
Stephanie Lin, Jacob Hilton, Owain Evans
TMLR 2022. [Paper] [Github] [TMLR] [Slide]
28 May 2022
Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation
Lorenz Kuhn, Yarin Gal, Sebastian Farquhar
ICLR 2023. [Paper]
19 Feb 2022
Cold-Start Data Selection for Few-shot Language Model Fine-tuning: A Prompt-Based Uncertainty Propagation Approach
Yue Yu, Rongzhi Zhang, Ran Xu, Jieyu Zhang, Jiaming Shen, Chao Zhang
arXiv 2022. [Paper][Github]
15 Sep 2022
Fine-Tuning Language Models via Epistemic Neural Networks
Ian Osband, Seyed Mohammad Asghari, Benjamin Van Roy, Nat McAleese, John Aslanides, Geoffrey Irving
arXiv 2022. [Paper][Github]
3 Nov 2022
Uncertainty Quantification with Pre-trained Language Models: A Large-Scale Empirical Analysis
Yuxin Xiao, Paul Pu Liang, Umang Bhatt, Willie Neiswanger, Ruslan Salakhutdinov, Louis-Philippe Morency
EMNLP 2022 (Findings). [Paper][Github]
10 Oct 2022
Uncertainty Estimation for Language Reward Models
Adam Gleave, Geoffrey Irving
arXiv 2022. [Paper]
14 Mar 2022
Uncertainty Estimation and Reduction of Pre-trained Models for Text Regression
Yuxia Wang, Daniel Beck, Timothy Baldwin, Karin Verspoor
TACL 2022. [Paper]
Jun 2022
Uncertainty Estimation in Autoregressive Structured Prediction
Andrey Malinin, Mark Gales
ICLR 2021. [Paper]
18 Feb 2020
Unsupervised Quality Estimation for Neural Machine Translation
Marina Fomicheva, Shuo Sun, Lisa Yankovskaya, Frédéric Blain, Francisco Guzmán, Mark Fishel, Nikolaos Aletras, Vishrav Chaudhary, Lucia Specia
TACL 2020. [Paper][Dataset]
21 May 2020
Analyzing Uncertainty in Neural Machine Translation
Myle Ott, Michael Auli, David Grangier, Marc’Aurelio Ranzato
ICML 2018. [Paper]
2018
Uncertainty Quantification for Language Models: A Suite of Black-Box, White-Box, LLM Judge, and Ensemble Scorers
Dylan Bouchard, Mohit Singh Chauhan
arXiv 2025. [Paper][GitHub]
April 2025
Calibration
Fabrication on absent fields: a pre-registered six-arm document-extraction bake-off
Velrim
Zenodo 2026. [Article] [Code] [DOI]
Sep 2026
REFUTE: Reasoning Over Evidence — scientific critique & epistemic calibration
BGPT
HF 2026. [Technical report][Dataset][Leaderboard]
Jun 2026
The Asymmetric Burden of Proof: LLMs Show a Null-Result Asymmetry in a Matched-Vignette Benchmark
Rolando Bosch
Working paper 2026. [Paper]
4 Mar 2026
Similarity-Distance-Magnitude Universal Verification
Allen Schmaltz
arXiv 2025. [Paper] [Github]
27 Feb 2025
Calibrating Large Language Models Using Their Generations Only
Dennis Ulmer, Martin Gubri, Hwaran Lee, Sangdoo Yun, Seong Joon Oh
ACL 2024. [Paper][Github][Poster][Slides]
9 Mar 2024
Batch Calibration: Rethinking Calibration for In-Context Learning and Prompt Engineering
Han Zhou, Xingchen Wan, Lev Proleev, Diana Mincu, Jilin Chen, Katherine Heller, Subhrajit Roy
ICLR 2024. [Paper]
24 Jan 2024
Do Large Language Models Know What They Don’t Know?
Zhangyue Yin, Qiushi Sun, Qipeng Guo, Jiawen Wu, Xipeng Qiu, Xuanjing Huang
arXiv 2023. [Paper]
29 May 2023
Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback
Katherine Tian, Eric Mitchell, Allan Zhou, Archit Sharma, Rafael Rafailov, Huaxiu Yao, Chelsea Finn, Christopher D. Manning
arXiv 2023. [Paper]
24 May 2023
Towards Reliable Misinformation Mitigation: Generalization, Uncertainty, and GPT-4
Kellin Pelrine, Meilina Reksoprodjo, Caleb Gupta, Joel Christoph, Reihaneh Rabbany
arXiv 2023. [Paper]
24 May 2023
Calibrated Interpretation: Confidence Estimation in Semantic Parsing
Elias Stengel-Eskin, Benjamin Van Durme
arXiv 2022. [Paper] [Github]
14 Nov 2022.
Calibrating Sequence likelihood Improves Conditional Language Generation
Yao Zhao, Misha Khalman, Rishabh Joshi, Shashi Narayan, Mohammad Saleh, Peter J. Liu
ICLR 2023. [Paper]
30 Sep 2022
Calibrated Selective Classification
Adam Fisch, Tommi Jaakkola, Regina Barzilay
TMLR 2022. [Paper]
25 Aug 2022
Reducing conversational agents’ overconfidence through linguistic calibration
Sabrina J. Mielke, Arthur Szlam, Emily Dinan, Y-Lan Boureau
NAACL 2022. [Paper]
22 Jun 2022
Re-Examining Calibration: The Case of Question Answering
Chenglei Si, Chen Zhao, Sewon Min, Jordan Boyd-Graber
EMNLP 2022 Findings. [Paper]
25 May 2022
Towards Collaborative Neural-Symbolic Graph Semantic Parsing via Uncertainty
Zi Lin, Jeremiah Liu, Jingbo Shang
ACL Fingings 2022. [Paper]
22 May 2022
Uncertainty-aware machine translation evaluation
Taisiya Glushkova, Chrysoula Zerva, Ricardo Rei, André F. T. Martins
EMNLP 2021. [Paper]
13 Sep 2021
Calibrate Before Use: Improving Few-Shot Performance of Language Models
Tony Z. Zhao, Eric Wallace, Shi Feng, Dan Klein, Sameer Singh
ICML 2021. [Paper][Github
19 Feb 2021
How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answering
Zhengbao Jiang, Jun Araki, Haibo Ding, Graham Neubig
TACL 2021. [Paper][Github]
2 Dec 2020
Calibration of Pre-trained Transformers
Shrey Desai, Greg Durrett
EMNLP 2020. [Paper][Github]
17 May 2020
Ambiguity
Tree of Clarifications: Answering Ambiguous Questions with Retrieval-Augmented Large Language Models
Gangwoo Kim, Sungdong Kim, Byeongguk Jeon, Joonsuk Park, Jaewoo Kang
EMNLP 2023. [Paper][Github]
23 Oct 2023
Selectively Answering Ambiguous Questions
Jeremy R. Cole, Michael J.Q. Zhang, Daniel Gillick, Julian Martin Eisenschlos, Bhuwan Dhingra, Jacob Eisenstein \
arXiv 2023. [Paper]
24 May 2023
We’re Afraid Language Models Aren’t Modeling Ambiguity \
Alisa Liu, Zhaofeng Wu, Julian Michael, Alane Suhr, Peter West, Alexander Koller, Swabha Swayamdipta, Noah A. Smith, Yejin Choi
arXiv 2023. [Paper][Github]
24 Apr 2023
Task Ambiguity in Humans and Language Models
Alex Tamkin, Kunal Handa, Avash Shrestha, Noah Goodman
ICLR 2023. [Paper][Github]
20 Dec 2022
CLAM: Selective Clarification for Ambiguous Questions with Generative Language Models
Lorenz Kuhn, Yarin Gal, Sebastian Farquhar
arXiv 2022. [Paper]
15 Dec 2022
How to Approach Ambiguous Queries in Conversational Search: A Survey of Techniques, Approaches, Tools, and Challenges
Kimiya Keyvan, Jimmy Xiangji Huang
ACM Computing Survey, 2022. [Paper]
7 Dec 2022
Assistance with large language models
Dmitrii Krasheninnikov, Egor Krasheninnikov, David Krueger
NeurIPS MLSW Workshop 2022. [Paper]
5 Dec 2022
Why Did the Chicken Cross the Road? Rephrasing and Analyzing Ambiguous Questions in VQA
Elias Stengel-Eskin, Jimena Guallar-Blasco, Yi Zhou, Benjamin Van Durme
arXiv 2022. [Paper][Github]
14 Nov 2022
Abg-CoQA: Clarifying Ambiguity in Conversational Question Answering
Meiqi Guo, Mingda Zhang, Siva Reddy, Malihe Alikhani
AKBC 2021. [Paper]
22 Jun 2021
Confidence
The Confidence-Competence Gap in Large Language Models: A Cognitive Study
Aniket Kumar Singh, Suman Devkota, Bishal Lamichhane, Uttam Dhakal, Chandra Dhakal
arXiv 2023. [Paper]
28 Sep 2023
Strength in Numbers: Estimating Confidence of Large Language Models by Prompt Agreement
Gwenyth Portillo Wightman, Alexandra Delucia, Mark Dredze
ACL TrustNLP Workshop 2023. [Paper]
1 Jul 2023
What Are the Different Approaches for Detecting Content Generated by LLMs Such As ChatGPT? And How Do They Work and Differ?
Sebastian Raschka
[Link] [GPTZero]
1 Feb 2023
DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature
Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D. Manning, Chelsea Finn
arXiv 2023. [Paper][Website]
26 Jan 2023
Confident Adaptive Language Modeling
Tal Schuster, Adam Fisch, Jai Gupta, Mostafa Dehghani, Dara Bahri, Vinh Q. Tran, Yi Tay, Donald Metzler
NeurIPS 2022. [Paper]
25 Oct 2022
Conformal risk control
Anastasios N Angelopoulos, Stephen Bates, Adam Fisch, Lihua Lei, Tal Schuster
arXiv 2022. [Paper][Github]
4 Aug 2022
Active Learning
A Survey of Active Learning for Natural Language Processing
Zhisong Zhang, Emma Strubell, Eduard Hovy
EMNLP 2022. [Paper][Github]
18 Oct 2022
Active Prompting with Chain-of-Thought for Large Language Models
Shizhe Diao, Pengcheng Wang, Yong Lin, Tong Zhang
arXiv 2023. [Paper][Github]
23 Feb 2023
Low-resource Interactive Active Labeling for Fine-tuning Language Models
Seiji Maekawa, Dan Zhang, Hannah Kim, Sajjadur Rahman, Estevam Hruschka
EMNLP Findings 2022. [Paper]
7 Dec 2022
Can You Label Less by Using Out-of-Domain Data? Active & Transfer Learning with Few-shot Instructions
Rafal Kocielnik, Sara Kangaslahti, Shrimai Prabhumoye, Meena Hari, R. Michael Alvarez, Anima Anandkumar
NeurIPS Workshop 2022. [Paper]
21 Nov 2022
AfroLM: A Self-Active Learning-based Multilingual Pretrained Language Model for 23 African Languages
Bonaventure F. P. Dossou, Atnafu Lambebo Tonja, Oreen Yousuf, Salomey Osei, Abigail Oppong, Iyanuoluwa Shode, Oluwabusayo Olufunke Awoyomi, Chris Chinenye Emezue
EMNLP 2022. [Paper][Github]
7 Nov 2022
Active Learning Helps Pretrained Models Learn the Intended Task
Alex Tamkin, Dat Pham Nguyen, Salil Deshpande, Jesse Mu, Noah Goodman
NeurIPS 2022. [Paper][Github]
31 Oct 2022
Selective Annotation Makes Language Models Better Few-Shot Learners
Hongjin Su, Jungo Kasai, Chen Henry Wu, Weijia Shi, Tianlu Wang, Jiayi Xin, Rui Zhang, Mari Ostendorf, Luke Zettlemoyer, Noah A. Smith, Tao Yu
ICLR 2023. [Paper][Github]
5 Sep 2022
Multi-task Active Learning for Pre-trained Transformer-based Models
Guy Rotman, Roi Reichart
TACL 2022. [Paper] [Github]
10 Aug 2022
AcTune: Uncertainty-Based Active Self-Training for Active Fine-Tuning of Pretrained Language Models
Yue Yu, Lingkai Kong, Jieyu Zhang, Rongzhi Zhang, Chao Zhang
NAACL-HLT2022. [Paper] [Github]
10 Jul 2022
Towards Computationally Feasible Deep Active Learning
Akim Tsvigun, Artem Shelmanov, Gleb Kuzmin, Leonid Sanochkin, Daniil Larionov, Gleb Gusev, Manvel Avetisian, Leonid Zhukov
NAACL 2022. [Paper] [Github]
7 May 2022
FAMIE: A Fast Active Learning Framework for Multilingual Information Extraction
Minh Van Nguyen, Nghia Trung Ngo, Bonan Min, Thien Huu Nguyen
NAACL 2022. [Paper] [Github]
16 Feb 2022
On the Importance of Effectively Adapting Pretrained Language Models for Active Learning
Katerina Margatina, Loïc Barrault, Nikolaos Aletras
ACL 2022. [Paper]
2 Mar 2022
Limitations of Active Learning With Deep Transformer Language Models
Mike D’Arcy, Doug Downey
Arxiv 2022. [Paper]
28 Jan 2022
Active Learning by Acquiring Contrastive Examples
Katerina Margatina, Giorgos Vernikos, Loïc Barrault, Nikolaos Aletras
EMNLP 2021. [Paper][Github]
8 Sep 2021
Revisiting Uncertainty-based Query Strategies for Active Learning with Transformers
Christopher Schröder, Andreas Niekler, Martin Potthast
ACL 2022 Findings. [Paper][Github]
12 Jul 2021
Active Learning for Sequence Tagging with Deep Pre-trained Models and Bayesian Uncertainty Estimates
Artem Shelmanov, Dmitri Puzyrev, Lyubov Kupriyanova, Denis Belyakov, Daniil Larionov, Nikita Khromov, Olga Kozlova, Ekaterina Artemova, Dmitry V. Dylov, Alexander Panchenko
EACL 2021. [Paper]
18 Feb 2021
Fine-tuning BERT for Low-Resource Natural Language Understanding via Active Learning
Daniel Grießhaber, Johannes Maucher, Ngoc Thang Vu
COLING 2020. [Paper]
4 Dec 2020
Reliability
Hallucination
awesome hallucination detection
Cognometry v0: 8-Benchmark Cross-Validated Hallucination Detection in Production LLMs
Introduces cognometry — the empirical measurement of cognitive states in LLMs. 9-signal pooled LR (text, entity, knowledge grounding, 4 response-novelty variants, NLI contradiction via DeBERTa-v3-base-mnli) cross-validated on 8 benchmarks (HaluEval QA/Dialog/Summ, TruthfulQA, HaluBench DROP/PubMedQA/FinanceBench/RAGTruth). AUC 0.998 on HaluEval-QA; two below-chance results (DROP, FinanceBench) declared as published failure modes in the weights module.
[Paper] [Code] [Manifesto]
SPARK: Adaptive Parametric Knowledge Participation for Faithful Retrieval-Augmented Generation
Ruochang Li, Pengcheng Huang, Zhenghao Liu, Yukun Yan, Huiyuan Xie, Yu Gu, Ge Yu, Maosong Sun
arXiv 2026. [Paper][Github]
26 Jun 2026
DRIFT: Detecting Representational Inconsistencies for Factual Truthfulness
Rohan Bhatnagar, Youran Sun, Chi Andrew Zhang, Yixin Wen, Haizhao Yang
arXiv 2026. [Paper]
24 Jan 2026
QuCo-RAG: Quantifying Uncertainty from the Pre-training Corpus for Dynamic Retrieval-Augmented Generation
Dehai Min, Kailin Zhang, Tongtong Wu, Lu Cheng
ACL Findings 2026. [Paper][Github]
22 Dec 2025
ParamMute: Suppressing Knowledge-Critical FFNs for Faithful Retrieval-Augmented Generation
Pengcheng Huang, Zhenghao Liu, Yukun Yan, Haiyan Zhao, Xiaoyuan Yi, Hao Chen, Zhiyuan Liu, Maosong Sun, Tong Xiao, Ge Yu, Chenyan Xiong
NeurIPS 2025. [Paper][Github]
21 Feb 2025
HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Models
Tianrui Guan*, Fuxiao Liu*, Xiyang Wu, Ruiqi Xian, Zongxia Li, Xiaoyu Liu, Xijun Wang, Lichang Chen, Furong Huang, Yaser Yacoob, Dinesh Manocha, Tianyi Zhou
CVPR 2024. [Paper][Github]
18 Mar 2024
SAC$^3$: Reliable Hallucination Detection in Black-Box Language Models via Semantic-aware Cross-check Consistency
Jiaxin Zhang, Zhuohang Li, Kamalika Das, Bradley A. Malin, Sricharan Kumar
EMNLP 2023. [Paper][Github]
3 Nov 2023
Hallucination Leaderboard
Vectara
[Link]
2 Nov 2023
Beyond Factuality: A Comprehensive Evaluation of Large Language Models as Knowledge Generators
Liang Chen, Yang Deng, Yatao Bian, Zeyu Qin, Bingzhe Wu, Tat-Seng Chua, Kam-Fai Wong
EMNLP 2023. [Paper][Github]
12 Oct 2023
Chain-of-Verification Reduces Hallucination in Large Language Models
Shehzaad Dhuliawala, Mojtaba Komeili, Jing Xu, Roberta Raileanu, Xian Li, Asli Celikyilmaz, Jason Weston
arXiv 2023. [Paper]
20 Sep 2023
Do Language Models Know When They’re Hallucinating References?
Ayush Agrawal, Lester Mackey, Adam Tauman Kalai
arXiv 2023. [Paper]
29 May 2023.
Self-contradictory Hallucinations of Large Language Models: Evaluation, Detection and Mitigation
Niels Mündler, Jingxuan He, Slobodan Jenko, Martin Vechev
arXiv 2023. [Paper]
25 May 2023
Why Does ChatGPT Fall Short in Providing Truthful Answers?
Shen Zheng, Jie Huang, Kevin Chen-Chuan Chang
arXiv 2023. [Paper]
24 May 2023
How Language Model Hallucinations Can Snowball
Muru Zhang, Ofir Press, William Merrill, Alisa Liu, Noah A. Smith
arXiv 2023. [Paper]
22 May 2023
LM vs LM: Detecting Factual Errors via Cross Examination
Roi Cohen, May Hamri, Mor Geva, Amir Globerson
arXiv 2023. [Paper]
22 May 2023
HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models
Junyi Li, Xiaoxue Cheng, Wayne Xin Zhao, Jian-Yun Nie, Ji-Rong Wen
arXiv 2023. [Paper]
19 May 2023
SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models
Potsawee Manakul, Adian Liusie, Mark J. F. Gales
arXiv 2023. [Paper] [Github]
8 Mar 2023
Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedback
Baolin Peng, Michel Galley, Pengcheng He, Hao Cheng, Yujia Xie, Yu Hu, Qiuyuan Huang, Lars Liden, Zhou Yu, Weizhu Chen, Jianfeng Gao
arXiv 2023. [Paper]
23 Feb 2023
RHO (ρ): Reducing Hallucination in Open-domain Dialogues with Knowledge Grounding
Ziwei Ji, Zihan Liu, Nayeon Lee, Tiezheng Yu, Bryan Wilie, Min Zeng, Pascale Fung
arXiv 2022. [Paper]
3 Dec 2022
FaithDial: A Faithful Benchmark for Information-Seeking Dialogue
Nouha Dziri, Ehsan Kamalloo, Sivan Milton, Osmar Zaiane, Mo Yu, Edoardo M. Ponti, Siva Reddy
TACL 2022. [Paper]
22 Apr 2022
Survey of Hallucination in Natural Language Generation
Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Yejin Bang, Wenliang Dai, Andrea Madotto, Pascale Fung
arXiv 2022. [Paper]
8 Feb 2022
Mechanistic Interpretability
FG-TRACER: Tracing Information Flow in Multimodal Large Language Models in Free-Form Generation
Alessia Saporita, Vittorio Pipoli, Federico Bolelli, Lorenzo Baraldi, Andrea Acquaviva, Elisa Ficarra
WACV 2026. [Paper] [Github]
Introduces a mechanistic interpretability framework for tracing cross-modal information flow between visual and textual modalities in MLLMs during free-form generation, leveraging attention masking interventions and a novel stable formulation for quantifying information flow.
Truthfulness
TruthX: Alleviating Hallucinations by Editing Large Language Models in Truthful Space \
Shaolei Zhang, Tian Yu, Yang Feng
arXiv 2024. [Paper] [Github]
27 Feb 2024
Inference-Time Intervention: Eliciting Truthful Answers from a Language Model \
Kenneth Li, Oam Patel, Fernanda Viégas, Hanspeter Pfister, Martin Wattenberg
arXiv 2023. [Paper] [Github]
6 June 2023
The Internal State of an LLM Knows When its Lying
Amos Azaria, Tom Mitchell
arXiv 2023. [Paper]
26 Apr 2023
TruthfulQA: Measuring How Models Mimic Human Falsehoods
Stephanie Lin, Jacob Hilton, Owain Evans
ACL 2022. [Paper] [Github] [Blog]
8 Sep 2021
Truthful AI: Developing and governing AI that does not lie
Owain Evans, Owen Cotton-Barratt, Lukas Finnveden, Adam Bales, Avital Balwit, Peter Wills, Luca Righetti, William Saunders
arXiv 2021. [Paper] [Blog]
13 Oct 2021
Measuring Reliability of Large Language Models through Semantic Consistency
Harsh Raj, Domenic Rosati, Subhabrata Majumdar
NeurIPS 2022 ML Safety Workshop. [Paper]
10 Nov 2022
Reasoning
REFINER: Reasoning Feedback on Intermediate Representations
Debjit Paul, Mete Ismayilzada, Maxime Peyrard, Beatriz Borges, Antoine Bosselut, Robert West, Boi Faltings
arXiv 2023. [Paper]
4 Apr 2023
OpenICL: An Open-Source Framework for In-context Learning
Zhenyu Wu, YaoXiang Wang, Jiacheng Ye, Jiangtao Feng, Jingjing Xu, Yu Qiao, Zhiyong Wu
arXiv 2023. [Paper] [Github]
6 Mar 2023
Reliable Natural Language Understanding with Large Language Models and Answer Set Programming
Abhiramon Rajasekharan, Yankai Zeng, Parth Padalkar, Gopal Gupta
arXiv 2023. [Paper]
7 Feb 2023
Self-Consistency Improves Chain of Thought Reasoning in Language Models
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, Denny Zhou
ICLR 2023. [Paper]
21 Mar 2022
Chain of Thought Prompting Elicits Reasoning in Large Language Models.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed Chi, Quoc Le, Denny Zhou
arXiv 2022. [Paper]
28 Jan 2022
STaR: Self-Taught Reasoner Bootstrapping Reasoning With Reasoning.
Eric Zelikman, Yuhuai Wu, Noah D. Goodman
NeurIPS 2022. [Paper][Github]
28 Mar 2022
The Unreliability of Explanations in Few-shot Prompting for Textual Reasoning
Xi Ye, Greg Durrett
NeurIPS 2022. [Paper] [Github]
6 May 2022
Rationale-Augmented Ensembles in Language Models
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Denny Zhou
arXiv 2022. [Paper]
2 Jul 2022
ReAct: Synergizing Reasoning and Acting in Language Models
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, Yuan Cao
ICLR 2023. [Paper][Github] [Project]
6 Oct 2022
On Second Thought, Let’s Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoning
Omar Shaikh, Hongxin Zhang, William Held, Michael Bernstein, Diyi Yang
arXiv 2022. [Paper]
15 Dec 2022
On the Advance of Making Language Models Better Reasoners
Yifei Li, Zeqi Lin, Shizhuo Zhang, Qiang Fu, Bei Chen, Jian-Guang Lou, Weizhu Chen
arXiv 2022. [Paper][Github]
6 Jun 2022
Ask Me Anything: A simple strategy for prompting language models
Simran Arora, Avanika Narayan, Mayee F. Chen, Laurel Orr, Neel Guha, Kush Bhatia, Ines Chami, Frederic Sala, Christopher Ré
arXiv 2022. [Paper][Github]
5 Oct 2022
MathPrompter: Mathematical Reasoning using Large Language Models
Shima Imani, Liang Du, Harsh Shrivastava
arXiv 2023. [Paper]
4 Mar 2023
Complexity-Based Prompting for Multi-Step Reasoning
Yao Fu, Hao Peng, Ashish Sabharwal, Peter Clark, Tushar Khot
arXiv 2022. [Paper][Github]
3 Oct 2022
Measuring and Narrowing the Compositionality Gap in Language Models
Ofir Press, Muru Zhang, Sewon Min, Ludwig Schmidt, Noah A. Smith, Mike Lewis
arXiv 2022. [Paper][Github]
7 Oct 2022
Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questions
Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, Ashish Sabharwal
arXiv 2023. [Paper][Github]
20 Dec 2022
Prompt tuning, optimization and design
Large Language Models as Optimizers
Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoc V. Le, Denny Zhou, Xinyun Chen
arXiv 2023. [Paper]
Sep 7 2023
InstructZero: Efficient Instruction Optimization for Black-Box Large Language Models
Lichang Chen, Jiuhai Chen, Tom Goldstein, Heng Huang, Tianyi Zhou
arXiv 2023. [Paper] [Github]
5 Jun 2023
Promptboosting: Black-box text classification with ten forward passes
Bairu Hou, Joe O’Connor, Jacob Andreas, Shiyu Chang, Yang Zhang
ICML 2023. [Paper][Github]
23 Jan 2023
GrIPS: Gradient-free, Edit-based Instruction Search for Prompting Large Language Models
Archiki Prasad, Peter Hase, Xiang Zhou, Mohit Bansal
EACL 2023. [Paper][Github]
Mar 14 2022
RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learning
Mingkai Deng, Jianyu Wang, Cheng-Ping Hsieh, Yihan Wang, Han Guo, Tianmin Shu, Meng Song, Eric P. Xing, Zhiting Hu
EMNLP 2022. [Paper][Github]
25 May 2022
Black-box Prompt Learning for Pre-trained Language Models
Shizhe Diao, Zhichao Huang, Ruijia Xu, Xuechun Li, Yong Lin, Xiao Zhou, Tong Zhang
TMLR 2023. [Paper][Github]
22 Jan 2022
Black-Box Tuning for Language-Model-as-a-Service
Tianxiang Sun, Yunfan Shao, Hong Qian, Xuanjing Huang, Xipeng Qiu
ICML 2022. [Paper][Github]
10 Jan 2022
BBTv2: towards a gradient-free future with large language models
Tianxiang Sun, Zhengfu He, Hong Qian, Yunhua Zhou, Xuanjing Huang, Xipeng Qiu \
EMNLP 2022. [Paper] [Github]
7 Dec 2022
Automatic Chain of Thought Prompting in Large Language Models
Zhuosheng Zhang, Aston Zhang, Mu Li, Alex Smola
ICLR 2023. [Paper][Github]
7 Oct 2022
Automatic Prompt Augmentation and Selection with Chain-of-Thought from Labeled Data
KaShun Shum, Shizhe Diao, Tong Zhang
arXiv 2023. [Paper][Github]
24 Feb 2023
Large Language Models Are Human-Level Prompt Engineers
Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, Jimmy Ba
ICLR 2023. [Paper] [Github]
3 Nov 2022
Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity
Yao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel, Pontus Stenetorp
ACL 2022. [Paper]
Active Example Selection for In-Context Learning
Yiming Zhang, Shi Feng, Chenhao Tan
EMNLP 2022. [Paper][Github]
8 Nov 2022
Selective Annotation Makes Language Models Better Few-Shot Learners
Hongjin Su, Jungo Kasai, Chen Henry Wu, Weijia Shi, Tianlu Wang, Jiayi Xin, Rui Zhang, Mari Ostendorf, Luke Zettlemoyer, Noah A. Smith, Tao Yu
ICLR 2023. [Paper][Github]
5 Sep 2022
Learning To Retrieve Prompts for In-Context Learning
Ohad Rubin, Jonathan Herzig, Jonathan Berant
NAACL-HLT 2022. [Paper][Github]
16 Dec 2021
Instruction and RLHF
LaMini-LM: A Diverse Herd of Distilled Models from Large-Scale Instructions
Minghao Wu, Abdul Waheed, Chiyu Zhang, Muhammad Abdul-Mageed, Alham Fikri Aji
arXiv 2023. [Paper][Github]
27 Apr 2023
Self-Refine: Iterative Refinement with Self-Feedback
Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Sean Welleck, Bodhisattwa Prasad Majumder, Shashank Gupta, Amir Yazdanbakhsh, Peter Clark
arXiv 2023. [Paper][Github] [Website]
30 Mar 2023
Is Prompt All You Need? No. A Comprehensive and Broader View of Instruction Learning
Renze Lou, Kai Zhang, Wenpeng Yin
arXiv 2023. [Paper][Github]
18 Mar 2023
Self-Instruct: Aligning Language Model with Self Generated Instructions
Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, Hannaneh Hajishirzi
arXiv 2022. [Paper] [Github]
20 Dec 2022
Constitutional AI: Harmlessness from AI Feedback
Yuntao Bai, et al (Anthropic)
arXiv 2022. [Paper]
15 Dec 2022
Discovering Language Model Behaviors with Model-Written Evaluations
Ethan Perez et al.
arXiv 2022. [Paper]
19 Dec 2022
In-Context Instruction Learning
Seonghyeon Ye, Hyeonbin Hwang, Sohee Yang, Hyeongu Yun, Yireun Kim, Minjoon Seo
arXiv 2023. [Paper][Github]
28 Feb 2023
Teaching LLMs to Learn Tool Trialing and Execution through Environment Interaction
Xingjie Gao, Pengcheng Huang, Zhenghao Liu, Yukun Yan, Shuo Wang, Zulong Chen, Chen Qian, Ge Yu, Yu Gu
arXiv 2026. [Paper][Github]
19 Jan 2026
Internet-augmented language models through few-shot prompting for open-domain question answering
Angeliki Lazaridou, Elena Gribovskaya, Wojciech Stokowiec, Nikolai Grigorev
arXiv 2023. [Paper]
10 Mar 2023
Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks
Wenhu Chen, Xueguang Ma, Xinyi Wang, William W. Cohen
arXiv 2022. [Paper][Github]
22 Nov 2022
PAL: Program-aided Language Models
Luyu Gao, Aman Madaan, Shuyan Zhou, Uri Alon, Pengfei Liu, Yiming Yang, Jamie Callan, Graham Neubig
arXiv 2022. [Paper] [Github] [Project]
18 Nov 2022
TALM: Tool Augmented Language Models
Aaron Parisi, Yao Zhao, Noah Fiedel
arXiv 2022. [Paper]
24 May 2022
Toolformer: Language Models Can Teach Themselves to Use Tool
Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, Thomas Scialom
arXiv 2023. [Paper]
9 Feb 2023
Fine-tuning
Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizes
Cheng-Yu Hsieh, Chun-Liang Li, Chih-Kuan Yeh, Hootan Nakhost, Yasuhisa Fujii, Alexander Ratner, Ranjay Krishna, Chen-Yu Lee, Tomas Pfister
arXiv 2023. [Paper]
3 May 2023
FreeLM: Fine-Tuning-Free Language Model
Xiang Li1, Xin Jiang, Xuying Meng, Aixin Sun, Yequan Wang
arXiv 2023. [Paper]
2 May 2023
Automated Data Curation for Robust Language Model Fine-Tuning
Jiuhai Chen, Jonas Mueller
arXiv 2024. [Paper]
19 Mar 2024
Robustness
Invariance
Consistency Is Not Understanding: Orbit Audits for Neural Field Weight Readers
Sungguk Cha, Mintae Kim, Youngsub Han, Byoung-Ki Jeon, Sangyeob Lee
arXiv 2026. [Paper]
4 Nov 2026
Invariant Language Modeling \
Maxime Peyrard, Sarvjeet Singh Ghotra, Martin Josifoski, Vidhan Agarwal, Barun Patra, Dean Carignan, Emre Kiciman, Robert West
EMNLP 2022. [Paper][Github]
16 Oct 2021
Towards Robust Personalized Dialogue Generation via Order-Insensitive Representation Regularization
Liang Chen, Hongru Wang, Yang Deng, Wai-Chung Kwan, Kam-Fai Wong
Findings of ACL 2023. [Paper][Github]
22 May 2023
Distribution Shift
Exploring Distributional Shifts in Large Language Models for Code Analysis
Shushan Arakelyan, Rocktim Jyoti Das, Yi Mao, Xiang Ren
arXiv 2023. [Paper]
16 Mar 2023
Out-of-Distribution
Out-of-Distribution Detection and Selective Generation for Conditional Language Models
Jie Ren, Jiaming Luo, Yao Zhao, Kundan Krishna, Mohammad Saleh, Balaji Lakshminarayanan, Peter J. Liu
ICLR 2023. [Paper]
30 Sep 2022
Adaptation and Generalization
On the Domain Adaptation and Generalization of Pretrained Language Models: A Survey
Xu Guo, Han Yu
arXiv 2022. [Paper]
6 Nov 2022
Adversarial
Internal Safety Collapse in Frontier Large Language Models
Yutao Wu, Jie Zhang, Tianwei Zhang, Xingjun Ma
arXiv 2026. [Paper][Github]
4 Mar 2026
PEARL: Towards Permutation-Resilient LLMs
Liang Chen, Li Shen, Yang Deng, Xiaoyan Zhao, Bin Liang, Kam-Fai Wong
ICLR 2025. [Paper][Github]
27 Feb 2025
Jailbreaking LLMs’ Safeguard with Universal Magic Words for Text Embedding Models
Haoyu Liang, Youran Sun, Yunfeng Cai, Jun Zhu, Bo Zhang
arXiv 2025. [Paper]
23 Jan 2025
Adversarial Attacks on LLMs
Lilian Weng
[Blog]
25 Oct 2023
PromptBench: Towards Evaluating the Robustness of Large Language Models on Adversarial Prompts
Kaijie Zhu, Jindong Wang, Jiaheng Zhou, Zichen Wang, Hao Chen, Yidong Wang, Linyi Yang, Wei Ye, Neil Zhenqiang Gong, Yue Zhang, Xing Xie
arXiv 2023. [Paper][Github]
7 Jun 20223
On the Robustness of ChatGPT: An Adversarial and Out-of-distribution Perspective
Jindong Wang, Xixu Hu, Wenxin Hou, Hao Chen, Runkai Zheng, Yidong Wang, Linyi Yang, Haojun Huang, Wei Ye, Xiubo Geng, Binxin Jiao, Yue Zhang, Xing Xie
arXiv 2023. [Paper] [Github]
22 Feb 2023
Reliability Testing for Natural Language Processing Systems
Samson Tan, Shafiq Joty, Kathy Baxter, Araz Taeihagh, Gregory A. Bennett, Min-Yen Kan
ACL-IJCNLP 2021. [Paper]
06 May 2021
Attribution
Attributed Question Answering: Evaluation and Modeling for Attributed Large Language Models
Bernd Bohnet, Vinh Q. Tran, Pat Verga, Roee Aharoni, Daniel Andor, Livio Baldini Soares, Massimiliano Ciaramita, Jacob Eisenstein, Kuzman Ganchev, Jonathan Herzig, Kai Hui, Tom Kwiatkowski, Ji Ma, Jianmo Ni, Lierni Sestorain Saralegui, Tal Schuster, William W. Cohen, Michael Collins, Dipanjan Das, Donald Metzler, Slav Petrov, Kellie Webster
arXiv 2022. [Paper]
15 Dec 2022
Causality
Can Large Language Models Infer Causation from Correlation?
Zhijing Jin, Jiarui Liu, Zhiheng Lyu, Spencer Poff, Mrinmaya Sachan, Rada Mihalcea, Mona Diab, Bernhard Schölkopf
arXiv 2023. [Paper] [Github]
9 Jun 2023
Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoning
Antonia Creswell, Murray Shanahan, Irina Higgins
ICLR 2023. [Paper]
19 May 2022
Investigating causal understanding in LLMs
Marius Hobbhahn, Tom Lieberum, David Seiler
NeurIPS 2022 Workshop. [Paper][Blog]
3 Oct 2022
评论