Probing reasoning with verbal probabilities in NLP models. We test probabilistic logical reasoning when using words estimative of probability (probably, surely...) in language models.
The enclosed notebook contains the code to build the dataset.
🤗 https://hf.co/datasets/sileod/probability_words_nli
@inproceedings{sileo-moens-2023-probing,
title = "Probing neural language models for understanding of words of estimative probability",
author = "Sileo, Damien and
Moens, Marie-francine",
booktitle = "Proceedings of the The 12th Joint Conference on Lexical and Computational Semantics (*SEM 2023)",
month = jul,
year = "2023",
address = "Toronto, Canada",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2023.starsem-1.41",
pages = "469--476",
}