This reporsitory contains the code for our LREC-Coling 2024 paper "Action and Reaction go hand in hand! A Multi-modal Dialogue Act aided Sarcasm Identification".
An example dialogue from our dataset is illustrated in the figure below:
Sarcasm primarily involves saying something but "meaning the opposite" or "meaning something completely different" in order to convey a particular tone or mood. In both the above cases, the "meaning" is reflected by the communicative intention of the speaker, known as dialogue acts. In this paper, we seek to investigate a novel phenomenon of analyzing sarcasm in the context of dialogue acts with the hypothesis that the latter helps to understand the former better. Toward this aim, we extend the multi-modal MUStARD dataset to enclose dialogue acts for each dialogue. To demonstrate the utility of our hypothesis, we develop a dialogue act-aided multi-modal transformer network for sarcasm identification (MM-SARDAC), leveraging interrelation between these tasks. In addition, we introduce an order-infused, multi-modal infusion mechanism into our proposed model, which allows for a more intuitive combined modality representation by selectively focusing on relevant modalities in an ordered manner. Extensive empirical results indicate that dialogue act-aided sarcasm identification achieved better performance compared to performing sarcasm identification alone.
- Authors: Mohit Tomar, Tulika Saha, Abhisek Tiwari, Sriparna Saha
- Our (MUStARD2) dataset is curated by annotating Dialogue Act (DA) labels to already existing multi-modal sarcasm identification MUStARD dataset.
- The dataset consists of multi-party multi-modal conversation where we label the final utterance of a dialogue with Dialogue Acts (DAs). Thus final utterance has both Sarcasm and DAs labels.
- We selected 12 DAs for labeling dialogues and these are Greeting (g), Question (q), Answer (ans), Statement-Opinion (o), Statement-Non-Opinion (s), Apology (ap), Command (c), Agreement (ag), Dis-agreement (dag), Acknowledge (a), Backchannel (b), and Others (oth).
Here is the link to our dataset - https://docs.google.com/spreadsheets/d/1bUEa0nkWn5w4NsZpadE0EprW4YQLkWZUWxsSCZBXD2Y/edit?usp=sharing
If you consider this work to be useful, please cite it as
@inproceedings{tomar2024action,
title={Action and Reaction Go Hand in Hand! a Multi-modal Dialogue Act Aided Sarcasm Identification},
author={Tomar, Mohit Singh and Saha, Tulika and Tiwari, Abhisek and Saha, Sriparna},
booktitle={Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)},
pages={298--309},
year={2024}
}This code is adapted from the following Github repository https://github.com/LCS2-IIITD/MAF.
For any queries, feel free to contact Mohit Tomar ([email protected])