UNSUPERVISED MULTI-REVIEW SUMMARIZATION USING FINE-TUNED TRANSFORMER LANGUAGE MODELS

Automatic summarization is the task of generating concise, correct, and factual summaries.The task can be applied to different textual styles, including news, academic publications, and product or place reviews.This dissertation addresses the summary of multiple evaluations.This type of application stands out for its unsupervised nature and the need to deal with the redundancy of the information present in the reviews.The automatic summarization works are evaluated using the ROUGE metric, which is based on the comparison of n-grans between the reference text and the generated summary.The lack of supervised data motivated the creation of the MeanSum architecture, which was the first neural network architecture based on an unsupervised model for this task.It is based on auto-encoder and has been extended to other works, but none explored the effects of using the attention mechanism and auxiliary tasks during training.The present work is divided into two parts: the first deals with an experiment in which we make extensions to the MeanSum architecture, adding attention mechanisms and auxiliary sentiment classification tasks.In the same experiment, we explore synthetic data to adapt supervised models for unsupervised tasks.In the second part, we used the results previously obtained to carry out a second study on fine-tuning pre-trained Transformer language models.The use of these models showed a promising alternative to the unsupervised nature of the problem, outperforming previous works by + 4 ROUGE.