Extractive & Abstractive Summarization with Transformer Language Models | Research Paper Walkthrough

#textsummarization #transformers #languagemodel #researchpaperwalkthrough
In this video, we will go through an interesting work that proposes a text summarisation method that does abstractive summarisation by doing an extractive summarisation first. This technique has been seen to work really good when dealing with long documents.

P.S. Sorry for little bad audio at certain points in the video.

Please feel free to share out the content and subscribe to my channel :)

⏩ Abstract: We present a method to produce abstractive summaries of long documents that exceed several thousand words via neural abstractive summarization. We perform a simple extractive step before generating a summary, which is then used to condition the transformer language model on relevant information before being tasked with generating a summary. We show that this extractive step significantly improves summarization results. We also show that this approach produces more abstractive summaries compared to prior work that employs a copy mechanism while still achieving higher rouge scores. Note: The abstract above was not written by the authors, it was generated by one of the models presented in this paper.

⏩ OUTLINE:
0:00 - Abstract and Introduction
2:30 - Main contributions of the paper
3:28 - Pictorial flow of the summarisation process
4:36 - Extractive Summarisation Models
13:00 - Transformer Language Model (TLM)
16:25 - Ending Remarks

⏩ Paper Title: On Extractive and Abstractive Neural Document Summarization with Transformer Language Models
⏩ Paper Authors: Sandeep Subramanian, Raymond Li, Jonathan Pilault, Christopher Pal
⏩ Organization: Element AI, Montréal Institute for Learning Algorithms, Université de Montréal, École Polytechnique de Montréal, Canada CIFAR AI Chair

IMPORTANT LINKS
*********************

*********************************************
If you want to support me financially which totally optional and voluntary :) ❤️

*********************************************
*********************************************

Tools I use for making videos :)

Please feel free to share out the content and subscribe to my channel :)

About Me:
I am Prakhar Mishra and this channel is my passion project. I am currently pursuing my MS (by research) in Data Science. I have an industry work-ex of 3 years in the field of Data Science and Machine Learning with a particular focus in Natural Langauge Processing (NLP).

#extractive #abstractive #documentsummarisation #techviz #datascienceguy #naturallanguageprocessing #nlp #deeplearning

[daniilornat]

Thank you for explanation. But I can't understand this thing on 8:03. Does the documen-encoder takes as an input the entire sequence s1..sN (each sentence at a time) to produce d1? If it does, how we recieve d2, d3, ..., dN? Can you explain this, please?

[urvilpatel]

hi! Do you have code for this model. or maybe code for any hybrid based text summarization model

[KrithikaRajendran]

Where to learn the linear algebra necessary to understand this paper?