How to Paraphrase Documents using Transformers

Example of a paraphrasing model in documents

George Pipis

6 min read

·

Jan 26, 2022

--

Listen

Share

Press enter or click to view image in full size

In this tutorial, we will show you how to paraphrase a whole document using Transformers. In this example, we will work with Colab so that you can easily follow along. We will extract programmatically the blog post “ How to get a Data Science Job without Experience “ using the “newspaper3k” library, and then we will generate a paraphrased document. Let’s start by installing the required libraries.

!pip install transformers
!pip install torch
!pip install sentencepiece
!pip install newspaper3k

Once you run this cell above in the Colab you will get something similar to this:

Press enter or click to view image in full size

Extract the Web Document

We can easily extract the text of the document and store it to a variable called input_text.

from newspaper import Article
 
#…