Topic Modeling 63,000+ Research Papers with BERTopic + GPT-4o-mini

Khushi Dubey

4 min read

·

Sep 1, 2025

--

Listen

Share

Hey there! 👋 I’m super excited to share my first blog post and even more excited that it’s about my very first NLP project. I dove deep into topic modeling using BERTopic , and ended up analyzing over 63,000 research papers from arXiv (yes, sixty-three thousand!!).

This blog post is a walk-through of what I built, why I built it, and what cool insights I found along the way. Let’s dive in!

What I Worked On

I applied BERTopic on the neuralwork/arxiver dataset of over 63,000 research papers to discover and label meaningful topics using embeddings and clustering.

Here’s what I did:

• Preprocessed the abstracts
• Ran BERTopic on them
• Used GPT-4o-mini to assign descriptive topic names
• Saved outputs (topics, names, visualizations) to avoid recomputation
• Analyzed trends like top topics and paper counts over time

I learned a lot about how BERTopic clusters papers based on semantic meaning and how GPT-generated names can make topics more intuitive.

Dataset at a Glance

• Dataset: neuralwork/arxiver
• Fields: Title, Abstract, Authors, Date, Markdown, Link
• Size: 63,357 papers from Sept 2022 to Oct 2023

Project Workflow (Modular & Reusable!)

To keep things clean, fast, and reusable, I split the project into 3 neat parts:

Part 1: Topic Modeling Pipeline + Caching

• Loaded the neuralwork/arxiver dataset
• Preprocessed the abstracts
• Ran BERTopic to generate topics
• Used GPT-4o-mini to give each topic a meaningful name
• Saved everything to a CSV (so I don’t have to rerun the heavy stuff)

This step is compute-heavy but only needs to be done once — super helpful if you want to experiment later.

Part 2: Building the Final Dataset

• Fetched full paper metadata (title, authors, date, etc.)
• Merged it with the topic modeling results
• The final dataset includes:
  id | title | abstract | authors | published_date | link | markdown | topic | Topic Name

Part 3: Visualizations + Topic Insights

I then explored the data with fun, interactive, informative visualizations!

Key Highlights:

1. Top 10 Topics by Paper Count

• Most frequent: Astrophysics of Neutrinos and Black Holes (9561 papers!)
• Least frequent: Renewable Energy and Grid Management (509 papers)

Press enter or click to view image in full size

1. Papers Published Per Month

• Peak: May 2023 (4701 papers)
• Low: Sept 2022 (1 paper)

Press enter or click to view image in full size

3. Monthly Trends for Top 5 Topics

Tracked trends for topics like Quantum Phase Transitions, Medical Imaging, and more!

Press enter or click to view image in full size

Model Settings

Here’s what I used under the hood:

Press enter or click to view image in full size

Visual Goodies

You can also explore the BERTopic visualizations in the Colab notebook.

• Bar Chart

Press enter or click to view image in full size

• Heatmap

Press enter or click to view image in full size

Intertopic Distance Map

Sample Topic Analysis

Press enter or click to view image in full size

Why I Loved This Project

• Fast Iteration: Thanks to caching, I could explore ideas quickly
• Readable Results: GPT-4o made topics actually understandable
• Emerging Trends: I saw how research areas evolve month-by-month
• Easily Extendable: You can plug in more data, tweak models, or explore new fields

Want to Try It Out?

Check out the full Colab notebook and GitHub repo here:
👉 GitHub Repo
👉 Colab Notebook

Thanks for reading! I’m just starting out in NLP and AI, and this project taught me so much about pipelines, embeddings, visualizations, and model efficiency. Hope it inspired you to explore BERTopic too! 😊

If you have feedback or ideas to improve it , I’d love to hear from you!