# Substack Leaderboard Scraper πŸ“Š (`easyapi/substack-leaderboard-scraper`) Actor Scrape detailed publication data from Substack leaderboards. Get comprehensive insights about top newsletters including subscriber counts, pricing, author details, and more. Perfect for newsletter research and market analysis. - **URL**: https://apify.com/easyapi/substack-leaderboard-scraper.md - **Developed by:** [EasyApi](https://apify.com/easyapi) (community) - **Categories:** Lead generation, Integrations, Other - **Stats:** 72 total users, 10 monthly users, 100.0% runs succeeded, 3 bookmarks - **User rating**: No ratings yet ## Pricing from $2.99 / 1,000 results This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Substack Leaderboard Scraper πŸ“Š Powerful scraper for extracting comprehensive data from Substack leaderboards. Get detailed insights about top newsletters, their performance, and monetization metrics. ### πŸ”₯ Features - Scrapes complete publication data from Substack leaderboards - Extracts detailed metrics including subscriber counts, pricing plans, and engagement data - Captures author information, publication details, and monetization settings - Supports multiple currency pricing information - Handles pagination automatically - Built-in stealth mechanisms to ensure reliable scraping ### πŸ“‹ Use Cases - Newsletter market research - Competitive analysis - Content strategy planning - Pricing strategy research - Author and publication discovery - Media industry analysis ### πŸ’Ž Output Data Each scraped publication includes: - Publication name, description, and URL - Subscriber counts and rankings - Author details and social media links - Pricing plans across multiple currencies - Publication settings and features - Podcast and community information - Monetization configurations - Theme and customization settings ### πŸ” Input Parameters The actor accepts the following inputs: - `leaderboardUrls`: Array of Substack leaderboard URLs to scrape - `maxItems`: Maximum number of items to scrape (optional) - `proxyConfiguration`: Proxy settings (optional) ### πŸ’‘ Tips - Use proxy configuration for better reliability - Adjust `maxItems` based on your needs - Consider rate limiting for large-scale scraping ### πŸ“ Notes - Respects Substack's robots.txt - Built with Puppeteer for reliable data extraction - Includes automatic retries and error handling #### Input Example A full explanation of an input example in JSON. ```json { "leaderboardUrls": [ "https://substack.com/leaderboard/technology/paid" ], "maxItems": 50 } ```` #### Output sample The results will be wrapped into a dataset which you can always find in theΒ **Storage**Β tab. Here's an excerpt from the data you'd get if you apply the input parameters above: And here is the same data but in JSON. You can choose in which format to download your data: JSON, JSONL, Excel spreadsheet, HTML table, CSV, or XML. ```json [ { "leaderboardUrl": "https://substack.com/leaderboard/technology/paid", "apple_pay_disabled": false, "apex_domain": null, "author_id": 30107029, "byline_images_enabled": true, "bylines_enabled": true, "chartable_token": null, "community_enabled": true, "copyright": "Gergely Orosz", "cover_photo_url": "https://bucketeer-e05bbc84-baa3-437e-9518-adb32be77984.s3.amazonaws.com/public/images/5a9e5545-0407-4cd2-9fd1-cc2c3882fe59_1279x268.png", "created_at": "2021-08-25T13:08:12.798Z", "custom_domain_optional": false, "custom_domain": "newsletter.pragmaticengineer.com", "custom_publication_theme_id": null, "default_comment_sort": "best_first", "default_coupon": null, "default_group_coupon": "ce270f3a", "default_show_guest_bios": true, "email_banner_url": "https://substack-post-media.s3.amazonaws.com/public/images/3f6c431d-310a-45a0-90fe-3dc43354ef38_1100x220.png", "email_from_name": "The Pragmatic Engineer", "email_from": null, "embed_tracking_disabled": false, "explicit": false, "expose_paywall_content_to_search_engines": true, "fb_pixel_id": null, "fb_site_verification_token": null, "flagged_as_spam": false, "founding_subscription_benefits": [], "free_subscription_benefits": [ "Shorter articles every Tuesday - with a full article once a month" ], "ga_pixel_id": null, "google_site_verification_token": null, "google_tag_manager_token": null, "hero_image": null, "hero_text": "Big Tech and startups, from the inside. Highly relevant for software engineers and managers, useful for those working in tech.", "hide_intro_subtitle": null, "hide_intro_title": null, "hide_podcast_feed_link": false, "homepage_type": "newspaper", "id": 458709, "image_thumbnails_always_enabled": false, "invite_only": false, "language": "en", "logo_url_wide": null, "logo_url": "https://substackcdn.com/image/fetch/f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fbucketeer-e05bbc84-baa3-437e-9518-adb32be77984.s3.amazonaws.com%2Fpublic%2Fimages%2F5ecbf7ac-260b-423b-8493-26783bf01f06_600x600.png", "minimum_group_size": 4, "moderation_enabled": true, "name": "The Pragmatic Engineer", "paid_subscription_benefits": [ "Full articles every Tuesday and Thursday", "Access to resources and templates for engineering managers and engineers", "Access to the complete archive, see all comments and comment on articles" ], "parsely_pixel_id": null, "payments_state": "enabled", "paywall_free_trial_enabled": false, "podcast_art_url": "https://substack-post-media.s3.amazonaws.com/public/images/2a3d0e54-d48a-4d6e-aa28-d834942d4167_3000x3000.png", "paid_podcast_episode_art_url": null, "podcast_byline": "Gergely Orosz", "podcast_description": "Software engineering at Big Tech and startups, from the inside. Deepdives with experienced engineers and tech professionals who share their hard-earned lessons, interesting stories and advice they have on building software. \n\nEspecially relevant for software engineers and engineering leaders: useful for those working in tech.", "podcast_enabled": true, "podcast_feed_url": null, "podcast_title": "The Pragmatic Engineer", "post_preview_limit": null, "require_clickthrough": false, "rss_feed_url": null, "rss_website_url": null, "show_pub_podcast_tab": true, "show_recs_on_homepage": false, "subdomain": "pragmaticengineer", "subscriber_invites": 0, "support_email": null, "theme_var_background_pop": "#FF6B00", "theme_var_color_links": false, "theme_var_cover_bg_color": null, "trial_end_override": null, "twitter_pixel_id": null, "type": "newsletter", "post_reaction_faces_enabled": true, "is_personal_mode": false, "plans": [ { "id": "yearly150usd", "object": "plan", "active": true, "aggregate_usage": null, "amount": 15000, "amount_decimal": "15000", "billing_scheme": "per_unit", "created": 1633977225, "currency": "usd", "interval": "year", "interval_count": 1, "livemode": true, "metadata": { "substack": "yes" }, "meter": null, "nickname": "$150 a year", "product": "prod_KOFpMRj8emeVAU", "tiers": null, "tiers_mode": null, "transform_usage": null, "trial_period_days": null, "usage_type": "licensed", "currency_options": { "aud": { "custom_unit_amount": null, "tax_behavior": "inclusive", "unit_amount": 24500, "unit_amount_decimal": "24500" }, "brl": { "custom_unit_amount": null, "tax_behavior": "inclusive", "unit_amount": 93000, "unit_amount_decimal": "93000" }, "cad": { "custom_unit_amount": null, "tax_behavior": "inclusive", "unit_amount": 22000, "unit_amount_decimal": "22000" }, "chf": { "custom_unit_amount": null, "tax_behavior": "inclusive", "unit_amount": 14000, "unit_amount_decimal": "14000" }, "dkk": { "custom_unit_amount": null, "tax_behavior": "inclusive", "unit_amount": 108000, "unit_amount_decimal": "108000" }, "eur": { "custom_unit_amount": null, "tax_behavior": "inclusive", "unit_amount": 14500, "unit_amount_decimal": "14500" }, "gbp": { "custom_unit_amount": null, "tax_behavior": "inclusive", "unit_amount": 12000, "unit_amount_decimal": "12000" }, "mxn": { "custom_unit_amount": null, "tax_behavior": "inclusive", "unit_amount": 313500, "unit_amount_decimal": "313500" }, "nok": { "custom_unit_amount": null, "tax_behavior": "inclusive", "unit_amount": 171000, "unit_amount_decimal": "171000" }, "nzd": { "custom_unit_amount": null, "tax_behavior": "inclusive", "unit_amount": 27000, "unit_amount_decimal": "27000" }, "pln": { "custom_unit_amount": null, "tax_behavior": "inclusive", "unit_amount": 62000, "unit_amount_decimal": "62000" }, "sek": { "custom_unit_amount": null, "tax_behavior": "inclusive", "unit_amount": 166000, "unit_amount_decimal": "166000" }, "usd": { "custom_unit_amount": null, "tax_behavior": "inclusive", "unit_amount": 15000, "unit_amount_decimal": "15000" } } }, { "id": "monthly15usd", "object": "plan", "active": true, "aggregate_usage": null, "amount": 1500, "amount_decimal": "1500", "billing_scheme": "per_unit", "created": 1630008931, "currency": "usd", "interval": "month", "interval_count": 1, "livemode": true, "metadata": { "substack": "yes" }, "meter": null, "nickname": "$15 a month", "product": "prod_K735TEUdeGQPmt", "tiers": null, "tiers_mode": null, "transform_usage": null, "trial_period_days": null, "usage_type": "licensed", "currency_options": { "aud": { "custom_unit_amount": null, "tax_behavior": "inclusive", "unit_amount": 2500, "unit_amount_decimal": "2500" }, "brl": { "custom_unit_amount": null, "tax_behavior": "inclusive", "unit_amount": 9500, "unit_amount_decimal": "9500" }, "cad": { "custom_unit_amount": null, "tax_behavior": "inclusive", "unit_amount": 2200, "unit_amount_decimal": "2200" }, "chf": { "custom_unit_amount": null, "tax_behavior": "inclusive", "unit_amount": 1400, "unit_amount_decimal": "1400" }, "dkk": { "custom_unit_amount": null, "tax_behavior": "inclusive", "unit_amount": 11000, "unit_amount_decimal": "11000" }, "eur": { "custom_unit_amount": null, "tax_behavior": "inclusive", "unit_amount": 1500, "unit_amount_decimal": "1500" }, "gbp": { "custom_unit_amount": null, "tax_behavior": "inclusive", "unit_amount": 1200, "unit_amount_decimal": "1200" }, "mxn": { "custom_unit_amount": null, "tax_behavior": "inclusive", "unit_amount": 31500, "unit_amount_decimal": "31500" }, "nok": { "custom_unit_amount": null, "tax_behavior": "inclusive", "unit_amount": 17500, "unit_amount_decimal": "17500" }, "nzd": { "custom_unit_amount": null, "tax_behavior": "inclusive", "unit_amount": 2700, "unit_amount_decimal": "2700" }, "pln": { "custom_unit_amount": null, "tax_behavior": "inclusive", "unit_amount": 6500, "unit_amount_decimal": "6500" }, "sek": { "custom_unit_amount": null, "tax_behavior": "inclusive", "unit_amount": 17000, "unit_amount_decimal": "17000" }, "usd": { "custom_unit_amount": null, "tax_behavior": "inclusive", "unit_amount": 1500, "unit_amount_decimal": "1500" } } } ], "stripe_user_id": "acct_1JSopsA7n83QB5wA", "stripe_country": "NL", "stripe_publishable_key": "pk_live_51JSopsA7n83QB5wAw9IUzfjqUCZGQg4SiKPHVTvZ9kYb7CCt6Ef1740RiT4hrkCLTlyQiFY1xYkVzJwauwGjRw0L00fCfhzbwF", "stripe_platform_account": "CA", "automatic_tax_enabled": true, "author_name": "Gergely Orosz", "author_handle": "pragmaticengineer", "author_photo_url": "https://substackcdn.com/image/fetch/f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fbucketeer-e05bbc84-baa3-437e-9518-adb32be77984.s3.amazonaws.com%2Fpublic%2Fimages%2F802a32bb-2048-428b-bdb5-d6acd1e2b2d5_48x48.png", "author_bio": "Writing The Pragmatic Engineer. Previously at Uber, Skype, Microsoft. Author of The Software Engineer's Guidebook.", "twitter_screen_name": "GergelyOrosz", "twitter_share_on_publish_opt_in": false, "has_custom_tos": false, "has_custom_privacy": false, "theme": { "background_pop_color": null, "web_bg_color": "#ffffff", "cover_bg_color": null, "publication_id": 458709, "color_links": null, "font_preset_heading": "fancy_serif", "font_preset_body": "sans", "font_family_headings": null, "font_family_body": null, "font_family_ui": null, "font_size_body_desktop": null, "print_secondary": null, "custom_css_web": null, "custom_css_email": null, "home_hero": "newspaper", "home_posts": "list", "home_show_top_posts": false, "hide_images_from_list": false, "home_hero_alignment": "left", "home_hero_show_podcast_links": true, "default_post_header_variant": null }, "threads_v2_settings": null, "default_group_coupon_percent_off": "10.00", "pause_return_date": null, "has_posts": true, "has_recommendations": true, "first_post_date": "2021-08-30T11:24:02.082Z", "has_podcast": true, "has_free_podcast": true, "has_subscriber_only_podcast": false, "has_community_content": true, "twitter_permissions": "write", "rankingDetail": "Tens of thousands of paid subscribers", "rankingDetailFreeIncluded": "Hundreds of thousands of subscribers", "rankingDetailOrderOfMagnitude": 10000, "rankingDetailFreeIncludedOrderOfMagnitude": 100000, "rankingDetailFreeSubscriberCount": "Over 931,000 subscribers", "rankingDetailByLanguage": { "de": { "rankingDetail": "Zehntausende von Paid-Abonnenten", "rankingDetailFreeIncluded": "Hunderttausende von Abonnenten", "rankingDetailOrderOfMagnitude": 10000, "rankingDetailFreeIncludedOrderOfMagnitude": 100000, "rankingDetailFreeSubscriberCount": "Über 931,000 Abonnenten", "freeSubscriberCount": "931,000", "freeSubscriberCountOrderOfMagnitude": "931K+" }, "es": { "rankingDetail": "Decenas de miles de suscriptores de pago", "rankingDetailFreeIncluded": "Cientos de miles de suscriptores", "rankingDetailOrderOfMagnitude": 10000, "rankingDetailFreeIncludedOrderOfMagnitude": 100000, "rankingDetailFreeSubscriberCount": "MΓ‘s de 931,000 suscriptores", "freeSubscriberCount": "931,000", "freeSubscriberCountOrderOfMagnitude": "931K+" }, "fr": { "rankingDetail": "Des dizaines de milliers d'abonnΓ©s payants", "rankingDetailFreeIncluded": "Des centaines de milliers d'abonnΓ©s", "rankingDetailOrderOfMagnitude": 10000, "rankingDetailFreeIncludedOrderOfMagnitude": 100000, "rankingDetailFreeSubscriberCount": "Plus de 931,000 abonnΓ©s", "freeSubscriberCount": "931,000", "freeSubscriberCountOrderOfMagnitude": "931K+" }, "pt": { "rankingDetail": "Dezenas de milhares de subscritores pagos", "rankingDetailFreeIncluded": "Centenas de milhares de subscritores", "rankingDetailOrderOfMagnitude": 10000, "rankingDetailFreeIncludedOrderOfMagnitude": 100000, "rankingDetailFreeSubscriberCount": "Mais de 931,000 subscritores", "freeSubscriberCount": "931,000", "freeSubscriberCountOrderOfMagnitude": "931K+" }, "pt-br": { "rankingDetail": "Dezenas de milhares de assinantes pagantes", "rankingDetailFreeIncluded": "Centenas de milhares de assinantes", "rankingDetailOrderOfMagnitude": 10000, "rankingDetailFreeIncludedOrderOfMagnitude": 100000, "rankingDetailFreeSubscriberCount": "Mais de 931,000 assinantes", "freeSubscriberCount": "931,000", "freeSubscriberCountOrderOfMagnitude": "931K+" }, "it": { "rankingDetail": "Decine di migliaia di abbonati a pagamento", "rankingDetailFreeIncluded": "Centinaia di migliaia di abbonati", "rankingDetailOrderOfMagnitude": 10000, "rankingDetailFreeIncludedOrderOfMagnitude": 100000, "rankingDetailFreeSubscriberCount": "Oltre 931,000 abbonati", "freeSubscriberCount": "931,000", "freeSubscriberCountOrderOfMagnitude": "931K+" }, "en": { "rankingDetail": "Tens of thousands of paid subscribers", "rankingDetailFreeIncluded": "Hundreds of thousands of subscribers", "rankingDetailOrderOfMagnitude": 10000, "rankingDetailFreeIncludedOrderOfMagnitude": 100000, "rankingDetailFreeSubscriberCount": "Over 931,000 subscribers", "freeSubscriberCount": "931,000", "freeSubscriberCountOrderOfMagnitude": "931K+" } }, "freeSubscriberCount": "931,000", "freeSubscriberCountOrderOfMagnitude": "931K+", "author_bestseller_tier": 0, "disable_monthly_subscriptions": false, "disable_annual_subscriptions": false, "hide_post_restacks": false, "notes_feed_enabled": false, "last_chat_post_at": null, "no_follow": false, "paywall_chat": "free", "sections": [ { "id": 24961, "created_at": "2022-08-18T13:19:56.809Z", "updated_at": "2024-06-18T17:22:31.340Z", "publication_id": 458709, "name": "The Pulse", "description": "Keeping a pulse on the software engineering industry, every Thursday. Insights, patterns and trends within Big Tech and startups. Exclusive analysis of what is happening, and why.", "slug": "the-pulse", "is_podcast": false, "is_live": true, "is_default_on": true, "sibling_rank": 1, "port_status": "success", "logo_url": null, "hide_from_navbar": false, "email_from_name": "", "hide_posts_from_pub_listings": false, "email_banner_url": null, "cover_photo_url": null, "hide_intro_title": false, "hide_intro_subtitle": false, "ignore_publication_email_settings": false, "podcastSettings": null, "showLinks": [], "spotifyPodcastSettings": null, "pageTheme": null, "podcastPalette": { "DarkMuted": { "population": 72, "rgb": [ 73, 153, 137 ] }, "DarkVibrant": { "population": 6013, "rgb": [ 4, 100, 84 ] }, "LightMuted": { "population": 7, "rgb": [ 142, 198, 186 ] }, "LightVibrant": { "population": 3, "rgb": [ 166, 214, 206 ] }, "Muted": { "population": 6, "rgb": [ 92, 164, 156 ] }, "Vibrant": { "population": 5, "rgb": [ 76, 164, 146 ] } }, "spotify_podcast_settings": null }, { "id": 132248, "created_at": "2024-06-18T15:49:33.192Z", "updated_at": "2024-06-18T16:23:15.199Z", "publication_id": 458709, "name": "Deepdives", "description": "Long-form educational articles every Tuesday. These articles typically go deep into a topic relevant for software engineers or engineering managers, and tend to offer actionable advice.", "slug": "deepdives", "is_podcast": false, "is_live": true, "is_default_on": true, "sibling_rank": 2, "port_status": "success", "logo_url": null, "hide_from_navbar": false, "email_from_name": "", "hide_posts_from_pub_listings": false, "email_banner_url": null, "cover_photo_url": null, "hide_intro_title": false, "hide_intro_subtitle": false, "ignore_publication_email_settings": false, "podcastSettings": null, "showLinks": [], "spotifyPodcastSettings": null, "pageTheme": null, "podcastPalette": { "DarkMuted": { "population": 72, "rgb": [ 73, 153, 137 ] }, "DarkVibrant": { "population": 6013, "rgb": [ 4, 100, 84 ] }, "LightMuted": { "population": 7, "rgb": [ 142, 198, 186 ] }, "LightVibrant": { "population": 3, "rgb": [ 166, 214, 206 ] }, "Muted": { "population": 6, "rgb": [ 92, 164, 156 ] }, "Vibrant": { "population": 5, "rgb": [ 76, 164, 146 ] } }, "spotify_podcast_settings": null }, { "id": 182752, "created_at": "2025-01-27T17:55:26.493Z", "updated_at": "2025-01-27T21:26:37.087Z", "publication_id": 458709, "name": "Announcements", "description": "Issues that are neither deepdives, nor The Pulse issues", "slug": "announcements", "is_podcast": false, "is_live": true, "is_default_on": true, "sibling_rank": 3, "port_status": "success", "logo_url": null, "hide_from_navbar": false, "email_from_name": "", "hide_posts_from_pub_listings": false, "email_banner_url": null, "cover_photo_url": null, "hide_intro_title": false, "hide_intro_subtitle": false, "ignore_publication_email_settings": false, "podcastSettings": null, "showLinks": [], "spotifyPodcastSettings": null, "pageTheme": null, "podcastPalette": { "DarkMuted": { "population": 72, "rgb": [ 73, 153, 137 ] }, "DarkVibrant": { "population": 6013, "rgb": [ 4, 100, 84 ] }, "LightMuted": { "population": 7, "rgb": [ 142, 198, 186 ] }, "LightVibrant": { "population": 3, "rgb": [ 166, 214, 206 ] }, "Muted": { "population": 6, "rgb": [ 92, 164, 156 ] }, "Vibrant": { "population": 5, "rgb": [ 76, 164, 146 ] } }, "spotify_podcast_settings": null } ], "multipub_migration": null, "navigationBarItems": [ { "id": "bc2883cd-a436-40ba-ad79-bb9944b34f1b", "publication_id": 458709, "sibling_rank": 0, "link_title": null, "link_url": null, "section_id": 24961, "post_id": null, "is_hidden": null, "standard_key": null, "post_tag_id": null, "post": null, "postTag": null, "section": { "id": 24961, "created_at": "2022-08-18T13:19:56.809Z", "updated_at": "2024-06-18T17:22:31.340Z", "publication_id": 458709, "name": "The Pulse", "description": "Keeping a pulse on the software engineering industry, every Thursday. Insights, patterns and trends within Big Tech and startups. Exclusive analysis of what is happening, and why.", "slug": "the-pulse", "is_podcast": false, "is_live": true, "is_default_on": true, "sibling_rank": 1, "port_status": "success", "logo_url": null, "hide_from_navbar": false, "email_from_name": "", "hide_posts_from_pub_listings": false, "email_banner_url": null, "cover_photo_url": null, "hide_intro_title": false, "hide_intro_subtitle": false, "ignore_publication_email_settings": false } }, { "id": "13219a85-602a-4315-a13a-e8cf8114b672", "publication_id": 458709, "sibling_rank": 9999, "link_title": null, "link_url": null, "section_id": 132248, "post_id": null, "is_hidden": null, "standard_key": null, "post_tag_id": null, "post": null, "postTag": null, "section": { "id": 132248, "created_at": "2024-06-18T15:49:33.192Z", "updated_at": "2024-06-18T16:23:15.199Z", "publication_id": 458709, "name": "Deepdives", "description": "Long-form educational articles every Tuesday. These articles typically go deep into a topic relevant for software engineers or engineering managers, and tend to offer actionable advice.", "slug": "deepdives", "is_podcast": false, "is_live": true, "is_default_on": true, "sibling_rank": 2, "port_status": "success", "logo_url": null, "hide_from_navbar": false, "email_from_name": "", "hide_posts_from_pub_listings": false, "email_banner_url": null, "cover_photo_url": null, "hide_intro_title": false, "hide_intro_subtitle": false, "ignore_publication_email_settings": false } }, { "id": "91c2c7ff-d4f9-4f8a-8ca2-c269ddb299e7", "publication_id": 458709, "sibling_rank": 9999, "link_title": null, "link_url": null, "section_id": 182752, "post_id": null, "is_hidden": null, "standard_key": null, "post_tag_id": null, "post": null, "postTag": null, "section": { "id": 182752, "created_at": "2025-01-27T17:55:26.493Z", "updated_at": "2025-01-27T21:26:37.087Z", "publication_id": 458709, "name": "Announcements", "description": "Issues that are neither deepdives, nor The Pulse issues", "slug": "announcements", "is_podcast": false, "is_live": true, "is_default_on": true, "sibling_rank": 3, "port_status": "success", "logo_url": null, "hide_from_navbar": false, "email_from_name": "", "hide_posts_from_pub_listings": false, "email_banner_url": null, "cover_photo_url": null, "hide_intro_title": false, "hide_intro_subtitle": false, "ignore_publication_email_settings": false } } ], "contributors": [ { "name": "Gergely Orosz", "handle": "pragmaticengineer", "role": "admin", "owner": true, "user_id": 30107029, "photo_url": "https://substackcdn.com/image/fetch/f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fbucketeer-e05bbc84-baa3-437e-9518-adb32be77984.s3.amazonaws.com%2Fpublic%2Fimages%2F802a32bb-2048-428b-bdb5-d6acd1e2b2d5_48x48.png", "bio": "Writing The Pragmatic Engineer. Previously at Uber, Skype, Microsoft. Author of The Software Engineer's Guidebook." }, { "name": "Pen Name Consulting", "handle": "pennameconsulting", "role": "contributor", "owner": false, "user_id": 98424238, "photo_url": "https://substackcdn.com/image/fetch/f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fbucketeer-e05bbc84-baa3-437e-9518-adb32be77984.s3.amazonaws.com%2Fpublic%2Fimages%2F396242ba-4052-49f1-9591-aa0cb88022ea_144x144.png", "bio": null } ], "threads_v2_enabled": false, "viralGiftsConfig": { "id": "4f27054e-d0f8-4806-ac93-003d99963c07", "publication_id": 458709, "enabled": true, "gifts_per_user": 5, "gift_length_months": 1, "send_extra_gifts": true, "message": "Big Tech and high-growth startups, from the inside. Highly relevant for software engineers and managers, useful for those working in tech.", "created_at": "2022-10-06T13:48:22.068838+00:00", "updated_at": "2022-10-06T13:48:22.068838+00:00", "days_til_invite": 14, "send_emails": true, "show_link": null, "grant_email_body": "{\"type\":\"doc\",\"content\":[{\"type\":\"paragraph\",\"content\":[{\"type\":\"text\",\"text\":\"πŸ‘‹ Hi, this is Gergely,\"}]},{\"type\":\"paragraph\",\"content\":[{\"type\":\"text\",\"text\":\"I am grateful that you are a subscriber to The Pragmatic Engineer. You’re support has helped make the publication what it is today!\"}]},{\"type\":\"paragraph\",\"content\":[{\"type\":\"text\",\"text\":\"As a thank you, you now have three FREE 1-month gift subscriptions to give to family and friends (usual value = $45). Know someone who might love The Pragmatic Engineer? Click on the link below to share one of your gift subscriptions with them.\"}]},{\"type\":\"button\",\"attrs\":{\"url\":\"%%grant_gift_url%%\",\"text\":\"Give a subscription\",\"action\":null,\"class\":\"left\"}},{\"type\":\"paragraph\",\"content\":[{\"type\":\"text\",\"text\":\"Thanks again for reading and sharing The Pragmatic Engineer!\"}]}]}", "grant_email_subject": "The Pragmatic Engineer subscriptions for you to give away" }, "tier": 2, "no_index": false, "can_set_google_site_verification": true, "can_have_sitemap": true, "founding_plan_name_english": "Founding Member", "draft_plans": [ { "id": "yearly150usd", "object": "plan", "active": true, "aggregate_usage": null, "amount": 15000, "amount_decimal": "15000", "billing_scheme": "per_unit", "created": 1633977225, "currency": "usd", "interval": "year", "interval_count": 1, "livemode": true, "metadata": { "substack": "yes" }, "meter": null, "nickname": "$150 a year", "product": "prod_KOFpMRj8emeVAU", "tiers": null, "tiers_mode": null, "transform_usage": null, "trial_period_days": null, "usage_type": "licensed", "currency_options": { "aud": { "custom_unit_amount": null, "tax_behavior": "inclusive", "unit_amount": 24500, "unit_amount_decimal": "24500" }, "brl": { "custom_unit_amount": null, "tax_behavior": "inclusive", "unit_amount": 93000, "unit_amount_decimal": "93000" }, "cad": { "custom_unit_amount": null, "tax_behavior": "inclusive", "unit_amount": 22000, "unit_amount_decimal": "22000" }, "chf": { "custom_unit_amount": null, "tax_behavior": "inclusive", "unit_amount": 14000, "unit_amount_decimal": "14000" }, "dkk": { "custom_unit_amount": null, "tax_behavior": "inclusive", "unit_amount": 108000, "unit_amount_decimal": "108000" }, "eur": { "custom_unit_amount": null, "tax_behavior": "inclusive", "unit_amount": 14500, "unit_amount_decimal": "14500" }, "gbp": { "custom_unit_amount": null, "tax_behavior": "inclusive", "unit_amount": 12000, "unit_amount_decimal": "12000" }, "mxn": { "custom_unit_amount": null, "tax_behavior": "inclusive", "unit_amount": 313500, "unit_amount_decimal": "313500" }, "nok": { "custom_unit_amount": null, "tax_behavior": "inclusive", "unit_amount": 171000, "unit_amount_decimal": "171000" }, "nzd": { "custom_unit_amount": null, "tax_behavior": "inclusive", "unit_amount": 27000, "unit_amount_decimal": "27000" }, "pln": { "custom_unit_amount": null, "tax_behavior": "inclusive", "unit_amount": 62000, "unit_amount_decimal": "62000" }, "sek": { "custom_unit_amount": null, "tax_behavior": "inclusive", "unit_amount": 166000, "unit_amount_decimal": "166000" }, "usd": { "custom_unit_amount": null, "tax_behavior": "inclusive", "unit_amount": 15000, "unit_amount_decimal": "15000" } } }, { "id": "monthly15usd", "object": "plan", "active": true, "aggregate_usage": null, "amount": 1500, "amount_decimal": "1500", "billing_scheme": "per_unit", "created": 1630008931, "currency": "usd", "interval": "month", "interval_count": 1, "livemode": true, "metadata": { "substack": "yes" }, "meter": null, "nickname": "$15 a month", "product": "prod_K735TEUdeGQPmt", "tiers": null, "tiers_mode": null, "transform_usage": null, "trial_period_days": null, "usage_type": "licensed", "currency_options": { "aud": { "custom_unit_amount": null, "tax_behavior": "inclusive", "unit_amount": 2500, "unit_amount_decimal": "2500" }, "brl": { "custom_unit_amount": null, "tax_behavior": "inclusive", "unit_amount": 9500, "unit_amount_decimal": "9500" }, "cad": { "custom_unit_amount": null, "tax_behavior": "inclusive", "unit_amount": 2200, "unit_amount_decimal": "2200" }, "chf": { "custom_unit_amount": null, "tax_behavior": "inclusive", "unit_amount": 1400, "unit_amount_decimal": "1400" }, "dkk": { "custom_unit_amount": null, "tax_behavior": "inclusive", "unit_amount": 11000, "unit_amount_decimal": "11000" }, "eur": { "custom_unit_amount": null, "tax_behavior": "inclusive", "unit_amount": 1500, "unit_amount_decimal": "1500" }, "gbp": { "custom_unit_amount": null, "tax_behavior": "inclusive", "unit_amount": 1200, "unit_amount_decimal": "1200" }, "mxn": { "custom_unit_amount": null, "tax_behavior": "inclusive", "unit_amount": 31500, "unit_amount_decimal": "31500" }, "nok": { "custom_unit_amount": null, "tax_behavior": "inclusive", "unit_amount": 17500, "unit_amount_decimal": "17500" }, "nzd": { "custom_unit_amount": null, "tax_behavior": "inclusive", "unit_amount": 2700, "unit_amount_decimal": "2700" }, "pln": { "custom_unit_amount": null, "tax_behavior": "inclusive", "unit_amount": 6500, "unit_amount_decimal": "6500" }, "sek": { "custom_unit_amount": null, "tax_behavior": "inclusive", "unit_amount": 17000, "unit_amount_decimal": "17000" }, "usd": { "custom_unit_amount": null, "tax_behavior": "inclusive", "unit_amount": 1500, "unit_amount_decimal": "1500" } } } ], "base_url": "https://newsletter.pragmaticengineer.com", "hostname": "newsletter.pragmaticengineer.com", "is_on_substack": false, "show_links": [ { "id": "23747", "publication_id": 458709, "section_id": null, "url": "https://podcasts.apple.com/us/podcast/the-pragmatic-engineer-podcast/id1769051199", "platform": "apple_podcasts" }, { "id": "23797", "publication_id": 458709, "section_id": null, "url": "https://open.spotify.com/show/2Bho9xCbOQMWMJ7UKmqCzD", "platform": "spotify" }, { "id": "23798", "publication_id": 458709, "section_id": null, "url": "https://open.spotify.com/show/2Bho9xCbOQMWMJ7UKmqCzD", "platform": "spotify_for_paid_users" }, { "id": "23799", "publication_id": 458709, "section_id": null, "url": "https://www.youtube.com/@pragmaticengineer", "platform": "youtube" }, { "id": "25077", "publication_id": 458709, "section_id": null, "url": "https://music.youtube.com/playlist?list=PLzwJJv8h-iciW53inSOkQA4mkG8TuQAUh", "platform": "youtube_music" } ], "spotify_podcast_settings": { "id": "21280", "publication_id": 458709, "section_id": null, "spotify_access_token": "bd775e35-7384-4fc4-bf47-ae24ae58d79b", "spotify_uri": "spotify:show:2wfSTS6OzM2iOQJWwbL6U0", "spotify_podcast_title": null, "created_at": "2024-09-14T18:44:10.144Z", "updated_at": "2024-09-17T17:59:03.311Z", "currently_published_on_spotify": false, "feed_url_for_spotify": "https://api.substack.com/feed/podcast/spotify/bd775e35-7384-4fc4-bf47-ae24ae58d79b/458709.rss", "spotify_show_url": "https://open.spotify.com/show/2wfSTS6OzM2iOQJWwbL6U0" }, "podcastPalette": { "Vibrant": { "rgb": [ 244, 67, 60 ], "population": 1469 }, "DarkVibrant": { "rgb": [ 125.51941747572819, 11.586407766990265, 7.0805825242718115 ], "population": 0 }, "LightVibrant": { "rgb": [ 244, 140, 132 ], "population": 33 }, "Muted": { "rgb": [ 158, 158, 158 ], "population": 82 }, "DarkMuted": { "rgb": [ 84, 84, 84 ], "population": 261 }, "LightMuted": { "rgb": [ 188, 188, 188 ], "population": 14 } }, "pageThemes": { "podcast": { "id": "13711", "publication_id": 458709, "section_id": null, "page": "podcast", "page_hero": "feature-media", "page_posts": "grid", "show_podcast_links": true, "hero_alignment": "left" } }, "live_subscriber_counts": false, "scrapedAt": "2025-02-10T06:17:32.349Z" }, ... ] ``` ### Related Actors - πŸ“š [Substack People Scraper](https://apify.com/easyapi/substack-people-scraper) - Extract comprehensive Substack author and publication data using keywords - πŸ“š [Substack Posts Scraper](https://apify.com/easyapi/substack-posts-scraper) - Scrape Substack posts and articles with comprehensive post data - πŸ“š [Substack Publications Scraper](https://apify.com/easyapi/substack-publications-scraper) - Get detailed publication information from Substack based on keywords - πŸ” [Substack Notes Scraper](https://apify.com/easyapi/substack-notes-scraper) - Extract notes and comments from Substack's search results - πŸ“Š [Medium User Search Scraper](https://apify.com/easyapi/medium-user-search-scraper) - Extract detailed Medium user profiles from search results - πŸ“Š [Medium Posts Search Scraper](https://apify.com/easyapi/medium-posts-search-scraper) - Extract comprehensive article data from Medium's search results - πŸ“Š [Medium Publications Search Scraper](https://apify.com/easyapi/medium-publications-search-scraper) - Scrape Medium publications by keywords - πŸ“Š [Patreon Analytics](https://apify.com/easyapi/patreon-analytic) - Extract comprehensive analytics for top Patreon creators - πŸ”₯ [Patreon Hot Creators Scraper](https://apify.com/easyapi/patreon-hot-creators-scraper) - Track trending Patreon creators with 300+ members - 🎯 [Patreon Creator Scraper](https://apify.com/easyapi/patreon-creator-scraper) - Scrape Patreon creator profiles across 30+ categories - πŸ“Š [Patreon Monthly Top Creators Scraper](https://apify.com/easyapi/patreon-monthly-top-creators-scraper) - Extract detailed rankings and earnings data for top Patreon creators - πŸ“± [Social Media Influencer Scraper](https://apify.com/easyapi/social-media-influencer-scraper) - Discover top influencers across TikTok, Instagram, and YouTube - πŸ” [Keyword Discovery Tool](https://apify.com/easyapi/keyword-discovery-tool) - Discover new keyword ideas and uncover valuable search insights - πŸ“„ [Article Content Extractor](https://apify.com/easyapi/article-content-extractor) - Extract clean article content and metadata from any web page - πŸ” [Keyword Density Checker](https://apify.com/easyapi/keyword-density-checker) - Analyze webpage content to calculate keyword density and frequency # Actor input Schema ## `leaderboardUrls` (type: `array`): Array of Substack leaderboard URLs to scrape ## `maxItems` (type: `integer`): Maximum number of items to scrape ## `proxyConfiguration` (type: `object`): Proxy settings for the scraper ## Actor input object example ```json { "leaderboardUrls": [ "https://substack.com/leaderboard/technology/paid" ], "maxItems": 50, "proxyConfiguration": { "useApifyProxy": true } } ``` # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = {}; // Run the Actor and wait for it to finish const run = await client.actor("easyapi/substack-leaderboard-scraper").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`πŸ’Ύ Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // πŸ“š Want to learn more πŸ“–? Go to β†’ https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = {} # Run the Actor and wait for it to finish run = client.actor("easyapi/substack-leaderboard-scraper").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("πŸ’Ύ Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # πŸ“š Want to learn more πŸ“–? Go to β†’ https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{}' | apify call easyapi/substack-leaderboard-scraper --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=easyapi/substack-leaderboard-scraper", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Substack Leaderboard Scraper πŸ“Š", "description": "Scrape detailed publication data from Substack leaderboards. Get comprehensive insights about top newsletters including subscriber counts, pricing, author details, and more. Perfect for newsletter research and market analysis.", "version": "0.0", "x-build-id": "cBtjSkQPBKehwwcTB" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/easyapi~substack-leaderboard-scraper/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-easyapi-substack-leaderboard-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/easyapi~substack-leaderboard-scraper/runs": { "post": { "operationId": "runs-sync-easyapi-substack-leaderboard-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/easyapi~substack-leaderboard-scraper/run-sync": { "post": { "operationId": "run-sync-easyapi-substack-leaderboard-scraper", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "required": [ "leaderboardUrls" ], "properties": { "leaderboardUrls": { "title": "Leaderboard URLs", "uniqueItems": true, "type": "array", "description": "Array of Substack leaderboard URLs to scrape", "default": [ "https://substack.com/leaderboard/technology/paid" ], "items": { "type": "string" } }, "maxItems": { "title": "Maximum Items", "minimum": 1, "type": "integer", "description": "Maximum number of items to scrape", "default": 50 }, "proxyConfiguration": { "title": "Proxy Configuration", "type": "object", "description": "Proxy settings for the scraper", "default": { "useApifyProxy": true } } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```