# Themuse Job Scraper (`shahidirfan/themuse-job-scraper`) Actor

Meet TheMuse Job Scraper! This lightweight Apify actor efficiently extracts job listings directly from TheMuse. Get the latest job data, including titles, companies, and locations, quickly and easily. For best results and reliable scraping, using residential proxies is highly recommended.

- **URL**: https://apify.com/shahidirfan/themuse-job-scraper.md
- **Developed by:** [Shahid Irfan](https://apify.com/shahidirfan) (community)
- **Categories:** Jobs, Developer tools, Other
- **Stats:** 10 total users, 3 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: 5.00 out of 5 stars

## Pricing

Pay per usage

This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-usage

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## The Muse Jobs Scraper

Extract comprehensive job listings from TheMuse.com into clean, structured data for recruiting, market research, and career trend analysis. This scraper delivers complete job postings with company details, descriptions, categories, and location data at scale. Perfect for HR teams, job boards, analysts, and researchers tracking career opportunities.

### Features

- **Complete Job Data** - Collect titles, companies, descriptions, categories, locations, levels, and more from every listing
- **Flexible Search** - Filter by keyword, location, category, date posted, or provide direct URLs
- **Rich Structured Fields** - Includes normalized values plus source objects for companies, locations, categories, levels, refs, and pagination metadata
- **Configurable Limits** - Set maximum results and pages to control run time
- **Deduplication** - Built-in duplicate detection prevents repeated records
- **Export-Ready Output** - Clean, normalized JSON with consistent field names

### Use Cases

#### Recruiting and Talent Sourcing

Collect job listings from The Muse to identify companies hiring for specific roles. Build targeted outreach lists based on job types, levels, and locations.

#### Market Research

Track hiring trends across industries, locations, and company sizes. Analyze which skills and roles are in demand, what compensation ranges appear, and how job requirements evolve over time.

#### Competitive Intelligence

Monitor job postings from specific companies or industries. Understand hiring patterns, team growth, and strategic priorities based on open positions.

#### Data Analysis

Build comprehensive datasets for labor market analysis, salary research, and career trend forecasting. Export clean data for BI tools, spreadsheets, or custom pipelines.

#### Career Research

Collect job listings for personal career exploration. Compare roles, companies, and requirements across your target field in one structured dataset.

***

### Input Parameters

| Parameter | Type | Required | Default | Description |
|-----------|------|----------|---------|-------------|
| `startUrls` | Array | No | `[]` | List of TheMuse search URLs or public API URLs |
| `keyword` | String | No | - | Job search keyword (e.g., "software engineer") |
| `location` | String | No | - | Location filter (city, region, or Remote) |
| `datePosted` | String | No | `""` | Date filter: `last_7d`, `last_30d`, `last_month` |
| `maxItems` | Integer | No | `20` | Maximum jobs to collect (0 = unlimited) |
| `maxPages` | Integer | No | `0` | Maximum pages to paginate (0 = unlimited) |
| `proxyConfiguration` | Object | No | - | Apify Proxy settings |

***

### Output Data

Each dataset item is a single job record with these fields:

| Field | Type | Description |
|-------|------|-------------|
| `api_url` | String | Direct API URL for the job |
| `api_model_type` | String | Source model type |
| `api_page` | Number | Search results page where the job was found |
| `api_page_count` | Number | Total pages reported for the search |
| `api_total_results` | Number | Total matching jobs reported for the search |
| `categories` | Array | Job category names |
| `category_count` | Number | Number of categories on the job |
| `category_objects` | Array | Original category objects from the source |
| `company` | String | Company name |
| `company_data` | Object | Original company object, including ID, name, and slug |
| `company_id` | Number | Company identifier |
| `company_short_name` | String | Company URL slug |
| `date_posted` | String | Publication date (ISO 8601) |
| `description_html` | String | Full job description in HTML |
| `description_length` | Number | Character length of the plain text description |
| `description_text` | String | Plain text job description |
| `job_category` | String | Primary job category |
| `job_id` | Number | Unique job identifier |
| `job_type` | String | Employment type (Full time, Part time, etc.) |
| `landing_page` | String | Job page URL on The Muse |
| `level_objects` | Array | Original career level objects |
| `level_short_names` | Array | Short career level labels |
| `levels` | Array | Career levels (e.g., Senior Level) |
| `location` | String | Primary location |
| `location_count` | Number | Number of locations on the job |
| `location_objects` | Array | Original location objects |
| `locations` | Array | All locations for this job |
| `publication_date` | String | ISO 8601 publication date |
| `refs` | Object | Source reference links |
| `search_timed_out` | Boolean | Whether the search response timed out |
| `search_took_ms` | Number | Search response time reported by the source |
| `slug` | String | URL-friendly job name |
| `source` | String | Data source (`api`) |
| `tag_objects` | Array | Original tag objects |
| `tags` | Array | Job tags |
| `title` | String | Job title |
| `url` | String | Job page URL |

***

### Usage Examples

#### Keyword and Location Search

Search for software engineering jobs in a specific city:

```json
{
    "keyword": "software engineer",
    "location": "New York City",
    "maxItems": 50
}
```

#### Filter by Date Posted

Get the most recent listings from the past week:

```json
{
    "keyword": "marketing",
    "location": "Remote",
    "datePosted": "last_7d",
    "maxItems": 100
}
```

#### Direct API Job URL

Collect data from a specific job posting:

```json
{
    "startUrls": ["https://www.themuse.com/api/public/jobs/20020349"]
}
```

#### Multiple Search URLs

Run several searches in one go:

```json
{
    "startUrls": [
        "https://www.themuse.com/search/keyword/data_science",
        "https://www.themuse.com/search/location/remote"
    ],
    "maxItems": 200,
    "maxPages": 5
}
```

***

### Sample Output

```json
{
    "job_id": 20020349,
    "title": "Operations Manager",
    "company": "CVS Health",
    "company_id": 12096,
    "company_short_name": "cvshealth",
    "company_data": {
        "id": 12096,
        "short_name": "cvshealth",
        "name": "CVS Health"
    },
    "location": "Greenville, SC",
    "locations": ["Greenville, SC"],
    "location_count": 1,
    "job_category": "Retail",
    "categories": ["Retail"],
    "category_count": 1,
    "job_type": "Full time",
    "levels": ["Senior Level"],
    "level_short_names": ["senior"],
    "tags": [],
    "publication_date": "2025-09-08T19:37:04Z",
    "date_posted": "2025-09-08T19:37:04Z",
    "url": "https://www.themuse.com/jobs/cvshealth/operations-manager-b547da",
    "landing_page": "https://www.themuse.com/jobs/cvshealth/operations-manager-b547da",
    "slug": "operations-manager-b547da",
    "description_html": "<p>Full job description...</p>",
    "description_text": "Full job description...",
    "description_length": 2476,
    "api_page": 1,
    "api_page_count": 5,
    "api_total_results": 100
}
```

***

### Tips for Best Results

#### Choose Effective Keywords

- Use specific job titles like "software engineer" or "product manager"
- Try broader terms like "marketing" or "data" for wider results
- Combine with location for targeted searches

#### Optimize Collection Size

- Start with 20-50 results for testing
- Increase `maxItems` for production runs
- Use `maxPages` to cap API pagination during larger runs

#### Use Date Filters

- Apply `last_7d` for recent listings
- Use `last_30d` for broader time windows
- Leave empty to collect all available jobs

#### Proxy Configuration

For large-scale collection, residential proxies improve reliability:

```json
{
    "proxyConfiguration": {
        "useApifyProxy": true,
        "apifyProxyGroups": ["RESIDENTIAL"]
    }
}
```

***

### Integrations

Connect your data with:

- **Google Sheets** - Export for analysis and sharing
- **Airtable** - Build searchable job databases
- **Slack** - Get notifications for new listings
- **Webhooks** - Send to custom endpoints
- **Make** - Create automated workflows
- **Zapier** - Trigger actions based on new data

#### Export Formats

- **JSON** - For developers and APIs
- **CSV** - For spreadsheet analysis
- **Excel** - For business reporting
- **XML** - For system integrations

***

### Frequently Asked Questions

#### How many jobs can I collect?

You can collect all available jobs matching your filters. The practical limit depends on the total results on The Muse. Use `maxItems` to cap the dataset size.

#### Can I scrape job details?

Yes. The actor automatically fetches each job's full description and metadata from the detail endpoint.

#### What if data is missing?

Some fields may be empty if the public API doesn't provide that information. The actor fetches full descriptions from each API detail endpoint by default.

#### Does this actor use proxies?

Proxy usage is optional. For large-scale collection, residential proxies are recommended for better reliability.

#### How fast is the collection?

The actor uses optimized HTTP requests with connection pooling for fast data retrieval. A typical run of 20 results completes in under a minute.

#### Can I search by category?

Yes. Use the `keyword` field with category names, or provide a direct search URL via `startUrls`.

***

### Support

For issues or feature requests, contact support through the Apify Console.

#### Resources

- [Apify Documentation](https://docs.apify.com/)
- [API Reference](https://docs.apify.com/api/v2)
- [Scheduling Runs](https://docs.apify.com/schedules)

***

### Legal Notice

This actor is designed for legitimate data collection purposes. Users are responsible for ensuring compliance with The Muse Terms of Service and applicable laws. Use data responsibly and respect rate limits.

# Actor input Schema

## `startUrls` (type: `array`):

Optional list of TheMuse search URLs or public API URLs. API detail URLs like https://www.themuse.com/api/public/jobs/20020349 are supported.

## `keyword` (type: `string`):

Job search keyword (e.g., 'software engineer', 'marketing'). Used to construct API search parameters when startUrl is not provided.

## `location` (type: `string`):

Location filter for constructing search URLs (e.g., 'New York', 'Remote'). Leave empty for all locations.

## `datePosted` (type: `string`):

Date filter for TheMuse URLs. Examples: 'last\_7d', 'last\_30d', 'last\_month'. Leave empty for all dates.

## `maxItems` (type: `integer`):

Maximum number of jobs to collect. Set to 0 for unlimited (scrape until no more results).

## `maxPages` (type: `integer`):

Safety cap on pagination. Set to 0 to paginate until empty or maxItems is reached.

## `proxyConfiguration` (type: `object`):

Apify Proxy configuration. Residential proxies recommended for high-volume or stealth mode.

## Actor input object example

```json
{
  "startUrls": [],
  "keyword": "software engineer",
  "datePosted": "",
  "maxItems": 20,
  "maxPages": 0,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

# Actor output Schema

## `overview` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "keyword": "software engineer",
    "maxItems": 20
};

// Run the Actor and wait for it to finish
const run = await client.actor("shahidirfan/themuse-job-scraper").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "keyword": "software engineer",
    "maxItems": 20,
}

# Run the Actor and wait for it to finish
run = client.actor("shahidirfan/themuse-job-scraper").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "keyword": "software engineer",
  "maxItems": 20
}' |
apify call shahidirfan/themuse-job-scraper --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com/?tools=shahidirfan/themuse-job-scraper",
                "--header",
                "Authorization: Bearer <YOUR_API_TOKEN>"
            ]
        }
    }
}

```

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/98A9pVzRsVHYUgUYl/builds/0yiKHYOIPOdWk2Hts/openapi.json