# Weather Scraper (`epctex/weather-scraper`) Actor
Get data about weather forecasts or analyze conditions in any location in the world. This scraper downloads information about forecasts, humidity, temperature, wind, etc. and delivers it in HTML, JSON, XLSX, and more.
- **URL**: https://apify.com/epctex/weather-scraper.md
- **Developed by:** [epctex](https://apify.com/epctex) (community)
- **Categories:** Travel, News
- **Stats:** 340 total users, 4 monthly users, 100.0% runs succeeded, 15 bookmarks
- **User rating**: 5.00 out of 5 stars
## Pricing
$10.00/month + usage
To use this Actor, you pay a monthly rental fee to the developer. The rent is subtracted from your prepaid usage every month after the free trial period.You also pay for the Apify platform usage, which gets cheaper the higher Apify subscription plan you have.
Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#rental-actors
## What's an Apify Actor?
Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".
## How to integrate an Actor?
If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.
In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md):
```bash
npm install apify-client
```
In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md):
```bash
pip install apify-client
```
In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md):
````bash
# MacOS / Linux
curl -fsSL https://apify.com/install-cli.sh | bash
# Windows
irm https://apify.com/install-cli.ps1 | iex
```bash
In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md).
If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md).
For usage examples, see the [API](#api) section below.
For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).
# README
## Weather Scraper
Weather Scraper is an [Apify actor](https://apify.com/actors) for extracting weather information from [Weather.com](https://weather.com). It gives you access to various available weather data in structured form. It is build on top of [Apify SDK](https://sdk.apify.com/) and you can run it both on [Apify platform](https://my.apify.com) and locally.
- [Input](#input)
- [Output](#output)
- [Compute units consumption](#compute-units-consumption)
- [Extend output function](#extend-output-function)
### Bugs, fixes, updates, and changelog
This scraper is under active development. If you have any feature requests you can create an issue from [here](https://github.com/epctex-support/weather-scraper/issues).
### Input Parameters
| Field | Type | Description | Default value
| ----- | ---- | ----------- | -------------|
| startUrls | array | List of place urls to be processed | `[]` |
| units | string | Unit system to use for the results | `metric` |
| timeFrame | string | Future time frame you want to extract data for | `today` |
| maxItems | number | Maximum number of actor pages that will be scraped | all found |
| locations | array | List of cities / addresses to be processed | `[]` |
| locationIds | array | List of location ids to be processed | `[]` |
| extendOutputFunction | string | Function that takes a JQuery handle ($) as argument and returns data that will be merged with the default output. More information in [Extend output function](#extend-output-function) | |
| customMapFunction | string | Function that takes each of the output rows as argument and returns data that will be mapped, changed formatting, of the each rows. More information in [Custom map function](#custom-map-function) | |
| proxyConfiguration | object | Proxy settings of the run. If you have access to Apify proxy, leave the default settings. If not, you can set `{ "useApifyProxy": false" }` to disable proxy usage | `{ "useApifyProxy": true }`|
#### Input example
```json
{
"startUrls": [
{"url":"https://weather.com/weather/today/l/5aea1d50a6d6b9e99cf89ba79f463d67dcf21ea5061990aae1ffc1c7fa8911a9"},
{"url":"https://weather.com/weather/today/l/ada030b2de8db495da2d93d5a2ecf30de1ce8b54cb09725d19c803543685646d"}
],
"locationIds": [
"e60256f3426acd3c1b3380921210fcffeab628a120c41d1de03b59a0f0dd32ad",
"bf217d537cc1c8074ec195ce07fb74de3c1593caa6033b7c3be4645ccc5b01de"
],
"locations": [
"Montgomery, Alabama",
"Juneau, Alaska"
],
"timeFrame": "ten_day",
"units": "metric",
"maxItems": 10,
"proxyConfiguration": {
"useApifyProxy": true
},
"extendOutputFunction": "($) => {}",
"customMapFunction": "(object) => {return {...object, }}"
}
````
#### Determining locations
In advanced input section, you can provide `locations` parameter - list of addresses you want to scrape. The actual scraped place is the first result in search box on weather.com. In order to get relevant results (which is not always guaranteed in this case), try to input both city and country name (eg. `Paris, France`, `Vienna, Austria`).
If specific locations are needed, you can provide `locationIds` parameter. Location id is used by weather.com to identify specific location. It can be found in the url of a place, it is the last parameter in path. Few examples:
For New York, NY, which url is `https://weather.com/cs-CZ/weather/tenday/l/f892433d7660da170347398eb8e3d722d8d362fe7dd15af16ce88324e1b96e70` the location id is `f892433d7660da170347398eb8e3d722d8d362fe7dd15af16ce88324e1b96e70`. For London, England with url `https://weather.com/en-UK/weather/today/l/7517a52d4d1815e639ae1001edb8c5fda2264ea579095b0f28f55c059599e074` the location id is `7517a52d4d1815e639ae1001edb8c5fda2264ea579095b0f28f55c059599e074`.
### Compute Unit Consumption
Compute unit consumption was measured on a dataset of 50 US capitals.
When the search was done using `locations` parameter (given as `City, State`), the average compute units consumption was 0.02682.
For the same locations, but given in form of `startUrls`, the average consumption was 0.0152.
And finally, when using `locationIds`, the average consumption was 0.01916.
Keeping in mind that the dataset is small, the conclusion is to prefer `startUrls` or `locationIds` over `locations`, if you need to scrape larger amounts of locations. This is due to the fact that the search for location id is omitted in the first two cases.
### Output
Output is stored in a dataset. Each item is information about weather in a location. The items come in two forms - day/night values for daily items and current values for moment items. Example for day/night values:
```
{
"city": "Třeboň",
"state": "South Bohemia",
"country": "Czech Republic",
"zipCode": "379 01",
"time": "2020-08-15T07:00:00+0200",
"temperature": "24/16",
"forecast": "Thunderstorms/Scattered Thunderstorms",
"humidity": "79/88",
"windDirection": "W/WNW",
"windSpeed": "9/6"
}
```
Example for current values:
```
{
"city": "Třeboň",
"state": "South Bohemia",
"country": "Czech Republic",
"zipCode": "379 01",
"time": "2020-08-12T19:00:00+0200",
"temperature": 27,
"forecast": "Sunny",
"humidity": 45,
"windDirection": "E",
"windSpeed": 7
}
```
### Extend output function
You can use this function to update the default output of this actor. This function gets a JQuery handle `$` as an argument so you can choose what data from the page you want to scrape. The output from this will function will get merged with the default output.
The return value of this function has to be an object!
You can return fields to achive 3 different things:
- Add a new field - Return object with a field that is not in the default output
- Change a field - Return an existing field with a new value
- Remove a field - Return an existing field with a value `undefined`
### Custom map function
You can use this function to change the output of each of the rows that you will get from this actor. This function gets each of the rows on the output as an argument so you can change the formatting, pick which attributes you want to get as the output. The output from this function will be converted into the format you will return.
The return value of this function has to be an object!
You can return fields to achive 3 different things:
- Add a new field - Return object with a field that is not in the default output
- Change a field - Return an existing field with a new value
- Remove a field - Return an existing field with a value `undefined`
```
(object) => {
return {
title: object.title.replace(" - The Weather Channel | Weather.com", "") ,
windSpeed: 7 + "knots",
}
}
```
This example will change the output in the following way:
```
{
"title": "Třeboň, South Bohemia, Czech Republic 10-Day Weather Forecast"
"windSpeed": "7 knots"
}
```
Note that all the data are scraped from today page (eg. `https://weather.com/weather/today/l/81cbe8a06fd80171651aef7a414bce1e599aa05082d82f4e319f94b4b60602e0`).
### During the Run
During the run, the actor will output messages letting you know what is going on. Each message always contains a short label specifying which page from the provided list is currently specified.
When items are loaded from the page, you should see a message about this event with a loaded item count and total item count for each page.
If you provide incorrect input to the actor, it will immediately stop with a failure state and output an explanation of what is wrong.
### Weather Scraper Export
During the run, the actor stores results into a dataset. Each item is a separate item in the dataset.
You can manage the results in any language (Python, PHP, Node JS/NPM). See the FAQ or our API reference to learn more about getting results from this Weather Scraper actor.
### Contact
Please visit us through [epctex.com](https://epctex.com) to see all the products that are available for you. If you are looking for any custom integration or so, please reach out to us through the chat box in [epctex.com](https://epctex.com). In need of support? is at your service.
# Actor input Schema
## `startUrls` (type: `array`):
URLs to start with. It should be a Weather.com full URLs.
## `locations` (type: `array`):
Input must be in: City, State, Country or City, Country
## `locationIds` (type: `array`):
ID is in the last part of URL on weather.com
## `timeFrame` (type: `string`):
Select time frame
## `units` (type: `string`):
Select units
## `maxItems` (type: `integer`):
Limit the number of locations to be scraped
## `proxyConfiguration` (type: `object`):
Select proxies to be used by your actor.
## `extendOutputFunction` (type: `string`):
Function that takes a JQuery handle ($) as argument and returns data that will be merged with the default output
## `customMapFunction` (type: `string`):
Function that takes each of the objects as argument and returns data that will be mapped by the function itself.
## Actor input object example
```json
{
"startUrls": [
{
"url": "https://weather.com/weather/today/l/5aea1d50a6d6b9e99cf89ba79f463d67dcf21ea5061990aae1ffc1c7fa8911a9"
},
{
"url": "https://weather.com/weather/today/l/ada030b2de8db495da2d93d5a2ecf30de1ce8b54cb09725d19c803543685646d"
}
],
"locations": [
"Montgomery, Alabama",
"Juneau, Alaska"
],
"locationIds": [
"e60256f3426acd3c1b3380921210fcffeab628a120c41d1de03b59a0f0dd32ad",
"bf217d537cc1c8074ec195ce07fb74de3c1593caa6033b7c3be4645ccc5b01de"
],
"timeFrame": "today",
"units": "imperial",
"maxItems": 10,
"proxyConfiguration": {
"useApifyProxy": true
},
"extendOutputFunction": "($) => { return {} }",
"customMapFunction": "(object) => { return {...object} }"
}
```
# API
You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.
## JavaScript example
```javascript
import { ApifyClient } from 'apify-client';
// Initialize the ApifyClient with your Apify API token
// Replace the '' with your token
const client = new ApifyClient({
token: '',
});
// Prepare Actor input
const input = {
"startUrls": [
{
"url": "https://weather.com/weather/today/l/5aea1d50a6d6b9e99cf89ba79f463d67dcf21ea5061990aae1ffc1c7fa8911a9"
},
{
"url": "https://weather.com/weather/today/l/ada030b2de8db495da2d93d5a2ecf30de1ce8b54cb09725d19c803543685646d"
}
],
"locations": [
"Montgomery, Alabama",
"Juneau, Alaska"
],
"locationIds": [
"e60256f3426acd3c1b3380921210fcffeab628a120c41d1de03b59a0f0dd32ad",
"bf217d537cc1c8074ec195ce07fb74de3c1593caa6033b7c3be4645ccc5b01de"
],
"timeFrame": "today",
"units": "imperial",
"maxItems": 10,
"proxyConfiguration": {
"useApifyProxy": true
},
"extendOutputFunction": ($) => { return {} },
"customMapFunction": (object) => { return {...object} }
};
// Run the Actor and wait for it to finish
const run = await client.actor("epctex/weather-scraper").call(input);
// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
console.dir(item);
});
// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs
```
## Python example
```python
from apify_client import ApifyClient
# Initialize the ApifyClient with your Apify API token
# Replace '' with your token.
client = ApifyClient("")
# Prepare the Actor input
run_input = {
"startUrls": [
{ "url": "https://weather.com/weather/today/l/5aea1d50a6d6b9e99cf89ba79f463d67dcf21ea5061990aae1ffc1c7fa8911a9" },
{ "url": "https://weather.com/weather/today/l/ada030b2de8db495da2d93d5a2ecf30de1ce8b54cb09725d19c803543685646d" },
],
"locations": [
"Montgomery, Alabama",
"Juneau, Alaska",
],
"locationIds": [
"e60256f3426acd3c1b3380921210fcffeab628a120c41d1de03b59a0f0dd32ad",
"bf217d537cc1c8074ec195ce07fb74de3c1593caa6033b7c3be4645ccc5b01de",
],
"timeFrame": "today",
"units": "imperial",
"maxItems": 10,
"proxyConfiguration": { "useApifyProxy": True },
"extendOutputFunction": "($) => { return {} }",
"customMapFunction": "(object) => { return {...object} }",
}
# Run the Actor and wait for it to finish
run = client.actor("epctex/weather-scraper").call(run_input=run_input)
# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
print(item)
# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start
```
## CLI example
```bash
echo '{
"startUrls": [
{
"url": "https://weather.com/weather/today/l/5aea1d50a6d6b9e99cf89ba79f463d67dcf21ea5061990aae1ffc1c7fa8911a9"
},
{
"url": "https://weather.com/weather/today/l/ada030b2de8db495da2d93d5a2ecf30de1ce8b54cb09725d19c803543685646d"
}
],
"locations": [
"Montgomery, Alabama",
"Juneau, Alaska"
],
"locationIds": [
"e60256f3426acd3c1b3380921210fcffeab628a120c41d1de03b59a0f0dd32ad",
"bf217d537cc1c8074ec195ce07fb74de3c1593caa6033b7c3be4645ccc5b01de"
],
"timeFrame": "today",
"units": "imperial",
"maxItems": 10,
"proxyConfiguration": {
"useApifyProxy": true
},
"extendOutputFunction": "($) => { return {} }",
"customMapFunction": "(object) => { return {...object} }"
}' |
apify call epctex/weather-scraper --silent --output-dataset
```
## MCP server setup
```json
{
"mcpServers": {
"apify": {
"command": "npx",
"args": [
"mcp-remote",
"https://mcp.apify.com/?tools=epctex/weather-scraper",
"--header",
"Authorization: Bearer "
]
}
}
}
```
## OpenAPI specification
```json
{
"openapi": "3.0.1",
"info": {
"title": "Weather Scraper",
"description": "Get data about weather forecasts or analyze conditions in any location in the world. This scraper downloads information about forecasts, humidity, temperature, wind, etc. and delivers it in HTML, JSON, XLSX, and more.",
"version": "0.0",
"x-build-id": "8RN8DKkaTTa5gcCTq"
},
"servers": [
{
"url": "https://api.apify.com/v2"
}
],
"paths": {
"/acts/epctex~weather-scraper/run-sync-get-dataset-items": {
"post": {
"operationId": "run-sync-get-dataset-items-epctex-weather-scraper",
"x-openai-isConsequential": false,
"summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.",
"tags": [
"Run Actor"
],
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/inputSchema"
}
}
}
},
"parameters": [
{
"name": "token",
"in": "query",
"required": true,
"schema": {
"type": "string"
},
"description": "Enter your Apify token here"
}
],
"responses": {
"200": {
"description": "OK"
}
}
}
},
"/acts/epctex~weather-scraper/runs": {
"post": {
"operationId": "runs-sync-epctex-weather-scraper",
"x-openai-isConsequential": false,
"summary": "Executes an Actor and returns information about the initiated run in response.",
"tags": [
"Run Actor"
],
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/inputSchema"
}
}
}
},
"parameters": [
{
"name": "token",
"in": "query",
"required": true,
"schema": {
"type": "string"
},
"description": "Enter your Apify token here"
}
],
"responses": {
"200": {
"description": "OK",
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/runsResponseSchema"
}
}
}
}
}
}
},
"/acts/epctex~weather-scraper/run-sync": {
"post": {
"operationId": "run-sync-epctex-weather-scraper",
"x-openai-isConsequential": false,
"summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.",
"tags": [
"Run Actor"
],
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/inputSchema"
}
}
}
},
"parameters": [
{
"name": "token",
"in": "query",
"required": true,
"schema": {
"type": "string"
},
"description": "Enter your Apify token here"
}
],
"responses": {
"200": {
"description": "OK"
}
}
}
}
},
"components": {
"schemas": {
"inputSchema": {
"type": "object",
"required": [
"proxyConfiguration",
"timeFrame",
"units"
],
"properties": {
"startUrls": {
"title": "Start urls",
"type": "array",
"description": "URLs to start with. It should be a Weather.com full URLs.",
"items": {
"type": "object",
"required": [
"url"
],
"properties": {
"url": {
"type": "string",
"title": "URL of a web page",
"format": "uri"
}
}
}
},
"locations": {
"title": "Search Locations by name",
"type": "array",
"description": "Input must be in: City, State, Country or City, Country",
"items": {
"type": "string"
}
},
"locationIds": {
"title": "Search Locations by location id",
"type": "array",
"description": "ID is in the last part of URL on weather.com",
"items": {
"type": "string"
}
},
"timeFrame": {
"title": "Time Frame for the weather forecast.",
"enum": [
"today",
"hour_by_hour",
"ten_day",
"weekend",
"month"
],
"type": "string",
"description": "Select time frame",
"default": "today"
},
"units": {
"title": "Units",
"enum": [
"imperial",
"metric"
],
"type": "string",
"description": "Select units",
"default": "imperial"
},
"maxItems": {
"title": "Maximum number of location items",
"type": "integer",
"description": "Limit the number of locations to be scraped",
"default": 10
},
"proxyConfiguration": {
"title": "Proxy configuration",
"type": "object",
"description": "Select proxies to be used by your actor."
},
"extendOutputFunction": {
"title": "Extend output function",
"type": "string",
"description": "Function that takes a JQuery handle ($) as argument and returns data that will be merged with the default output"
},
"customMapFunction": {
"title": "Custom map function",
"type": "string",
"description": "Function that takes each of the objects as argument and returns data that will be mapped by the function itself."
}
}
},
"runsResponseSchema": {
"type": "object",
"properties": {
"data": {
"type": "object",
"properties": {
"id": {
"type": "string"
},
"actId": {
"type": "string"
},
"userId": {
"type": "string"
},
"startedAt": {
"type": "string",
"format": "date-time",
"example": "2025-01-08T00:00:00.000Z"
},
"finishedAt": {
"type": "string",
"format": "date-time",
"example": "2025-01-08T00:00:00.000Z"
},
"status": {
"type": "string",
"example": "READY"
},
"meta": {
"type": "object",
"properties": {
"origin": {
"type": "string",
"example": "API"
},
"userAgent": {
"type": "string"
}
}
},
"stats": {
"type": "object",
"properties": {
"inputBodyLen": {
"type": "integer",
"example": 2000
},
"rebootCount": {
"type": "integer",
"example": 0
},
"restartCount": {
"type": "integer",
"example": 0
},
"resurrectCount": {
"type": "integer",
"example": 0
},
"computeUnits": {
"type": "integer",
"example": 0
}
}
},
"options": {
"type": "object",
"properties": {
"build": {
"type": "string",
"example": "latest"
},
"timeoutSecs": {
"type": "integer",
"example": 300
},
"memoryMbytes": {
"type": "integer",
"example": 1024
},
"diskMbytes": {
"type": "integer",
"example": 2048
}
}
},
"buildId": {
"type": "string"
},
"defaultKeyValueStoreId": {
"type": "string"
},
"defaultDatasetId": {
"type": "string"
},
"defaultRequestQueueId": {
"type": "string"
},
"buildNumber": {
"type": "string",
"example": "1.0.0"
},
"containerUrl": {
"type": "string"
},
"usage": {
"type": "object",
"properties": {
"ACTOR_COMPUTE_UNITS": {
"type": "integer",
"example": 0
},
"DATASET_READS": {
"type": "integer",
"example": 0
},
"DATASET_WRITES": {
"type": "integer",
"example": 0
},
"KEY_VALUE_STORE_READS": {
"type": "integer",
"example": 0
},
"KEY_VALUE_STORE_WRITES": {
"type": "integer",
"example": 1
},
"KEY_VALUE_STORE_LISTS": {
"type": "integer",
"example": 0
},
"REQUEST_QUEUE_READS": {
"type": "integer",
"example": 0
},
"REQUEST_QUEUE_WRITES": {
"type": "integer",
"example": 0
},
"DATA_TRANSFER_INTERNAL_GBYTES": {
"type": "integer",
"example": 0
},
"DATA_TRANSFER_EXTERNAL_GBYTES": {
"type": "integer",
"example": 0
},
"PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
"type": "integer",
"example": 0
},
"PROXY_SERPS": {
"type": "integer",
"example": 0
}
}
},
"usageTotalUsd": {
"type": "number",
"example": 0.00005
},
"usageUsd": {
"type": "object",
"properties": {
"ACTOR_COMPUTE_UNITS": {
"type": "integer",
"example": 0
},
"DATASET_READS": {
"type": "integer",
"example": 0
},
"DATASET_WRITES": {
"type": "integer",
"example": 0
},
"KEY_VALUE_STORE_READS": {
"type": "integer",
"example": 0
},
"KEY_VALUE_STORE_WRITES": {
"type": "number",
"example": 0.00005
},
"KEY_VALUE_STORE_LISTS": {
"type": "integer",
"example": 0
},
"REQUEST_QUEUE_READS": {
"type": "integer",
"example": 0
},
"REQUEST_QUEUE_WRITES": {
"type": "integer",
"example": 0
},
"DATA_TRANSFER_INTERNAL_GBYTES": {
"type": "integer",
"example": 0
},
"DATA_TRANSFER_EXTERNAL_GBYTES": {
"type": "integer",
"example": 0
},
"PROXY_RESIDENTIAL_TRANSFER_GBYTES": {
"type": "integer",
"example": 0
},
"PROXY_SERPS": {
"type": "integer",
"example": 0
}
}
}
}
}
}
}
}
}
}
```