# UK Planning Applications Lead Finder (`dazrave/uk-planning-leads`) Actor

Find recent UK planning applications that look useful for trades, kitchens, extensions and renovation lead generation. Returns scored lead records with reasons from verified council portals.

- **URL**: https://apify.com/dazrave/uk-planning-leads.md
- **Developed by:** [Darren Ravenscroft](https://apify.com/dazrave) (community)
- **Categories:** Lead generation, Business, Marketing
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $20.00 / 1,000 planning application leads

This Actor is paid per event and usage. You are charged both the fixed price for specific events and for Apify platform usage.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## UK Planning Applications Lead Finder (Apify actor)

Finds **public UK planning applications** that look useful for trades,
kitchens, extensions and renovation lead generation.

The actor searches supported council planning portals, normalises each result,
then adds a relevance score and plain-English reasons such as `extension`,
`single-storey extension`, `kitchen`, `renovation` or `loft conversion`.

Use it to build a weekly prospecting list for kitchen fitters, builders,
architectural services, glazing, roofing, landscaping and other home-improvement
businesses.

### What you get

Each dataset row includes:

`leadId, councilId, councilName, sourceUrl, applicationRef, address, proposal,
status, submittedDate, decisionDate, applicantName, relevanceScore,
relevanceReasons, matchedKeywords, category, scrapedAt`

Typical result:

- council: Bury Council
- proposal: `Single storey rear extension`
- category: `extension`
- relevance score: `6`
- reasons: matched `extension` and `single-storey extension`

### Current coverage

Default input uses `councilSelection: ["north-west-pack"]`, focused on Daz's
North West lead-gen area. Verified active councils currently include:

- Bury
- Trafford
- Stockport
- Wigan
- Knowsley
- St Helens
- Cheshire West and Chester
- Lancaster
- Blackpool

`midlands-pack` is also available for Warwick, Solihull, Wolverhampton and
Bromsgrove/Redditch.

### Supported portal styles

| `portalType` | What it is | Search | Parsing |
|---|---|---|---|
| `idox` | Idox PublicAccess (`/online-applications/`, `.do` endpoints) — the most common UK portal | GET `advancedSearchResults.do` with keyword + validated date range | result cards (`li.searchresult`), pagination via the `next` pager link |
| `searchresults-aspx` | Legacy ASP.NET portals with `SearchResults.aspx`-style result tables | GET with configurable query param names (`aspxParams` per council) | result tables, columns matched by header text |

### Output schema (dataset records)

`leadId, councilId, councilName, sourceUrl, applicationRef, address, proposal, status,
submittedDate, decisionDate, applicantName, relevanceScore, relevanceReasons,
matchedKeywords, category, scrapedAt`

- Dates are normalized to `YYYY-MM-DD`.
- `applicantName` is only filled where the portal publishes it on the results
  page (Idox shows it only on detail pages, so it is `null` for Idox in this MVP).
- `category` is one of `kitchen, extension, loft, conversion, renovation, new-build, other`.
- `relevanceScore` combines your input keywords (+2 each) with built-in trade
  signals (kitchen +5, extensions +3–4, refurb/renovation +3, …) and negative
  signals (tree works, adverts, telecoms −5) so junk can be filtered with
  `minRelevanceScore`. `relevanceReasons` explains every point.

### Running on Apify

Use the default North West preset for a quick run, or select specific council IDs
in `councilSelection`. Keep `sendKeywordToPortal` off unless you deliberately
want a narrow exact search; the default date-range search catches more relevant
applications and filters locally.

Recommended starter input:

```json
{
  "councilSelection": ["north-west-pack"],
  "keywords": ["kitchen", "extension", "refurbishment", "alterations"],
  "minRelevanceScore": 1,
  "maxPagesPerCouncil": 2,
  "maxConcurrency": 1,
  "requestDelaySecs": 5,
  "sendKeywordToPortal": false
}
```

Legacy inputs such as `councils: ["bury"]` and `councilPreset:
"north-west-pack"` are also accepted to avoid brittle buyer-facing failures.

### Local run

```bash
npm install
npm test              # offline — fixtures only, no network
npm run check:councils # live compatibility matrix, writes AVAILABILITY.md
apify run             # local Apify-style run using src/sampleInput.json
npm start             # same actor entrypoint without the CLI wrapper
npm run smoke         # optional: one polite live request to the first active sample council
```

Results land in `./storage/datasets/default/*.json`. To run with your own
input locally, create `./storage/key_value_stores/default/INPUT.json` matching
`.actor/input_schema.json`, or edit `src/sampleInput.json`.

### Politeness / rate limiting

Defaults are deliberately conservative and configurable via input:
`maxConcurrency: 1`, `requestDelaySecs: 5` between requests to the same
council, `maxPagesPerCouncil: 3`, 1 retry, and an identifying user agent.
Please keep them low — council portals are small public services.

### Publishing / monetisation notes

The actor is designed to run as a small paid utility on Apify. Good first-public
positioning:

- title: `UK Planning Applications Lead Finder`
- audience: trades, kitchen firms, builders, renovators and local lead-gen teams
- promise: `Find recent planning applications likely to indicate upcoming home-improvement work`
- pricing: start with Apify usage-based pricing while coverage is still growing
- caveat: it is not a marketing-compliance tool; users remain responsible for lawful basis, council terms, and opt-outs

### Legal & privacy caution

- Planning registers are public, but **check each council's terms of use and
  robots.txt** before scraping, and keep request rates minimal.
- Records contain **personal data** (addresses, sometimes applicant names).
  Under UK GDPR you need a lawful basis (likely legitimate interests) to
  process them for marketing; run and document an LIA before contacting anyone.
- Direct mail to addresses from the register is generally lower-risk than
  calls/emails; screen against mail preference services and honour opt-outs.
- Do not resell raw register data; some councils assert database rights.
- Applicant/agent details published on portals must not be used contrary to
  the council's stated purpose notices.

### Pilot councils & next ones to add

Sample input uses `councilSelection: ["north-west-pack"]`, which expands to every active North West council in `src/councils.js`. Current verified active North West councils are Bury, Trafford, Stockport, Wigan, Knowsley, St Helens, Cheshire West and Chester, Lancaster, and Blackpool; `midlands-pack` remains available for Warwick/Solihull/Wolverhampton/Bromsgrove-Redditch. `npm run check:councils` writes the latest live matrix to `AVAILABILITY.md`. Council portal URLs change often — verify the base URL loads before relying on it.

Good next candidates (all Idox PublicAccess at the time of writing):

- **South Kesteven / Boston / East Lindsey** (Lincolnshire cluster)
- **Maidstone, Medway, Canterbury** (Kent)
- **Trafford, Stockport, Bolton** (Greater Manchester)
- Councils on Northgate/Planning Explorer or custom portals need either a new
  `portalType` parser or per-council `aspxParams` mapping — the header-matching
  table parser (`parseAspxSearchResults`) is the starting point.

To add a council: append `{ name, portalType, baseUrl }` to the input (or
`src/sampleInput.json`), run `npm run smoke`-style single fetch to confirm the
markup parses, and add a fixture + test if the markup differs.

### Project layout

```
.actor/            actor.json + input_schema.json (Apify metadata)
src/main.js        entrypoint: input → crawl → score → dataset
src/scraper.js     URL builders + HTML parsers (Idox, ASPX table)
src/leadScoring.js keyword/relevance scoring + categorisation
src/sampleInput.json  fallback input for local dev
test/              offline tests + HTML fixtures (node --test)
scripts/smoke.js   optional single live fetch, non-fatal offline
```

# Actor input Schema

## `councilSelection` (type: `array`):

Use north-west-pack for Daz's current focus area, midlands-pack for the earlier Midlands set, or choose individual council IDs. Active North West IDs currently include bury, trafford, stockport, wigan, knowsley, st-helens, cheshire-west, lancaster, blackpool.

## `customCouncils` (type: `array`):

Optional custom council objects: { id, name, portalType: 'idox' | 'searchresults-aspx', baseUrl }. Use for testing extra portals before adding them to the registry.

## `keywords` (type: `array`):

Keywords used for local relevance scoring. By default the actor searches by date range then scores locally, which catches more applications than exact-phrase portal search.

## `dateFrom` (type: `string`):

Start of the validated/received date range. Defaults to 14 days ago.

## `dateTo` (type: `string`):

End of the validated/received date range. Defaults to today.

## `minRelevanceScore` (type: `integer`):

Leads scoring below this are dropped. 0 keeps everything.

## `maxPagesPerCouncil` (type: `integer`):

Politeness cap on paginated search results per council.

## `maxConcurrency` (type: `integer`):

Keep low to be polite to council servers.

## `requestDelaySecs` (type: `integer`):

Polite delay between requests sent to the same council portal.

## `sendKeywordToPortal` (type: `boolean`):

When false, searches by date range then scores/filter locally. This avoids exact-phrase portal searches missing relevant applications.

## Actor input object example

```json
{
  "councilSelection": [
    "north-west-pack"
  ],
  "customCouncils": [],
  "keywords": [
    "kitchen",
    "extension",
    "refurbishment",
    "alterations"
  ],
  "minRelevanceScore": 1,
  "maxPagesPerCouncil": 3,
  "maxConcurrency": 1,
  "requestDelaySecs": 5,
  "sendKeywordToPortal": false
}
```

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "councilSelection": [
        "north-west-pack"
    ],
    "customCouncils": [],
    "keywords": [
        "kitchen",
        "extension",
        "refurbishment",
        "alterations"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("dazrave/uk-planning-leads").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "councilSelection": ["north-west-pack"],
    "customCouncils": [],
    "keywords": [
        "kitchen",
        "extension",
        "refurbishment",
        "alterations",
    ],
}

# Run the Actor and wait for it to finish
run = client.actor("dazrave/uk-planning-leads").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "councilSelection": [
    "north-west-pack"
  ],
  "customCouncils": [],
  "keywords": [
    "kitchen",
    "extension",
    "refurbishment",
    "alterations"
  ]
}' |
apify call dazrave/uk-planning-leads --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com/?tools=dazrave/uk-planning-leads",
                "--header",
                "Authorization: Bearer <YOUR_API_TOKEN>"
            ]
        }
    }
}

```

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/wkaiamEGJGdl4u04H/builds/hTLJlfO01rxaTca4o/openapi.json