DEV Community

Aurelix Solutions
Aurelix Solutions

Posted on

A daily list of new building permits from 12 US cities, with the contractor named

A building permit is a public notice that work is about to start, often with the address, the value of the job and the contractor's company. That makes permits one of the best lead sources in construction: a roofing supplier wants every re-roof permit, a solar installer wants to know which ZIP codes are busy, a materials rep wants the contractors pulling the most permits this month. The catch is that every city publishes them differently, in its own portal with its own column names.

This post pulls permits from 12 US cities in one format, keeps only new or changed ones on a daily schedule, and ranks contractors by recent work.

Disclosure: I'm Aurelix, and I built the New York Building Permits Scraper, which despite the name covers all 12 cities below. It's a paid tool (prices below).

Where the data comes from

The cities' own open-data permit records: Seattle, Chicago, New York City (DOB NOW work permits), Los Angeles, San Francisco, Austin, Baton Rouge, Cincinnati, Mesa, New Orleans, Norfolk and Orlando. Every row has the same 30 fields:

  • permit_key (stable, for joins and dedupe), city, state, permit_number, permit_type, permit_subtype, description.
  • address, zip_code, neighborhood, latitude, longitude.
  • valuation (the value declared by the applicant), status, and applied_date, issued_date, expires_date, completed_date where the city publishes them.
  • contractor_company in nine of the cities, contractor_phone in Austin, Mesa and Orlando, contractor_email in Mesa, and contractor_name and contractor_address where published.
  • owner_name and applicant_name in five cities, and Chicago's full contacts list.

Names are returned as each city publishes them; Los Angeles records carry none, because its portal terms bar collecting personal information.

A daily contractor lead feed

pip install apify-client
Enter fullscreen mode Exit fullscreen mode
import csv
import os
from collections import Counter

from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])
run = client.actor("aurelix/building-permits-scraper").call(
    run_input={
        "cities": ["austin", "chicago", "mesa", "orlando"],
        "daysBack": 7,
        "contractorOnly": True,
        "minValuation": 50000,
        "maxResultsPerCity": 200,
        "incremental": True,
        "stateKey": "contractor-leads",
    }
)
permits = list(client.dataset(run["defaultDatasetId"]).iterate_items())

fields = ["city", "issued_date", "permit_type", "description", "address", "zip_code",
          "valuation", "status", "contractor_company", "contractor_phone", "url"]
with open("new_permits.csv", "w", newline="", encoding="utf-8") as f:
    w = csv.DictWriter(f, fieldnames=fields, extrasaction="ignore")
    w.writeheader()
    w.writerows(sorted(permits, key=lambda p: p["valuation"] or 0, reverse=True))

busy = Counter((p["contractor_company"], p["city"]) for p in permits)
value = Counter()
for p in permits:
    value[(p["contractor_company"], p["city"])] += p["valuation"] or 0
print(f"{len(permits)} new or changed permits")
for (company, city), n in busy.most_common(10):
    print(f"  {company} ({city}): {n} permits, ${value[(company, city)]:,.0f} declared")
Enter fullscreen mode Exit fullscreen mode

With incremental on, the first run returns everything that matches; each later run with the same stateKey returns only permits that are new or whose status changed, so a daily schedule pays only for fresh rows. To run it without a script, save the input as a task in the Apify Console and schedule it daily; a Slack webhook URL in slackWebhookUrl posts the count and the top 10 with links, and nothing when there is nothing new.

Other ways to cut the same data:

  • Solar, roofing, ADUs: "keywords": ["solar", "roof", "ADU"] matches the description of work or the address. Group by zip_code to see where the work is.
  • Large projects: "permitTypes": ["new"] with "minValuation": 1000000 across NYC, Chicago and Los Angeles, then follow status and completed_date on later runs.
  • Find the contractor's website and phone: search the contractor_company names with the Google Maps Phone Scraper.

Working in Canada? The Canada Building Permits tool does the same for Toronto and six other Canadian cities.

Why this tool

  • $1.50 per 1,000 permits ($1.05 on paid plans) plus $0.006 per run, cheaper than the leading tools.
  • All twelve cities in one run, in one schema.
  • Built for a daily feed: new-or-changed mode, a contractor-only filter, keyword and minimum value filters, and a Slack or webhook digest.
  • What it doesn't cover: cities outside the twelve above.

Limits

  • Twelve cities. Open an issue on the Store page to ask for another.
  • Publishing lag (as measured in our checks): New York City runs about 2 to 5 days behind, Los Angeles refreshes weekly and can be up to about 10 days behind; the other ten were 0 to 2 days behind. A look-back shorter than the lag returns nothing for that city, and the run status says so.
  • Contractor names are published by nine cities; Los Angeles, San Francisco and Norfolk publish none. Phones only in Austin, Mesa and Orlando.
  • Issued permits only, filtered by issue date; applications not yet issued are not included.
  • Valuation is what the applicant declared; some permits carry none.
  • Up to 90 days back and 50,000 rows per city per run; by default 1,000 per city, newest first.

Cost

You pay per permit returned. Cities that return nothing are free, and in new-or-changed mode, permits already returned are not charged again. The current price per permit is on the Store page; Apify's free plan includes $5 of credit a month, about 3,300 permits.

Links

Top comments (0)