DEV Community

novosai0day-a11y
novosai0day-a11y

Posted on

A Google News alert with the real article URLs (not news.google.com redirects)

Google News RSS is the easiest way to monitor a brand, a competitor or a topic in any country — until you notice that every link is a news.google.com/rss/articles/CBMi… redirect. You can't dedupe by publisher URL, you can't fetch the article, and your Slack alerts look like spam.

Google News Scraper with Original Article URLs (disclosure: I built it) resolves those links to the publisher's real URL and returns headline, publisher, date and snippet for any query, language and country.

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")
run = client.actor("fujiwaranosai/google-news-scraper").call(run_input={
    "queries": ['"Anthropic" OR "OpenAI"', "Tesla -recall"],
    "timeRange": "1d",
    "language": "en",
    "country": "us",
    "onlyNew": True,
})
for a in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(a["published_at"], a["publisher"], a["title"], a["article_url"])
Enter fullscreen mode Exit fullscreen mode

Ideas

  • PR monitoring: one query per brand/executive, schedule hourly with timeRange: "1h" and onlyNew.
  • Market research: track tickers or sectors per country edition (es/mx, pt/br, de/de).
  • AI digests: pass article_url to your content crawler and summarize the day's news with an LLM.

Google News operators work too: "exact phrase", OR, -exclude, site:reuters.com. The first 100 articles of every account are free.

More ready-to-run examples: https://github.com/novosai0day-a11y/public-data-apis

Top comments (0)