Showing posts with label Web Scraping. Show all posts
Showing posts with label Web Scraping. Show all posts

Friday, 17 April 2026

n8n and Apify Tutorial: How to Automate Web Scraping Workflows

n8n + Apify Tutorial

How to Use n8n with Apify for Web Scraping and Automation

Learn how to connect n8n and Apify to run Actors, scrape websites, collect structured data, and send results to apps like Google Sheets, Telegram, and CRMs.

If you want to automate web scraping and data extraction, n8n + Apify is a powerful combination. Apify handles the scraping using Actors, while n8n lets you route the output into your own workflow.

What you can build: website scrapers, lead collectors, competitor monitoring, Google Maps data extraction, content pipelines, and alert systems.

What You’ll Learn

  • What Apify does
  • How to connect Apify to n8n
  • How to run an Actor from n8n
  • How to fetch scraped results
  • How to build a simple real workflow
  • How to use HTTP Request instead of a dedicated node

What Is Apify?

Apify is a platform for running automation and web scraping tools called Actors. These Actors can extract website data, browse pages, collect structured results, and save outputs into datasets or key-value stores.

In simple terms, Apify does the scraping work, and n8n handles the automation around it.

💡 Simple explanation: Apify gets the data. n8n moves the data where you need it.

Why Use n8n with Apify?

This combo is useful because you can let Apify scrape websites and let n8n handle the business logic afterward.

  • Run scrapers on demand
  • Store results in Google Sheets or databases
  • Send Telegram or email alerts
  • Trigger AI analysis of scraped content
  • Build no-code or low-code data pipelines

How n8n Connects to Apify

There are two main ways to connect n8n and Apify:

  1. Use the Apify integration/node
  2. Use the n8n HTTP Request node with the Apify API

Apify’s official n8n integration supports actions and triggers, including running Actors and reacting to Actor or task events. If you prefer raw API control, you can also use n8n’s HTTP Request node.

Step 1: Create Your Apify Account

First, create an Apify account and choose an Actor you want to run. You can use Actors from the Apify Store or your own custom Actor.

For authentication, Apify documentation says you can connect using OAuth or an Apify API token.

Step 2: Connect Apify to n8n

In n8n, add the Apify node if available in your environment, then create the credential using your Apify login or token.

Basic flow:

Trigger → Apify → Google Sheets / Telegram / Database

If you are not using the Apify node, you can connect through HTTP Request instead.

Step 3: Run an Apify Actor from n8n

The standard Apify API pattern is:

  1. Send a POST request to run the Actor
  2. Get the run response
  3. Read the defaultDatasetId
  4. Fetch the dataset items

Example Actor run endpoint:

POST https://api.apify.com/v2/acts/YOUR-ACTOR-ID/runs?token=YOUR_API_TOKEN

Sample JSON input:

{
  "startUrls": [
    { "url": "https://example.com" }
  ],
  "maxRequestsPerCrawl": 20
}
Tip: The POST payload becomes the Actor input, usually as application/json.

Step 4: Get the Scraped Results

After the Actor starts, the response includes information about the run. Apify’s API documentation explains that you typically monitor the run and then fetch results from the dataset using the defaultDatasetId.

Dataset items example:

GET https://api.apify.com/v2/datasets/YOUR_DATASET_ID/items?token=YOUR_API_TOKEN

These items can then be sent to Google Sheets, Airtable, Telegram, Supabase, or any app connected to n8n.

n8n Workflow Example

Here is a simple workflow idea:

Manual Trigger
   ↓
HTTP Request (Run Apify Actor)
   ↓
Wait
   ↓
HTTP Request (Get Dataset Items)
   ↓
Google Sheets
   ↓
Telegram

This workflow runs an Apify scraper, waits for it to finish, fetches the scraped items, stores them in Google Sheets, and sends a Telegram notification.

Example Use Cases

Google Maps lead scraping
E-commerce product monitoring
News and article collection
AI content research pipelines

Using HTTP Request Instead of a Dedicated Apify Node

n8n’s HTTP Request node can call any REST API, so it works well with Apify if you want more direct control.

Typical HTTP Request setup:

  • Method: POST
  • URL: Apify Actor run endpoint
  • Headers: Content-Type: application/json
  • Body: Your Actor input JSON

This is a good option if you want a custom workflow or if your installed n8n environment does not include the Apify node.

Common Problems and Fixes

1. Invalid API token
Double-check your Apify API token or credential setup.

2. Actor runs but no data appears
Make sure the Actor actually writes output into the dataset.

3. Dataset fetch returns empty
The run may not be finished yet. Add a Wait node or poll the run status.

4. HTTP Request errors
Verify method, URL, headers, and JSON body.

5. Wrong Actor input
Every Actor has its own input schema, so follow the specific Actor’s documentation.

Best Practices

  • Start with one Actor and one destination app
  • Test the Actor in Apify first before wiring it into n8n
  • Log raw results before transforming them
  • Use Wait or polling when the scraper takes time
  • Store API tokens securely in credentials

Watch My n8n tuts on YouTube

I also share videos about automation, n8n workflows, and AI systems on my YouTube channel.

▶ Visit My YouTube

FAQ

Can n8n connect to Apify directly?
Yes. Apify provides an n8n integration for actions and triggers, and you can also use HTTP Request for raw API calls.

Do I need coding to use Apify with n8n?
Not much. You mainly need to understand Actor inputs, API tokens, and JSON bodies.

What is the easiest starter workflow?
Run an Actor, wait for completion, fetch dataset items, and send them to Google Sheets.

n8n and Apify Tutorial: How to Automate Web Scraping Workflows

n8n + Apify Tutorial How to Use n8n with Apify for Web Scraping and Automation Learn how to connect ...