Skip to content
FlowHubFluxonLab
A
AI Automationfree

Autonomous AI crawler

by Oskaradapted from n8n official workflow galleryUpdated Aug 2026
RequiresAAI AgentCall n8n Workflow ToolCall n8n Workflow ToolHTMLHTMLHTTP RequestHTTP RequestOpenAI Chat ModelOpenAI Chat ModelStructured Output ParserStructured Output ParserSupabaseSupabase
Share Post Share
TWTextTWURLsOpenAI Chat ModelOpenAI Chat Mod…OPJSON ParserSeMap company name and websiteMap company nam…MaExecute workflowGet companiesSeSelect company name and websiteSelect company …SeSet social media arraySet social medi…MeMerge all dataInsert new rowMaConvert HTML to MarkdownConvert HTML to…HtRetrieve URLsSOSplit out URLsRDRemove duplicatedRemove duplicat…SeSet domain to pathSet domain to p…FiFilter out invalid URLsFilter out inva…AgAggregate URLsFiFilter out empty hrefsFilter out empt…SeSet domain (text)Set domain (tex…SeAdd protocool to domain (text)Add protocool t…HRGet website (text)Get website (te…SeSet response (text)Set response (t…SeSet domain (URL)HRGet website (URL)Get website (UR…SeSet response (URL)Set response (U…SeAdd protocool to domain (URL)Add protocool t…AgCrawl website123456789101112
1/5
FLOWS
STEPS · 12
Run manually by an operator

This workflow with AI agent is designed to navigate through the page to retrieve specific type of information (in this example: social media profile links). The agent is equipped with 2 tools: text tool:** to retrieve all the text from the page, URLs tool:** to extract all possible links from the page. 💡 You can edit prompt and JSON schema connected to the agent in order to return other data then social media profile links. 👉 This workflow uses Supabase as storage (input/output). Feel free to change it to any other database of your choice. 🎬 See this workflow in action in my YouTube video. How it works? The workflow uses the input URL (website) as a starting point to retrieve the data (e.g. example.com). Using the "URLs tool", the agent is able to retrieve all links from the page and navigate to them. For example, if you want to retrieve contact information, agent will try to find a subpage that might contain this information (e.g. example.com/contact) and extract the information using the text tool. Set up steps Connect database with input data (website addresses) or pin sample data to trigger node. Configure the crawling agent to retrieve the desired data (e.g.

Tags

n8nreference-onlyagenthtmllm-chat-open-aioutput-parser-structuredsupabasetool-workflow
Connects
AAI Agentcalln8nworkflowtoolCall n8n Workflow ToolhtmlHTMLWHTTP RequestopenaichatmodelOpenAI Chat ModelstructuredoutputparserStructured Output Parser
CategoryAI Automation
Triggermanual
Complexityadvanced
Nodes28
AddedJul 5, 2024

Related workflows

See all AI Automation
DhtmlWopenaichatmodel
free

Scrape and summarize webpages with AI

This workflow integrates both web scraping and NLP functionalities. It uses HTML parsing to extract links, HTTP requests to fetch essay content, and AI-based summarization using GPT-4o. It's an excellent example of an end-to-end automated task that is not only efficient but also provides real value by summarizing valuable content. Note that to use this template, you need to be on n8n version 1.50.0 or later.

by n8n Team
BsheetsWopenaichatmodel
free

Scrape and score Amazon products with Apify, OpenAI, and Google Sheets

Quick overview This workflow manually scrapes product results from an Amazon search URL via the Apify Amazon Crawler actor, uses OpenAI to generate a structured product analysis (category, key feature, verdict, value score), and appends the enriched results to a Google Sheets spreadsheet. How it works Runs when you manually click “Test workflow” in n8n. Sends the configured Amazon search URL and scraping limits to the apify Amazon Crawler actor and retrieves the dataset items. Filters out any scraper items that are missing a title or contain an error. Uses OpenAI to analyze each product’s title, brand, price, rating, and review count and returns a structured category, key feature, buyer verdict, and 1–10 value score. Combines the scraped fields and AI analysis into a single row format and adds a timestamp. Appends each enriched product as a new row in Google Sheets. Setup Create an apify API token and add it as an HTTP Header Auth credential used by the Apify request. Add an OpenAI API key as an OpenAI credential and choose the model to use in the configuration. Connect a Google Sheets OAuth2 account and select the target spreadsheet, sheet tab, and columns for appending rows. Upda

by Dr. Firas
Agmailopenaichatmodeltextclassifier
free

Suggest meeting slots using AI

The purpose of this n8n workflow is to automate the process of identifying incoming Gmail emails that are requesting an appointment, evaluating their content, checking calendar availability, and then composing and sending a response email. Note that to use this template, you need to be on n8n version 1.19.4 or later.

by n8n Team