agentql

What is AgentQL?

AgentQL is a suite of tools for extracting data and automating workflows on live web sites featuring an AI-powered query language, Python and JavaScript SDKs, a browser-based debugger, and a REST API endpoint. It uses natural language queries to pinpoint data and elements on any web page, including authenticated and dynamically generated content. Users can define structured data output and apply transforms within queries. AgentQL’s natural language selectors find elements intuitively based on the content of the web page and work across similar web sites, self-healing as UI changes over time.

Get started in 5 minutes with our quick start!

Features

• Playwright AgentQL’s Python SDK and JavaScript SDK seamlessly integrates with Playwright for advanced automation and testing.
• Cross-site compatibility lets you use the same query across different sites with similar content.
• Structured output defined by the shape of your query.
• Natural language selectors find elements and data anywhere on a site using intuitive queries.
• Transforms and extracts data in your queries.
• Works on any page, public or private, any site, any URL, even behind authentication.
• Resilience to UI changes means queries work regardless of how a page’s structure changes over time.

Tools

• Python SDK for running automation and scraping scripts with AgentQL queries in Python.
• JavaScript SDK for running automation and scraping scripts with AgentQL queries in JavaScript.
• REST API for executing queries without an SDK.
• Integrations with agent frameworks and automation tools like Langchain and Zapier and our MCP server.
• Debugger Browser Extension lets you debug and finesse queries in real-time on live sites.
• AgentQL Query Language lets you define queries with natural language.
• Playground for playing with AgentQL lets you export python scripts and optimize queries with prompts.

Examples

Example Name	Python	JavaScript
Getting Started	Script	Script
Close Cookie Dialog	Script / Colab	Script
Close Popup Windows	Script / Colab	Script
Compare Product Prices	Script / Colab	Script
Get Element by Prompt	Script	Script
Infinite Scroll	Script / Colab	Script
Use Remote Browser	Script	Script
Use Existing Browser	Script	Script
Query List Items	Script	Script
Site Login	Script / Colab	Script
Headless Browser	Script / Colab	Script
Save/Load Auth Session	Script	Script
Stealth Mode	Script	Script
Wait for Page Load	Script / Colab	Script
E-commerce Pricing Data	Script	Script
Sentiment Analysis	Script / Colab	Script
Get XPath	Script	Script
Submit Form	Script	Script
Collect YouTube Comments	N/A	Script
Use Humanlike Behavior to Avoid Antibot	Script	Script
Run in Google Colab	Script	N/A
Collect Ecommerce Paginated Data	Script / Colab	Script
Collect Paginated Data From HackerNews	Script / Colab	Script
Scrape Google Maps and save as a CSV	Script	Script
News Aggregator	Script	Script

For comprehensive guides and API references, check out our official documentation.

Show Your Support 🌟

If you find AgentQL helpful, please consider giving us a star on GitHub! It helps us reach more developers and continue improving the project.

Get in touch

For questions, feedback, or support, join our Discord community. You can follow us on GitHub, Twitter, and LinkedIn!