🎉 Crawl4AI v0.8.9 Released!

📦 Installation

PyPI:

pip install crawl4ai==0.8.9

Docker:

docker pull unclecode/crawl4ai:0.8.9
docker pull unclecode/crawl4ai:latest

Note: Docker images are being built and will be available shortly.
Check the Docker Release workflow for build status.

📝 What's Changed

See CHANGELOG.md for details.

🎉 Crawl4AI v0.8.8 Released!

📦 Installation

PyPI:

pip install crawl4ai==0.8.8

Docker:

docker pull unclecode/crawl4ai:0.8.8
docker pull unclecode/crawl4ai:latest

Note: Docker images are being built and will be available shortly.
Check the Docker Release workflow for build status.

📝 What's Changed

See CHANGELOG.md for details.

🎉 Crawl4AI v0.8.7 Released!

📦 Installation

PyPI:

pip install crawl4ai==0.8.7

Docker:

docker pull unclecode/crawl4ai:0.8.7
docker pull unclecode/crawl4ai:latest

Note: Docker images are being built and will be available shortly.
Check the Docker Release workflow for build status.

📝 What's Changed

See CHANGELOG.md for details.

🎉 Crawl4AI v0.8.5 Released!

📦 Installation

PyPI:

pip install crawl4ai==0.8.5

Docker:

docker pull unclecode/crawl4ai:0.8.5
docker pull unclecode/crawl4ai:latest

Note: Docker images are being built and will be available shortly.
Check the Docker Release workflow for build status.

📝 What's Changed

See CHANGELOG.md for details.

🎉 Crawl4AI v0.8.0 Released!

📦 Installation

PyPI:

pip install crawl4ai==0.8.0

Docker:

docker pull unclecode/crawl4ai:0.8.0
docker pull unclecode/crawl4ai:latest

Note: Docker images are being built and will be available shortly.
Check the Docker Release workflow for build status.

📝 What's Changed

See CHANGELOG.md for details.

🎉 Crawl4AI v0.7.8 Released!

📦 Installation

PyPI:

pip install crawl4ai==0.7.8

Docker:

docker pull unclecode/crawl4ai:0.7.8
docker pull unclecode/crawl4ai:latest

Note: Docker images are being built and will be available shortly.
Check the Docker Release workflow for build status.

📝 What's Changed

See CHANGELOG.md for details.

🎉 Crawl4AI v0.7.7 Released!

This release introduces a complete self-hosting platform with enterprise-grade real-time monitoring. This release transforms Crawl4AI Docker from a simple containerized crawler into a production-ready platform with full operational transparency and control.

🚀 What's New

Major Feature: Real-time Monitoring & Self-Hosting Platform

Docker deployment now includes:

• 📊 Interactive Monitoring Dashboard (/dashboard)
• 🔌 Comprehensive Monitor API
• ⚡ WebSocket Streaming
• 🔥 Smart Browser Pool (3-tier architecture)
• 🧹 Janitor System
• 📈 Production-Ready

🐛 Critical Bug Fixes

• Fixed async LLM extraction blocking issue (#1055) - now supports true parallel processing
• Fixed CDP endpoint verification with exponential backoff (#1445)
• Fixed arun_many to always return a list, even on exception

Configuration & Features

• Updated browser and crawler config documentation to match implementation
• Enhanced DFS deep crawl strategy with seen URL tracking
• Fixed sitemap parsing and URL normalization in AsyncUrlSeeder (#1559)
• Fixed viewport configuration in managed browsers (#1490)
• Fixed remove_overlay_elements functionality (#1396)

Docker & Infrastructure

• Fixed LLM API key handling for multi-provider support
• Standardized Docker port to 11235 across all configs
• Improved error handling with comprehensive status codes
• Fixed fit_html serialization in /crawl and /crawl/stream endpoints

Security

• Updated pyOpenSSL from >=24.3.0 to >=25.3.0 (security vulnerability fix)
• Added verification tests for security updates

📦 Installation

PyPI:

pip install crawl4ai==0.7.7

Docker:

docker pull unclecode/crawl4ai:0.7.7
docker pull unclecode/crawl4ai:latest

Note: Docker images are being built and will be available shortly.
Check the Docker Release workflow for build status.

📝 What's Changed

See CHANGELOG.md for details.

🎉 Crawl4AI v0.7.6 Released!

Crawl4AI v0.7.6 - Webhook Support for Docker Job Queue API

Users can now:

• Use webhooks with both /crawl/job and /llm/job endpoints
• Get real-time notifications instead of polling
• Configure webhook delivery with custom headers
• Include full data in webhook payloads
• Set global webhook URLs in config.yml
• Benefit from automatic retry with exponential backoff

📦 Installation

PyPI:

pip install crawl4ai==0.7.6

Docker:

docker pull unclecode/crawl4ai:0.7.6
docker pull unclecode/crawl4ai:latest

Note: Docker images are being built and will be available shortly.
Check the Docker Release workflow for build status.

📝 What's Changed

See CHANGELOG.md for details.

🚀 Crawl4AI v0.7.5: Docker Hooks & Security Update

🎯 What's New

🔧 Docker Hooks System

Inject custom Python functions at 8 key pipeline points for authentication, performance optimization, and content processing.

Function-Based API with IDE support:

from crawl4ai import hooks_to_string

async def on_page_context_created(page, context, **kwargs):
    """Block images to speed up crawling"""
    await context.route("**/*.{png,jpg,jpeg,gif,webp}", lambda route: route.abort())
    return page

hooks_code = hooks_to_string({"on_page_context_created": on_page_context_created})

8 Available Hook Points:
on_browser_created, on_page_context_created, before_goto, after_goto, on_user_agent_updated, on_execution_started, before_retrieve_html, before_return_html

🤖 Enhanced LLM Integration

• Custom temperature parameter for creativity control
• Multi-provider support (OpenAI, Gemini, custom endpoints)
• base_url configuration for self-hosted models
• Improved Docker API integration

🔒 HTTPS Preservation

New preserve_https_for_internal_links option maintains secure protocols throughout crawling — critical for authenticated sessions and security-conscious applications.

🛠️ Major Bug Fixes

• URL Processing: Fixed '+' sign preservation in query parameters (#1332)
• JWT Authentication: Resolved Docker JWT validation issues (#1442)
• Playwright Stealth: Fixed stealth features integration (#1481)
• Proxy Configuration: Enhanced parsing with new proxy_config structure
• Memory Management: Fixed leaks in long-running sessions
• Docker Serialization: Resolved JSON encoding errors (#1419)
• LLM Providers: Fixed custom provider integration for adaptive crawler (#1291)
• Performance: Resolved backoff strategy failures (#989)

📦 Installation

PyPI:
pip install crawl4ai==0.7.5

Docker:
docker pull unclecode/crawl4ai:0.7.5
docker pull unclecode/crawl4ai:latest

Platforms Supported: Linux/AMD64, Linux/ARM64 (Apple Silicon, AWS Graviton)

⚠️ Breaking Changes

1. Python 3.10+ Required (upgraded from 3.9)
2. Proxy Parameter Deprecated - Use new proxy_config structure
3. New Dependency - cssselect added for better CSS handling

📚 Resources

• 📖 Full Release Notes: https://github.com/unclecode/crawl4ai/blob/main/docs/blog/release-v0.7.5.md
• 📘 Documentation: https://docs.crawl4ai.com
• 💬 Discord Community: https://discord.gg/jP8KfhDhyN
• 🐛 Issues: https://github.com/unclecode/crawl4ai/issues

🙏 Contributors

Thank you to everyone who reported issues, provided feedback, and contributed to this release!

Full Changelog: v0.7.4...v0.7.5

🎉 Crawl4AI v0.7.4 Released!

📦 Installation

PyPI:

pip install crawl4ai==0.7.4

Docker:

docker pull unclecode/crawl4ai:0.7.4
docker pull unclecode/crawl4ai:latest

📝 What's Changed

See CHANGELOG.md for details.