This example demonstrates how to set up a complete Paddler cluster using Docker Compose.

This example sets up:

• Paddler load balancer - Distributes requests across the agents with web admin panel enabled
• Two Paddler agents - Each responsible for inference tasks

1. Navigate to the example directory:

   cd example/

2. Start the cluster:

   docker compose up

3. Access the services:

   • Management dashboard: http://localhost:8062/dashboard

4. Stop the cluster:

   docker compose down

   To remove all data including cached models:

   docker compose down -v

To use GPU acceleration with CUDA, follow these steps:

Install the NVIDIA Container Toolkit on your host system by following the official installation guide:

https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html

After starting the containers, you can verify GPU usage with:

nvidia-smi

You should see the Docker containers listed in the GPU processes.