🪣 Data Bucket

Lightweight API service for capturing and storing HTTP requests

Get Started Data Bucket Login →

Overview

Data Bucket is a powerful yet lightweight API service designed for testing webhooks, debugging API integrations, and rapid prototyping. Capture any HTTP request, configure mock responses, and inspect payloads with ease.

💻 Minimum Hardware Requirements

Data Bucket is designed to be lightweight and efficient. Below are the minimum hardware requirements for running the Docker service:

Component Minimum Recommended
CPU 1 vCPU / Core 2 vCPU / Cores
RAM 512 MB 1 GB
Storage 500 MB 2 GB (for logs and data)
Network Any network connection Stable internet for image pulls
📝 Notes:
  • The service includes both the Data Bucket application and an Nginx reverse proxy
  • Storage requirements grow based on the number of buckets and captured requests
  • Performance may vary depending on request volume and payload sizes
  • These requirements are for the Docker containers only - the host system needs additional resources for the Docker engine itself
⚠️ Production Considerations: For production environments handling high traffic volumes, consider scaling resources based on your specific workload. Monitor CPU and memory usage to optimize performance.

🚀 Docker Deployment

Quick Start with Docker Compose

Deploy Data Bucket in minutes using Docker Compose with our pre-built image from Docker Hub.

📋 Prerequisites:
  • Docker installed on your system
  • Docker Compose (included with Docker Desktop)

Create docker-compose.yml

Create a docker-compose.yml file with the following content:

services:
  data-bucket:
    image: ajoealex/data-bucket:latest
    container_name: data-bucket
    restart: unless-stopped
    environment:
      - port=7894
      - user_name=admin
      - password=your_secure_password
      - auth_ignored_method_pids=3
      - requests_per_bucket=50
      - enable_request_logging=false
    volumes:
      - bucket-data:/app/app_data
    networks:
      - data-bucket-network
    expose:
      - "7894"

  nginx:
    image: nginx:alpine
    container_name: data-bucket-nginx
    restart: unless-stopped
    ports:
      - "7895:80"
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf:ro
      - nginx-logs:/var/log/nginx
    depends_on:
      - data-bucket
    networks:
      - data-bucket-network

volumes:
  bucket-data:
    driver: local
  nginx-logs:
    driver: local

networks:
  data-bucket-network:
    driver: bridge

Create nginx.conf

Create an nginx.conf file in the same directory:

events {
    worker_connections 1024;
}

http {
    access_log /var/log/nginx/access.log;
    error_log /var/log/nginx/error.log;

    log_format detailed '$remote_addr - $remote_user [$time_local] '
                       '"$request" $status $body_bytes_sent '
                       '"$http_referer" "$http_user_agent" '
                       'rt=$request_time';

    upstream data_bucket_app {
        server data-bucket:7894;
    }

    server {
        listen 80;
        server_name localhost;

        access_log /var/log/nginx/access.log detailed;
        client_max_body_size 100M;

        location / {
            proxy_pass http://data_bucket_app;
            proxy_http_version 1.1;

            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;

            proxy_connect_timeout 60s;
            proxy_send_timeout 60s;
            proxy_read_timeout 60s;
        }

        location /health {
            access_log off;
            return 200 "healthy\n";
            add_header Content-Type text/plain;
        }
    }
}

Start the Services

docker-compose up -d

Verify Deployment

Test that the service is running:

curl http://localhost:7895/api/v1/ping

Expected response: {"status":"ok"}

🐳 Essential Docker Commands

Start services

docker-compose up -d

Stop services

docker-compose down

View logs (follow mode)

docker-compose logs -f data-bucket

Restart services

docker-compose restart

Pull latest image

docker-compose pull

Update and restart

docker-compose pull && docker-compose up -d

Remove everything (including volumes)

docker-compose down -v

View running containers

docker-compose ps

⚙️ Configuration

Environment Variables

Variable Description Default
port Internal port for the data-bucket service 7894
user_name Basic auth username (leave empty to disable auth) admin
password Basic auth password (leave empty to disable auth) pwd
auth_ignored_method_pids Comma-separated PIDs to skip authentication
PID 3 = data capture endpoint		 1,2,3,4,5,6
requests_per_bucket Maximum requests stored per bucket 50
enable_request_logging Enable/disable request logging to console false

Using .env File

For easier configuration management, create a .env file:

PORT=7894
USER_NAME=admin
PASSWORD=your_secure_password
AUTH_IGNORED_PIDS=3
REQUESTS_PER_BUCKET=50
ENABLE_REQUEST_LOGGING=false

Update your docker-compose.yml to reference the .env file:

environment:
  - port=${PORT:-7894}
  - user_name=${USER_NAME:-admin}
  - password=${PASSWORD:-pwd}
  - auth_ignored_method_pids=${AUTH_IGNORED_PIDS:-3}
  - requests_per_bucket=${REQUESTS_PER_BUCKET:-50}
  - enable_request_logging=${ENABLE_REQUEST_LOGGING:-false}
⚠️ Security Note: Always use strong passwords in production and never commit your .env file to version control.

API Endpoint PIDs

Each API endpoint is assigned a PID (Process ID) for authentication control. Use these PIDs in auth_ignored_method_pids to bypass authentication for specific endpoints.

PID Method Endpoint Description
0 GET /ping Health check - Always public (no auth required)
- POST /auth Authentication test - Always public (no auth required)
1 POST /create_bucket Create a new bucket
2 PUT /bucket/:bucket_id Update bucket configuration
3 ANY /bucket_data/:bucket_id/data/* Send data to bucket (supports wildcard paths)
4 GET /bucket_data/:bucket_id Retrieve bucket data
5 DELETE /bucket_data/:bucket_id Delete bucket data only
6 DELETE /bucket/:bucket_id/clean Delete entire bucket
7 DELETE /bucket/remove_all Delete all buckets
8 GET /buckets List all buckets
💡 Example Use Cases:
  • Public webhook receiver: Set auth_ignored_method_pids=3 to allow anyone to send data to buckets while protecting management endpoints
  • Open testing environment: Set auth_ignored_method_pids=1,2,3,4,5,6 to disable auth for most operations
  • Fully secured: Set auth_ignored_method_pids= (empty) to require authentication on all endpoints

📚 API Reference

Base URL

http://localhost:7895/api/v1

Authentication

Most endpoints require Basic Authentication. Include the Authorization header:

Authorization: Basic base64(username:password)

Endpoints

🏥 Health Check

GET /ping

Check if the service is running. No authentication required.

Response:

{"status": "ok"}

🆕 Create Bucket

POST /create_bucket

Create a new bucket for capturing requests with customizable mock responses.

Request Body:

{
  "name": "my-webhook",
  "mock_response": {"success": true, "message": "Data received"},
  "mock_headers": {"X-Custom-Header": "value"},
  "mock_status_code": 200,
  "mock_response_type": "json"
}

Response:

{
  "bucket_id": "5260566a-b74d-4e14-812c-2b495c7d3385"
}

📨 Send Data to Bucket

ANY /bucket_data/:bucket_id/data/*

Capture any HTTP request to this endpoint. Supports wildcard paths for flexible routing.

Examples:

  • /bucket_data/abc123/data
  • /bucket_data/abc123/data/users
  • /bucket_data/abc123/data/users/123/profile
  • /bucket_data/abc123/data/webhooks/stripe/payment

Supported Content Types:

  • application/json
  • application/x-www-form-urlencoded
  • multipart/form-data
  • text/plain
  • text/xml / application/xml
  • Binary files (images, PDFs, etc.)

Response: Returns the configured mock response for the bucket

📥 Get Bucket Data

GET /bucket_data/:bucket_id?cleanup=true&latest=false

Retrieve all captured requests from a bucket.

Query Parameters:

  • cleanup (boolean): Clear data after retrieval (default: true)
  • latest (boolean): Get only the most recent request (default: false)

Response:

{
  "data": [
    {
      "endpoint": "/api/v1/bucket_data/abc123/data/users",
      "method": "POST",
      "headers": {...},
      "payload": {...},
      "payload_type": "json",
      "query": {},
      "timestamp": "2025-12-29T10:30:00.000Z",
      "ip": "127.0.0.1"
    }
  ]
}

📋 List All Buckets

GET /buckets

Get information about all buckets including request counts and configuration.

Response:

{
  "buckets": {
    "abc123": {
      "name": "my-webhook",
      "mock_response": {...},
      "mock_headers": {...},
      "mock_status_code": 200,
      "mock_response_type": "json",
      "created_at": "2025-12-29T10:00:00.000Z",
      "request_count": 5,
      "max_requests": 50,
      "last_request_at": "2025-12-29T10:30:00.000Z"
    }
  }
}

✏️ Update Bucket Configuration

PUT /bucket/:bucket_id

Update bucket mock response, headers, status code, or response type.

Request Body:

{
  "mock_response": {"updated": true},
  "mock_status_code": 201,
  "mock_headers": {"X-New-Header": "value"}
}

🧹 Delete Bucket Data

DELETE /bucket_data/:bucket_id

Clear all captured requests from a bucket without deleting the bucket itself.

🗑️ Delete Bucket

DELETE /bucket/:bucket_id/clean

Completely remove a bucket and all its data.

💣 Delete All Buckets

DELETE /bucket/remove_all

Remove all buckets and their data. Use with caution!