BitDive Infrastructure Deployment Guide
⚠️ Access Required: To access the BitDive infrastructure repository and deployment files, you'll need to register for one of our plans. Check out our pricing options to get started.
This guide outlines the steps to deploy the BitDive environment using Docker Compose. The setup includes essential services for running the BitDive platform.
Video Tutorial
Repository: The infrastructure setup is available in our GitHub repository.
Services Overview
The BitDive infrastructure consists of the following services:
- Vault: Secrets management system
- PostgreSQL: Database server with SSL configuration
- MinIO: Object storage service
- Keycloak: Identity and access management system
- Monitoring API: Monitors application performance
- Flink Load: Loads data to MinIO
- File Acceptor: Processes incoming files
- Frontend: React-based user interface
Prerequisites
Ensure you have installed:
- Docker (v20.10+)
- Docker Compose (v1.29+)
- Git (for cloning the repository)
Environment Setup
-
Clone the infrastructure repository if you haven't already:
git clone https://github.com/bitDive/infrastructure.git
cd infrastructure/docker-compose -
Create a
.env
file in the root directory with the following variables:
SERVER_IP=your-server-ip
SERVER_NAME=your-server-URL
# Vault Configuration
VAULT_ADDR=https://${SERVER_IP}:8200
VAULT_ADDR_CONTAINER=https://vault-server:8200
# PostgreSQL Configuration
POSTGRES_USER=citizix_user
POSTGRES_PASSWORD=your_postgres_password
POSTGRES_DB=data-bitdive
POSTGRES_HOST=postgres-bitdive
POSTGRES_PORT=5432
# MinIO Configuration
MINIO_ROOT_USER=your_minio_user
MINIO_ROOT_PASSWORD=your_minio_passwor
MINIO_DOMAIN=http://${SERVER_IP}
MINIO_CONSOLE_ADDRESS=:9001
MINIO_ENDPOINT=http://minio:9000
# Keycloak Configuration
KEYCLOAK_DB_USERNAME=${POSTGRES_USER}
KEYCLOAK_DB_PASSWORD=${POSTGRES_PASSWORD}
KEYCLOAK_ADMIN=your_keycloak_user
KEYCLOAK_ADMIN_PASSWORD=your_keycloak_password
KEYCLOAK_HTTP_ENABLED=false
KEYCLOAK_HTTP_SSL_PORT=8443
KEYCLOAK_KEY_STORE_PASSWORD=your-keycloak-keystore-password
KEYCLOAK_TRUST_STORE_PASSWORD=your-keycloak-truststore-password
JAVA_KEYSTORE_PASSWORD=your-keystore-password
JAVA_TRUSTSTORE_PASSWORD=your-truststore-password
KEYCLOAK_DB_URL=jdbc:postgresql://${POSTGRES_HOST}:${POSTGRES_PORT}/keycloak?ssl=true&sslmode=verify-full&&sslfactory=org.postgresql.ssl.DefaultJavaSSLFactory
# Vault User Credentials
VAULT_LOGIN=your_vault_login
VAULT_PASSWORD=your_vault_password
# Vault Certificates Configuration
VAULT_CERT_DB_COMMON_NAME=${POSTGRES_USER}
VAULT_CERT_DB_ALT_NAME=${POSTGRES_HOST}
VAULT_CERT_DB_TTL=24h
VAULT_CERT_SERVICE_COMMON_NAME=file-acceptor.bitdive
VAULT_CERT_SERVICE_ALT_NAMES=${SERVER_IP}
VAULT_CERT_SERVICE_TTL=24h
KEYCLOAK_FRONTEND_URL_NOT_SSL=https://${SERVER_IP}:8999
KEYCLOAK_FRONTEND_URL=https://${SERVER_NAME}:8999
VAULT_CERT_KEYCLOAK_COMMON_NAME=${SERVER_IP}
VAULT_CERT_KEYCLOAK_ALT_NAME=${SERVER_IP}
VAULT_CERT_KEYCLOAK_TTL=24h
# Keycloak Realm URL
KEYCLOAK_REALM_URL=https://keycloak:${KEYCLOAK_HTTP_SSL_PORT}/realms/bitdive/protocol/openid-connect/certs
KEYCLOAK_REALM_URL_NOT_SSL=http://keycloak:8080/realms/bitdive/protocol/openid-connect/certs
# Frontend Configuration
REACT_APP_API_URL=https://${SERVER_NAME}:8081
REACT_APP_KEYCLOAK_URL=https://${SERVER_NAME}:9999/
REACT_APP_KEYCLOAK_REALM=bitdive
REACT_APP_KEYCLOAK_CLIENT_ID=react-client
GENERATE_SOURCEMAP=false
Deployment Steps
Step 1: Configure Environment
Edit the .env
file with your specific values:
- Update
SERVER_IP
andSERVER_NAME
- Set secure passwords for all services
- Configure SSL certificates parameters
Step 2: Deploy Services
-
Start Vault:
docker-compose up -d vault
Wait until Vault is fully initialized.
-
Initialize Database SSL:
docker-compose up -d init-db-ssl
Wait for the initialization to complete.
-
Deploy remaining services:
docker-compose up -d init-container-ssl
-
Verify deployment:
docker-compose ps
Service Access
After successful deployment, access the services at:
- Vault: https://your-server-ip:8200
- MinIO Console: http://your-server-ip:9001
- Keycloak Admin: https://your-server-ip:9999
- BitDive Frontend: http://your-server-ip
Post-Deployment Configuration
After deploying the infrastructure, you'll need to configure Keycloak for authentication:
- Access the Keycloak Admin Console at https://your-server-ip:9999
- Follow the Keycloak Configuration Guide to set up your realm and client
Data Persistence
- Vault data:
./vault/
- PostgreSQL data:
./postgresql/
- MinIO data:
./data/
Troubleshooting
Common Issues
-
Vault Startup Issues
- Check
./configVault/config.json
- Verify SSL certificates
- Review Vault logs:
docker-compose logs vault
- Check
-
PostgreSQL SSL Issues
- Verify certificates in
./vault/ssl/postgres-server
- Check permissions on SSL files
- Review PostgreSQL logs:
docker-compose logs postgres-bitdive
- Verify certificates in
-
Service Connection Issues
- Ensure all ports are available
- Check network connectivity
- Verify SSL configurations