From b5c1ac95b7c96fab75dcd7a7197cf59758230345 Mon Sep 17 00:00:00 2001 From: Julien Gautier Date: Tue, 28 Apr 2026 21:24:57 +0200 Subject: [PATCH 1/6] dbdump initial commit --- dbdump_mongo/.env.example | 18 ++ dbdump_mongo/.gitignore | 15 ++ dbdump_mongo/README.md | 219 +++++++++++++++++++ dbdump_mongo/db-dump.sh | 450 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 702 insertions(+) create mode 100644 dbdump_mongo/.env.example create mode 100644 dbdump_mongo/.gitignore create mode 100644 dbdump_mongo/README.md create mode 100755 dbdump_mongo/db-dump.sh diff --git a/dbdump_mongo/.env.example b/dbdump_mongo/.env.example new file mode 100644 index 0000000..eeeb808 --- /dev/null +++ b/dbdump_mongo/.env.example @@ -0,0 +1,18 @@ +# MongoDB Dump Environment Configuration Example +# Copy this pattern to create environments: .env.prod, .env.uat, etc. +# DO NOT commit actual configuration files to version control + +# Environment: example +# Created: 2026-04-14 + +# MongoDB credentials (no defaults - must be provided) +MONGO_USERNAME=admin_user +MONGO_PASSWORD=secure_password_here +MONGO_HOST=cluster0.mongodb.net +MONGO_PORT=27017 +APP_NAME=my-app + +# Note: The connection string will be constructed as: +# mongodb://:@:/ +# ?ssl=true&replicaSet=globaldb&retrywrites=false +# &maxIdleTimeMS=120000&appName=@@ diff --git a/dbdump_mongo/.gitignore b/dbdump_mongo/.gitignore new file mode 100644 index 0000000..d518a48 --- /dev/null +++ b/dbdump_mongo/.gitignore @@ -0,0 +1,15 @@ +# Environment configuration files (contain sensitive credentials) +.env +.env.* +!.env.example + +# Dump directories (can be large) +dumps/ +*.dump + +# OS files +.DS_Store +Thumbs.db + +# Logs +*.log diff --git a/dbdump_mongo/README.md b/dbdump_mongo/README.md new file mode 100644 index 0000000..9f05435 --- /dev/null +++ b/dbdump_mongo/README.md @@ -0,0 +1,219 @@ +# MongoDB Dump Utility + +This directory contains a standalone tool for backing up MongoDB databases from remote servers. +It manages multiple environment configurations for easy switching between prod, uat, and other environments. + +## Quick Start + +### First Run + +```bash +./db-dump.sh dump +``` + +The script will detect no environments and guide you through creating one. Answer the prompts with: +- **Environment name** (required): e.g., `prod`, `uat`, `staging` +- **MongoDB username** (required): The admin user +- **MongoDB password** (required): The admin password +- **MongoDB host** (required): e.g., `cluster0.mongodb.net` +- **MongoDB port** (required): Usually `27017` or `10255` +- **Application name** (required): Used in connection string and dump folder name + +### Subsequent Runs + +```bash +./db-dump.sh dump +``` + +The script will show you a list of configured environments and let you choose which one to backup, +or create a new environment. + +## Commands + +```bash +# Perform a backup of a selected environment +./db-dump.sh dump + +# List all configured environments +./db-dump.sh list + +# Add a new environment +./db-dump.sh add + +# Delete an environment +./db-dump.sh delete + +# Show help +./db-dump.sh help +``` + +## How It Works + +1. **Environment Selection**: On each run, you select from existing environments or create new ones +2. **Credential Storage**: Each environment's credentials are stored in `.env.` files +3. **MongoDB Connection**: Connects using your credentials with MongoDB Atlas parameters (ssl, replicaSet, etc.) +4. **Dump Creation**: Uses `mongodump` to create a backup +5. **Timestamped Output**: Dumps are organized as `./dumps///` + +## Configuration Files + +Each environment is stored in a separate file: + +``` +.env.prod # Production environment +.env.uat # UAT environment +.env.staging # Staging environment (if created) +``` + +Each file contains: +``` +MONGO_USERNAME= +MONGO_PASSWORD= +MONGO_HOST= +MONGO_PORT= +APP_NAME= +``` + +## Directory Structure + +``` +dbdump/ +├── db-dump.sh # Main script +├── .env.prod # Production config (created by user) +├── .env.uat # UAT config (created by user) +├── .env.example # Example configuration +├── .gitignore # Git ignore rules +├── README.md # This file +└── dumps/ # Created automatically + ├── prod/ + │ ├── 20260414_1632/ + │ │ ├── admin/ + │ │ └── myapp/ + │ └── 20260414_1645/ + └── uat/ + └── 20260414_1632/ +``` + +## Dump Output + +Dumps are stored with the following structure: + +``` +dumps/// + admin/ # System databases + config/ + myapp/ # Application databases + system.profile/ +``` + +Each backup preserves the complete database structure and is ready to be restored using `mongorestore`. + +## Accessing Dumps + +To view or restore a dump: + +```bash +# List recent dumps +ls -lh dumps/prod/ + +# Restore a dump to local MongoDB +mongorestore --uri "mongodb://localhost:27017" dumps/prod/20260414_1632/ + +# Copy dump to another location +cp -r dumps/prod/20260414_1632/ /backup/prod-backup-20260414/ +``` + +## Security + +- Configuration files have restricted permissions (600) +- Credentials are stored locally in `.env.*` files +- These files are git-ignored for security +- Never commit `.env.*` files to version control + +## Requirements + +- **MongoDB Database Tools**: Download from https://www.mongodb.com/try/download/database-tools + - Specifically requires `mongodump` command +- Bash shell +- Network access to MongoDB server + +## Installation of MongoDB Database Tools + +### macOS +```bash +brew tap mongodb/brew +brew install mongodb-database-tools +``` + +### Linux (Ubuntu/Debian) +```bash +wget https://fastdl.mongodb.org/tools/db/mongodb-database-tools-ubuntu2004-x86_64-100.8.0.tgz +tar -xzf mongodb-database-tools-ubuntu2004-x86_64-100.8.0.tgz +sudo cp mongodb-database-tools-ubuntu2004-x86_64-100.8.0/bin/* /usr/local/bin/ +``` + +### Windows +Download the MSI installer from https://www.mongodb.com/try/download/database-tools and run it. + +## Troubleshooting + +### mongodump not found +```bash +# Check if MongoDB Database Tools is installed +mongodump --version + +# If not installed, see "Installation" section above +``` + +### Connection refused +- Verify host and port are correct +- Check network connectivity to MongoDB server +- Ensure firewall allows access + +### Authentication failed +- Verify username and password +- Check the user has admin permissions +- Confirm the `authSource=admin` parameter + +### Port already displaying in error +The connection string is checked for errors. If you see connection errors, the script displays the exact MongoDB error message. + +## Advanced Usage + +### Manual mongodump command (for reference) + +The script constructs and runs this command automatically: + +```bash +mongodump --uri "mongodb://:@:/?ssl=true&replicaSet=globaldb&retrywrites=false&maxIdleTimeMS=120000&appName=@@" -o ./dumps// +``` + +### Scheduling automated backups + +You can schedule regular backups using cron (Linux/macOS): + +```bash +# Add to crontab +crontab -e + +# Daily dump at 3 AM for production environment +0 3 * * * cd /path/to/dbdump && ./db-dump.sh dump << EOF +1 +EOF +``` + +## Support + +For MongoDB connection issues, refer to: +- MongoDB Documentation: https://docs.mongodb.com/ +- Connection String Reference: https://docs.mongodb.com/manual/reference/connection-string/ + +## About + +**Author:** Julien Gautier ([jgautier.webdev@gmail.com](mailto:jgautier.webdev@gmail.com)) + +**Organization:** AdAstra + +**Project:** adastra_scripts + +This utility was created to enable efficient backup and versioning of MongoDB databases from remote servers for AdAstra projects. diff --git a/dbdump_mongo/db-dump.sh b/dbdump_mongo/db-dump.sh new file mode 100755 index 0000000..0755581 --- /dev/null +++ b/dbdump_mongo/db-dump.sh @@ -0,0 +1,450 @@ +#!/bin/bash + +# Author: Julien Gautier +# Organization: AdAstra +# Project: adastra_scripts +# +################################################################################ +# MongoDB Dump Utility Script (Standalone) +# +# This script performs backups (dumps) of MongoDB databases from remote or local servers. +# It manages multiple environments (prod, uat, etc.) with stored credentials. +# +# Usage: ./db-dump.sh +# +# Commands: +# dump Select environment and perform a dump +# list List all configured environments +# add Add a new environment +# delete Delete an environment +# help Show this help message +# +# Examples: +# ./db-dump.sh dump +# ./db-dump.sh list +# ./db-dump.sh add +# +# Configuration: +# Each environment is stored in a separate .env. file. +# All dumps are stored in ./dumps/// +# +################################################################################ + +set -e + +# Configuration +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +DUMPS_DIR="${SCRIPT_DIR}/dumps" + +# Colors +RED='\033[0;31m' +GREEN='\033[0;32m' +YELLOW='\033[1;33m' +BLUE='\033[0;34m' +CYAN='\033[0;36m' +NC='\033[0m' + +# Helper functions +print_info() { + echo -e "${BLUE}┣━${NC} ${CYAN}${1}${NC}" +} + +print_success() { + echo -e "${GREEN}✓ ${1}${NC}" +} + +print_error() { + echo -e "${RED}✗ ${1}${NC}" +} + +print_warning() { + echo -e "${YELLOW}⚠ ${1}${NC}" +} + +print_separator() { + echo -e "${BLUE}┣━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━${NC}" +} + +print_separator_up() { + echo -e "${BLUE}┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━${NC}" +} + +print_separator_down() { + echo -e "${BLUE}┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━${NC}" +} + +print_section() { + echo + print_separator_up + print_info "$1" + print_separator_down + echo +} + +check_mongodump() { + if ! command -v mongodump &> /dev/null; then + print_error "mongodump is not installed or not in PATH" + print_info "Install MongoDB Database Tools from: https://www.mongodb.com/try/download/database-tools" + exit 1 + fi + print_success "mongodump found: $(mongodump --version)" +} + +get_environments() { + local envs=() + for env_file in "$SCRIPT_DIR"/.env.*; do + if [[ -f "$env_file" ]]; then + local env_name=$(basename "$env_file" | sed 's/^\.env\.//') + # Exclude the example file + if [[ "$env_name" != "example" ]]; then + envs+=("$env_name") + fi + fi + done + echo "${envs[@]}" +} + +env_file_path() { + echo "${SCRIPT_DIR}/.env.$1" +} + +create_environment() { + print_section "Create New Environment" + + # Check if environment name already exists + read -p "Enter environment name (prod, uat, etc.): " env_name + env_name="${env_name,,}" # Convert to lowercase + + if [[ -z "$env_name" ]]; then + print_error "Environment name cannot be empty" + return 1 + fi + + if [[ -f "$(env_file_path "$env_name")" ]]; then + print_error "Environment '$env_name' already exists" + return 1 + fi + + print_info "Enter MongoDB connection details for '$env_name'" + echo + + # Read credentials (no defaults) + read -p "MongoDB username: " mongo_username + if [[ -z "$mongo_username" ]]; then + print_error "Username cannot be empty" + return 1 + fi + + read -sp "MongoDB password: " mongo_password + echo + if [[ -z "$mongo_password" ]]; then + print_error "Password cannot be empty" + return 1 + fi + + read -p "MongoDB host (e.g., cluster0.mongodb.net): " mongo_host + if [[ -z "$mongo_host" ]]; then + print_error "Host cannot be empty" + return 1 + fi + + read -p "MongoDB port (e.g., 27017): " mongo_port + if [[ -z "$mongo_port" ]]; then + print_error "Port cannot be empty" + return 1 + fi + + read -p "Application name (appName): " app_name + if [[ -z "$app_name" ]]; then + print_error "Application name cannot be empty" + return 1 + fi + + # Create .env file for this environment + local env_file=$(env_file_path "$env_name") + cat > "$env_file" << EOF +# Environment: $env_name +# Created: $(date) +MONGO_USERNAME=${mongo_username} +MONGO_PASSWORD=${mongo_password} +MONGO_HOST=${mongo_host} +MONGO_PORT=${mongo_port} +APP_NAME=${app_name} +EOF + chmod 600 "$env_file" # Restrict permissions for security + + print_success "Environment '$env_name' created successfully" + print_info "Configuration stored in: $(basename "$env_file")" +} + +# Global variable to hold selected environment +SELECTED_ENVIRONMENT="" + +select_environment() { + SELECTED_ENVIRONMENT="" + local envs=($(get_environments)) + local count=${#envs[@]} + + print_separator_up + + if [[ $count -eq 0 ]]; then + print_info "No Environments Configured" + print_separator + echo + print_warning "No existing environments found" + echo + echo " 1) Create a new environment" + echo " 2) Exit" + echo + echo -n "Enter your choice (1-2): " + read choice + + if [[ "$choice" == "1" ]]; then + if create_environment; then + # After creating an environment, recursively call to select it + select_environment + return $? + else + return 1 + fi + else + return 1 + fi + fi + + print_info "Select Environment" + print_separator_down + echo + + for i in "${!envs[@]}"; do + echo " $((i + 1))) ${envs[$i]}" + done + echo " $((count + 1))) Create new environment" + echo + echo -n "Enter your choice (1-$((count + 1))): " + read choice + + if [[ "$choice" =~ ^[0-9]+$ ]] && [[ $choice -ge 1 ]] && [[ $choice -le $((count + 1)) ]]; then + if [[ $choice -eq $((count + 1)) ]]; then + if create_environment; then + # After creating an environment, recursively call to select it + select_environment + return $? + else + return 1 + fi + else + SELECTED_ENVIRONMENT="${envs[$((choice - 1))]}" + return 0 + fi + else + print_error "Invalid choice" + return 1 + fi +} + +load_environment() { + local env_name="$1" + local env_file=$(env_file_path "$env_name") + + if [[ ! -f "$env_file" ]]; then + print_error "Environment file not found: $env_file" + return 1 + fi + + # shellcheck source=/dev/null + source "$env_file" +} + +perform_dump() { + local env_name="$1" + + load_environment "$env_name" || return 1 + + print_section "MongoDB Dump: $env_name" + + # Create output directory with timestamp + local timestamp=$(date +%Y%m%d_%H%M) + local output_dir="${DUMPS_DIR}/${APP_NAME}/${timestamp}" + + mkdir -p "$output_dir" + + print_separator_up + print_info "Environment: $env_name" + print_info "App Name: $APP_NAME" + print_info "Host: $MONGO_HOST:$MONGO_PORT" + print_info "Output: $output_dir" + print_separator_down + + # Construct connection string + local connection_uri="mongodb://${MONGO_USERNAME}:${MONGO_PASSWORD}@${MONGO_HOST}:${MONGO_PORT}/?ssl=true&replicaSet=globaldb&retrywrites=false&maxIdleTimeMS=120000&appName=@${APP_NAME}@" + + # Perform dump + if mongodump --uri "$connection_uri" -o "$output_dir" 2>&1; then + print_success "Dump completed successfully" + print_info "Output directory: $output_dir" + + # Show dump summary + local dump_size=$(du -sh "$output_dir" | cut -f1) + print_info "Total size: $dump_size" + else + print_error "Dump failed" + print_warning "Removing incomplete dump directory..." + rm -rf "$output_dir" + return 1 + fi +} + +cmd_dump() { + print_separator_up + print_info "MongoDB Dump Utility" + print_separator_down + + check_mongodump + + select_environment + if [[ $? -ne 0 ]]; then + print_error "No environment selected" + return 1 + fi + + if perform_dump "$SELECTED_ENVIRONMENT"; then + print_success "Dump operation completed" + else + print_error "Dump operation failed" + return 1 + fi +} + +cmd_list() { + print_section "Configured Environments" + + local envs=($(get_environments)) + + if [[ ${#envs[@]} -eq 0 ]]; then + print_warning "No environments configured yet" + return 0 + fi + + for env in "${envs[@]}"; do + local env_file=$(env_file_path "$env") + # shellcheck source=/dev/null + source "$env_file" + echo " • $env" + echo " Host: $MONGO_HOST:$MONGO_PORT" + echo " App: $APP_NAME" + echo + done +} + +cmd_add() { + if create_environment; then + print_success "Environment added successfully" + else + print_error "Failed to add environment" + return 1 + fi +} + +cmd_delete() { + print_section "Delete Environment" + + local envs=($(get_environments)) + + if [[ ${#envs[@]} -eq 0 ]]; then + print_warning "No environments configured" + return 0 + fi + + for i in "${!envs[@]}"; do + echo " $((i + 1))) ${envs[$i]}" + done + echo + + read -p "Enter environment number to delete: " choice + + if [[ "$choice" =~ ^[0-9]+$ ]] && [[ $choice -ge 1 ]] && [[ $choice -le ${#envs[@]} ]]; then + local env_name="${envs[$((choice - 1))]}" + read -p "Confirm deletion of '$env_name'? (yes/no): " confirm + + if [[ "$confirm" == "yes" ]]; then + local env_file=$(env_file_path "$env_name") + rm -f "$env_file" + print_success "Environment '$env_name' deleted" + else + print_info "Deletion cancelled" + fi + else + print_error "Invalid choice" + return 1 + fi +} + +cmd_help() { + cat << EOF + +MongoDB Dump Utility Script (Standalone) + +Usage: ./db-dump.sh + +Commands: + dump Select an environment and perform a backup dump + list List all configured environments + add Add a new environment configuration + delete Delete an environment configuration + help Show this help message + +Examples: + ./db-dump.sh dump # Interactive environment selection and dump + ./db-dump.sh list # Show all environments + ./db-dump.sh add # Add new environment + ./db-dump.sh delete # Remove an environment + +Environment Configuration: + - Each environment is stored in a separate .env. file + - Credentials are encrypted (file permissions: 600) + - Supported environments: prod, uat, dev, staging, etc. + +Dump Output: + - Dumps are stored in: ./dumps/// + - Each dump is timestamped for easy identification + - Connection string includes MongoDB Atlas parameters + +Requirements: + - MongoDB Database Tools (mongodump) + - Bash shell + +MongoDB Connection String Format: + mongodb://:@:/ + ?ssl=true&replicaSet=globaldb&retrywrites=false + &maxIdleTimeMS=120000&appName=@@ + +EOF +} + +# Main +COMMAND="${1:-help}" + +case "$COMMAND" in + dump) + cmd_dump + ;; + list) + cmd_list + ;; + add) + cmd_add + ;; + delete) + cmd_delete + ;; + help|--help|-h) + cmd_help + ;; + *) + print_error "Unknown command: $COMMAND" + echo + cmd_help + exit 1 + ;; +esac From dfd8f7b22b8101de3db532fa8e001c4488e3c5ff Mon Sep 17 00:00:00 2001 From: Julien Gautier Date: Tue, 28 Apr 2026 21:26:23 +0200 Subject: [PATCH 2/6] dbrestore initial commit --- dbrestore_mongo/.env.example | 14 ++ dbrestore_mongo/.gitignore | 10 ++ dbrestore_mongo/README.md | 277 ++++++++++++++++++++++++++++++ dbrestore_mongo/db-restore.sh | 312 ++++++++++++++++++++++++++++++++++ 4 files changed, 613 insertions(+) create mode 100644 dbrestore_mongo/.env.example create mode 100644 dbrestore_mongo/.gitignore create mode 100644 dbrestore_mongo/README.md create mode 100755 dbrestore_mongo/db-restore.sh diff --git a/dbrestore_mongo/.env.example b/dbrestore_mongo/.env.example new file mode 100644 index 0000000..0385e8a --- /dev/null +++ b/dbrestore_mongo/.env.example @@ -0,0 +1,14 @@ +# MongoDB Restore Configuration +# Copy this file to .env and update with your values +# This configuration should match your local MongoDB instance + +# MongoDB credentials (for the target database) +MONGO_USERNAME=admin +MONGO_PASSWORD=password +MONGO_HOST=localhost +MONGO_PORT=27017 +MONGO_DATABASE=adastradb +MONGO_AUTHSOURCE=admin + +# Note: The connection string will be constructed as: +# mongodb://:@:/?authSource= diff --git a/dbrestore_mongo/.gitignore b/dbrestore_mongo/.gitignore new file mode 100644 index 0000000..1a857c0 --- /dev/null +++ b/dbrestore_mongo/.gitignore @@ -0,0 +1,10 @@ +# Environment configuration file (contains sensitive credentials) +.env +!.env.example + +# OS files +.DS_Store +Thumbs.db + +# Logs +*.log diff --git a/dbrestore_mongo/README.md b/dbrestore_mongo/README.md new file mode 100644 index 0000000..bc25847 --- /dev/null +++ b/dbrestore_mongo/README.md @@ -0,0 +1,277 @@ +# MongoDB Restore Utility (Standalone) + +This directory contains a standalone tool for restoring MongoDB database backups created with `db-dump.sh`. +It provides interactive navigation through backup directories and restores them to a configured MongoDB instance. + +## Quick Start + +### 1. Configure Connection Details + +Copy the example configuration: +```bash +cp .env.example .env +``` + +Edit `.env` with your MongoDB connection details: +``` +MONGO_USERNAME=admin +MONGO_PASSWORD=password +MONGO_HOST=localhost +MONGO_PORT=27017 +MONGO_DATABASE=adastradb +MONGO_AUTHSOURCE=admin +``` + +### 2. Restore a Backup + +```bash +./db-restore.sh +``` + +The script will: +1. Verify MongoDB tools are installed +2. Load your configuration +3. Ask for the path to your backups directory +4. Guide you through the directory structure +5. Show a summary of what will be restored +6. Ask for confirmation +7. Restore the backup + +## How It Works + +### Configuration + +The script requires a `.env` file with these variables: +- `MONGO_USERNAME` - MongoDB admin username +- `MONGO_PASSWORD` - MongoDB admin password +- `MONGO_HOST` - MongoDB server hostname/IP +- `MONGO_PORT` - MongoDB server port +- `MONGO_DATABASE` - Target database name +- `MONGO_AUTHSOURCE` - Authentication source (usually `admin`) + +### Interactive Navigation + +The script intelligently navigates through your backup directory structure: + +``` +dumps/ +├── prod/ ← Choose this +│ ├── 20260414_1632/ ← Then this +│ │ └── (database folders) ← Then this +│ └── 20260414_1645/ +└── uat/ + └── 20260414_1700/ +``` + +At each level, the script: +1. Shows available subdirectories +2. Asks you to choose one +3. Continues until it finds `.bson` files (backup data) + +### Backup Detection + +The script automatically stops navigation when it finds a directory containing `.bson` files, which are the actual backup data files. + +Example backup structure: +``` +~/Works/dbdump/dumps/ +└── bo-uat-cosmodb/ + └── 20260414_1732/ + └── admin/ ← BSON files here + ├── system.profile.bson + └── system.version.bson +``` + +## Example Use Case + +Given this backup structure: +``` +~/Works/dbdump/dumps/ +├── bo-prod-cosmodb/ +│ └── 20260414_1600/ +│ └── AdAstra/ +│ ├── collection1.bson +│ └── collection1.metadata.json +└── bo-uat-cosmodb/ + └── 20260414_1732/ + └── AdAstra/ + ├── orders.bson + └── orders.metadata.json +``` + +Restore process: +```bash +./db-restore.sh + +# Script asks: Dumps path: ~/Works/dbdump/dumps +# Shows: 1) bo-prod-cosmodb 2) bo-uat-cosmodb +# You choose: 2 +# Shows: 1) 20260414_1732 +# You choose: 1 +# Shows: 1) AdAstra +# You choose: 1 +# Script finds BSON files and shows summary +``` + +## Restore Summary + +Before executing the restore, the script displays: + +``` +MongoDB Connection: + Host: localhost + Port: 27017 + Username: admin + Database: adastradb + AuthSource: admin + +Backup Source: + Location: /home/user/Works/dbdump/dumps/bo-uat-cosmodb/20260414_1732 + Backup files: 42 + Total size: 250M +``` + +You must confirm with "yes" to proceed. + +## Connection String + +The script constructs the MongoDB connection string as: + +``` +mongodb://:@:/?authSource= +``` + +Example: +``` +mongodb://admin:password@localhost:27017/adastradb?authSource=admin +``` + +## Directory Structure + +``` +dbrestore/ +├── db-restore.sh # Main script +├── .env # Configuration (created by you, git-ignored) +├── .env.example # Example configuration +├── .gitignore # Git ignore rules +└── README.md # This file +``` + +## Security + +- Configuration file (`.env`) has restricted permissions +- Contains sensitive credentials (username, password) +- Never commit actual `.env` file to version control +- Use `.env.example` as template + +## Requirements + +- **MongoDB Database Tools**: Download from https://www.mongodb.com/try/download/database-tools + - Specifically requires `mongorestore` command +- Bash shell +- Network access to MongoDB server +- Backup files created by `db-dump.sh` or `mongodump` + +## Installation of MongoDB Database Tools + +### macOS +```bash +brew tap mongodb/brew +brew install mongodb-database-tools +``` + +### Linux (Ubuntu/Debian) +```bash +wget https://fastdl.mongodb.org/tools/db/mongodb-database-tools-ubuntu2004-x86_64-100.8.0.tgz +tar -xzf mongodb-database-tools-ubuntu2004-x86_64-100.8.0.tgz +sudo cp mongodb-database-tools-ubuntu2004-x86_64-100.8.0/bin/* /usr/local/bin/ +``` + +### Windows +Download the MSI installer from https://www.mongodb.com/try/download/database-tools and run it. + +## Troubleshooting + +### mongorestore not found +```bash +# Check if MongoDB Database Tools is installed +mongorestore --version + +# If not installed, see "Installation" section above +``` + +### .env file not found +```bash +# Copy the example configuration +cp .env.example .env + +# Edit with your values +nano .env # or your favorite editor +``` + +### Connection refused +- Verify host and port are correct +- Check network connectivity to MongoDB server +- Ensure firewall allows access +- Verify MongoDB service is running (if local) + +### Authentication failed +- Verify username and password in `.env` +- Check user has sufficient permissions +- Confirm `authSource` is correct (usually `admin`) + +### No BSON files found +- Make sure you're pointing to the root of dumps directory +- Navigate through subdirectories until you reach the folder with actual backup files +- BSON files are typically deep in the directory structure + +## Manual Restore Command + +For reference, here's the command the script executes: + +```bash +mongorestore --uri "mongodb://user:pass@host:port/db?authSource=admin" /path/to/dumps +``` + +## Related Tools + +- **`dbutils/db-start.sh`** - Start MongoDB in Docker for restoring locally +- **`dbdump/db-dump.sh`** - Create backups from remote MongoDB servers +- **`dbrestore/db-restore.sh`** - Restore backups to a MongoDB instance (this tool) + +## Workflow Example + +Complete workflow to backup and restore: + +```bash +# 1. Backup from production MongoDB (in dbdump folder) +cd ~/Works/dbdump +./db-dump.sh dump +# Select prod environment and wait for backup + +# 2. Start local MongoDB (in dbutils folder) +cd ~/Works/dbutils +./db-start.sh up + +# 3. Restore backup locally (in dbrestore folder) +cd ~/Works/dbrestore +./db-restore.sh +# Navigate to the backup created in step 1 and restore +``` + +## Documentation + +For more information: +- MongoDB Documentation: https://docs.mongodb.com/ +- mongorestore Manual: https://docs.mongodb.com/database-tools/mongorestore/ +- Connection String Reference: https://docs.mongodb.com/manual/reference/connection-string/ + +## About + +**Author:** Julien Gautier ([jgautier.webdev@gmail.com](mailto:jgautier.webdev@gmail.com)) + +**Organization:** AdAstra + +**Project:** adastra_scripts + +This utility was created to simplify the restoration of MongoDB database backups to local development environments for AdAstra projects. diff --git a/dbrestore_mongo/db-restore.sh b/dbrestore_mongo/db-restore.sh new file mode 100755 index 0000000..2018ad9 --- /dev/null +++ b/dbrestore_mongo/db-restore.sh @@ -0,0 +1,312 @@ +#!/bin/bash + +# Author: Julien Gautier +# Organization: AdAstra +# Project: adastra_scripts +# +################################################################################ +# MongoDB Restore Utility Script (Standalone) +# +# This script restores MongoDB database dumps from backup files. +# It provides interactive navigation through dump directories and restores +# to a configured MongoDB instance. +# +# Usage: ./db-restore.sh +# +# Configuration: +# Requires .env file with MongoDB connection details: +# - MONGO_USERNAME +# - MONGO_PASSWORD +# - MONGO_HOST +# - MONGO_PORT +# - MONGO_DATABASE +# - MONGO_AUTHSOURCE +# +# Example command (what the script executes): +# mongorestore --uri "mongodb://user:pass@host:port/db?authSource=admin" /path/to/dumps +# +################################################################################ + +set -e + +# Configuration +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +ENV_FILE="${SCRIPT_DIR}/.env" + +# Global variables +SELECTED_DUMP_PATH="" + +# Colors +RED='\033[0;31m' +GREEN='\033[0;32m' +YELLOW='\033[1;33m' +BLUE='\033[0;34m' +CYAN='\033[0;36m' +NC='\033[0m' + +# Helper functions +print_info() { + echo -e "${BLUE}┣━${NC} ${CYAN}${1}${NC}" +} + +print_success() { + echo -e "${GREEN}✓ ${1}${NC}" +} + +print_error() { + echo -e "${RED}✗ ${1}${NC}" +} + +print_warning() { + echo -e "${YELLOW}⚠ ${1}${NC}" +} + +print_separator() { + echo -e "${BLUE}┣━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━${NC}" +} + +print_separator_up() { + echo -e "${BLUE}┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━${NC}" +} + +print_separator_down() { + echo -e "${BLUE}┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━${NC}" +} + +print_section() { + echo + print_separator_up + print_info "$1" + print_separator_down + echo +} + +check_mongorestore() { + if ! command -v mongorestore &> /dev/null; then + print_error "mongorestore is not installed or not in PATH" + print_info "Install MongoDB Database Tools from: https://www.mongodb.com/try/download/database-tools" + exit 1 + fi + print_success "mongorestore found: $(mongorestore --version | head -1)" +} + +check_env_file() { + if [[ ! -f "$ENV_FILE" ]]; then + print_error "Configuration file not found: $ENV_FILE" + print_info "Please create .env file with the following variables:" + print_info " MONGO_USERNAME=" + print_info " MONGO_PASSWORD=" + print_info " MONGO_HOST=" + print_info " MONGO_PORT=" + print_info " MONGO_DATABASE=" + print_info " MONGO_AUTHSOURCE=" + print_info "You can copy from .env.example" + exit 1 + fi +} + +load_config() { + # shellcheck source=/dev/null + source "$ENV_FILE" + + # Validate required variables + if [[ -z "$MONGO_USERNAME" ]] || [[ -z "$MONGO_PASSWORD" ]] || [[ -z "$MONGO_HOST" ]] || \ + [[ -z "$MONGO_PORT" ]] || [[ -z "$MONGO_DATABASE" ]] || [[ -z "$MONGO_AUTHSOURCE" ]]; then + print_error "Missing required configuration variables in .env file" + exit 1 + fi +} + +has_bson_files() { + local dir="$1" + [[ -n $(find "$dir" -maxdepth 1 -type f -name "*.bson" 2>/dev/null | head -1) ]] +} + +has_subdirectories() { + local dir="$1" + [[ -n $(find "$dir" -maxdepth 1 -type d ! -name "." ! -name ".." 2>/dev/null | head -1) ]] +} + +navigate_to_dump() { + SELECTED_DUMP_PATH="" + local current_path="$1" + + # Expand ~ in path + current_path="${current_path/#\~/$HOME}" + + # Check if path exists + if [[ ! -d "$current_path" ]]; then + print_error "Path does not exist: $current_path" + return 1 + fi + + # If this directory has BSON files, we're at the right level + if has_bson_files "$current_path"; then + SELECTED_DUMP_PATH="$current_path" + return 0 + fi + + # If no subdirectories and no BSON files, this isn't a valid dump directory + if ! has_subdirectories "$current_path"; then + print_error "No BSON files found and no subdirectories in: $current_path" + return 1 + fi + + # Navigate through subdirectories + while true; do + print_separator_up + print_info "Select Dump Directory" + print_separator_down + echo + print_info "Current location: $current_path" + echo + + # Get list of subdirectories + local subdirs=() + local count=0 + + # Read subdirectories into array (null-terminated, excluding . and ..) + while IFS= read -r -d '' subdir; do + subdirs+=("$subdir") + echo " $((++count))) $(basename "$subdir")" + done < <(find "$current_path" -maxdepth 1 -mindepth 1 -type d -print0 | sort -z) + + if [[ $count -eq 0 ]]; then + print_error "No subdirectories found in: $current_path" + return 1 + fi + + echo + echo -n "Enter your choice (1-$count): " + read choice + + if [[ "$choice" =~ ^[0-9]+$ ]] && [[ $choice -ge 1 ]] && [[ $choice -le $count ]]; then + current_path="${subdirs[$((choice - 1))]}" + + # Check if this directory has BSON files + if has_bson_files "$current_path"; then + SELECTED_DUMP_PATH="$current_path" + return 0 + fi + + # If no more subdirectories and no BSON files, error + if ! has_subdirectories "$current_path"; then + print_error "No BSON files found in: $current_path" + return 1 + fi + # Continue loop to show next level + else + print_error "Invalid choice" + fi + done +} + +get_backup_summary() { + local dump_path="$1" + local bson_count=$(find "$dump_path" -name "*.bson" -type f 2>/dev/null | wc -l) + local total_size=$(du -sh "$dump_path" 2>/dev/null | cut -f1) + + echo "Backup files: $bson_count" + echo "Total size: $total_size" +} + +show_summary() { + local dump_path="$1" + + print_section "Restore Summary" + + echo "MongoDB Connection:" + echo " Host: $MONGO_HOST" + echo " Port: $MONGO_PORT" + echo " Username: $MONGO_USERNAME" + echo " Database: $MONGO_DATABASE" + echo " AuthSource: $MONGO_AUTHSOURCE" + echo + echo "Backup Source:" + echo " Location: $dump_path" + echo " $(get_backup_summary "$dump_path")" + echo + echo "Connection string:" + echo " mongodb://:@$MONGO_HOST:$MONGO_PORT/$MONGO_DATABASE?authSource=$MONGO_AUTHSOURCE" + echo +} + +confirm_restore() { + echo -n "Do you want to proceed with the restore? (yes/no): " + read response + [[ "$response" == "yes" ]] +} + +perform_restore() { + local dump_path="$1" + + print_separator_up + print_info "Starting MongoDB Restore..." + print_separator_down + echo + + # Construct connection string + local connection_uri="mongodb://${MONGO_USERNAME}:${MONGO_PASSWORD}@${MONGO_HOST}:${MONGO_PORT}/${MONGO_DATABASE}?authSource=${MONGO_AUTHSOURCE}" + + # Perform restore + if mongorestore --uri "$connection_uri" "$dump_path" 2>&1; then + print_success "Restore completed successfully" + return 0 + else + print_error "Restore operation failed" + return 1 + fi +} + +main() { + print_separator_up + print_info "MongoDB Restore Utility" + print_separator_down + + # Initial checks + check_mongorestore + check_env_file + load_config + + print_success "Configuration loaded" + echo + + # Ask for initial dump path + print_info "Enter the path to the dumps directory" + echo " Example: ~/Works/dbdump/dumps" + echo + echo -n "Dumps path: " + read dumps_path + + if [[ -z "$dumps_path" ]]; then + print_error "Dumps path cannot be empty" + return 1 + fi + + # Navigate to the correct dump directory + print_info "Navigating to backup location..." + navigate_to_dump "$dumps_path" + if [[ $? -ne 0 ]]; then + print_error "Failed to select backup directory" + return 1 + fi + + # Show summary of what will be restored + show_summary "$SELECTED_DUMP_PATH" + + # Ask for confirmation + if ! confirm_restore; then + print_info "Restore operation cancelled" + return 1 + fi + + # Perform the restore + if perform_restore "$SELECTED_DUMP_PATH"; then + print_success "Restore operation completed" + else + print_error "Restore operation failed" + return 1 + fi +} + +main "$@" From 5ba3645e2a192e17fd65ad2d4cb9d2c2c3dc82b0 Mon Sep 17 00:00:00 2001 From: Julien Gautier Date: Tue, 28 Apr 2026 21:27:46 +0200 Subject: [PATCH 3/6] dbutils initial commit --- dbutils_mongo/.env.example | 11 + dbutils_mongo/.gitignore | 14 + dbutils_mongo/README.md | 154 +++++++++ dbutils_mongo/db-start.sh | 412 +++++++++++++++++++++++ dbutils_mongo/docker-compose-mongodb.yml | 48 +++ dbutils_mongo/mongo-init.js | 17 + 6 files changed, 656 insertions(+) create mode 100644 dbutils_mongo/.env.example create mode 100644 dbutils_mongo/.gitignore create mode 100644 dbutils_mongo/README.md create mode 100755 dbutils_mongo/db-start.sh create mode 100644 dbutils_mongo/docker-compose-mongodb.yml create mode 100644 dbutils_mongo/mongo-init.js diff --git a/dbutils_mongo/.env.example b/dbutils_mongo/.env.example new file mode 100644 index 0000000..1a6cd3e --- /dev/null +++ b/dbutils_mongo/.env.example @@ -0,0 +1,11 @@ +# MongoDB Configuration Example +# Copy this file to .env and update with your values +# The script will prompt for these values on first run + +MONGO_USERNAME=admin +MONGO_PASSWORD=password +MONGO_DATABASE=adastradb +MONGO_PORT=27017 +MONGO_EXPRESS_PORT=8081 +MONGO_EXPRESS_USERNAME=dev +MONGO_EXPRESS_PASSWORD=dev diff --git a/dbutils_mongo/.gitignore b/dbutils_mongo/.gitignore new file mode 100644 index 0000000..bc19f33 --- /dev/null +++ b/dbutils_mongo/.gitignore @@ -0,0 +1,14 @@ +# Environment variables +.env +*.env + +# Docker volumes +mongodb-data/ +mongo-config/ + +# OS files +.DS_Store +Thumbs.db + +# Logs +*.log diff --git a/dbutils_mongo/README.md b/dbutils_mongo/README.md new file mode 100644 index 0000000..8f62382 --- /dev/null +++ b/dbutils_mongo/README.md @@ -0,0 +1,154 @@ +# MongoDB Database Utility (Standalone) + +This directory contains a standalone MongoDB setup for local development. It is completely independent from the main backend and frontend projects. + +## Quick Start + +### First Run (Setup) + +```bash +./db-start.sh up +``` + +The script will prompt you for: +- **MongoDB username** (default: `admin`) +- **MongoDB password** (default: `password`) +- **Database name** (default: `adastradb`) +- **MongoDB port** (default: `27017`) +- **Mongo Express port** (default: `8081`) + +Your configuration will be stored in `.env` file (automatically git-ignored). + +### Common Commands + +```bash +# Start MongoDB (prompts for credentials on first run) +./db-start.sh up + +# View logs +./db-start.sh logs + +# Stop MongoDB +./db-start.sh down + +# Restart MongoDB +./db-start.sh restart + +# Check container status +./db-start.sh status + +# Reconfigure credentials +./db-start.sh reconfigure + +# Remove all containers and volumes (DATA LOSS!) +./db-start.sh clean + +# Show help +./db-start.sh help +``` + +## Accessing MongoDB + +### From Your Application +- **Connection String**: `mongodb://:@localhost:27017/?authSource=admin` +- **Example**: `mongodb://admin:password@localhost:27017/adastradb?authSource=admin` + +### Using Mongo Express (GUI) +- **URL**: http://localhost:8081 +- **Username**: `dev` +- **Password**: `dev` + +## Data Persistence + +All data is stored in Docker volumes and persists across container restarts. To permanently delete data, use: + +```bash +./db-start.sh clean +``` + +## Collections + +The database is automatically initialized with the following collection: + +- **dataImportHistory**: Stores records of all data imports for audit and recovery tracking + - Indexed on `importDate` (descending) and `status` + - Ready for future data import functionality + +## File Structure + +``` +dbutils/ +├── db-start.sh # Main script +├── docker-compose-mongodb.yml # Docker Compose configuration +├── mongo-init.js # Database initialization script +├── .env # Configuration (generated on first run, git-ignored) +├── .env.example # Example configuration +├── .gitignore # Git ignore rules +└── README.md # This file +``` + +## Configuration + +The script uses a `.env` file to store configuration. You can: + +1. **Let the script generate it**: Run `./db-start.sh up` and answer the prompts +2. **Pre-create it**: Copy `.env.example` to `.env` and modify values +3. **Edit it manually**: Edit `.env` directly and run `./db-start.sh up` +4. **Reconfigure**: Run `./db-start.sh reconfigure` to re-run the setup wizard + +## Requirements + +- Docker +- Docker Compose (included with Docker Desktop) +- Bash shell + +## Standalone Usage + +This utility can be used independently from AdAstra projects: + +```bash +# From anywhere +cd /path/to/dbutils +./db-start.sh up + +# Or add to PATH and use globally (optional) +export PATH=$PATH:/path/to/dbutils +db-start.sh up +``` + +## Troubleshooting + +### Port already in use +If MongoDB or Mongo Express port is already in use: +1. Run `./db-start.sh reconfigure` +2. Choose a different port (e.g., 27018 for MongoDB, 8082 for Mongo Express) + +### Docker not running +```bash +# Check Docker status +docker info +``` + +### Permission denied +```bash +# Make script executable +chmod +x db-start.sh +``` + +## Notes + +- Credentials are stored in `.env` (git-ignored for security) +- Mongo Express is included for easy database management +- Data persists in Docker volumes +- No initialization scripts are run (unlike the backend docker-compose) +- This is standalone and independent from the main project configuration + +## About + +**Author:** Julien Gautier ([jgautier.webdev@gmail.com](mailto:jgautier.webdev@gmail.com)) + +**Organization:** AdAstra + +**Project:** adastra_scripts + +These utilities were created to simplify MongoDB management during local development of AdAstra projects. diff --git a/dbutils_mongo/db-start.sh b/dbutils_mongo/db-start.sh new file mode 100755 index 0000000..0de319e --- /dev/null +++ b/dbutils_mongo/db-start.sh @@ -0,0 +1,412 @@ +#!/bin/bash + +# Author: Julien Gautier +# Organization: AdAstra +# Project: adastra_scripts +# +################################################################################ +# MongoDB Database Utility Script (Standalone) +# +# This script manages a standalone MongoDB instance for local development. +# It runs independently and requires no configuration from the backend project. +# +# Usage: ./db-start.sh +# +# Commands: +# up Start MongoDB and Mongo Express (interactive setup) +# down Stop MongoDB and Mongo Express +# logs Show database logs (follow mode) +# restart Restart MongoDB and Mongo Express +# status Show container status +# clean Remove database containers and volumes +# help Show this help message +# +# Configuration: +# The script will prompt for MongoDB credentials on first run. +# Configuration is stored in .env file. +# +# Examples: +# ./db-start.sh up +# ./db-start.sh logs +# ./db-start.sh down +# +################################################################################ + +set -e + +# Configuration +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +COMPOSE_FILE="${SCRIPT_DIR}/docker-compose-mongodb.yml" +ENV_FILE="${SCRIPT_DIR}/.env" +PROJECT_NAME="dbutils" + +# Colors +RED='\033[0;31m' +GREEN='\033[0;32m' +YELLOW='\033[1;33m' +BLUE='\033[0;34m' +CYAN='\033[0;36m' +NC='\033[0m' + +# Helper functions +print_info() { + echo -e "${BLUE}┣━${NC} ${CYAN}${1}${NC}" +} + +print_success() { + echo -e "${GREEN}✓ ${1}${NC}" +} + +print_error() { + echo -e "${RED}✗ ${1}${NC}" +} + +print_warning() { + echo -e "${YELLOW}⚠ ${1}${NC}" +} + +print_separator() { + echo -e "${BLUE}┣━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━${NC}" +} + +print_separator_up() { + echo -e "${BLUE}┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━${NC}" +} + +print_separator_down() { + echo -e "${BLUE}┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━${NC}" +} + +check_docker() { + if ! command -v docker &> /dev/null; then + print_error "Docker is not installed or not in PATH" + exit 1 + fi + print_success "Docker found: $(docker --version)" +} + +check_docker_running() { + if ! docker info > /dev/null 2>&1; then + print_error "Docker daemon is not running" + exit 1 + fi + print_success "Docker daemon is running" +} + +check_docker_compose() { + if ! command -v docker-compose &> /dev/null && ! docker compose version &> /dev/null 2>&1; then + print_error "Docker Compose is not installed or not in PATH" + exit 1 + fi + print_success "Docker Compose is available" +} + +check_compose_file() { + if [[ ! -f "$COMPOSE_FILE" ]]; then + print_error "Docker Compose file not found at $COMPOSE_FILE" + exit 1 + fi +} + +get_compose_cmd() { + if command -v docker-compose &> /dev/null; then + echo "docker-compose" + else + echo "docker compose" + fi +} + +setup_credentials() { + print_separator_up + print_info "MongoDB Configuration Setup" + print_separator_down + echo + + # Read username + read -p "Enter MongoDB root username (default: admin): " mongo_username + mongo_username="${mongo_username:-admin}" + + # Read password + read -sp "Enter MongoDB root password (default: password): " mongo_password + echo + mongo_password="${mongo_password:-password}" + + # Read database name + read -p "Enter MongoDB database name (default: adastradb): " mongo_database + mongo_database="${mongo_database:-adastradb}" + + echo + print_info "MongoDB port (default: 27017): " + read -p "Enter MongoDB port or press Enter for default: " mongo_port + mongo_port="${mongo_port:-27017}" + + print_info "Mongo Express port (default: 8081): " + read -p "Enter Mongo Express port or press Enter for default: " mongo_express_port + mongo_express_port="${mongo_express_port:-8081}" + + echo + + # Create .env file + cat > "$ENV_FILE" << EOF +# MongoDB Configuration +MONGO_USERNAME=${mongo_username} +MONGO_PASSWORD=${mongo_password} +MONGO_DATABASE=${mongo_database} +MONGO_PORT=${mongo_port} +MONGO_EXPRESS_PORT=${mongo_express_port} +MONGO_EXPRESS_USERNAME=dev +MONGO_EXPRESS_PASSWORD=dev +EOF + + print_success "Configuration saved to .env" + echo +} + +load_credentials() { + if [[ -f "$ENV_FILE" ]]; then + print_info "Loading configuration from .env" + # shellcheck source=/dev/null + source "$ENV_FILE" + else + print_warning "No configuration found. Running setup..." + setup_credentials + # shellcheck source=/dev/null + source "$ENV_FILE" + fi +} + +cmd_up() { + print_separator_up + print_info "Starting MongoDB Database..." + print_separator_down + echo + + check_docker + check_docker_running + check_compose_file + check_docker_compose + + load_credentials + + COMPOSE_CMD=$(get_compose_cmd) + + # Export variables for docker-compose + export MONGO_USERNAME + export MONGO_PASSWORD + export MONGO_DATABASE + export MONGO_PORT + export MONGO_EXPRESS_PORT + export MONGO_EXPRESS_USERNAME + export MONGO_EXPRESS_PASSWORD + + $COMPOSE_CMD -f "$COMPOSE_FILE" -p "$PROJECT_NAME" up -d + + print_success "MongoDB and Mongo Express have been started" + print_separator_up + print_info "MongoDB Connection Details:" + print_info " Host: localhost:${MONGO_PORT}" + print_info " Username: ${MONGO_USERNAME}" + print_info " Password: ${MONGO_PASSWORD}" + print_info " Database: ${MONGO_DATABASE}" + print_separator + print_info "Mongo Express (GUI) available at: http://localhost:${MONGO_EXPRESS_PORT}" + print_info " Username: ${MONGO_EXPRESS_USERNAME}" + print_info " Password: ${MONGO_EXPRESS_PASSWORD}" + print_separator_down +} + +cmd_down() { + print_separator_up + print_info "Stopping MongoDB and Mongo Express..." + print_separator_down + + check_docker + check_compose_file + check_docker_compose + + COMPOSE_CMD=$(get_compose_cmd) + + $COMPOSE_CMD -f "$COMPOSE_FILE" -p "$PROJECT_NAME" down + + print_success "MongoDB and Mongo Express have been stopped" +} + +cmd_logs() { + print_separator_up + print_info "Showing MongoDB logs..." + print_separator_down + + check_docker + check_compose_file + check_docker_compose + + COMPOSE_CMD=$(get_compose_cmd) + + $COMPOSE_CMD -f "$COMPOSE_FILE" -p "$PROJECT_NAME" logs -f +} + +cmd_restart() { + print_separator_up + print_info "Restarting MongoDB and Mongo Express..." + print_separator_down + + check_docker + check_docker_running + check_compose_file + check_docker_compose + + load_credentials + + COMPOSE_CMD=$(get_compose_cmd) + + # Export variables for docker-compose + export MONGO_USERNAME + export MONGO_PASSWORD + export MONGO_DATABASE + export MONGO_PORT + export MONGO_EXPRESS_PORT + export MONGO_EXPRESS_USERNAME + export MONGO_EXPRESS_PASSWORD + + $COMPOSE_CMD -f "$COMPOSE_FILE" -p "$PROJECT_NAME" restart + + print_success "MongoDB and Mongo Express have been restarted" +} + +cmd_status() { + print_separator_up + print_info "Container Status" + print_separator_down + + check_docker + check_compose_file + check_docker_compose + + COMPOSE_CMD=$(get_compose_cmd) + + $COMPOSE_CMD -f "$COMPOSE_FILE" -p "$PROJECT_NAME" ps +} + +cmd_clean() { + print_separator_up + print_warning "This will remove database containers and volumes" + print_warning "All data will be PERMANENTLY LOST!" + echo + read -p "Are you sure? Type 'yes' to confirm: " response + echo + + if [[ "$response" == "yes" ]]; then + print_separator + print_info "Removing containers and volumes..." + print_separator + + check_docker + check_compose_file + check_docker_compose + + COMPOSE_CMD=$(get_compose_cmd) + + $COMPOSE_CMD -f "$COMPOSE_FILE" -p "$PROJECT_NAME" down -v --remove-orphans + + print_success "All containers and volumes have been removed" + print_info "Configuration file (.env) has been preserved" + else + print_info "Clean operation cancelled" + fi + print_separator_down +} + +cmd_reconfigure() { + print_separator_up + print_warning "This will reset the MongoDB configuration" + echo + read -p "Are you sure? Type 'yes' to confirm: " response + echo + + if [[ "$response" == "yes" ]]; then + print_info "Stopping services before reconfiguration..." + check_docker 2>/dev/null || true + check_compose_file 2>/dev/null || true + + COMPOSE_CMD=$(get_compose_cmd) + if command -v $COMPOSE_CMD &> /dev/null; then + $COMPOSE_CMD -f "$COMPOSE_FILE" -p "$PROJECT_NAME" down 2>/dev/null || true + fi + + rm -f "$ENV_FILE" + print_success "Configuration cleared" + + setup_credentials + print_info "You can now run './db-start.sh up' to start with the new configuration" + else + print_info "Reconfiguration cancelled" + fi + print_separator_down +} + +cmd_help() { + cat << EOF + +MongoDB Database Utility Script (Standalone) + +Usage: ./db-start.sh + +Commands: + up Start MongoDB and Mongo Express (interactive setup on first run) + down Stop MongoDB and Mongo Express + logs Show database and container logs (follow mode) + restart Restart MongoDB and Mongo Express + status Show container status + reconfigure Reset and reconfigure MongoDB credentials + clean Remove all containers and volumes (DATA LOSS!) + help Show this help message + +Examples: + ./db-start.sh up + ./db-start.sh logs + ./db-start.sh down + ./db-start.sh reconfigure + +Configuration: + On first run, 'up' command will prompt for MongoDB credentials. + Configuration is stored in .env file (git-ignored). + You can reconfigure anytime with 'reconfigure' command. + +EOF +} + +# Main +COMMAND="${1:-help}" + +case "$COMMAND" in + up|start) + cmd_up + ;; + down|stop) + cmd_down + ;; + logs) + cmd_logs + ;; + restart) + cmd_restart + ;; + status|ps) + cmd_status + ;; + reconfigure) + cmd_reconfigure + ;; + clean|remove) + cmd_clean + ;; + help|--help|-h) + cmd_help + ;; + *) + print_error "Unknown command: $COMMAND" + echo + cmd_help + exit 1 + ;; +esac diff --git a/dbutils_mongo/docker-compose-mongodb.yml b/dbutils_mongo/docker-compose-mongodb.yml new file mode 100644 index 0000000..d18f183 --- /dev/null +++ b/dbutils_mongo/docker-compose-mongodb.yml @@ -0,0 +1,48 @@ +services: + mongodb: + image: mongo:latest + container_name: dbutils-mongodb + ports: + - "${MONGO_PORT:-27017}:27017" + environment: + MONGO_INITDB_ROOT_USERNAME: ${MONGO_USERNAME} + MONGO_INITDB_ROOT_PASSWORD: ${MONGO_PASSWORD} + MONGO_INITDB_DATABASE: ${MONGO_DATABASE} + volumes: + - mongodb-data:/data/db + - mongodb-config:/data/configdb + - ./mongo-init.js:/docker-entrypoint-initdb.d/mongo-init.js + networks: + - dbutils-network + restart: unless-stopped + healthcheck: + test: echo 'db.runCommand("ping").ok' | mongosh localhost:27017/test --quiet + interval: 10s + timeout: 5s + retries: 5 + start_period: 40s + + mongo-express: + image: mongo-express:latest + container_name: dbutils-mongo-express + ports: + - "${MONGO_EXPRESS_PORT:-8081}:8081" + environment: + ME_CONFIG_MONGODB_URL: mongodb://${MONGO_USERNAME}:${MONGO_PASSWORD}@mongodb:27017/?authSource=admin + ME_CONFIG_MONGODB_ADMINUSERNAME: ${MONGO_USERNAME} + ME_CONFIG_MONGODB_ADMINPASSWORD: ${MONGO_PASSWORD} + ME_CONFIG_BASICAUTH_USERNAME: ${MONGO_EXPRESS_USERNAME:-dev} + ME_CONFIG_BASICAUTH_PASSWORD: ${MONGO_EXPRESS_PASSWORD:-dev} + depends_on: + - mongodb + networks: + - dbutils-network + restart: unless-stopped + +volumes: + mongodb-data: + mongodb-config: + +networks: + dbutils-network: + driver: bridge diff --git a/dbutils_mongo/mongo-init.js b/dbutils_mongo/mongo-init.js new file mode 100644 index 0000000..04253eb --- /dev/null +++ b/dbutils_mongo/mongo-init.js @@ -0,0 +1,17 @@ +// Initialize MongoDB database + +// Switch to the application database +db = db.getSiblingDB(process.env.MONGO_INITDB_DATABASE || 'adastradb'); + +// Create a collection for data import history +// This collection will track all data imports for audit and recovery purposes +db.createCollection('dataImportHistory', { + capped: false +}); + +// Create an index on the importDate for better query performance +db.dataImportHistory.createIndex({ importDate: -1 }); +db.dataImportHistory.createIndex({ status: 1 }); + +print(`✓ Database '${db.getName()}' initialized successfully`); +print(`✓ Collection 'dataImportHistory' created`); From f804b7abad578ee42fb1cc8e2f693ed04dff3bfd Mon Sep 17 00:00:00 2001 From: Julien Gautier Date: Tue, 28 Apr 2026 21:30:54 +0200 Subject: [PATCH 4/6] (docs): initial docs commit --- docs/dbdump_mongo.html | 500 ++++++++++++++++++++++++++++++++++ docs/dbrestore_mongo.html | 546 ++++++++++++++++++++++++++++++++++++++ docs/dbutils_mongo.html | 513 +++++++++++++++++++++++++++++++++++ 3 files changed, 1559 insertions(+) create mode 100644 docs/dbdump_mongo.html create mode 100644 docs/dbrestore_mongo.html create mode 100644 docs/dbutils_mongo.html diff --git a/docs/dbdump_mongo.html b/docs/dbdump_mongo.html new file mode 100644 index 0000000..69013fb --- /dev/null +++ b/docs/dbdump_mongo.html @@ -0,0 +1,500 @@ + + + + + + MongoDB Dump Utility + + + +
+
+

MongoDB Dump Utility

+

Database Server Backup Management

+
+ +
+

Table of Contents

+ +
+ +
+
+

+ This directory contains a standalone tool for backing up MongoDB databases from remote or local servers. It manages multiple environment configurations for easy switching between prod, uat, and other environments. +

+
+ +
+

Quick Start

+ +

First Run

+ 
./db-dump.sh dump
+ +

The script will detect no environments and guide you through creating one. Answer the prompts with:

+
    +
  • Environment name (required): e.g., prod, uat, staging
    • +
  • MongoDB username (required): The admin user
    • +
  • MongoDB password (required): The admin password
    • +
  • MongoDB host (required): e.g., cluster0.mongodb.net
    • +
  • MongoDB port (required): Usually 27017 or 10255
    • +
  • Application name (required): Used in connection string and dump folder name
    • +
+ +

Subsequent Runs

+ 
./db-dump.sh dump
+ +

The script will show you a list of configured environments and let you choose which one to backup, or create a new environment.

+
+ +
+

Commands

+ 
# Perform a backup of a selected environment
+./db-dump.sh dump
+
+# List all configured environments
+./db-dump.sh list
+
+# Add a new environment
+./db-dump.sh add
+
+# Delete an environment
+./db-dump.sh delete
+
+# Show help
+./db-dump.sh help
+
+ +
+

How It Works

+
    +
  1. Environment Selection: On each run, you select from existing environments or create new ones
    2. +
  2. Credential Storage: Each environment's credentials are stored in .env.<name> files
    3. +
  3. MongoDB Connection: Connects using your credentials with MongoDB Atlas parameters (ssl, replicaSet, etc.)
    4. +
  4. Dump Creation: Uses mongodump to create a backup
    5. +
  5. Timestamped Output: Dumps are organized as ./dumps/<appName>/<yyyymmdd_hhmm>/
    6. +
+
+ +
+

Configuration Files

+ +

Each environment is stored in a separate file:

+ 
.env.prod       # Production environment
+.env.uat        # UAT environment
+.env.staging    # Staging environment (if created)
+ +

Each file contains:

+ 
MONGO_USERNAME=<username>
+MONGO_PASSWORD=<password>
+MONGO_HOST=<host>
+MONGO_PORT=<port>
+APP_NAME=<appName>
+
+ +
+

Directory Structure

+
dbdump/ +├── db-dump.sh # Main script +├── .env.prod # Production config (created by user) +├── .env.uat # UAT config (created by user) +├── .env.example # Example configuration +├── .gitignore # Git ignore rules +├── README.md # This file +└── dumps/ # Created automatically + ├── prod/ + │ ├── 20260414_1632/ + │ │ ├── admin/ + │ │ └── myapp/ + │ └── 20260414_1645/ + └── uat/ + └── 20260414_1632/
+
+ +
+

Dump Output

+ +

Dumps are stored with the following structure:

+ 
dumps/<appName>/<yyyymmdd_hhmi>/
+  admin/           # System databases
+  config/
+  myapp/           # Application databases
+  system.profile/
+ +

Each backup preserves the complete database structure and is ready to be restored using mongorestore.

+
+ +
+

Accessing Dumps

+ +

To view or restore a dump:

+ 
# List recent dumps
+ls -lh dumps/prod/
+
+# Restore a dump to local MongoDB
+mongorestore --uri "mongodb://localhost:27017" dumps/prod/20260414_1632/
+
+# Copy dump to another location
+cp -r dumps/prod/20260414_1632/ /backup/prod-backup-20260414/
+
+ +
+

Security

+
    +
  • Configuration files have restricted permissions (600)
    • +
  • Credentials are stored locally in .env.* files
    • +
  • These files are git-ignored for security
    • +
  • Never commit .env.* files to version control
    • +
+
+ +
+

Requirements

+ +
+ +
+

Installation of MongoDB Database Tools

+ +

macOS

+ 
brew tap mongodb/brew
+brew install mongodb-database-tools
+ +

Linux (Ubuntu/Debian)

+ 
wget https://fastdl.mongodb.org/tools/db/mongodb-database-tools-ubuntu2004-x86_64-100.8.0.tgz
+tar -xzf mongodb-database-tools-ubuntu2004-x86_64-100.8.0.tgz
+sudo cp mongodb-database-tools-ubuntu2004-x86_64-100.8.0/bin/* /usr/local/bin/
+ +

Windows

+

Download the MSI installer from https://www.mongodb.com/try/download/database-tools and run it.

+
+ +
+

Troubleshooting

+ +

mongodump not found

+ 
# Check if MongoDB Database Tools is installed
+mongodump --version
+
+# If not installed, see "Installation" section above
+ +

Connection refused

+
    +
  • Verify host and port are correct
    • +
  • Check network connectivity to MongoDB server
    • +
  • Ensure firewall allows access
    • +
+ +

Authentication failed

+
    +
  • Verify username and password
    • +
  • Check the user has admin permissions
    • +
  • Confirm the authSource=admin parameter
    • +
+ +

Port already displaying in error

+

The connection string is checked for errors. If you see connection errors, the script displays the exact MongoDB error message.

+
+ +
+

Advanced Usage

+ +

Manual mongodump command (for reference)

+ +

The script constructs and runs this command automatically:

+ 
mongodump --uri "mongodb://<username>:<password>@<host>:<port>/?ssl=true&replicaSet=globaldb&retrywrites=false&maxIdleTimeMS=120000&appName=@<appName>@" -o ./dumps/<appName>/<yyyymmdd_hhmm>
+ +

Scheduling automated backups

+

You can schedule regular backups using cron (Linux/macOS):

+ 
# Add to crontab
+crontab -e
+
+# Daily dump at 3 AM for production environment
+0 3 * * * cd /path/to/dbdump && ./db-dump.sh dump << EOF
+1
+EOF
+
+ +
+

Support

+ +

For MongoDB connection issues, refer to:

+ +
+ +
+

About

+

+ Author: Julien Gautier (jgautier.webdev@gmail.com) +

+

+ Organization: AdAstra +

+

+ Project: adastra_scripts +

+

+ This utility was created to enable efficient backup and versioning of MongoDB databases from remote or local servers for AdAstra projects. +

+
+
+ +
+

MongoDB Dump Utility Documentation

+

Generated with care for AdAstra projects

+

+ © 2026 AdAstra - All rights reserved +

+
+
+ + \ No newline at end of file diff --git a/docs/dbrestore_mongo.html b/docs/dbrestore_mongo.html new file mode 100644 index 0000000..4933d0d --- /dev/null +++ b/docs/dbrestore_mongo.html @@ -0,0 +1,546 @@ + + + + + + MongoDB Restore Utility + + + +
+
+

MongoDB Restore Utility

+

Standalone Backup Restoration Tool

+
+ +
+

Table of Contents

+ +
+ +
+
+

+ This directory contains a standalone tool for restoring MongoDB database backups created with db-dump.sh. It provides interactive navigation through backup directories and restores them to a configured MongoDB instance. +

+
+ +
+

Quick Start

+ +

1. Configure Connection Details

+ 
cp .env.example .env
+ +

Edit .env with your MongoDB connection details:

+ 
MONGO_USERNAME=admin
+MONGO_PASSWORD=password
+MONGO_HOST=localhost
+MONGO_PORT=27017
+MONGO_DATABASE=adastradb
+MONGO_AUTHSOURCE=admin
+ +

2. Restore a Backup

+ 
./db-restore.sh
+ +

The script will:

+
    +
  1. Verify MongoDB tools are installed
    2. +
  2. Load your configuration
    3. +
  3. Ask for the path to your backups directory
    4. +
  4. Guide you through the directory structure
    5. +
  5. Show a summary of what will be restored
    6. +
  6. Ask for confirmation
    7. +
  7. Restore the backup
    8. +
+
+ +
+

How It Works

+ +

Configuration

+ +

The script requires a .env file with these variables:

+
    +
  • MONGO_USERNAME - MongoDB admin username
    • +
  • MONGO_PASSWORD - MongoDB admin password
    • +
  • MONGO_HOST - MongoDB server hostname/IP
    • +
  • MONGO_PORT - MongoDB server port
    • +
  • MONGO_DATABASE - Target database name
    • +
  • MONGO_AUTHSOURCE - Authentication source (usually admin)
    • +
+
+ +
+

Interactive Navigation

+ +

The script intelligently navigates through your backup directory structure:

+
dumps/ +├── prod/ - Choose this +│ ├── 20260414_1632/ - Then this +│ │ └── (database folders) - Then this +│ └── 20260414_1645/ +└── uat/ + └── 20260414_1700/
+ +

At each level, the script:

+
    +
  1. Shows available subdirectories
    2. +
  2. Asks you to choose one
    3. +
  3. Continues until it finds .bson files (backup data)
    4. +
+
+ +
+

Backup Detection

+ +

The script automatically stops navigation when it finds a directory containing .bson files, which are the actual backup data files.

+ +

Example backup structure:

+ 
~/Works/dbdump/dumps/
+└── bo-uat-cosmodb/
+    └── 20260414_1732/
+        └── admin/                 - BSON files here
+            ├── system.profile.bson
+            └── system.version.bson
+
+ +
+

Example Use Case

+ +

Given this backup structure:

+ 
~/Works/dbdump/dumps/
+├── bo-prod-cosmodb/
+│   └── 20260414_1600/
+│       └── AdAstra/
+│           ├── collection1.bson
+│           └── collection1.metadata.json
+└── bo-uat-cosmodb/
+    └── 20260414_1732/
+        └── AdAstra/
+            ├── orders.bson
+            └── orders.metadata.json
+ +

Restore process:

+ 
./db-restore.sh
+
+# Script asks: Dumps path: ~/Works/dbdump/dumps
+# Shows: 1) bo-prod-cosmodb  2) bo-uat-cosmodb
+# You choose: 2
+# Shows: 1) 20260414_1732
+# You choose: 1
+# Shows: 1) AdAstra
+# You choose: 1
+# Script finds BSON files and shows summary
+
+ +
+

Restore Summary

+ +

Before executing the restore, the script displays:

+ 
MongoDB Connection:
+  Host:       localhost
+  Port:       27017
+  Username:   admin
+  Database:   adastradb
+  AuthSource: admin
+
+Backup Source:
+  Location: /home/user/Works/dbdump/dumps/bo-uat-cosmodb/20260414_1732
+  Backup files: 42
+  Total size: 250M
+ +

You must confirm with "yes" to proceed.

+
+ +
+

Connection String

+ +

The script constructs the MongoDB connection string as:

+ 
mongodb://<username>:<password>@<host>:<port>/<database>?authSource=<authSource>
+ +

Example:

+ 
mongodb://admin:password@localhost:27017/adastradb?authSource=admin
+
+ +
+

Directory Structure

+
dbrestore/ +├── db-restore.sh # Main script +├── .env # Configuration (created by you, git-ignored) +├── .env.example # Example configuration +├── .gitignore # Git ignore rules +└── README.md # This file
+
+ +
+

Security

+
    +
  • Configuration file (.env) has restricted permissions
    • +
  • Contains sensitive credentials (username, password)
    • +
  • Never commit actual .env file to version control
    • +
  • Use .env.example as template
    • +
+
+ +
+

Requirements

+ +
+ +
+

Installation of MongoDB Database Tools

+ +

macOS

+ 
brew tap mongodb/brew
+brew install mongodb-database-tools
+ +

Linux (Ubuntu/Debian)

+ 
wget https://fastdl.mongodb.org/tools/db/mongodb-database-tools-ubuntu2004-x86_64-100.8.0.tgz
+tar -xzf mongodb-database-tools-ubuntu2004-x86_64-100.8.0.tgz
+sudo cp mongodb-database-tools-ubuntu2004-x86_64-100.8.0/bin/* /usr/local/bin/
+ +

Windows

+

Download the MSI installer from https://www.mongodb.com/try/download/database-tools and run it.

+
+ +
+

Troubleshooting

+ +

mongorestore not found

+ 
# Check if MongoDB Database Tools is installed
+mongorestore --version
+
+# If not installed, see "Installation" section above
+ +

.env file not found

+ 
# Copy the example configuration
+cp .env.example .env
+
+# Edit with your values
+nano .env  # or your favorite editor
+ +

Connection refused

+
    +
  • Verify host and port are correct
    • +
  • Check network connectivity to MongoDB server
    • +
  • Ensure firewall allows access
    • +
  • Verify MongoDB service is running (if local)
    • +
+ +

Authentication failed

+
    +
  • Verify username and password in .env
    • +
  • Check user has sufficient permissions
    • +
  • Confirm authSource is correct (usually admin)
    • +
+ +

No BSON files found

+
    +
  • Make sure you're pointing to the root of dumps directory
    • +
  • Navigate through subdirectories until you reach the folder with actual backup files
    • +
  • BSON files are typically deep in the directory structure
    • +
+
+ +
+

Manual Restore Command

+ +

For reference, here's the command the script executes:

+ 
mongorestore --uri "mongodb://user:pass@host:port/db?authSource=admin" /path/to/dumps
+
+ + + +
+

Workflow Example

+ +

Complete workflow to backup and restore:

+ 
# 1. Backup from production MongoDB (in dbdump folder)
+cd ~/Works/dbdump
+./db-dump.sh dump
+# Select prod environment and wait for backup
+
+ +
+

About

+

+ Author: Julien Gautier (jgautier.webdev@gmail.com) +

+

+ Organization: AdAstra +

+

+ Project: adastra_scripts +

+

+ These utilities were created to simplify MongoDB management during local development of AdAstra projects. +

+
+
+ +
+

MongoDB Restore Utility Documentation

+

Generated with care for AdAstra projects

+

+ © 2026 AdAstra - All rights reserved +

+
+
+ + \ No newline at end of file diff --git a/docs/dbutils_mongo.html b/docs/dbutils_mongo.html new file mode 100644 index 0000000..c744e78 --- /dev/null +++ b/docs/dbutils_mongo.html @@ -0,0 +1,513 @@ + + + + + + MongoDB Database Utility (Standalone) + + + +
+
+

MongoDB Database Utility

+

Standalone Local Development Setup

+
+ +
+

Table of Contents

+ +
+ +
+
+

+ This directory contains a standalone MongoDB setup for local development. It is completely independent from the main backend and frontend projects. +

+
+ +
+

Quick Start

+ +

First Run (Setup)

+ 
./db-start.sh up
+ +

The script will prompt you for:

+
    +
  • MongoDB username (default: admin)
    • +
  • MongoDB password (default: password)
    • +
  • Database name (default: adastradb)
    • +
  • MongoDB port (default: 27017)
    • +
  • Mongo Express port (default: 8081)
    • +
+ +

Your configuration will be stored in .env file (automatically git-ignored).

+
+ +
+

Common Commands

+ 
# Start MongoDB (prompts for credentials on first run)
+./db-start.sh up
+
+# View logs
+./db-start.sh logs
+
+# Stop MongoDB
+./db-start.sh down
+
+# Restart MongoDB
+./db-start.sh restart
+
+# Check container status
+./db-start.sh status
+
+# Reconfigure credentials
+./db-start.sh reconfigure
+
+# Remove all containers and volumes (DATA LOSS!)
+./db-start.sh clean
+
+# Show help
+./db-start.sh help
+
+ +
+

Accessing MongoDB

+ +

From Your Application

+
    +
  • Connection String: mongodb://<username>:<password>@localhost:27017/<database>?authSource=admin
    • +
  • Example: mongodb://admin:password@localhost:27017/adastradb?authSource=admin
    • +
+ +

Using Mongo Express (GUI)

+ +
+ +
+

Data Persistence

+

All data is stored in Docker volumes and persists across container restarts. To permanently delete data, use:

+ 
./db-start.sh clean
+
+ +
+

Collections

+

The database is automatically initialized with the following collection:

+
    +
  • dataImportHistory: Stores records of all data imports for audit and recovery tracking +
      +
    • Indexed on importDate (descending) and status
      • +
    • Ready for future data import functionality
      • +
    +
    • +
+
+ +
+

File Structure

+
dbutils/ +├── db-start.sh # Main script +├── docker-compose-mongodb.yml # Docker Compose configuration +├── mongo-init.js # Database initialization script +├── .env # Configuration (generated on first run, git-ignored) +├── .env.example # Example configuration +├── .gitignore # Git ignore rules +└── README.md # This file
+
+ +
+

Configuration

+

The script uses a .env file to store configuration. You can:

+
    +
  1. Let the script generate it: Run ./db-start.sh up and answer the prompts
    2. +
  2. Pre-create it: Copy .env.example to .env and modify values
    3. +
  3. Edit it manually: Edit .env directly and run ./db-start.sh up
    4. +
  4. Reconfigure: Run ./db-start.sh reconfigure to re-run the setup wizard
    5. +
+
+ +
+

Requirements

+
    +
  • Docker
    • +
  • Docker Compose (included with Docker Desktop)
    • +
  • Bash shell
    • +
+
+ +
+

Standalone Usage

+

This utility can be used independently from AdAstra projects:

+ 
# From anywhere
+cd /path/to/dbutils
+./db-start.sh up
+
+# Or add to PATH and use globally (optional)
+export PATH=$PATH:/path/to/dbutils
+db-start.sh up
+
+ +
+

Troubleshooting

+ +

Port already in use

+

If MongoDB or Mongo Express port is already in use:

+
    +
  1. Run ./db-start.sh reconfigure
    2. +
  2. Choose a different port (e.g., 27018 for MongoDB, 8082 for Mongo Express)
    3. +
+ +

Docker not running

+ 
# Check Docker status
+docker info
+ +

Permission denied

+ 
# Make script executable
+chmod +x db-start.sh
+
+ +
+

Notes

+
    +
  • Credentials are stored in .env (git-ignored for security)
    • +
  • Mongo Express is included for easy database management
    • +
  • Data persists in Docker volumes
    • +
  • No initialization scripts are run (unlike the backend docker-compose)
    • +
  • This is standalone and independent from the main project configuration
    • +
+
+ +
+

About

+

+ Author: Julien Gautier (jgautier.webdev@gmail.com) +

+

+ Organization: AdAstra +

+

+ Project: adastra_scripts +

+

+ These utilities were created to simplify MongoDB management during local development of AdAstra projects. +

+
+
+ +
+

MongoDB Database Utility Documentation

+

Generated with care for AdAstra projects

+

+ © 2026 AdAstra - All rights reserved +

+
+
+ + \ No newline at end of file From 8949a33481b2838f4f53f229f6e425c5d4f327b5 Mon Sep 17 00:00:00 2001 From: Julien Gautier Date: Wed, 29 Apr 2026 00:53:58 +0200 Subject: [PATCH 5/6] docs: add CLAUDE.md with project guidance for AI assistants --- CLAUDE.md | 64 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 64 insertions(+) create mode 100644 CLAUDE.md diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..8c93a3f --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,64 @@ +# CLAUDE.md + +This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. + +## Project Overview + +A collection of standalone Bash utility scripts for MongoDB operations (backup, restore, local dev environment). No build system, package manager, or test suite — each module is a self-contained shell script. + +## Modules + +| Module | Script | Purpose | +|---|---|---| +| `dbdump_mongo/` | `db-dump.sh` | Dump MongoDB databases across multiple named environments | +| `dbrestore_mongo/` | `db-restore.sh` | Restore a MongoDB dump to a target instance | +| `dbutils_mongo/` | `db-start.sh` | Spin up a local MongoDB + Mongo Express via Docker | + +## Running the Scripts + +```bash +# Backup — interactive environment picker +cd dbdump_mongo && bash db-dump.sh + +# Restore — interactive backup directory navigator +cd dbrestore_mongo && bash db-restore.sh + +# Local dev database — Docker-based +cd dbutils_mongo && bash db-start.sh up # start +cd dbutils_mongo && bash db-start.sh down # stop +cd dbutils_mongo && bash db-start.sh status +cd dbutils_mongo && bash db-start.sh logs +cd dbutils_mongo && bash db-start.sh clean # remove volumes +cd dbutils_mongo && bash db-start.sh reconfigure +``` + +## Environment Setup + +Each module requires a `.env` file (copied from the corresponding `.env.example`). The `.env` files are git-ignored. `dbdump_mongo` supports multiple named environments stored as `.env.` files with `chmod 600` permissions. + +## Architecture Patterns + +All three scripts share the same conventions: + +- **Sub-command dispatch**: a `main()` function routes `$1` to `cmd_()` functions. +- **Colored output**: ANSI escape codes (`\033[...m`) for styled terminal output; consistent separator lines. +- **Prerequisite checks**: each script validates its required tools (`mongodump`, `mongorestore`, `docker`, `docker compose`) at startup and exits early with a clear message if missing. +- **Interactive prompts**: `read -p` / `select` menus guide the user through environment selection, directory navigation, and confirmation steps before any destructive action. + +### dbdump_mongo — multi-environment credential model + +Named environments are stored as `.env.` files alongside the main `.env`. Credentials include `MONGO_USERNAME`, `MONGO_PASSWORD`, `MONGO_HOST`, `MONGO_PORT`, `APP_NAME`. Dumps land in `./dumps///`. The script generates MongoDB Atlas-compatible connection strings (ssl, replicaSet flags). + +### dbrestore_mongo — backup directory navigation + +The script walks through backup directory hierarchies interactively. It detects valid restore targets by checking for `.bson` files (`has_bson_files()`). Before executing `mongorestore`, it shows a summary of connection details and the selected backup path and requires explicit confirmation. + +### dbutils_mongo — Docker Compose orchestration + +Runs two containers on a `dbutils-network` bridge: MongoDB (configurable port, default `MONGO_PORT`) and Mongo Express (default port `8081`). Data is persisted in named volumes `mongodb-data` and `mongodb-config`. On first run, `setup_credentials()` prompts interactively and writes `.env`. The `mongo-init.js` initializer creates a `dataImportHistory` collection with indexes on `importDate` and `status`. + +## External Dependencies + +- `mongodump` / `mongorestore` — MongoDB Database Tools (must be installed separately) +- `docker` + `docker compose` (v2 plugin syntax) +- `bash` (scripts use `bash`-specific features, not POSIX `sh`) From ff6c2205d1000c6234e44aa702003490a9d5c0b8 Mon Sep 17 00:00:00 2001 From: Julien Gautier Date: Wed, 29 Apr 2026 00:55:26 +0200 Subject: [PATCH 6/6] docs: update README with projects decriptions --- README.md | 39 +++++++++++++++++++++++++++++++++++++-- 1 file changed, 37 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index f122317..f0e04fe 100644 --- a/README.md +++ b/README.md @@ -1,3 +1,38 @@ -# adastra_scripts +# AdAstra Scripts -AdAstra Utility Scripts \ No newline at end of file +A collection of utility scripts for AdAstra projects. Each script is self-contained and can be used independently. + +## Scripts + +### MongoDB + +| Script | Description | +|---|---| +| [`dbdump_mongo/`](dbdump_mongo/) | Backup MongoDB databases with multi-environment support (prod, uat, staging, …) | +| [`dbrestore_mongo/`](dbrestore_mongo/) | Restore a MongoDB backup with interactive directory navigation | +| [`dbutils_mongo/`](dbutils_mongo/) | Spin up a local MongoDB + Mongo Express instance via Docker | + +#### Typical MongoDB workflow + +```bash +# 1. Backup from a remote MongoDB +cd dbdump_mongo && ./db-dump.sh dump + +# 2. Start a local MongoDB instance +cd dbutils_mongo && ./db-start.sh up + +# 3. Restore the backup locally +cd dbrestore_mongo && ./db-restore.sh +``` + +## Requirements + +Each script lists its own requirements in its README. Common dependencies: + +- **Bash** +- **MongoDB Database Tools** (`mongodump` / `mongorestore`) — [download](https://www.mongodb.com/try/download/database-tools) +- **Docker & Docker Compose** (for `dbutils_mongo` only) + +## Author + +Julien Gautier — AdAstra