Added new rom import system utilizing WAL to avoid locking the database and freezing the frontend

Also added new logging setup to hopefully stream the scrape process

DOCKER.md

@@ -1,112 +0,0 @@
-# DosVault Docker Deployment
-
-## Quick Start
-
-1. **Copy the environment template:**
-   ```bash
-   cp .env.example .env
-   ```
-
-2. **Edit `.env` with your configuration:**
-   - Set `IGDB_CLIENT_ID` and `IGDB_SECRET_KEY` (required)
-   - Set `ROMS_PATH` to your ROM collection directory
-	 - Set `DOSVAULT_ADMIN_USERNAME` to you admin username
-	 - Set `DOSVAULT_ADMIN_EMAIL` to your admin email
-	 - Set `DOSVAULT_ADMIN_PASSWORD` to your admin password
-   - Optionally customize host/port settings
-
-3. **Start the application:**
-   ```bash
-   docker-compose up -d
-   ```
-
-4. **Create admin user:**
-   ```bash
-   docker-compose exec dosvault python src/create_admin.py
-   ```
-
-5. **Access the application:**
-   - Web interface: http://localhost:8080
-   - Admin panel: http://localhost:8080/admin
-
-## Configuration
-
-### Environment Variables
-
-| Variable | Required | Description |
-|----------|----------|-------------|
-| `IGDB_CLIENT_ID` | Yes | Twitch API Client ID |
-| `IGDB_SECRET_KEY` | Yes | Twitch API Client Secret |
-| `ROMS_PATH` | No | Path to ROM collection (default: ./roms) |
-| `DOSFRONTEND_CONFIG_DIR` | No | Application data directory (default: /app/data) |
-
-### Configuration Persistence
-
-Configuration changes made through the web interface are automatically persisted to the mounted volume:
-
-- **In Docker**: Configuration is stored in `/app/data/config.json` (mounted volume)
-- **Regular install**: Configuration is stored in `~/.config/dosfrontend/config.json`
-- **File structure**: All application data uses the same base directory:
-  - `config.json` - Main configuration file
-  - `roms.db` - SQLite database
-  - `images/` - Downloaded game artwork
-  - `logs/` - Application logs
-
-### Volume Mounts
-
-- `dosvault_data:/app/data` - Application data (database, images, logs)
-- `${ROMS_PATH}:/app/data/roms:ro` - ROM collection (read-only)
-
-## Database Management
-
-### Initialize Database
-```bash
-docker-compose exec dosvault python src/migrate.py init
-```
-
-### Run Migrations
-```bash
-docker-compose exec dosvault python src/migrate.py upgrade
-```
-
-### Scrape ROM Metadata
-```bash
-docker-compose exec dosvault python -m src
-```
-
-## Maintenance
-
-### View Logs
-```bash
-docker-compose logs -f dosvault
-```
-
-### Backup Database
-```bash
-docker-compose exec dosvault cp /app/data/roms.db /app/data/backup.db
-docker cp $(docker-compose ps -q dosvault):/app/data/backup.db ./backup.db
-```
-
-### Update Application
-```bash
-docker-compose pull
-docker-compose up -d
-```
-
-## Troubleshooting
-
-### Check Container Health
-```bash
-docker-compose ps
-```
-
-### Access Container Shell
-```bash
-docker-compose exec dosvault bash
-```
-
-### Reset Data
-```bash
-docker-compose down -v
-docker-compose up -d
-```