🧪 Security Fixes Testing Guide

Quick Test Checklist

Run these tests to verify all security fixes are working correctly.

---

✅ Test 1: Verify SSH Keys Are Ignored by Git

Goal: Confirm SSH keys won't be committed

Steps:

# 1. Check if keys are tracked by git
git ls-files | Select-String "key|\.key|_key"

# 2. Create a test key file (to verify .gitignore works)
New-Item -Path "config/deploy/test_key" -ItemType File -Force
New-Item -Path "config/deploy/test_key.pub" -ItemType File -Force

# 3. Check git status - these files should NOT appear
git status

# 4. Clean up test files
Remove-Item "config/deploy/test_key"
Remove-Item "config/deploy/test_key.pub"

Expected Result: - No key files should appear in git status - If you see key files, .gitignore isn't working properly

---

✅ Test 2: Verify Hardcoded Secrets Are Removed

Goal: Confirm docker-compose.yml requires environment variables

Steps:

# 1. Check docker-compose.yml for hardcoded defaults
Select-String -Path "docker-compose.yml" -Pattern "API_KEY.*:-|POSTGRES_PASSWORD.*:-"

# 2. Try to start without env vars (should fail)
# First, backup your .env
Copy-Item .env .env.backup

# Temporarily rename .env
Rename-Item .env .env.temp

# Try docker-compose (should fail or show errors)
docker-compose config

# Restore .env
Rename-Item .env.temp .env

Expected Result: - No matches in step 1 (no hardcoded defaults) - Step 2 should show errors about missing environment variables

---

✅ Test 3: Verify Supabase Key Requires Environment Variable

Goal: Confirm frontend fails gracefully without env var

Steps:

# 1. Check if hardcoded key is removed
Select-String -Path "frontend/src/config/supabase.ts" -Pattern "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9"

# 2. Temporarily remove env var and test
# In frontend/.env or root .env, comment out:
# VITE_SUPABASE_ANON_KEY=...

# 3. Try to build/start frontend
cd frontend
npm run dev
# Should show error about missing VITE_SUPABASE_ANON_KEY

Expected Result: - No hardcoded key found in step 1 - Frontend should fail with clear error message about missing env var

---

✅ Test 4: Verify Authentication Is Always Required

Goal: Confirm no auth bypass in development mode

Steps:

# 1. Check middleware.py for dev bypass
Select-String -Path "backend/middleware.py" -Pattern "development.*unauthenticated|ENVIRONMENT.*development.*call_next"

# 2. Test API endpoint without auth token
# Start your backend server
# Then in another terminal:
$headers = @{
    "Content-Type" = "application/json"
}
# Don't include Authorization header
Invoke-RestMethod -Uri "http://localhost:8000/api/health" -Method GET -Headers $headers

# 3. Test with invalid token
$headers = @{
    "Authorization" = "Bearer invalid_token_12345"
    "Content-Type" = "application/json"
}
try {
    Invoke-RestMethod -Uri "http://localhost:8000/api/components/search" -Method POST -Headers $headers -Body '{}'
} catch {
    Write-Host "Status: $($_.Exception.Response.StatusCode.value__)"
    Write-Host "Expected: 401 Unauthorized"
}

Expected Result: - Step 1: No matches (dev bypass removed) - Step 2: Should get 401 Unauthorized (or endpoint requires auth) - Step 3: Should get 401 Unauthorized

---

✅ Test 5: Verify CSP Headers Are Applied

Goal: Confirm Content Security Policy is working

Steps:

# 1. Check CSP doesn't have unsafe-inline/unsafe-eval for scripts
Select-String -Path "backend/main.py" -Pattern "script-src.*unsafe-inline|script-src.*unsafe-eval"

# 2. Start your backend server
# Then check headers:
$response = Invoke-WebRequest -Uri "http://localhost:8000/health" -Method GET
$response.Headers["Content-Security-Policy"]

# 3. Check browser console (if frontend is running)
# Open browser DevTools → Console
# Look for CSP violations (should be minimal/none)

Expected Result: - Step 1: Should NOT find unsafe-inline or unsafe-eval in script-src - Step 2: Should see CSP header in response - Step 3: No CSP violation errors in console

---

✅ Test 6: Verify CORS Restrictions

Goal: Confirm CORS only allows specific origins

Steps:

# 1. Check CORS middleware doesn't allow all origins
Select-String -Path "backend/middleware/cors.js" -Pattern "callback\(null, true\)" -Context 2

# 2. Test from unauthorized origin
$headers = @{
    "Origin" = "https://malicious-site.com"
    "Content-Type" = "application/json"
}
try {
    Invoke-RestMethod -Uri "http://localhost:8000/api/health" -Method GET -Headers $headers
} catch {
    Write-Host "CORS blocked: $($_.Exception.Message)"
}

# 3. Test from allowed origin (localhost)
$headers = @{
    "Origin" = "http://localhost:5173"
    "Content-Type" = "application/json"
}
$response = Invoke-RestMethod -Uri "http://localhost:8000/api/health" -Method GET -Headers $headers
Write-Host "Allowed origin works: $($response -ne $null)"

Expected Result: - Step 1: Should show restricted origins, not wildcard - Step 2: Should be blocked (CORS error) - Step 3: Should work (allowed origin)

---

✅ Test 7: Verify Webhook Signature Enforcement

Goal: Confirm webhooks require valid signatures

Steps:

# 1. Check webhook code requires signature
Select-String -Path "backend/routes/webhooks.py" -Pattern "SUPABASE_WEBHOOK_SECRET.*not set.*reject|raise.*401.*webhook"

# 2. Test webhook without signature
$headers = @{
    "Content-Type" = "application/json"
}
$body = @{
    type = "INSERT"
    table = "users"
    record = @{
        id = "test-123"
        email = "test@example.com"
    }
} | ConvertTo-Json

try {
    Invoke-RestMethod -Uri "http://localhost:8000/api/webhooks" -Method POST -Headers $headers -Body $body
} catch {
    Write-Host "Status: $($_.Exception.Response.StatusCode.value__)"
    Write-Host "Expected: 401 if secret not set, or 401 if signature invalid"
}

Expected Result: - Step 1: Should find code that rejects webhooks without secret - Step 2: Should get 401 error

---

✅ Test 8: Verify Error Messages Are Sanitized

Goal: Confirm production doesn't expose internal errors

Steps:

# 1. Check error handler code
Select-String -Path "backend/main.py" -Pattern "is_production.*HTTPException|detail.*internal error"

# 2. Set ENVIRONMENT=production temporarily
$env:ENVIRONMENT = "production"

# 3. Trigger an error (if you have a test endpoint that throws)
# Check response - should be generic message, not full traceback

# 4. Restore environment
$env:ENVIRONMENT = "development"

Expected Result: - Step 1: Should find production error sanitization code - Step 3: Error response should be generic, not exposing internals

---

✅ Test 9: Run Package Security Scan

Goal: Check for vulnerable dependencies

Steps:

# 1. Scan root package.json
cd .
npm audit

# 2. Scan frontend package.json
cd frontend
npm audit

# 3. Fix automatically (if safe)
npm audit fix

# 4. Check for high/critical issues
npm audit --audit-level=high

Expected Result: - Should show list of vulnerabilities - Fix any high/critical issues

---

✅ Test 10: Full Integration Test

Goal: Verify everything works together

Steps:

# 1. Start backend with proper env vars
cd backend
# Make sure .env has all required variables
python main.py

# 2. Start frontend
cd ../frontend
npm run dev

# 3. Test authentication flow
# - Try to access protected endpoint without token → should fail
# - Login with valid credentials → should work
# - Access protected endpoint with token → should work

# 4. Check browser console for errors
# - No CSP violations
# - No CORS errors (from allowed origins)
# - No authentication errors (when properly authenticated)

Expected Result: - All services start without errors - Authentication works correctly - No security-related errors in console

---

🎯 Quick Verification Script

Run this PowerShell script to test everything at once:

# Security Fixes Verification Script
Write-Host "🔒 Testing Security Fixes..." -ForegroundColor Cyan
Write-Host ""

# Test 1: SSH Keys
Write-Host "Test 1: Checking SSH keys in git..." -ForegroundColor Yellow
$keysInGit = git ls-files | Select-String "key|\.key"
if ($keysInGit) {
    Write-Host "❌ FAIL: SSH keys found in git!" -ForegroundColor Red
    $keysInGit
} else {
    Write-Host "✅ PASS: No SSH keys in git" -ForegroundColor Green
}

# Test 2: Hardcoded secrets
Write-Host "`nTest 2: Checking for hardcoded secrets..." -ForegroundColor Yellow
$hardcoded = Select-String -Path "docker-compose.yml" -Pattern "API_KEY.*:-|POSTGRES_PASSWORD.*:-"
if ($hardcoded) {
    Write-Host "❌ FAIL: Hardcoded secrets found!" -ForegroundColor Red
    $hardcoded
} else {
    Write-Host "✅ PASS: No hardcoded secrets" -ForegroundColor Green
}

# Test 3: Supabase key
Write-Host "`nTest 3: Checking Supabase key..." -ForegroundColor Yellow
$supabaseKey = Select-String -Path "frontend/src/config/supabase.ts" -Pattern "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9"
if ($supabaseKey) {
    Write-Host "❌ FAIL: Hardcoded Supabase key found!" -ForegroundColor Red
} else {
    Write-Host "✅ PASS: No hardcoded Supabase key" -ForegroundColor Green
}

# Test 4: Auth bypass
Write-Host "`nTest 4: Checking auth bypass..." -ForegroundColor Yellow
$authBypass = Select-String -Path "backend/middleware.py" -Pattern "ENVIRONMENT.*development.*call_next|development.*unauthenticated"
if ($authBypass) {
    Write-Host "❌ FAIL: Auth bypass found!" -ForegroundColor Red
    $authBypass
} else {
    Write-Host "✅ PASS: No auth bypass" -ForegroundColor Green
}

# Test 5: CSP
Write-Host "`nTest 5: Checking CSP..." -ForegroundColor Yellow
$unsafeCSP = Select-String -Path "backend/main.py" -Pattern "script-src.*unsafe-inline|script-src.*unsafe-eval"
if ($unsafeCSP) {
    Write-Host "⚠️  WARNING: Unsafe CSP directives found" -ForegroundColor Yellow
    $unsafeCSP
} else {
    Write-Host "✅ PASS: CSP is secure" -ForegroundColor Green
}

# Test 6: CORS
Write-Host "`nTest 6: Checking CORS..." -ForegroundColor Yellow
$corsWildcard = Select-String -Path "backend/middleware/cors.js" -Pattern "callback\(null, true\)" -Context 0,2 | Where-Object { $_.Context.PostContext -match "allow.*all|development.*all" }
if ($corsWildcard) {
    Write-Host "⚠️  WARNING: CORS may allow all origins in dev" -ForegroundColor Yellow
} else {
    Write-Host "✅ PASS: CORS is restricted" -ForegroundColor Green
}

Write-Host "`n✅ Security verification complete!" -ForegroundColor Cyan

---

📋 Test Results Checklist

After running all tests, check off what works:

• SSH keys are ignored by git
• No hardcoded secrets in docker-compose.yml
• Supabase key requires environment variable
• Authentication is always required
• CSP headers are applied (no unsafe-inline/eval for scripts)
• CORS only allows specific origins
• Webhooks require signatures
• Error messages are sanitized in production
• No critical package vulnerabilities
• Full integration test passes

---

🚨 If Tests Fail

1. SSH keys in git: Check .gitignore and remove files from git history
2. Hardcoded secrets: Remove defaults from docker-compose.yml
3. Auth bypass: Remove development mode bypass code
4. CSP issues: Check if frontend needs inline scripts (may need refactoring)
5. CORS issues: Verify allowed origins list is correct

---

💡 Manual Browser Testing

1. Open DevTools Console:
2. Check for CSP violations
3. Check for CORS errors

4. Check for authentication errors

5. Test Authentication:

6. Try accessing protected routes without login → should redirect/fail
7. Login with valid credentials → should work

8. Access protected routes after login → should work

9. Test API Calls:

10. Make request without token → should get 401
11. Make request with invalid token → should get 401
12. Make request with valid token → should work