SHADOW METRICS & DISSONANCE RESEARCH SERVICES RUNBOOK

🎯 EXECUTIVE SUMMARY

Two complementary monitoring services:

• Shadow Metrics (9010): Production-grade, simple monitoring
• Dissonance Research (9011): Experimental, rigorous time-series research platform

---

🔧 SERVICE 1: HELIX SHADOW METRICS (PRODUCTION)

Basic Info

• Port: 9010
• Purpose: Basic system monitoring and dissonance tracking
• Status: Production, stable
• Config: /etc/systemd/system/helix-shadow-metrics.service

API Endpoints

bash

# Health/status check
curl http://127.0.0.1:9010/status

# Response:
{
  "server_time_utc": "2025-11-11T20:01:57.881672Z",
  "version": "1.1.0", 
  "uptime_seconds": 1192,
  "dissonance": {
    "value": 0.12,
    "timestamp": "2025-11-11T19:42:45Z",
    "source": "auto-seed",
    "ip": "127.0.0.1"
  }
}

Troubleshooting

bash

# Check service status
sudo systemctl status helix-shadow-metrics.service

# Common issue: Permission denied on /opt/helix/
sudo chown helix:helix /opt/helix/

# Restart service
sudo systemctl restart helix-shadow-metrics.service

# Check logs
sudo journalctl -u helix-shadow-metrics.service -n 10

Configuration

• Seed Value: SEED_VALUE=0.12 in env.conf
• Auto-heal: AUTO_HEAL=0 (disabled)
• Dissonance Threshold: Δ_t > 0.3 triggers audit trails

---

🔬 SERVICE 2: HELIX DISSONANCE RESEARCH (EXPERIMENTAL)

Basic Info

• Port: 9011
• Purpose: Advanced time-series research and experimentation
• Status: Experimental, strict data validation
• Config: /etc/systemd/system/helix-dissonance.service

API Endpoints

bash

# Status check
curl http://127.0.0.1:9011/status

# Submit time data (STRICT SCHEMA REQUIRED)
curl -X POST http://127.0.0.1:9011/time \
  -H "Content-Type: application/json" \
  -d '{"iso_timestamp": "2025-11-11T20:20:00Z", "source": "test", "timezone": "UTC", "human": "experiment_name"}'

# Retrieve last time entry
curl http://127.0.0.1:9011/last_time

# API Documentation
curl http://127.0.0.1:9011/docs

Required Data Schema

json

{
  "iso_timestamp": "2025-11-11T20:20:00Z",  // ISO 8601 required
  "timezone": "UTC",                         // Timezone context
  "source": "experiment_source",             // Data provenance
  "human": "Experiment description"          // Human-readable ID
}

Troubleshooting

bash

# Common issue: Port conflict (uses 9011 to avoid conflict with 9010)
sudo systemctl edit helix-dissonance.service
# Change port in ExecStart: --port 9011

# Common issue: Truncated ExecStart command
# Fix in: /etc/systemd/system/helix-dissonance.service.d/z99-execstart.conf
# Ensure line ends with: --workers 1

# Restart service
sudo systemctl restart helix-dissonance.service

---

🚨 COMMON ISSUES & SOLUTIONS

Service Won't Start

bash

# Check directory permissions
sudo ls -la /opt/helix/

# Check if user can access directory  
sudo -u helix ls -la /opt/helix/

# Fix permissions if needed
sudo chown helix:helix /opt/helix/

Port Conflicts

bash

# Check what's using port 9010/9011
sudo netstat -tlnp | grep 901

# Change port in service config
sudo systemctl edit [service-name]

Missing Dissonance Data

bash

# Seed file might be missing
sudo /opt/helix/bin/seed-dissonance.sh

# Check if /run/helix/ exists
sudo ls -la /run/helix/

---

📊 MONITORING & HEALTH CHECKS

Daily Health Check Script

bash

#!/bin/bash
# Check both services
curl -s http://127.0.0.1:9010/status | grep -q '"version":"1.1.0"' && echo "Shadow Metrics: OK" || echo "Shadow Metrics: FAILED"
curl -s http://127.0.0.1:9011/status | grep -q '"ok":true' && echo "Dissonance Research: OK" || echo "Dissonance Research: FAILED"

Key Metrics to Monitor

• Service uptime
• Dissonance values
• API response times
• Error rates in logs

---

🔄 DEPLOYMENT & MAINTENANCE

Service Updates

bash

# Reload systemd after config changes
sudo systemctl daemon-reload

# Restart services
sudo systemctl restart helix-shadow-metrics.service
sudo systemctl restart helix-dissonance.service

# Enable on boot
sudo systemctl enable helix-shadow-metrics.service
sudo systemctl enable helix-dissonance.service

Backup Considerations

• Research service stores time series data (check persistence)
• Shadow metrics is stateless
• Backup service configs in /etc/systemd/system/

---

🎯 QUICK REFERENCE

Command	Purpose
sudo systemctl status helix-shadow-metrics.service	Check production service
sudo systemctl status helix-dissonance.service	Check research service
curl http://127.0.0.1:9010/status	Test production API
curl http://127.0.0.1:9011/status	Test research API
sudo journalctl -u helix-*-service -n 10	Check recent logs