Helix Core Ethos – TPAF Runbook v1.0

From Helix Project Wiki

Helix Core Ethos – Runbook v1.0

Purpose-Built for the "Two-Party Approval Flow (TPAF)" All steps are deterministic, auditable, and require explicit human confirmation before any irreversible action is taken.

1. Overview

Item Description
Runbook ID HCE-TPAF-RB-001
Version 1.0 (2025-09-15)
Effective Date 2025-09-15
Owner Helix Core Ethos – Governance Team
Stakeholders Product Ops, Security, Compliance, Engineering, Legal, End-User Support
Scope End-to-end execution of a high-impact intent (e.g., data-model training, fund transfer, policy change) that requires two-party (requester + approver) consent before any irreversible effect occurs.
Related Docs
  • Helix Core Ethos – Principles (TTD-first, Consent & Least-Privilege)
  • API Spec – intent.json-ld schema
  • Smart-Contract ABI – TPAF.sol
  • Data-Retention Profiles (see Appendix A)
Assumptions
  • All participants have an HSM-backed signing key registered on the Besu ledger.
  • Network connectivity to the Besu node is verified (health-check passed).
  • The "Intent" payload is human-readable and stored in an immutable object store (e.g., IPFS, S3 with Object-Lock).
Exclusions Routine low-impact intents (Tier 0) that do not require two-party approval.

All actions are logged with a tamper-evident receipt (Merkle proof) that is automatically emailed to the requester, approver, and compliance officer.

2. Roles & Responsibilities

Role Primary Responsibilities Required Access
Requester Drafts the intent, signs the initial payload, submits to the approval queue. REQUESTER key, read-only ledger view.
Approver Reviews intent, validates compliance, signs to grant consent. APPROVER key, write-access to ledger for approval transaction.
Ops Engineer Executes the runbook, monitors ledger inclusion, triggers downstream actions after confirmation. Full node admin, CI/CD pipeline trigger.
Compliance Officer Verifies that the intent satisfies all regulatory & policy checks before approval. Read-only audit logs, policy-engine access.
Security Analyst Confirms that HSM keys are healthy, validates cryptographic material, monitors for anomalies. HSM admin, security monitoring dashboard.
Incident Commander (on-call) Handles any failure, escalates per the Incident Response Playbook. Full system access.

3. Prerequisites

1. Infrastructure

  • Besu private network node (v22.7.0+) reachable at https://besu.node.company:8545
  • Object-store bucket with Object-Lock enabled (write-once, read-many)
  • CI/CD pipeline (GitHub Actions / GitLab CI) with a deploy-tpa-job that can invoke the smart-contract

2. Credentials

  • Requester HSM key (req-key-<UUID>) loaded in the secure signing service
  • Approver HSM key (app-key-<UUID>) loaded in the secure signing service
  • Service-account token for the object store with least-privilege (write-once only)

3. Software

  • node >=18.0 with web3.js library
  • docker >=24 (used for deterministic container execution)
  • jq (for JSON-LD manipulation)

4. Documentation

  • Latest Intent JSON-LD schema (v2.1)
  • Compliance checklist (Appendix B)

5. Human Confirmation

  • A signed statement from the Approver acknowledging that they have read the intent and that the action is irreversible. This must be captured before any transaction is submitted to the ledger.

4. Procedure – Step-by-Step

All steps are to be executed in the order presented.
Any deviation must be recorded in the "Deviation Log" (Section 7).

4.1 Intent Creation (Requester)

# Action Command / UI Expected Output Verification
4.1.1 Populate intent JSON-LD template curl -X GET https://api.company/intents/template > intent.json intent.json containing placeholders Open file, confirm placeholders are replaced.
4.1.2 Fill required fields (purpose, tier, data-ids, retention-profile) Edit with vim/IDE Fully populated JSON-LD Run jq . intent.json → no null values.
4.1.3 Validate against schema npm run validate-intent intent.json Validation passed If failure → abort, fix, re-validate.
4.1.4 Store immutable copy aws s3api put-object --bucket core-intents --key $(uuidgen).json --body intent.json --object-lock-mode COMPLIANCE --object-lock-retain-until-date $(date -d "+5 years" +%Y-%m-%d) S3 URL (e.g., s3://core-intents/abcd.json) Confirm HTTP 201 response, record URL.
4.1.5 Generate initial signature hsm-sign --key req-key-<UUID> --payload intent.json > req.sig req.sig (Base64) Verify hsm-verify --key req-key-<UUID> --payload intent.json --signature req.sigvalid.
4.1.6 Assemble Intent Package (JSON-LD + signature + S3 URL) jq -s '{intent: .[0], requester_sig: .[1], storage_url: .[2]}' intent.json req.sig s3url.txt > intent_pkg.json intent_pkg.json Run cat intent_pkg.json → human-readable fields visible.
4.1.7 Notify Approver (email or ticket) sendmail -s "TPAF Approval Required" approver@company.com < intent_pkg.json Email delivered (track via mail logs) Approver acknowledges receipt.

4.2 Approver Review & Consent

# Action Command / UI Expected Output Verification
4.2.1 Download Intent Package curl -O https://storage.company/abcd.json Local copy intent_pkg.json Check file hash matches S3 ETag.
4.2.2 Verify Requester signature hsm-verify --key req-key-<UUID> --payload intent.json --signature req.sig valid If invalid → reject, flag.
4.2.3 Run Compliance Checklist (Appendix B) Manual UI checklist All items marked ✅ Any ❌ → abort, communicate to requester.
4.2.4 Human Confirmation – sign statement Open Consent Form (HTML) → tick "I have read the intent and understand the irreversible impact." → click Signhsm-sign --key app-key-<UUID> --payload intent_pkg.json > app.sig app.sig (Base64) Verify with hsm-verify – success required to proceed.
4.2.5 Assemble signed package jq -s '{intent_pkg: .[0], approver_sig: .[1]}' intent_pkg.json app.sig > signed_pkg.json signed_pkg.json Visual inspection of both signatures.
4.2.6 Notify Ops & archive signed package aws s3 cp signed_pkg.json s3://signed-intents/$(date +%Y)/$(uuidgen).json --storage-class GLACIER HTTP 200 Artefact archived immutably.

4.3 Ledger Transaction (Ops Engineer)

# Action Command Expected Output
4.3.1 Submit approval transaction web3 sendTransaction --from app-key-<UUID> --data 0x… Transaction hash (e.g., 0xabc123…)
4.3.2 Wait for inclusion watch -n 1 "eth_getTransactionReceipt 0xabc123" Receipt with "status": "1"
4.3.3 Verify receipt status = 1 audit-log insert --type receipt --metadata '{"txHash":"0xabc123","status":"1"}' Log entry ID
4.3.4 Send receipt email (to all parties) sendmail -s "TPAF Receipt" requester@company.com,approver@company.com,compliance@company.com < receipt.json Email sent, SMTP logs show success
4.3.5 Gate external irreversible API Only after (a) ledger receipt status = 1 and (b) receipt email confirmed and (c) Ops dashboard shows "Human verification OK" External API invoked once all three conditions are satisfied
4.3.6 Archive execution artefacts aws s3 cp receipt.json s3://retention-vault/$(date +%Y)/$(uuidgen).json --storage-class GLACIER HTTP 200
4.3.7 Update Dashboard status → "Completed" UI click → API PATCH /tpa/{runId} {"status":"completed"} UI reflects change

4.4 Post-Execution – Clean-up & Retention

# Action Command Expected Output
4.4.1 Archive all artefacts to Retention Vault aws s3 cp intent_pkg_signed.json s3://retention-vault/$(date +%Y)/$(uuidgen).json --storage-class GLACIER HTTP 200
4.4.2 Record Retention Profile (Appendix A) audit-log insert --type retention --metadata '{"profile":"high-impact","expires":"2029-09-15"}' Log entry ID
4.4.3 Schedule automated purge Cron entry: 0 0 1 * * /usr/local/bin/purge-retention.sh high-impact Script runs nightly
4.4.4 Update Dashboard status to "Closed" PATCH /tpa/{runId} {"status":"closed"} UI shows "Closed"

5. Monitoring & Alerts

Metric Threshold Alert Destination Description
Ledger inclusion latency > 30 seconds per block Ops Slack #tpa-alerts Indicates network congestion
Signature verification failures ≥ 1 per run Security PagerDuty Potential key compromise
Email delivery failure > 0 per run Compliance Email Queue Must be resolved before action
Retention-Vault storage error Any error code Ops Email Prevents data loss
Unauthorized access attempts Any Security SOC Audit-log breach detection

All alerts are automatically correlated with the Run ID (UUID generated at step 4.1.1) for traceability.

6. Failure Handling & Rollback

Failure Point Immediate Action Escalation Path Evidence to Capture
Schema validation error (4.1.3) Abort, return to Requester with error details None (requester handles) Validation log, intent JSON
Signature verification failure (4.2.2 / 4.3.2) Halt, flag key for rotation Security Analyst → Incident Commander HSM audit logs, failed signature bytes
Compliance checklist ❌ (4.2.3) Reject, send remediation guide Compliance Officer → Legal Checklist screenshot, notes
Ledger transaction not mined (4.3.5) Retry up to 3 times with increased gas (max +20%) Ops Engineer → Incident Commander Transaction hash, node logs
Receipt email bounce (4.3.7) Re-send using alternative channel (SMS, secure portal) Ops Engineer → Compliance SMTP bounce code, re-send logs
External API error (4.3.8) Do not retry automatically. Notify Approver & Requester Incident Commander → Product Owner API response body, request payload

No irreversible external action is ever taken unless all three success conditions (ledger receipt, email confirmation, human verification) are met.

If a failure occurs after the external API was called (unlikely because of gating), the Incident Commander must:

  1. Initiate a compensating transaction on the ledger (if contract supports reversal)
  2. Document the incident in the Post-mortem Report (Section 9)

7. Deviation Log

Run-ID Deviation Description Reason Approved By Date Mitigation
uuid-example-001 Used a higher gas limit (250 k) Network congestion flagged by monitoring Security Lead 2025-09-20 Updated CI script; recorded in audit

All deviations must be recorded here before the run is closed.

8. Auditing & Evidence

Artefact Storage Location Retention Period Access Controls
Original Intent JSON-LD S3 core-intents/ (Object-Lock) 5 years (per policy) Read-only to Compliance, Ops
Signed Package S3 signed-intents/ (Object-Lock) 5 years Same as above
Ledger Transaction Receipt Besu node archive + receipt.json in Vault 7 years Read-only to Auditors
Email receipts Secure mail archive (eDiscovery) 7 years Legal hold enabled
Audit-log entries PostgreSQL audit_log DB 10 years Role-based access (Auditor)

All artefacts are hash-linked (Merkle root stored on-chain) to guarantee integrity.

9. Post-mortem & Continuous Improvement

  1. Schedule a post-mortem meeting within 48 h of any "Failed" or "Exception" run
  2. Root-cause analysis using the Deviation Log, audit logs, and HSM logs
  3. Action items are recorded in the product backlog and tagged tpa-improvement
  4. Metrics review (average latency, failure rate) on a quarterly basis

10. Appendices

Appendix A – Data-Retention Profiles

Profile Description Minimum Retention Max Retention Delete-Trigger
low-impact Tier 0, non-critical metadata 30 days 1 year Manual request
medium-impact Tier 1-2, analytics results 180 days 3 years Automatic purge after 3 years
high-impact Tier 3-4, model weights, fund transfers 5 years 7 years Automatic purge after 7 years (unless legal hold)
regulatory-hold Any data subject to legal subpoena Indefinite Indefinite Legal department override

Appendix B – Compliance Checklist

(must be 100% satisfied before Approver signs)

  1. Purpose Limitation – Intent purpose aligns with documented business need
  2. Data Minimization – Only data IDs required for the action are listed
  3. Consent Verification – All data subjects have provided explicit consent (if PII)
  4. Regulatory Mapping – Confirmed mapping to GDPR, PIPEDA, HIPAA, etc. (if applicable)
  5. Risk-Severity Rating – Tier correctly assigned per "Risk-Severity Matrix"
  6. Retention Alignment – Retention profile matches the tier
  7. Security Review – No known vulnerabilities in the external API endpoint
  8. Audit Trail Confirmation – All prior steps logged with immutable hashes

Each item is checked off in the UI; the Approver cannot sign until the checklist is fully ✅.

Appendix C – Sample Receipt (JSON-LD)

{
  "@context": "https://schema.org/",
  "@type": "Receipt",
  "identifier": "0xabc123…",
  "dateIssued": "2025-09-15T14:23:07Z",
  "transactionHash": "0xabc123…",
  "requester": "req-key-7f9c-…",
  "approver": "app-key-3a1e-…",
  "signatures": {
    "requester": "MEUCIQ…",
    "approver": "MEUCIQ…"
  },
  "merkleProof": {
    "root": "0xdef456…",
    "path": ["0x01", "0x23", "0x45"]
  },
  "actionId": "ACT-2025-00057",
  "status": "completed"
}

The receipt is attached to the email and also stored in the Retention Vault.

Final Checklist Before Closing the Run

  • [ ] All signatures verified
  • [ ] Ledger receipt status = 1
  • [ ] Receipt email sent and logged
  • [ ] Human confirmation (Ops dashboard) recorded
  • [ ] External irreversible API invoked only after the three conditions above
  • [ ] All artefacts archived per retention policy
  • [ ] Deviation Log completed (if any)

When every checkbox is ticked, change the Run status to Closed and archive the Run-ID in the audit system.


Prepared by: Helix Core Ethos – Governance Team

Reviewed & Approved: Stephen Hope, Director – 2025-10-08

All actions in this runbook are performed under the explicit guardrails of Helix Core Ethos: no hidden training, no dark patterns, no unverifiable claims, and no irreversible actions without confirmed human consent.