Team:Internal Data
From Helix Project Wiki
🗂️ Team: Internal Data
This page is for coordinating and referencing **internal datasets, schemas, and storage assets** used within Helix AI systems. It is not public-facing and must adhere to access control policies.
🔐 Access Policy
- This page is restricted to team members in the `ops`, `research`, or `governance` groups.
- Do not include raw credentials or tokens — store these in the secure vault only.
- Always redact PII and sensitive data in examples or logs unless explicitly whitelisted.
📁 Internal Data Repositories
| Name | Description | Location | Retention Policy | Owner |
|---|---|---|---|---|
| Intent Archive | Historical intent payloads + signatures | `s3://core-intents/` | 5 years | Ops |
| TPAF Signed Packages | Approved, dual-signed JSON-LD bundles | `s3://signed-intents/` | 7 years (GLACIER) | Governance |
| Retention Vault | Final ledger receipts + artefacts | `s3://retention-vault/` | 10 years | Compliance |
| Dev Logs | Transient logs from CI/CD and preflight validators | `s3://temp-logs/` | 30 days | Engineering |
| Deviation Records | Logged exceptions during TPAF or runbook flow | `db://audit.deviations` | 3 years | Security |
📄 Key Schemas & Metadata Definitions
If schema evolution is underway, link to the current proposal page instead (e.g., `Proposal:Intent_v2.2`).
🧪 Internal Datasets (for Agent Training / Testing)
| Dataset | Use Case | Format | Source | Notes |
|---|---|---|---|---|
| `decision_chains_v1` | Test set for agentic rollback logic | JSON | Annotated synthetic data | Internal use only |
| `runbook_metrics_2025Q3` | Operational benchmarking | CSV | CI pipeline output | Anonymized |
| `conflict_resolution_cases` | Training signal for agent coordination | JSONL | Manual curation | All data is synthetic |
🔍 Ongoing Data Tasks
- [ ] Add redaction rules to CI logs before `s3://temp-logs` upload
- [ ] Normalize field types across `signed_pkg.json` versions
- [ ] Align retention metadata with compliance audit table
- [ ] Rotate encryption keys on `core-intents` bucket (due: Nov 1)
🧾 Change Log
| Date | Change | Author |
|---|---|---|
| 2025-10-06 | Page created and linked to team dashboard | @you |
| 2025-10-08 | Added dataset inventory and access policy | @you |
