Skip to main content

​
Overview

Evolver executes validation commands specified in Gene definitions to verify evolution correctness. To prevent arbitrary command execution, all validation commands are gated by safety checks. Location: src/gep/solidify.js:569-601

​
Safety Checks

​
1. Prefix Whitelist

Only commands starting with node, npm, or npx are allowed: 
const VALIDATION_ALLOWED_PREFIXES = ['node ', 'npm ', 'npx '];

if (!VALIDATION_ALLOWED_PREFIXES.some(p => c.startsWith(p))) return false;
Rejected: 
bash script.sh     # ❌ Not in whitelist
python test.py     # ❌ Not in whitelist
sh -c "npm test"   # ❌ Not in whitelist
Allowed: 
node test.js       # ✅
npm test           # ✅
npx jest           # ✅

​
2. No Command Substitution

Backticks and $(...) are rejected anywhere in the command string: 
if (/`|\$\(/.test(c)) return false;
Rejected: 
node -e "$(cat /etc/passwd)"  # ❌ Command substitution
npm run `malicious`           # ❌ Backticks
node test.js --arg=$(whoami)  # ❌ Command substitution

​
3. No Shell Operators

After stripping quoted content, ;, &, |, >, < are rejected: 
const stripped = c.replace(/"[^"]*"/g, '').replace(/'[^']*'/g, '');
if (/[;&|><]/.test(stripped)) return false;
Rejected: 
npm test && rm -rf /         # ❌ Contains &&
node test.js | grep error    # ❌ Contains |
node test.js > /tmp/out.txt  # ❌ Contains >
npm test; echo "done"        # ❌ Contains ;
Allowed (operators inside quotes): 
node -e 'console.log("a && b")'  # ✅ && is quoted
npm test -- --grep "a|b"         # ✅ | is quoted

​
4. No Eval Patterns

Direct node -e / node --eval is blocked to prevent arbitrary code execution: 
if (/^node\s+(-e|--eval|--print|-p)\b/.test(c)) return false;
Rejected: 
node -e "process.exit(1)"         # ❌ Eval pattern
node --eval "require('fs')"       # ❌ Eval pattern
node -p "1+1"                     # ❌ Print pattern

​
5. Timeout

Each validation command is limited to 180 seconds (3 minutes): 
const r = tryRunCmd(c, { cwd: repoRoot, timeoutMs: 180000 });
Location: src/gep/solidify.js:596

​
6. Scoped Execution

Commands run with cwd set to the repository root, preventing path traversal: 
function runValidations(gene, opts = {}) {
  const repoRoot = opts.repoRoot || getRepoRoot();
  // ...
  const r = tryRunCmd(c, { cwd: repoRoot, timeoutMs });
}
Location: src/gep/solidify.js:583-601

​
Implementation

​
isValidationCommandAllowed

// src/gep/solidify.js:569-581
const VALIDATION_ALLOWED_PREFIXES = ['node ', 'npm ', 'npx '];

function isValidationCommandAllowed(cmd) {
  const c = String(cmd || '').trim();
  if (!c) return false;
  
  // 1. Prefix whitelist
  if (!VALIDATION_ALLOWED_PREFIXES.some(p => c.startsWith(p))) return false;
  
  // 2. No command substitution
  if (/`|\$\(/.test(c)) return false;
  
  // 3. No shell operators (after stripping quotes)
  const stripped = c.replace(/"[^"]*"/g, '').replace(/'[^']*'/g, '');
  if (/[;&|><]/.test(stripped)) return false;
  
  // 4. Block eval-like patterns
  if (/^node\s+(-e|--eval|--print|-p)\b/.test(c)) return false;
  
  return true;
}

​
runValidations

// src/gep/solidify.js:583-601
function runValidations(gene, opts = {}) {
  const repoRoot = opts.repoRoot || getRepoRoot();
  const timeoutMs = Number.isFinite(Number(opts.timeoutMs)) ? Number(opts.timeoutMs) : 180000;
  const validation = Array.isArray(gene && gene.validation) ? gene.validation : [];
  const results = [];
  const startedAt = Date.now();
  
  for (const cmd of validation) {
    const c = String(cmd || '').trim();
    if (!c) continue;
    
    // Safety check
    if (!isValidationCommandAllowed(c)) {
      results.push({ 
        cmd: c, 
        ok: false, 
        out: '', 
        err: 'BLOCKED: validation command rejected by safety check (allowed prefixes: node/npm/npx; shell operators prohibited)' 
      });
      return { ok: false, results, startedAt, finishedAt: Date.now() };
    }
    
    // Execute
    const r = tryRunCmd(c, { cwd: repoRoot, timeoutMs });
    results.push({ cmd: c, ok: r.ok, out: String(r.out || ''), err: String(r.err || '') });
    
    // Fail fast
    if (!r.ok) return { ok: false, results, startedAt, finishedAt: Date.now() };
  }
  
  return { ok: true, results, startedAt, finishedAt: Date.now() };
}

​
Gene Validation Examples

​
Safe Gene

{
  "type": "Gene",
  "id": "gene_safe_validation",
  "category": "repair",
  "validation": [
    "node scripts/validate-modules.js ./src",
    "npm test",
    "npx eslint src/"
  ]
}

​
Unsafe Gene (Rejected)

{
  "type": "Gene",
  "id": "gene_unsafe_validation",
  "category": "repair",
  "validation": [
    "bash scripts/deploy.sh",  // ❌ Not node/npm/npx
    "npm test && rm -rf /"      // ❌ Shell operator &&
  ]
}
Result: Solidify fails with violation: 
constraint: validation_failed: bash scripts/deploy.sh => BLOCKED: validation command rejected by safety check

​
External Asset Promotion

When promoting external Genes via scripts/a2a_promote.js, all validation commands are audited before promotion: 
// Pseudo-code from a2a_promote.js
for (const cmd of gene.validation) {
  if (!isValidationCommandAllowed(cmd)) {
    console.error(`Unsafe validation command: ${cmd}`);
    process.exit(1);
  }
}
Location: README.md:214-216

​
Bypass Attempts

​
Attempt 1: Shell Escape via Arguments

node test.js "--arg='; rm -rf /'"
Blocked: The ; is outside quotes after stripping, detected by step 3.

​
Attempt 2: Pipe via Unquoted Args

node test.js | grep error
Blocked: | detected by step 3.

​
Attempt 3: Command Substitution in Args

node test.js --file=$(cat /etc/passwd)
Blocked: $( detected by step 2.

​
Attempt 4: Eval Injection

node -e "require('child_process').execSync('rm -rf /')"
Blocked: node -e detected by step 4.

​
Custom Validation Scripts

If you need custom validation logic, wrap it in a Node.js script:

​
Bad (Shell Script)

# scripts/custom-check.sh
#!/bin/bash
if [ -f "required.txt" ]; then
  echo "OK"
else
  exit 1
fi
Gene: 
{
  "validation": [
    "bash scripts/custom-check.sh"  // ❌ Rejected
  ]
}

​
Good (Node.js Script)

// scripts/custom-check.js
const fs = require('fs');

if (!fs.existsSync('required.txt')) {
  console.error('Missing required.txt');
  process.exit(1);
}

console.log('OK');
Gene: 
{
  "validation": [
    "node scripts/custom-check.js"  // ✅ Allowed
  ]
}

​
Validation Report

Validation results are recorded in a machine-readable ValidationReport: 
{
  "type": "ValidationReport",
  "id": "vr_1234567890",
  "gene_id": "gene_repair_001",
  "commands": [
    "npm test",
    "node scripts/validate.js"
  ],
  "results": [
    { "cmd": "npm test", "ok": true, "duration_ms": 3200 },
    { "cmd": "node scripts/validate.js", "ok": true, "duration_ms": 450 }
  ],
  "env_fingerprint": { "platform": "linux", "node_version": "v18.16.0" },
  "started_at": "2024-01-15T10:30:00.000Z",
  "finished_at": "2024-01-15T10:30:03.650Z"
}
Location: src/gep/solidify.js:1133-1140

​
Timeout Handling

If a validation command exceeds 180 seconds, it is killed: 
tryRunCmd(cmd, { cwd: repoRoot, timeoutMs: 180000 });
Result: 
{
  "cmd": "npm run slow-test",
  "ok": false,
  "err": "Command timed out after 180000ms"
}

​
Best Practices

  1. Use npm scripts: Define complex validation in package.json scripts: 
    {
  "scripts": {
    "validate": "eslint src/ && jest --coverage"
  }
}
    Then: 
    { "validation": ["npm run validate"] }
  2. Keep validations fast: Target < 60 seconds for responsiveness
  3. Fail fast: If a critical check fails early, don’t waste time on subsequent checks
  4. No side effects: Validation commands should be read-only (no writes, no deployments)
  5. Explicit exit codes: Ensure scripts exit with code 0 (success) or non-zero (failure)

​
Troubleshooting

​
Validation command blocked

Symptom: Solidify fails with “BLOCKED: validation command rejected by safety check”. Cause: Command contains shell operators or non-whitelisted prefix. Solution: Rewrite command using Node.js: 
# Before (blocked)
bash scripts/check.sh && npm test

# After (allowed)
node scripts/check.js
npm test

​
Validation timeout

Symptom: Validation hangs for 3 minutes then fails. Cause: Command is too slow or stuck. Solution: Optimize or split into smaller checks: 
# Slow
npm run full-integration-test  # Takes 5 minutes

# Fast
npm run unit-test              # Takes 30 seconds

​
External gene rejected during promotion

Symptom: a2a_promote.js rejects gene with unsafe validation command. Cause: Gene contains shell scripts or dangerous commands. Solution: Contact asset author to fix validation commands, or fork and fix locally.

Build docs developers (and LLMs) love

Get started for freeTalk to us