Skip to main content

​
Overview

The eval-server command starts a persistent HTTP daemon designed for benchmark evaluation environments like SWE-bench. It provides a faster alternative to the MCP server by eliminating process startup overhead for repeated tool calls.
This command is designed for automated evaluation and benchmarking, not for normal development usage. For development, use the standard MCP server via gitnexus mcp.

​
Syntax

gitnexus eval-server [options]

​
Options

​
--port
number
default:"4848"
Port number for the HTTP server.Default: 4848
​
--idle-timeout
number
default:"0"
Auto-shutdown after N seconds of inactivity.
  • 0 = disabled (server runs indefinitely)
  • Useful for resource cleanup in CI/CD environments
Default: 0 (disabled)

​
Use cases

​
SWE-bench evaluation

When running GitNexus in automated benchmarks: 
# Start persistent server
gitnexus eval-server --port 4848 --idle-timeout 300

# Evaluation script makes HTTP calls instead of spawning processes
curl -X POST http://localhost:4848/query -d '{"query": "auth"}'
Benefits:
  • 50-100x faster than spawning new processes for each tool call
  • Persistent KuzuDB connections (no reconnect overhead)
  • Shared memory for indexes
  • Auto-cleanup with idle timeout

​
CI/CD integration

In continuous integration pipelines: 
# Start server with auto-shutdown
gitnexus eval-server --idle-timeout 60

# Run tests that call GitNexus tools
npm test

# Server auto-shuts down after 60s of no requests

​
Performance testing

Benchmark GitNexus tool performance: 
# Start server
gitnexus eval-server

# Run performance tests
ab -n 1000 -c 10 http://localhost:4848/query

# Stop server
curl -X POST http://localhost:4848/shutdown

​
HTTP API

The eval server exposes the same tools as the MCP server, but via HTTP:
EndpointMethodDescription
/queryPOSTProcess-grouped hybrid search
/contextPOST360-degree symbol view
/impactPOSTBlast radius analysis
/detect_changesPOSTGit-diff impact
/renamePOSTMulti-file coordinated rename
/cypherPOSTRaw Cypher queries
/list_reposGETList indexed repositories
/shutdownPOSTGracefully stop the server

​
Example request

curl -X POST http://localhost:4848/query \
  -H "Content-Type: application/json" \
  -d '{
    "query": "authentication",
    "limit": 3
  }'

​
Example response

{
  "processes": [{
    "summary": "LoginFlow",
    "priority": 0.042,
    "symbol_count": 4
  }],
  "process_symbols": [...],
  "definitions": [...]
}

​
Performance comparison

MethodCold StartWarm CallUse Case
MCP stdio~200ms~200msDevelopment (editor integration)
eval-server~50ms~5msBenchmarks (persistent connection)
Direct CLI~150ms~150msScripting (one-off queries)

​
Differences from MCP server

FeatureMCP ServerEval Server
ProtocolstdioHTTP
ConnectionPer-requestPersistent
Editor support✓✗
Benchmark usage✗✓
Auto-shutdown✗✓ (optional)

​
Security

The eval server binds to 127.0.0.1 (localhost only) by default. Do not expose it to public networks.
No authentication is required — the server is designed for local evaluation environments, not production deployment.

​
Troubleshooting

​
Port already in use

# Check what's using the port
lsof -i :4848

# Use a different port
gitnexus eval-server --port 5000

​
Server not responding

# Check if server is running
curl http://localhost:4848/list_repos

# Check logs (stdout/stderr)
gitnexus eval-server 2>&1 | tee server.log

​
Memory leaks in long-running tests

# Use idle timeout to periodically restart
gitnexus eval-server --idle-timeout 300

Build docs developers (and LLMs) love

Get started for freeTalk to us