Skip to main content
Searchers are the fundamental building blocks for implementing custom query processing and result manipulation in Vespa. They participate in a chain of responsibility pattern where queries pass through a sequence of searchers, each potentially modifying the query or result.

​
Overview

A Searcher can:
  • Modify queries before they are executed (query rewriting)
  • Process results by altering, reorganizing, or adding hits
  • Federate to multiple search chains in series or parallel
  • Act as a source by creating results from internal or external data
  • Implement workflows by calling downstream searchers multiple times

​
Basic Searcher Structure

All custom searchers extend the com.yahoo.search.Searcher class and override the search method: 
package com.example;

import com.yahoo.search.Query;
import com.yahoo.search.Result;
import com.yahoo.search.Searcher;
import com.yahoo.search.searchchain.Execution;

public class MySearcher extends Searcher {

    @Override
    public Result search(Query query, Execution execution) {
        // Modify query before passing it on
        query.properties().set("myProperty", "value");
        
        // Pass query to next searcher in chain
        Result result = execution.search(query);
        
        // Process result before returning
        result.hits().add(createMyHit());
        
        return result;
    }
}

​
Searcher Lifecycle

1

Construction

The searcher is instantiated with its configuration. Build any required in-memory structures here.
public class ConfiguredSearcher extends Searcher {
    private final String myConfig;
    
    public ConfiguredSearcher(MyConfig config) {
        this.myConfig = config.value();
    }
}
2

In Service

The search() method is called by multiple threads in parallel. Keep shared data structures immutable or use proper synchronization.
3

Deconstruction

Override deconstruct() to clean up resources. This is called when the searcher is being removed.
@Override
public void deconstruct() {
    super.deconstruct();
    // Clean up resources
}

​
Common Searcher Patterns

​
Query Processor

Modifies the query before execution: 
import com.yahoo.prelude.query.WordItem;
import com.yahoo.prelude.query.CompositeItem;

public class QueryEnricherSearcher extends Searcher {

    @Override
    public Result search(Query query, Execution execution) {
        // Add a term to the query
        CompositeItem root = (CompositeItem) query.getModel().getQueryTree().getRoot();
        if (root != null) {
            root.addItem(new WordItem("boost", "title"));
        }
        
        // Continue processing
        return execution.search(query);
    }
}

​
Result Processor

Modifies results after execution: 
import com.yahoo.search.result.Hit;

public class ResultEnricherSearcher extends Searcher {

    @Override
    public Result search(Query query, Execution execution) {
        Result result = execution.search(query);
        
        // Enrich each hit with additional data
        for (Hit hit : result.hits()) {
            hit.setField("enriched", true);
            hit.setField("timestamp", System.currentTimeMillis());
        }
        
        return result;
    }
}

​
Federator

Searches multiple sources and combines results: 
import com.yahoo.search.searchchain.AsyncExecution;
import java.util.List;
import java.util.ArrayList;

public class FederatorSearcher extends Searcher {

    @Override
    public Result search(Query query, Execution execution) {
        // Create parallel executions to different chains
        AsyncExecution news = new AsyncExecution("news", execution);
        AsyncExecution images = new AsyncExecution("images", execution);
        
        // Start parallel searches
        news.search(query.clone());
        images.search(query.clone());
        
        // Combine results
        Result combined = new Result(query);
        combined.hits().addAll(news.get().hits());
        combined.hits().addAll(images.get().hits());
        
        return combined;
    }
}

​
Source Searcher

Generates results from custom data sources: 
import com.yahoo.search.result.Hit;
import java.util.List;

public class CustomSourceSearcher extends Searcher {

    @Override
    public Result search(Query query, Execution execution) {
        Result result = new Result(query);
        
        // Fetch from external source
        List<MyData> data = fetchFromExternalAPI(query.getModel().getQueryString());
        
        // Convert to hits
        for (MyData item : data) {
            Hit hit = new Hit(item.getId());
            hit.setField("title", item.getTitle());
            hit.setField("content", item.getContent());
            hit.setRelevance(item.getScore());
            result.hits().add(hit);
        }
        
        return result;
    }
    
    private List<MyData> fetchFromExternalAPI(String query) {
        // Implementation here
        return List.of();
    }
}

​
Constructor Injection

Vespa supports dependency injection in searcher constructors: 
import com.yahoo.component.ComponentId;
import com.yahoo.component.annotation.Inject;

public class InjectableSearcher extends Searcher {
    private final MyService service;
    
    @Inject
    public InjectableSearcher(ComponentId id, MyService service) {
        super(id);
        this.service = service;
    }
}
Constructor priority:
  1. (ComponentId, ConfigClass1, ConfigClass2, ...)
  2. (String, ConfigClass1, ConfigClass2, ...)
  3. (ConfigClass1, ConfigClass2, ...)
  4. (ComponentId)
  5. (String)
  6. Default no-argument constructor

​
Search Chain Configuration

Add your searcher to services.xml: 
<container id="default" version="1.0">
  <search>
    <chain id="default" inherits="vespa">
      <searcher id="com.example.MySearcher" bundle="my-bundle"/>
    </chain>
  </search>
</container>

​
Chain Dependencies

Control ordering with annotations: 
import com.yahoo.component.chain.dependencies.After;
import com.yahoo.component.chain.dependencies.Before;
import com.yahoo.component.chain.dependencies.Provides;

@After("OtherSearcher")
@Before("AnotherSearcher")
@Provides("MyFeature")
public class OrderedSearcher extends Searcher {
    // ...
}

​
Real-World Example: Stemming

Here’s a simplified version of Vespa’s StemmingSearcher from ~/workspace/source/container-search/src/main/java/com/yahoo/prelude/querytransform/StemmingSearcher.java:62: 
import com.yahoo.component.annotation.Inject;
import com.yahoo.language.Linguistics;
import com.yahoo.prelude.IndexFacts;
import com.yahoo.search.searchchain.PhaseNames;

@After(PhaseNames.UNBLENDED_RESULT)
@Provides("Stemming")
public class StemmingSearcher extends Searcher {
    
    private final Linguistics linguistics;
    
    @Inject
    public StemmingSearcher(ComponentId id, Linguistics linguistics) {
        super(id);
        this.linguistics = linguistics;
    }
    
    @Override
    public Result search(Query query, Execution execution) {
        if (query.properties().getBoolean("nostemming")) {
            return execution.search(query);
        }
        
        IndexFacts.Session indexFacts = 
            execution.context().getIndexFacts().newSession(query);
        
        // Replace query terms with stems
        Item newRoot = replaceTerms(query, indexFacts);
        query.getModel().getQueryTree().setRoot(newRoot);
        
        query.trace("Stemming", true, 2);
        
        return execution.search(query);
    }
    
    private Item replaceTerms(Query query, IndexFacts.Session indexFacts) {
        // Stemming logic here
        return query.getModel().getQueryTree().getRoot();
    }
}

​
Error Handling

Create a Result with an error message:
if (invalidCondition) {
    return new Result(query, 
        ErrorMessage.createBadRequest("Invalid parameter"));
}
Throw a RuntimeException:
if (criticalFailure) {
    throw new RuntimeException("Critical failure: " + details);
}
Add a FeedbackHit explaining the condition:
import com.yahoo.search.result.FeedbackHit;

FeedbackHit feedback = new FeedbackHit();
feedback.setField("message", "Please correct your query");
result.hits().add(feedback);

​
Fill Operations

For federating searchers, override fill() to fetch additional data: 
@Override
public void fill(Result result, String summaryClass, Execution execution) {
    // Custom fill logic
    for (Hit hit : result.hits()) {
        if (hit.isFilled(summaryClass)) continue;
        // Fetch and populate fields
        hit.setField("fullContent", fetchContent(hit.getId()));
    }
    
    // Continue to next searcher
    execution.fill(result, summaryClass);
}

​
Testing

Test searchers using the Execution framework: 
import com.yahoo.search.searchchain.Execution;
import com.yahoo.component.chain.Chain;
import org.junit.jupiter.api.Test;

public class MySearcherTest {
    
    @Test
    public void testSearcher() {
        MySearcher searcher = new MySearcher();
        Chain<Searcher> chain = new Chain<>(searcher);
        Execution execution = new Execution(chain, Execution.Context.createContextStub());
        
        Query query = new Query("test query");
        Result result = execution.search(query);
        
        assertEquals(1, result.getHitCount());
    }
}

​
Performance Tips

  • Avoid synchronization: Keep data structures built during construction read-only
  • Use tracing: Call query.trace() to add debug information
  • Minimize allocations: Reuse objects when possible
  • Profile carefully: Searchers are on the critical path for all queries

​
Next Steps

Build docs developers (and LLMs) love

Get started for freeTalk to us