Queue position is arguably the most critical component of high-frequency backtesting. When you place a limit order, you join a queue of orders at that price level. Your fill depends on where you are in that queue and how much trading activity occurs at that price.HftBacktest provides sophisticated queue models to estimate your position and determine when orders fill.
Bid Side at $100:- Order 1: 10 contracts (at front of queue)- Order 2: 15 contracts- YOUR ORDER: 5 contracts ← You're here- Order 3: 20 contracts- Order 4: 30 contractsTotal: 80 contracts
If 30 contracts trade at $100:
Order 1 fills completely (10 filled, 20 remaining)
Order 2 fills completely (15 filled, 5 remaining)
YOUR ORDER fills completely (5 filled, 0 remaining)
Order 3 doesn’t fill
You’re 25 contracts from the front. You fill when at least 25 contracts trade at your price.
Without queue modeling, backtests optimistically assume instant fills when your price is reached. This drastically overstates HFT profitability.
Queue behavior is defined by the QueueModel trait from backtest/models/queue.rs:24-40:
pub trait QueueModel<MD>where MD: MarketDepth,{ /// Initialize queue position when order is accepted fn new_order(&self, order: &mut Order, depth: &MD); /// Adjust position when trades occur at this price fn trade(&self, order: &mut Order, qty: f64, depth: &MD); /// Adjust position when depth changes at this price fn depth(&self, order: &mut Order, prev_qty: f64, new_qty: f64, depth: &MD); /// Check if order should fill and return fill quantity fn is_filled(&self, order: &mut Order, depth: &MD) -> f64;}
Queue state is stored in order.q, a boxed trait object allowing flexible state representation.
From queue.rs:473-1128, this maintains explicit queues:
Order Addition
Market Order Arrival
Order Fill
Add your order to the queue:
fn add_backtest_order(&mut self, mut order: Order, _depth: &MD) { order.q = Box::new(L3OrderSource::Backtest); let queue = self.bid_queue.entry(order.price_tick).or_default(); queue.push_back(order); // Add to back of queue}
Track market orders from the feed:
fn add_market_feed_order(&mut self, order: &Event, depth: &MD) { let queue = self.bid_queue.entry(price_tick).or_default(); queue.push_back(Order { /* from event */ });}
When a market order fills, check your position:
fn fill_market_feed_order<const DELETE: bool>( &mut self, order_id: OrderId, order: &Event, depth: &MD,) -> Result<Vec<Order>, BacktestError> { let queue = self.bid_queue.get_mut(&order_price_tick).unwrap(); let mut filled = Vec::new(); // Fill all orders ahead of the market order for order_in_q in queue { if order_in_q.order_id == order_id { break; // Reached the market order } if order_in_q.is_backtest_order() { filled.push(order_in_q.clone()); } } filled}
Estimate your fill probability based on queue position:
@njitdef estimate_fill_probability(hbt, asset_no, price, side): depth = hbt.depth(asset_no) tick_size = depth.tick_size price_tick = int(price / tick_size) # Get queue depth at this price if side == BUY: queue_qty = depth.bid_qty_at_tick(price_tick) else: queue_qty = depth.ask_qty_at_tick(price_tick) # Estimate: if you're at back, you need all queue_qty to trade # Historical trade rate at this price # (requires collecting statistics) trade_rate = estimate_trade_rate(price_tick) # contracts per second expected_fill_time = queue_qty / trade_rate return expected_fill_time
@njitdef manage_adverse_selection(hbt): """Cancel orders likely to suffer adverse selection""" asset_no = 0 depth = hbt.depth(asset_no) orders = hbt.orders(asset_no) order_values = orders.values() while order_values.has_next(): order = order_values.get() if order.side == BUY: # If best bid has moved away, we're likely at the front # and will get adversely filled if order.price_tick < depth.best_bid_tick: hbt.cancel(asset_no, order.order_id, False)
@njitdef queue_aware_quoting(hbt): asset_no = 0 depth = hbt.depth(asset_no) best_bid = depth.best_bid best_bid_qty = depth.best_bid_qty # If queue is thin, join it # If queue is thick, improve price (pay for priority) THIN_THRESHOLD = 100.0 # contracts if best_bid_qty < THIN_THRESHOLD: # Join the queue quote_price = best_bid else: # Improve price by one tick to get front position quote_price = best_bid + depth.tick_size hbt.submit_buy_order(asset_no, order_id, quote_price, qty, GTX, LIMIT, False)