What is Lake Framework?

NEAR Lake Framework is an ecosystem of library companions that read data from the NEAR Data Lake. They allow you to build your own indexer by simply subscribing to the stream of blocks that is being constantly pushed to the NEAR Lake.

Example of a simple indexer built on top of NEAR Lake Framework You can create your own logic to process the NEAR Protocol data in the programming languages of your choice:

Python

PyPI Package

JavaScript

npm Package

Rust

Crates.io Package

GitHub repo: https://github.com/near/near-lake-framework/

Data Lake

The NEAR Lake is a set of AWS S3 buckets which are constantly receiving new blocks from a running indexer maintained by Aurora. Events such as FT Events and NFT Events are constantly being written as JSON files in two AWS S3 buckets:

Mainnet
Testnet

Bucket: near-lake-data-mainnetRegion: eu-central-1Contains all production blockchain data from NEAR Protocol mainnet.

Bucket: near-lake-data-testnetRegion: eu-central-1Contains all blockchain data from NEAR Protocol testnet for development and testing.

Both buckets are set up the way the requester pays for the access. Anyone can read from these buckets by connecting to them with their own AWS credentials to be charged by Amazon.

Latency

Indexers based on the Lake Framework inherit the latency characteristics of a NEAR Indexer plus an extra 0.1-2.1s latency of dumping to and reading from AWS S3.

Most of the latency is there due to the finalization delay and both Indexer Framework and Lake Framework add quite a minimum overhead.

Typical latency breakdown:

Finalization delay: ~3-5 seconds
S3 write/read overhead: 0.1-2.1 seconds
Total end-to-end: 3.9-7.1 seconds (estimated average 6-8s)

Cost

Indexers based on NEAR Lake consume 100-500MB of RAM depending on the size of the preloading queue, it does not require any storage, and it can potentially run even on Raspberry PI. Getting the blockchain data from S3 will cost around $30.16 per month as NEAR Lake configured S3 buckets in such a way that the reader is paying the costs.

AWS S3 Cost Breakdown

Assuming NEAR Protocol produces 1 block every 600ms, on a full day the network can create up to 144000 blocks (86400s / 600ms per block).According to the Amazon S3 prices list requests are charged for

0.005 per 1000 requests and `get` is charged for

0.0004 per 1000 requests.Calculations (assuming we are following the tip of the network all the time):GET requests:

144000 blocks per day * 10 requests for each block / 1000 requests * $0.0004 per 1k requests = $0.576 * 30 days = $17.20

Note: 10 requests for each block means we have 9 shards (1 file for common block data and 9 separate files for each shard)LIST requests:

144000 blocks per day / 1000 requests * $0.005 per 1k list requests = $0.72 * 30 days = $21.60

Total monthly cost:

$17.20 + $21.60 = $38.80

Comparison with NEAR Indexer Framework

NEAR Lake Framework is reading data from AWS S3, while the NEAR Indexer is running a full node and reading data from the blockchain directly in real time.

Feature	Indexer Framework	Lake Framework
Follows blocks and transactions	Yes	Yes (mainnet and testnet only)
Decentralized	Yes	No (Pagoda Inc dumps blocks to AWS S3)
Reaction time (end-to-end)	Minimum 3.8s (estimated average 5-7s)	Minimum 3.9s (estimated average 6-8s)
Reaction time (framework overhead)	0.1s	0.2-2.2s
Infrastructure cost	$500+/mo	$20-40/mo
Ease of maintenance	Advanced (follow nearcore upgrades, sync state)	Easy (deploy once and forget)
Time to start	Days (on mainnet/testnet)	Seconds
Local development	Advanced (localnet option, but testnet/mainnet testing is heavy)	Easy
Programming languages	Rust only	Any (Python, JavaScript, Rust)

NEAR Lake Framework is the recommended solution for most use cases due to its ease of use, lower cost, and support for multiple programming languages.

Key Benefits

Cost-Efficient

Runs on minimal infrastructure (~

30/month) compared to running a full node (

500+/month)

Easy Setup

Start indexing in seconds, not days. No blockchain state sync required.

Multiple Languages

Build indexers in Python, JavaScript, or Rust based on your team’s expertise.

Low Maintenance

Deploy once and forget. No need to follow nearcore upgrades or manage node infrastructure.

How It Works

Choose your language

Install the NEAR Lake Framework library for Python, JavaScript, or Rust.

Configure AWS credentials

Set up AWS credentials to access the S3 buckets (you pay for access).

Define your indexing logic

Write code to filter and process the blockchain data you need.

Start streaming

Run your indexer and start receiving blocks in real-time.

Examples & Tutorials

Raw Printer

Simple example of a data printer built on top of NEAR Lake Framework

Accounts Watcher

Source code for a video tutorial on how to use the NEAR Lake Framework

Transaction Watcher

Indexer example that watches for transactions for specified accounts/contracts

Getting Started

Python
JavaScript
Rust

pip install near-lake-framework

from near_lake_framework import LakeConfig, streamer

async def handle_block(block):
    print(f"Block #{block.block_height}")

config = LakeConfig.mainnet()
config.start_block_height = 80000000

await streamer(config, handle_block)

npm install near-lake-framework

const { LakeConfig, startStream } = require('near-lake-framework');

async function handleBlock(block) {
  console.log(`Block #${block.blockHeight}`);
}

const config = LakeConfig.mainnet();
config.startBlockHeight = 80000000;

startStream(config, handleBlock);

cargo add near-lake-framework

use near_lake_framework::LakeConfigBuilder;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let config = LakeConfigBuilder::default()
        .mainnet()
        .start_block_height(80000000)
        .build()?;

    let stream = near_lake_framework::streamer(config);

    while let Some(block) = stream.recv().await {
        println!("Block #{}", block.block_height());
    }

    Ok(())
}

Build Your First Indexer

Follow our step-by-step tutorials to build your first NEAR Lake indexer

Getting Started

Protocol

Smart Contracts

Web3 Applications

AI and Agents

Chain Abstraction

Primitives

Data Infrastructure

Python

JavaScript

Rust

Data Lake

Latency

Cost

Comparison with NEAR Indexer Framework

Key Benefits

Cost-Efficient

Easy Setup

Multiple Languages

Low Maintenance

How It Works

Examples & Tutorials

Raw Printer

Accounts Watcher

Transaction Watcher

More Tutorials

Getting Started

Build Your First Indexer

Build docs developers (and LLMs) love

Getting Started

Protocol

Smart Contracts

Web3 Applications

AI and Agents

Chain Abstraction

Primitives

Data Infrastructure

Python

JavaScript

Rust

​Data Lake

​Latency

​Cost

​Comparison with NEAR Indexer Framework

​Key Benefits

Cost-Efficient

Easy Setup

Multiple Languages

Low Maintenance

​How It Works

​Examples & Tutorials

Raw Printer

Accounts Watcher

Transaction Watcher

More Tutorials

​Getting Started

Build Your First Indexer

Build docs developers (and LLMs) love

Data Lake

Latency

Cost

Comparison with NEAR Indexer Framework

Key Benefits

How It Works

Examples & Tutorials

Getting Started