Skip to main content

​
Overview

The data module provides functions to create special data types for use in TopK documents. These types include vectors, sparse vectors, matrices, and byte arrays. Import data functions from topk-js/data. 
import { 
  bytes,
  f32Vector,
  f16Vector,
  f8Vector,
  binaryVector,
  u8Vector,
  i8Vector,
  f32SparseVector,
  u8SparseVector,
  matrix,
  f32List,
  f64List,
  i32List,
  i64List,
  u32List,
  stringList
} from "topk-js/data";

​
Classes

​
List

Instances of the List class are used to represent lists of values in TopK. Usually created using data constructors such as f32List(), i32List(), etc.

​
SparseVector

Instances of the SparseVector class are used to represent sparse vectors in TopK. Usually created using data constructors such as f32SparseVector() or u8SparseVector().

​
Matrix

Instances of the Matrix class are used to represent matrices in TopK. Usually created using the matrix() constructor.

​
Dense Vector Functions

​
f32Vector(values)

Creates a List type containing a 32-bit float vector. This function is an alias for f32List().
​
values
Array<number>
required
Array of 32-bit float values
import { f32Vector } from "topk-js/data";

await client.collection("books").upsert([
  {
    _id: "1",
    title: "Example Book",
    embedding: f32Vector([0.12, 0.67, 0.82, 0.53])
  }
]);

​
f16Vector(values)

Creates a List type containing a 16-bit float vector.
​
values
Array<number>
required
Array of 16-bit float values
import { f16Vector } from "topk-js/data";

await client.collection("books").upsert([
  {
    _id: "1",
    title: "Example Book",
    embedding: f16Vector([0.12, 0.67, 0.82, 0.53])
  }
]);

​
f8Vector(values)

Creates a List type containing an 8-bit float vector.
​
values
Array<number>
required
Array of 8-bit float values
import { f8Vector } from "topk-js/data";

await client.collection("books").upsert([
  {
    _id: "1",
    title: "Example Book",
    embedding: f8Vector([0.12, 0.67, 0.82, 0.53])
  }
]);

​
binaryVector(values)

Creates a List type containing a binary vector. This function is an alias for binaryList().
​
values
Array<number>
required
Array of binary values (0 or 1)
import { binaryVector } from "topk-js/data";

await client.collection("books").upsert([
  {
    _id: "1",
    title: "Example Book",
    embedding: binaryVector([0, 1, 1, 0, 1, 0, 0, 1])
  }
]);

​
u8Vector(values)

Creates a List type containing an 8-bit unsigned integer vector. This function is an alias for u8List().
​
values
Array<number>
required
Array of 8-bit unsigned integer values (0-255)
import { u8Vector } from "topk-js/data";

await client.collection("books").upsert([
  {
    _id: "1",
    title: "Example Book",
    embedding: u8Vector([0, 255, 1, 2, 3])
  }
]);

​
i8Vector(values)

Creates a List type containing an 8-bit signed integer vector. This function is an alias for i8List().
​
values
Array<number>
required
Array of 8-bit signed integer values (-128 to 127)
import { i8Vector } from "topk-js/data";

await client.collection("books").upsert([
  {
    _id: "1",
    title: "Example Book",
    embedding: i8Vector([-128, 127, -1, 0, 1])
  }
]);

​
Sparse Vector Functions

​
f32SparseVector(vector)

Creates a SparseVector type containing a sparse vector of 32-bit floats. This function is an alias for f32SparseList().
​
vector
Record<number, number>
required
Object mapping dimension indices to values
import { f32SparseVector } from "topk-js/data";

await client.collection("books").upsert([
  {
    _id: "1",
    title: "Example Book",
    sparse_embedding: f32SparseVector({
      0: 0.12,
      6: 0.67,
      17: 0.82,
      97: 0.53
    })
  }
]);

​
u8SparseVector(vector)

Creates a SparseVector type containing a sparse vector of 8-bit unsigned integers. This function is an alias for u8SparseList().
​
vector
Record<number, number>
required
Object mapping dimension indices to values (0-255)
import { u8SparseVector } from "topk-js/data";

await client.collection("books").upsert([
  {
    _id: "1",
    title: "Example Book",
    sparse_embedding: u8SparseVector({
      0: 12,
      6: 67,
      17: 82,
      97: 53
    })
  }
]);

​
Matrix Functions

​
matrix(values, valueType?)

Creates a Matrix type containing matrix values.
​
values
Array<Array<number>>
required
Array of number arrays representing the matrix rows
​
valueType
MatrixValueType
The matrix element value type. Defaults to 'f32'. Can be:
  • 'f32' - 32-bit float
  • 'f16' - 16-bit float
  • 'f8' - 8-bit float
  • 'u8' - 8-bit unsigned integer
  • 'i8' - 8-bit signed integer
import { matrix } from "topk-js/data";

await client.collection("books").upsert([
  {
    _id: "1",
    title: "Example Book",
    token_embeddings: matrix([
      [1.0, 2.0, 3.0],
      [4.0, 5.0, 6.0]
    ])
  }
]);

​
List Functions

​
f32List(values)

Creates a List type containing a list of 32-bit floating point numbers.
​
values
Array<number>
required
Array of 32-bit float values
import { f32List } from "topk-js/data";

await client.collection("analytics").upsert([
  {
    _id: "1",
    measurements: f32List([0.12, 0.67, 0.82, 0.53])
  }
]);

​
f64List(values)

Creates a List type containing a list of 64-bit floating point numbers.
​
values
Array<number>
required
Array of 64-bit float values
import { f64List } from "topk-js/data";

await client.collection("analytics").upsert([
  {
    _id: "1",
    precise_measurements: f64List([0.12, 0.67, 0.82, 0.53])
  }
]);

​
i32List(values)

Creates a List type containing a list of 32-bit signed integers.
​
values
Array<number>
required
Array of 32-bit integer values
import { i32List } from "topk-js/data";

await client.collection("analytics").upsert([
  {
    _id: "1",
    counts: i32List([0, 1, 2, 3, -1, -2])
  }
]);

​
i64List(values)

Creates a List type containing a list of 64-bit signed integers.
​
values
Array<number>
required
Array of 64-bit integer values
import { i64List } from "topk-js/data";

await client.collection("analytics").upsert([
  {
    _id: "1",
    large_counts: i64List([0, 1, 2, 3])
  }
]);

​
u32List(values)

Creates a List type containing a list of 32-bit unsigned integers.
​
values
Array<number>
required
Array of 32-bit unsigned integer values
import { u32List } from "topk-js/data";

await client.collection("analytics").upsert([
  {
    _id: "1",
    positive_counts: u32List([0, 1, 2, 3])
  }
]);

​
stringList(values)

Creates a List type containing a list of strings.
​
values
Array<string>
required
Array of string values
import { stringList } from "topk-js/data";

await client.collection("books").upsert([
  {
    _id: "1",
    title: "Example Book",
    tags: stringList(["fiction", "adventure", "classic"])
  }
]);

​
Bytes Function

​
bytes(buffer)

Creates bytes data from an array of numbers or a Buffer.
​
buffer
Array<number> | Buffer
required
Array of byte values or a Buffer object
import { bytes } from "topk-js/data";

await client.collection("files").upsert([
  {
    _id: "1",
    data: bytes([0, 1, 1, 0])
  }
]);

​
Complete Examples

import { Client } from "topk-js";
import { f32Vector, vectorIndex } from "topk-js/schema";
import { f32Vector as dataF32Vector } from "topk-js/data";
import { select, field, fn } from "topk-js/query";

const client = new Client({
  apiKey: "YOUR_TOPK_API_KEY",
  region: "aws-us-east-1-elastica"
});

// Create collection with vector field
await client.collections().create("products", {
  title: text(),
  embedding: f32Vector({ dimension: 384 }).index(
    vectorIndex({ metric: "cosine" })
  )
});

// Insert documents with vectors
await client.collection("products").upsert([
  {
    _id: "1",
    title: "Product 1",
    embedding: dataF32Vector([0.1, 0.2, 0.3, /* ... 384 dimensions */])
  }
]);

// Query with vector distance
const results = await client.collection("products").query(
  select({
    title: field("title"),
    distance: fn.vectorDistance(
      "embedding",
      [0.2, 0.3, 0.4, /* ... */]
    )
  })
  .topk(field("distance"), 10)
);

Build docs developers (and LLMs) love

Get started for freeTalk to us