Function Signature
function defineDatasource<TSchema extends SchemaDefinition>(
name: string,
options: DatasourceOptions<TSchema>
): DatasourceDefinition<TSchema>
Parameters
The datasource name. Must start with a letter or underscore and contain only alphanumeric characters and underscores.
options
DatasourceOptions<TSchema>
required
Datasource configuration object.
Column schema definition using type validators from t.*
Human-readable description of the datasource
Table engine configuration (defaults to MergeTree if not specified)
Access tokens for this datasource. Can be inline definitions or references to defined tokens.
Workspaces to share this datasource with
Whether to generate JSON path expressions for columns. Set to false for datasources that are targets of materialized views.
Forward query used to evolve a datasource with incompatible schema changes. Should be the SELECT clause only (no FROM/WHERE).
indexes
readonly DatasourceIndex[]
Secondary indexes for MergeTree-family engines
Kafka ingestion configuration. Mutually exclusive with s3 and gcs.
S3 import configuration. Mutually exclusive with kafka and gcs.
GCS import configuration. Mutually exclusive with kafka and s3.
Return Type
DatasourceDefinition<TSchema>
SchemaDefinition
type SchemaDefinition = Record<string, AnyTypeValidator | ColumnDefinition>
t.*) or column definitions.
ColumnDefinition
interface ColumnDefinition<T extends AnyTypeValidator = AnyTypeValidator> {
type: T
jsonPath?: string
}
DatasourceIndex
interface DatasourceIndex {
name: string
expr: string
type: string
granularity: number
}
Usage Examples
Basic Datasource
import { defineDatasource, t, engine } from '@tinybirdco/sdk'
export const events = defineDatasource('events', {
description: 'User event tracking data',
schema: {
timestamp: t.dateTime(),
event_id: t.uuid(),
user_id: t.string(),
event_type: t.string().lowCardinality(),
properties: t.string(), // JSON as string
session_id: t.string().nullable(),
},
engine: engine.mergeTree({
sortingKey: ['user_id', 'timestamp'],
partitionKey: 'toYYYYMM(timestamp)',
ttl: 'timestamp + INTERVAL 90 DAY',
}),
})
With Kafka Ingestion
import { defineDatasource, t, engine } from '@tinybirdco/sdk'
import { eventsKafka } from './connections'
export const kafkaEvents = defineDatasource('kafka_events', {
schema: {
timestamp: t.dateTime(),
payload: t.string(),
},
engine: engine.mergeTree({ sortingKey: ['timestamp'] }),
kafka: {
connection: eventsKafka,
topic: 'events',
groupId: 'events-consumer',
autoOffsetReset: 'earliest',
},
})
With Access Tokens
import { defineDatasource, defineToken, t } from '@tinybirdco/sdk'
const readToken = defineToken('app_read')
const ingestToken = defineToken('ingest_token')
export const events = defineDatasource('events', {
schema: {
timestamp: t.dateTime(),
event_name: t.string(),
},
tokens: [
{ token: readToken, scope: 'READ' },
{ token: ingestToken, scope: 'APPEND' },
],
})
With S3 Import
import { defineDatasource, t, engine } from '@tinybirdco/sdk'
import { landingS3 } from './connections'
export const s3Landing = defineDatasource('s3_landing', {
schema: {
timestamp: t.dateTime(),
session_id: t.string(),
},
engine: engine.mergeTree({ sortingKey: ['timestamp'] }),
s3: {
connection: landingS3,
bucketUri: 's3://my-bucket/events/*.csv',
schedule: '@auto',
},
})
Type Inference
Use InferRow to extract the TypeScript type for a datasource row:
import { type InferRow } from '@tinybirdco/sdk'
import { events } from './datasources'
type EventsRow = InferRow<typeof events>
// { timestamp: string, event_id: string, user_id: string, ... }
defineEndpoint
Create an API endpoint
definePipe
Create a transformation pipe
defineMaterializedView
Create a materialized view
Type Validators (t.*)
Column type validators