Autograd

The autograd module implements reverse-mode automatic differentiation (backpropagation) for tensor operations.

GradTensor Class

GradTensor

class

Tensor wrapper that records a computation graph for reverse-mode autodiff.Tracks operations and enables automatic gradient computation through backpropagation.

import { parameter } from 'deepbox/ndarray';

const x = parameter([1, 2, 3], { requiresGrad: true });
const y = x.mul(x).sum();
y.backward();
console.log(x.grad);  // Gradient with respect to x

Properties

tensor

Tensor

The underlying tensor containing the data.

requiresGrad

boolean

Whether this tensor requires gradient computation.

grad

Tensor | null

Accumulated gradient for this tensor. Returns null if no gradient has been computed.

shape

Shape

The dimensions of the underlying tensor.

dtype

DType

Data type of the tensor elements.

size

number

Total number of elements.

ndim

number

Number of dimensions.

Gradient Operations

backward

function

Backpropagate gradients from this node through the recorded graph.Parameters:

grad (Tensor) - Optional seed gradient (default: ones)

const x = parameter([1, 2, 3], { requiresGrad: true });
const y = x.sum();
y.backward();
console.log(x.grad);  // [1, 1, 1]

zeroGrad

function

Reset accumulated gradients to zero.

x.zeroGrad();

detach

function

Create a new GradTensor that doesn’t track gradients.Returns: GradTensor with requiresGrad=false

const x = parameter([1, 2, 3], { requiresGrad: true });
const y = x.detach();  // No gradient tracking

setGrad

function

Manually set the gradient for this tensor.Parameters:

grad (Tensor) - Gradient tensor

x.setGrad(tensor([1, 1, 1]));

setRequiresGrad

function

Enable or disable gradient tracking.Parameters:

value (boolean) - Whether to track gradients

x.setRequiresGrad(false);

Arithmetic Operations

add

function

Element-wise addition with automatic differentiation.Parameters:

other (GradTensor) - Tensor to add

Returns: GradTensor result

const x = parameter([1, 2, 3], { requiresGrad: true });
const y = parameter([4, 5, 6], { requiresGrad: true });
const z = x.add(y);
z.backward(tensor([1, 1, 1]));
console.log(x.grad);  // [1, 1, 1]
console.log(y.grad);  // [1, 1, 1]

sub

function

Element-wise subtraction with automatic differentiation.Parameters:

other (GradTensor) - Tensor to subtract

Returns: GradTensor result

mul

function

Element-wise multiplication with automatic differentiation.Parameters:

other (GradTensor) - Tensor to multiply

Returns: GradTensor result

const x = parameter([2, 3], { requiresGrad: true });
const y = parameter([4, 5], { requiresGrad: true });
const z = x.mul(y);  // [8, 15]
z.backward(tensor([1, 1]));
console.log(x.grad);  // [4, 5]
console.log(y.grad);  // [2, 3]

div

function

Element-wise division with automatic differentiation.Parameters:

other (GradTensor) - Divisor tensor

Returns: GradTensor result

neg

function

Element-wise negation with automatic differentiation.Returns: GradTensor result

abs

function

Element-wise absolute value with automatic differentiation.Returns: GradTensor result

pow

function

Element-wise power with automatic differentiation.Parameters:

exponent (number) - Exponent value

Returns: GradTensor result

const x = parameter([2, 3], { requiresGrad: true });
const y = x.pow(2);  // [4, 9]
y.backward(tensor([1, 1]));
console.log(x.grad);  // [4, 6] (2*x)

sqrt

function

Element-wise square root with automatic differentiation.Returns: GradTensor result

square

function

Element-wise square with automatic differentiation.Returns: GradTensor result

clip

function

Clip values with automatic differentiation.Parameters:

minVal (number) - Minimum value
maxVal (number) - Maximum value

Returns: GradTensor result

Mathematical Functions

exp

function

Element-wise exponential with automatic differentiation.Returns: GradTensor result

const x = parameter([0, 1], { requiresGrad: true });
const y = x.exp();  // [1, 2.718...]
y.backward(tensor([1, 1]));
console.log(x.grad);  // [1, 2.718...] (exp(x))

log

function

Element-wise natural logarithm with automatic differentiation.Returns: GradTensor result

const x = parameter([1, 2], { requiresGrad: true });
const y = x.log();
y.backward(tensor([1, 1]));
console.log(x.grad);  // [1, 0.5] (1/x)

tanh

function

Element-wise hyperbolic tangent with automatic differentiation.Returns: GradTensor result

Activation Functions

relu

function

ReLU activation with automatic differentiation.Returns: GradTensor result

const x = parameter([-1, 2], { requiresGrad: true });
const y = x.relu();  // [0, 2]
y.backward(tensor([1, 1]));
console.log(x.grad);  // [0, 1]

leakyRelu

function

Leaky ReLU activation with automatic differentiation.Parameters:

negativeSlope (number) - Slope for negative values (default: 0.01)

Returns: GradTensor result

sigmoid

function

Sigmoid activation with automatic differentiation.Returns: GradTensor result

const x = parameter([0], { requiresGrad: true });
const y = x.sigmoid();  // [0.5]
y.backward(tensor([1]));
console.log(x.grad);  // [0.25] (sigmoid(x) * (1 - sigmoid(x)))

Reduction Operations

sum

function

Sum with automatic differentiation.Parameters:

axis (Axis) - Optional axis to reduce along
keepdims (boolean) - Keep reduced dimensions (default: false)

Returns: GradTensor result

const x = parameter([[1, 2], [3, 4]], { requiresGrad: true });
const y = x.sum();
y.backward();
console.log(x.grad);  // [[1, 1], [1, 1]]

mean

function

Mean with automatic differentiation.Parameters:

axis (Axis) - Optional axis to reduce along
keepdims (boolean) - Keep reduced dimensions (default: false)

Returns: GradTensor result

max

function

Maximum with automatic differentiation.Parameters:

axis (Axis) - Optional axis to reduce along
keepdims (boolean) - Keep reduced dimensions (default: false)

Returns: GradTensor resultNote: When multiple elements share the maximum value, all tied positions receive the gradient.

min

function

Minimum with automatic differentiation.Parameters:

axis (Axis) - Optional axis to reduce along
keepdims (boolean) - Keep reduced dimensions (default: false)

Returns: GradTensor resultNote: When multiple elements share the minimum value, all tied positions receive the gradient.

Shape Operations

reshape

function

Reshape with automatic differentiation.Parameters:

newShape (Shape) - Desired shape

Returns: GradTensor result

const x = parameter([1, 2, 3, 4], { requiresGrad: true });
const y = x.reshape([2, 2]);
y.backward(tensor([[1, 1], [1, 1]]));
console.log(x.grad);  // [1, 1, 1, 1]

flatten

function

Flatten to 1D with automatic differentiation.Returns: GradTensor result

transpose

function

Transpose with automatic differentiation.Parameters:

axes (readonly number[]) - Optional axis permutation

Returns: GradTensor result

view

function

Create a view with automatic differentiation.Parameters:

shape (Shape) - Desired shape
strides (readonly number[]) - Optional strides
offset (number) - Optional offset

Returns: GradTensor result

Indexing Operations

slice

function

Slice with automatic differentiation.Parameters:

...ranges (SliceRange[]) - Slice specifications

Returns: GradTensor result

const x = parameter([[1, 2, 3], [4, 5, 6]], { requiresGrad: true });
const y = x.slice(0);
y.backward(tensor([1, 1, 1]));
console.log(x.grad);  // [[1, 1, 1], [0, 0, 0]]

gather

function

Gather with automatic differentiation.Parameters:

indices (GradTensor) - Index tensor
axis (Axis) - Axis to gather along

Returns: GradTensor result

Linear Algebra

matmul

function

Matrix multiplication with automatic differentiation.Parameters:

other (GradTensor) - Right-hand tensor

Returns: GradTensor result

const A = parameter([[1, 2], [3, 4]], { requiresGrad: true });
const B = parameter([[5, 6], [7, 8]], { requiresGrad: true });
const C = A.matmul(B);
C.backward(tensor([[1, 1], [1, 1]]));
console.log(A.grad);
console.log(B.grad);

Creation Functions

parameter

function

Create a GradTensor from data (learnable parameter).Parameters:

data (number | number[] | number[][] | number[][][]) - Initial values
options (GradTensorOptions) - Optional { requiresGrad?, dtype? }

Returns: GradTensor

import { parameter } from 'deepbox/ndarray';

const weights = parameter([[0.1, 0.2], [0.3, 0.4]], {
  requiresGrad: true,
  dtype: 'float32'
});

GradTensor.fromTensor

function

Create a GradTensor from an existing Tensor.Parameters:

t (Tensor) - Input tensor
options (GradTensorOptions) - Optional { requiresGrad?, dtype? }

Returns: GradTensor

import { GradTensor, tensor } from 'deepbox/ndarray';

const t = tensor([1, 2, 3]);
const gt = GradTensor.fromTensor(t, { requiresGrad: true });

GradTensor.scalar

function

Create a scalar GradTensor.Parameters:

value (number) - Scalar value
options (GradTensorOptions) - Optional { requiresGrad?, dtype? }

Returns: GradTensor

import { GradTensor } from 'deepbox/ndarray';

const learningRate = GradTensor.scalar(0.01, { requiresGrad: false });

Context Management

noGrad

function

Temporarily disable gradient tracking.Parameters:

callback (() => T) - Synchronous function to execute without gradient tracking

Returns: Callback return value

import { noGrad, parameter } from 'deepbox/ndarray';

const x = parameter([1, 2, 3], { requiresGrad: true });

// Inference mode - no gradient tracking
const result = noGrad(() => {
  return x.mul(x).sum();
});

// Gradient tracking is restored here

Note: Only accepts synchronous callbacks. Async functions will throw an error.

Gradient-Aware Operations

These are specialized versions of operations with gradient support:

softmaxGrad

function

Softmax with gradient support.Parameters:

t (Tensor | GradTensor) - Input tensor
axis (Axis) - Axis to compute along (default: -1)

Returns: Tensor or GradTensor

logSoftmaxGrad

function

Log-softmax with gradient support.Parameters:

t (Tensor | GradTensor) - Input tensor
axis (Axis) - Axis to compute along (default: -1)

Returns: Tensor or GradTensor

varianceGrad

function

Variance with gradient support.Parameters:

t (Tensor | GradTensor) - Input tensor
axis (Axis) - Optional axis to reduce along
keepdims (boolean) - Keep reduced dimensions (default: false)

Returns: Tensor or GradTensor

dropoutGrad

function

Dropout with gradient support.Parameters:

t (Tensor | GradTensor) - Input tensor
rate (number) - Dropout probability
training (boolean) - Whether in training mode

Returns: Tensor or GradTensor

im2colGrad

function

Image-to-column transformation with gradient support.Parameters:

input (Tensor | GradTensor) - Input tensor
kernelSize ([number, number]) - Kernel dimensions
stride (number | [number, number]) - Stride
padding (number | [number, number]) - Padding

Returns: Tensor or GradTensor

Types

GradTensorOptions

type

Options for creating GradTensors.

type GradTensorOptions = {
  requiresGrad?: boolean;
  dtype?: Exclude<DType, 'string'>;
};

Static Methods

GradTensor.isGradTensor

function

Check if a value is a GradTensor.Parameters:

value (unknown) - Value to check

Returns: Type guard boolean

import { GradTensor, parameter } from 'deepbox/ndarray';

const x = parameter([1, 2, 3]);
console.log(GradTensor.isGradTensor(x));  // true

Example: Training Loop

import { parameter, noGrad } from 'deepbox/ndarray';

// Create learnable parameters
const W = parameter([[0.1, 0.2], [0.3, 0.4]], { requiresGrad: true });
const b = parameter([0, 0], { requiresGrad: true });

for (let epoch = 0; epoch < 100; epoch++) {
  // Forward pass
  const y = W.matmul(x).add(b);
  const loss = y.sub(target).square().sum();

  // Backward pass
  W.zeroGrad();
  b.zeroGrad();
  loss.backward();

  // Update parameters (without tracking gradients)
  noGrad(() => {
    W.tensor.data.forEach((val, i) => {
      W.tensor.data[i] = val - 0.01 * W.grad!.data[i];
    });
    b.tensor.data.forEach((val, i) => {
      b.tensor.data[i] = val - 0.01 * b.grad!.data[i];
    });
  });
}

NDArray

DataFrame

Linear Algebra

Statistics

Machine Learning

Neural Networks

Optimization

Preprocessing

Metrics

Random

Plotting

Datasets

GradTensor Class

Properties

Gradient Operations

Arithmetic Operations

Mathematical Functions

Activation Functions

Reduction Operations

Shape Operations

Indexing Operations

Linear Algebra

Creation Functions

Context Management

Gradient-Aware Operations

Types

Static Methods

Example: Training Loop

Build docs developers (and LLMs) love

NDArray

DataFrame

Linear Algebra

Statistics

Machine Learning

Neural Networks

Optimization

Preprocessing

Metrics

Random

Plotting

Datasets

​GradTensor Class

​Properties

​Gradient Operations

​Arithmetic Operations

​Mathematical Functions

​Activation Functions

​Reduction Operations

​Shape Operations

​Indexing Operations

​Linear Algebra

​Creation Functions

​Context Management

​Gradient-Aware Operations

​Types

​Static Methods

​Example: Training Loop

Build docs developers (and LLMs) love

GradTensor Class

Properties

Gradient Operations

Arithmetic Operations

Mathematical Functions

Activation Functions

Reduction Operations

Shape Operations

Indexing Operations

Linear Algebra

Creation Functions

Context Management

Gradient-Aware Operations

Types

Static Methods

Example: Training Loop