Skip to main content
The generate() function transforms an array of tokens (from tokenize()) into an array of line objects. Each line contains styled token elements with appropriate CSS classes.

​
Signature

function generate(tokens: Array<Token>): Line[]

​
Parameters

​
tokens
Array<Token>
required
An array of tokens produced by the tokenize() function. Each token is a tuple of [type: number, value: string].

​
Returns

​
Line[]
Array<Line>
An array of line objects. Each line has the following structure:
{
  type: "element",
  tagName: "span",
  children: Child[],
  properties: {
    className: "css__line"
  }
}
Each child token within a line has this structure:
{
  type: "element",
  tagName: "span",
  children: [{ type: "text", value: string }],
  properties: {
    className: `css__token--${tokenType}`,
    style: {
      color: `var(--css-${tokenType})`
    }
  }
}

​
Example

import { tokenize, generate } from 'code-syntactic-sugar';

const code = `const a = 10;
const b = 20;`;

const tokens = tokenize(code);
const lines = generate(tokens);

console.log(lines.length); // 2 (two lines of code)

​
Token to CSS Class Mapping

The generate() function applies CSS classes based on token types:
Token TypeCSS ClassCSS Variable
identifiercss__token--identifiervar(--css-identifier)
keywordcss__token--keywordvar(--css-keyword)
stringcss__token--stringvar(--css-string)
classcss__token--classvar(--css-class)
propertycss__token--propertyvar(--css-property)
entitycss__token--entityvar(--css-entity)
jsxliteralscss__token--jsxliteralsvar(--css-jsxliterals)
signcss__token--signvar(--css-sign)
commentcss__token--commentvar(--css-comment)
breakN/A (line separator)N/A
spacecss__token--spacevar(--css-space)

​
Line Structure Details

import { tokenize, generate } from 'code-syntactic-sugar';

const code = `let x = 1;`;
const tokens = tokenize(code);
const lines = generate(tokens);

// lines[0] structure:
// {
//   type: "element",
//   tagName: "span",
//   properties: { className: "css__line" },
//   children: [
//     {
//       type: "element",
//       tagName: "span",
//       properties: {
//         className: "css__token--keyword",
//         style: { color: "var(--css-keyword)" }
//       },
//       children: [{ type: "text", value: "let" }]
//     },
//     // ... more tokens
//   ]
// }

​
Multi-Line Token Handling

import { tokenize, generate } from 'code-syntactic-sugar';

const code = `/* This is a
multi-line
comment */`;

const tokens = tokenize(code);
const lines = generate(tokens);

console.log(lines.length); // 3 lines
// The multi-line comment token is automatically split
// across multiple line objects

​
Notes

The generate() function automatically splits tokens that contain newline characters into multiple lines, ensuring correct line-by-line rendering.
Break tokens (type 9) are used as line separators. They create a new line but don’t add visible content.
Each line has the CSS class css__line, which is useful for styling individual lines or adding line numbers with CSS counters.
The function uses CSS custom properties (variables) for token colors, allowing easy theme customization:
:root {
  --css-keyword: #f47067;
  --css-string: #00a99a;
  --css-identifier: #354150;
  /* ... etc */
}
This is a low-level API. Most users should use the highlight() function instead, which calls generate() internally after tokenization.

Build docs developers (and LLMs) love

Get started for freeTalk to us