Skip to main content

​
Overview

The github module provides helpers for interacting with the GitHub REST API to fetch repository metadata, file trees, and file contents. It supports both public repositories (with limited rate limits) and authenticated requests using a GitHub token. Location: apps/www/lib/github.ts

​
Features

  • Parse GitHub repository URLs in multiple formats
  • Fetch repository metadata (default branch, description, stars)
  • Retrieve complete file trees recursively
  • Download raw file contents
  • Automatic rate limit handling with optional token authentication

​
Authentication

Set the GITHUB_TOKEN environment variable to increase rate limits from 60 to 5,000 requests per hour. 
GITHUB_TOKEN=ghp_your_token_here

​
Functions

​
parseRepoUrl

Parses a GitHub repository URL or short form into structured components. 
function parseRepoUrl(input: string): {
  owner: string
  repo: string
  branch?: string
}
​
input
string
required
The repository URL or short form. Accepts:
  • Full URL: https://github.com/owner/repo
  • Full URL with branch: https://github.com/owner/repo/tree/main
  • Short form: owner/repo
  • Git URL: https://github.com/owner/repo.git
​
owner
string
The repository owner (username or organization)
​
repo
string
The repository name
​
branch
string | undefined
The branch name (only present if specified in URL)

​
Example

import { parseRepoUrl } from "@/lib/github"

// Full URL
const result1 = parseRepoUrl("https://github.com/facebook/react")
// { owner: "facebook", repo: "react" }

// Full URL with branch
const result2 = parseRepoUrl("https://github.com/vercel/next.js/tree/canary")
// { owner: "vercel", repo: "next.js", branch: "canary" }

// Short form
const result3 = parseRepoUrl("vuejs/vue")
// { owner: "vuejs", repo: "vue" }

​
Errors

Throws an error if the input format is invalid: 
// ❌ Throws: "Invalid repo URL. Expected https://github.com/owner/repo or owner/repo"
parseRepoUrl("invalid-input")

​
getRepoInfo

Fetches repository metadata from the GitHub API. Validates that the repository exists and is accessible. 
function getRepoInfo(
  owner: string,
  repo: string
): Promise<RepoInfo>
​
owner
string
required
The repository owner (username or organization)
​
repo
string
required
The repository name
​
owner
string
The repository owner
​
repo
string
The repository name
​
defaultBranch
string
The default branch name (e.g., “main” or “master”)
​
description
string | null
The repository description (null if not set)
​
stars
number
The number of stars (stargazers_count)

​
Example

import { getRepoInfo } from "@/lib/github"

const info = await getRepoInfo("vercel", "next.js")
console.log(info)
// {
//   owner: "vercel",
//   repo: "next.js",
//   defaultBranch: "canary",
//   description: "The React Framework",
//   stars: 120000
// }

​
Errors

​
getRepoTree

Fetches the complete file tree of a repository using the Git Trees API. Returns only files (blobs), not directories. 
function getRepoTree(
  owner: string,
  repo: string,
  branch: string
): Promise<TreeNode[]>
​
owner
string
required
The repository owner
​
repo
string
required
The repository name
​
branch
string
required
The branch name (e.g., “main”, “master”, “canary”)
​
TreeNode[]
array
Array of file nodes in the repository tree

​
Example

import { getRepoTree } from "@/lib/github"

const tree = await getRepoTree("vercel", "next.js", "canary")

// Filter for TypeScript files
const tsFiles = tree.filter(node => node.path.endsWith(".ts"))

// Find all locale files
const localeFiles = tree.filter(node => 
  node.path.includes("/locales/") && node.path.endsWith(".json")
)
The API uses recursive=1 to fetch the entire tree in one request. For repositories with more than 100,000 entries, the tree may be truncated. A warning is logged to the console in this case.

​
Usage in Scan API

From app/api/scan/route.ts:27: 
// Fetch file tree
const tree = await getRepoTree(owner, repo, branch)

// Detect locale files
const groups = detectLocaleFiles(tree)

​
getFileContent

Fetches the raw text content of a single file from the repository. 
function getFileContent(
  owner: string,
  repo: string,
  branch: string,
  path: string
): Promise<string>
​
owner
string
required
The repository owner
​
repo
string
required
The repository name
​
branch
string
required
The branch name
​
path
string
required
The file path relative to repository root
​
content
string
The raw text content of the file

​
Example

import { getFileContent } from "@/lib/github"

const content = await getFileContent(
  "vercel",
  "next.js",
  "canary",
  "package.json"
)

const pkg = JSON.parse(content)
console.log(pkg.version)

​
Usage in Scan API

From app/api/scan/route.ts:52: 
const fetchPromises = Object.entries(group.files).flatMap(
  ([locale, filePaths]) =>
    filePaths.map(async (filePath) => {
      const content = await getFileContent(owner, repo, branch, filePath)
      const keys = parseLocaleFile(content, filePath)
      if (!keyMaps[locale]) keyMaps[locale] = {}
      Object.assign(keyMaps[locale], keys)
    })
)
await Promise.all(fetchPromises)
Uses raw.githubusercontent.com for direct file access, which is faster than the Contents API and doesn’t require base64 decoding.

​
Types

​
RepoInfo

Repository metadata returned by getRepoInfo. 
interface RepoInfo {
  owner: string
  repo: string
  defaultBranch: string
  description: string | null
  stars: number
}

​
TreeNode

A single file node in the repository tree. 
interface TreeNode {
  path: string
  type: "blob" | "tree"
  size?: number
}

​
Rate Limits

Without Token

60 requests per hour per IP address

With Token

5,000 requests per hour per token

​
Handling Rate Limits

import { getRepoInfo } from "@/lib/github"

try {
  const info = await getRepoInfo("owner", "repo")
} catch (error) {
  if (error.message.includes("rate limit")) {
    // Handle rate limit - retry with exponential backoff
    // or prompt user to add GITHUB_TOKEN
  }
}

​
Complete Example

import { 
  parseRepoUrl, 
  getRepoInfo, 
  getRepoTree, 
  getFileContent 
} from "@/lib/github"

async function analyzeRepository(url: string) {
  // 1. Parse URL
  const { owner, repo, branch: urlBranch } = parseRepoUrl(url)
  
  // 2. Get repo info
  const info = await getRepoInfo(owner, repo)
  const branch = urlBranch ?? info.defaultBranch
  
  console.log(`📦 ${owner}/${repo} (⭐ ${info.stars})`)
  console.log(`📝 ${info.description}`)
  
  // 3. Fetch file tree
  const tree = await getRepoTree(owner, repo, branch)
  console.log(`📁 ${tree.length} files in repository`)
  
  // 4. Download specific files
  const packageJsonPath = tree.find(n => n.path === "package.json")?.path
  if (packageJsonPath) {
    const content = await getFileContent(owner, repo, branch, packageJsonPath)
    const pkg = JSON.parse(content)
    console.log(`📦 Package: ${pkg.name}@${pkg.version}`)
  }
}

analyzeRepository("https://github.com/vercel/next.js")

Build docs developers (and LLMs) love

Get started for freeTalk to us