Data management scripts

Overview

The Aero scripts directory contains data management utilities built with Bun and TypeScript. These scripts help you import airport data from CSV files, migrate airline information, and interact with the API programmatically.

Location

The scripts are located in the scripts/ directory:

scripts/
├── index.ts         # CSV airport import
├── airport.ts       # Airport data migration
├── flights.ts       # Flight data fetching
├── json.ts          # JSON airline import
├── pb.ts            # PocketBase client (legacy)
├── data.json        # Source data for imports
├── package.json     # Dependencies
└── tsconfig.json    # TypeScript config

Prerequisites

The scripts require:

Bun runtime (v1.1.26 or higher)
PostgreSQL database (for API imports)
API server running (for direct API imports)

Installation

Navigate to scripts directory

cd scripts

Install dependencies

bun install

This installs:

csv (v6.3.11) - CSV parsing library
pocketbase (v0.24.0) - Legacy PocketBase client

Prepare data files

Place your data files in the scripts/ directory:

airport-codes.csv - Airport data (for index.ts)
data.json - Airline data (for json.ts)

Available scripts

Airport CSV import (index.ts)

Imports airport data from a CSV file into the database. CSV format:

ident,type,name,elevation,continent,iso_country,iso_region,municipality,gps_code,iata_code,local_code,coordinates
KJFK,large_airport,John F Kennedy International Airport,13,NA,US,US-NY,New York,KJFK,JFK,JFK,"40.6398, -73.7789"

Usage:

bun run index.ts

Implementation:

import fs from 'node:fs';
import { parse } from 'csv-parse';
import { pb } from './pb';

const processFile = async () => {
    const records = [];
    const parser = fs
        .createReadStream(`${__dirname}/airport-codes.csv`)
        .pipe(parse({}));
    
    for await (const record of parser) {
        if (record[0].startsWith("ident")) continue
        const [lat, long] = record[record.length - 1].split(", ")
        records.push({
            ident: record[0],
            type: record[1],
            name: record[2],
            elevation: record[3],
            continent: record[4],
            iso_country: record[5],
            iso_region: record[6],
            municipality: record[7],
            gps_code: record[8],
            iata_code: record[9],
            local_code: record[10],
            long, lat
        })
    }
    return records;
};

This script was originally designed for PocketBase. The create calls are commented out but can be adapted for the NestJS API.

Airport data migration (airport.ts)

Migrates airport data from PocketBase to the NestJS API. Usage:

bun run airport.ts

Implementation:

import { pb } from "./pb";

const airports = await pb.collection("airports").getFullList();

for (const airport of airports) {
  const record = await pb.collection("airports").getOne(airport.id);
  const data = await fetch("http://localhost:5000/v1/airports", {
    method: "POST",
    body: JSON.stringify({
      ident: record.ident,
      type: record.type,
      elevation: Number(record.elevation),
      continent: record.continent,
      isoCountry: record.iso_country,
      municipality: record.municipality,
      gpsCode: record.gps_code,
      iataCode: record.iata_code,
      name: record.name,
      lat: record.lat,
      long: record.long,
      isoRegion: record.iso_region,
    }),
    headers: {
      "Content-Type": "application/json",
    },
  });
  console.log(await data.json());
}

Update the API URL from http://localhost:5000 to http://localhost:3000 if using the current NestJS API.

JSON airline import (json.ts)

Imports airline data from a JSON file (typically generated by the scraper). Usage:

bun run json.ts

Expected JSON format:

[
  {
    "name": "american airlines",
    "fleet": "AA / AAL"
  },
  {
    "name": "delta air lines",
    "fleet": "DL / DAL"
  }
]

Implementation:

import flights from "./data.json"
import { pb } from "./pb"

function capitalize(text: string) {
    if (!text) return "";
    return text[0].toUpperCase() + text.slice(1);
}

for (const flight of flights) {
    const name = flight.name
    const [iata, icao] = flight.fleet.split("/").map((e) => e.trim())

    await pb.collection("airlines").create({
        name: capitalize(name),
        iata,
        icao
    }, { requestKey: null })
}
console.log(flights.length)

This script:

Reads data.json from the scraper output
Capitalizes airline names
Splits IATA/ICAO codes from the fleet field
Creates airline records

Flight data fetching (flights.ts)

Fetches flight information from the API for testing. Usage:

bun run flights.ts

Implementation:

import { pb } from "./pb";

// Example: Search flights
const res = await pb.send("/flights", {
  query: {
    from: "DEL",
    to: "SYD",
    date: "2025-01-01",
  },
});

// Example: Get specific flight
const res = await pb.send("/flight", {
  query: {
    id: "MH141",
  },
  requestKey: null,
});
console.dir(res, { depth: null });

This script uses the legacy PocketBase client. Adapt it to use the NestJS API endpoints at http://localhost:3000.

Adapting scripts for NestJS API

The scripts were originally written for PocketBase. Here’s how to adapt them for the NestJS API:

Replace PocketBase client with fetch

import { pb } from "./pb";

const airports = await pb.collection("airports").getFullList();

Update API endpoints

PocketBase	NestJS
`/airports`	`/v1/airports`
`/airlines`	`/v1/airlines`
`/flights`	`/v1/flights/search`
`/flight`	`/v1/flights/:id`

Authentication

The NestJS API requires JWT authentication for most endpoints:

// Login first
const loginResponse = await fetch("http://localhost:3000/v1/auth/login", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({
    email: "[email protected]",
    password: "password",
  }),
});
const { accessToken } = await loginResponse.json();

// Use token in subsequent requests
const response = await fetch("http://localhost:3000/v1/airports", {
  headers: {
    "Authorization": `Bearer ${accessToken}`,
  },
});

Creating custom scripts

You can create custom scripts for specific data management tasks:

Example: Bulk airport update

// update-airports.ts
interface Airport {
  id: string;
  iataCode: string;
  name: string;
}

const airports: Airport[] = await fetch(
  "http://localhost:3000/v1/airports"
).then(r => r.json());

for (const airport of airports) {
  // Update each airport
  await fetch(`http://localhost:3000/v1/airports/${airport.id}`, {
    method: "PATCH",
    headers: {
      "Content-Type": "application/json",
      "Authorization": `Bearer ${token}`,
    },
    body: JSON.stringify({
      name: airport.name.toUpperCase(),
    }),
  });
  
  console.log(`Updated ${airport.iataCode}`);
}

Example: Data validation

// validate-data.ts
const airlines = await fetch(
  "http://localhost:3000/v1/airlines"
).then(r => r.json());

const invalid = airlines.filter(airline => 
  !airline.iata || !airline.icao || !airline.name
);

console.log(`Found ${invalid.length} invalid airlines:`);
invalid.forEach(airline => {
  console.log(`- ${airline.name}: IATA=${airline.iata}, ICAO=${airline.icao}`);
});

Running scripts with Bun

Bun provides fast execution and built-in TypeScript support:

bun run index.ts

Data sources

Airport data

OurAirports - Free airport database with ICAO/IATA codes
OpenFlights - Community-maintained airport data
FAA Database - Official US airport information

Airline data

Use the web scraper to collect airline data, or use official sources:

IATA Airline Database - Official airline codes
Wikipedia - List of airline IATA/ICAO codes
Aviation databases - Commercial aviation data providers

Best practices

Validate data before import - Check for required fields and correct formats
Use transactions - Wrap bulk operations in database transactions when possible
Log progress - Output progress to track long-running imports
Handle errors gracefully - Skip invalid records and log errors
Backup before import - Always backup your database before running bulk imports
Rate limit API calls - Add delays between requests to avoid overwhelming the API
Use environment variables - Store API URLs and tokens in environment variables

Troubleshooting

Module not found

Ensure dependencies are installed:

bun install

API connection errors

Verify the API server is running:

cd ../api
pnpm run start:dev

CSV parsing errors

Check the CSV format and delimiter:

const parser = fs
  .createReadStream(csvPath)
  .pipe(parse({
    delimiter: ',',
    skip_empty_lines: true,
  }));

Authentication errors

Most API endpoints require authentication. Get a JWT token first:

curl -X POST http://localhost:3000/v1/auth/login \
  -H "Content-Type: application/json" \
  -d '{"email":"[email protected]","password":"password"}'

Contributing Guidelines - Development workflow and standards
Web Scraper - Generate data.json for airline imports
OpenAPI Client - Use the typed API client in scripts

Getting Started

Mobile App

Wear OS

Backend

Development

Data management scripts

Overview

Location

Prerequisites

Installation

Available scripts

Airport CSV import (index.ts)

Airport data migration (airport.ts)

JSON airline import (json.ts)

Flight data fetching (flights.ts)

Adapting scripts for NestJS API

Replace PocketBase client with fetch

Update API endpoints

Authentication

Creating custom scripts

Example: Bulk airport update

Example: Data validation

Running scripts with Bun

Data sources

Airport data

Airline data

Best practices

Troubleshooting

Module not found

API connection errors

CSV parsing errors

Authentication errors

Build docs developers (and LLMs) love

Getting Started

Mobile App

Wear OS

Backend

Development

​Overview

​Location

​Prerequisites

​Installation

​Available scripts

​Airport CSV import (index.ts)

​Airport data migration (airport.ts)

​JSON airline import (json.ts)

​Flight data fetching (flights.ts)

​Adapting scripts for NestJS API

​Replace PocketBase client with fetch

​Update API endpoints

​Authentication

​Creating custom scripts

​Example: Bulk airport update

​Example: Data validation

​Running scripts with Bun

​Data sources

​Airport data

​Airline data

​Best practices

​Troubleshooting

​Module not found

​API connection errors

​CSV parsing errors

​Authentication errors

​Related documentation

Build docs developers (and LLMs) love

Overview

Location

Prerequisites

Installation

Available scripts

Airport CSV import (index.ts)

Airport data migration (airport.ts)

JSON airline import (json.ts)

Flight data fetching (flights.ts)

Adapting scripts for NestJS API

Replace PocketBase client with fetch

Update API endpoints

Authentication

Creating custom scripts

Example: Bulk airport update

Example: Data validation

Running scripts with Bun

Data sources

Airport data

Airline data

Best practices

Troubleshooting

Module not found

API connection errors

CSV parsing errors

Authentication errors

Related documentation