Skip to main content

​
Overview

The lncrawl CLI is built with Typer and organized into subcommands. Running lncrawl with no arguments starts the web server on 127.0.0.1:8080. 
$ lncrawl -h
Usage: lncrawl [OPTIONS] COMMAND [ARGS]...

╭─ Options ────────────────────────────────────────────────────────────────────╮
│ --verbose             -l            Log levels: -l = warn, -ll = info, -lll  │
│                                     = debug                                  │
│ --config              -c      PATH  Config file                              │
│ --install-completion                Install completion for the current       │
│                                     shell.                                   │
│ --show-completion                   Show completion for the current shell,   │
│                                     to copy it or customize the              │
│                                     installation.                            │
│ --help                -h            Show this message and exit.              │
╰──────────────────────────────────────────────────────────────────────────────╯
╭─ Commands ───────────────────────────────────────────────────────────────────╮
│ version  Show current version.                                               │
│ config   View and modify configuration settings.                             │
│ sources  Manage sources.                                                     │
│ crawl    Crawl from novel page URL.                                          │
│ search   Search for novels by query string.                                  │
│ server   Run web server.                                                     │
╰──────────────────────────────────────────────────────────────────────────────╯

​
Global options

FlagShortDescription
--verbose-lIncrease log verbosity. Use -l for warn, -ll for info, -lll for debug.
--config PATH-c PATHPath to a config file to load instead of the default.
--install-completionInstall shell tab-completion for the current shell.
--show-completionPrint the shell completion script to stdout.
--help-hShow the help message and exit.

​
crawl

Download chapters from a novel page URL and export them to one or more formats. 
lncrawl crawl [OPTIONS] [URL]

​
Flags

FlagDescription
--noinDisable interactive mode. Requires all options to be passed as flags.
--allDownload all chapters.
--first NDownload the first N chapters (N must be ≥ 1).
--last NDownload the latest N chapters (N must be ≥ 1).
-f / --format FORMATOutput format. Can be repeated for multiple formats. See Output formats for the full list.
--user TEXTUsername or email for sources that require login.
--pass TEXTPassword or token for sources that require login.
URL(Argument) Novel details page URL.

​
Examples

# Interactive mode — prompts for URL, chapter range, and formats
lncrawl crawl

# Download all chapters as EPUB
lncrawl crawl --all -f epub https://novelfull.com/my-novel.html

# Download first 50 chapters as EPUB and MOBI
lncrawl crawl --first 50 -f epub -f mobi https://novelfull.com/my-novel.html

# Download latest 10 chapters, non-interactive
lncrawl crawl --noin --last 10 -f epub https://novelfull.com/my-novel.html

# Crawl a source that requires login
lncrawl crawl --user [email protected] --pass secret -f epub https://some-site.com/novel
When --noin is set and no format is specified with -f, all available formats are produced.
Search for novels across all searchable sources. 
lncrawl search [OPTIONS] [QUERY]

​
Flags

FlagShortDefaultDescription
--source TEXT-sFilter which sources to search (matches against source URL).
--concurrency N-c15Maximum number of concurrent source searches (1–25).
--limit N-l10Maximum number of results to return (1–25).
--timeout SECONDS-t30Per-source timeout in seconds.
QUERY(Argument) Search query string. Must be at least 2 characters.

​
Examples

# Interactive prompt for query
lncrawl search

# Search all sources
lncrawl search "solo leveling"

# Search only within novelfull sources
lncrawl search "overlord" --source "novelfull"

# Return up to 20 results with 10 concurrent searches
lncrawl search "reincarnation" --limit 20 --concurrency 10

​
server

Start the Lightnovel Crawler web server with a browser UI and REST API. 
lncrawl server [OPTIONS]

​
Flags

FlagShortDefaultDescription
--host TEXT-h0.0.0.0Host address to bind the server to.
--port INTEGER-p8080Port to listen on.
--watch-wFalseRun in watch/reload mode (auto-restarts on code changes).
--worker INTEGER-n1Number of worker processes (only used with --watch).

​
Examples

# Start server on default port 8080
lncrawl server

# Start server with info-level logging on port 8080
lncrawl -ll server -p 8080

# Bind to localhost only
lncrawl server --host 127.0.0.1 --port 9000

# Watch mode (auto-reload)
lncrawl server --watch
See Web server for full setup instructions.

​
sources

Manage and inspect crawler sources. 
lncrawl sources COMMAND [ARGS]...
List all available crawler sources.
lncrawl sources list [OPTIONS]
FlagShortDefaultDescription
--query TEXT-qFilter sources by keyword in the URL.
--can-search-sShow only sources that support novel search.
--can-login-lShow only sources that support login.
--mtl-bShow only machine-translated sources.
--manga-mShow only manga/manhua sources.
--with-rejectedFalseInclude rejected or disabled sources.
--output-type TEXT-otableOutput type: table, json, yaml, csv, or text.
# List all sources as a table
lncrawl sources list

# Filter to searchable English sources
lncrawl sources list --query en --can-search

# Export full list as JSON
lncrawl sources list -o json
Inspect a single source crawler.
lncrawl sources view QUERY
The QUERY argument accepts a crawler name, URL, or file path.
lncrawl sources view novelfull
lncrawl sources view https://novelfull.com
Scaffold a new source crawler (optionally using OpenAI for auto-generation).
lncrawl sources create [OPTIONS] [URL]
FlagShortDescription
--noinDisable interactive mode.
--locale TEXT-lContent language ISO 639-1 code (e.g. en, zh).
--features FEATURE-fCrawler features to enable. Repeatable: search, login, manga, mtl, volumes.
--openaiUse OpenAI to auto-generate the crawler implementation.
--overwriteReplace an existing crawler file.
URLTarget website URL.
# Interactive scaffold
lncrawl sources create

# Non-interactive with OpenAI auto-generation
lncrawl sources create --noin --locale en -f search --openai https://example.com

​
config

View and modify configuration settings. 
lncrawl config COMMAND [ARGS]...
Display all configuration sections and values.
lncrawl config view [--format yaml|json]

lncrawl config view
lncrawl config view --format yaml
lncrawl config view --format json
Read a single configuration value.
lncrawl config get [SECTION] [KEY]

lncrawl config get app openai_api_key
lncrawl config get crawler runner_concurrency
Write a configuration value.
lncrawl config set [SECTION] [KEY] [VALUE]

lncrawl config set server base_url "https://example.com"
lncrawl config set crawler runner_concurrency "10"
lncrawl config set mail smtp_port "587"

​
version

Print the installed version and exit. 
lncrawl version

Build docs developers (and LLMs) love

Get started for freeTalk to us