Skip to main content
The Text File Loader enables you to load and process text-based files in various formats, including plain text, code files, markdown, HTML, and more.

​
Overview

This versatile loader uses LangChain’s TextLoader to read text content from files. It supports a wide range of text-based file formats, making it ideal for loading documentation, code, configuration files, and other text content.

​
Supported File Types

The Text File Loader supports an extensive list of file formats:
  • .txt - Plain text files
  • .md / .markdown - Markdown files
  • .html - HTML documents
  • .xml - XML documents
  • .rst - reStructuredText
  • .tex / .ltx - LaTeX documents

​
Configuration

​
txtFile
file
required
The text file(s) to load. Supports multiple file upload.Supported Extensions: See the comprehensive list above
​
textSplitter
TextSplitter
Optional text splitter to chunk the file content into smaller segments.
Recommended for large files or when processing with LLMs that have context limits.

​
Advanced Parameters

​
metadata
json
Additional metadata to attach to the extracted documents.
{
  "project": "flowise",
  "file_type": "documentation",
  "language": "markdown",
  "last_updated": "2024-01-15"
}
​
omitMetadataKeys
string
Comma-separated list of metadata keys to exclude from the output.Example: source, blobTypeSpecial value: Use * to omit all default metadata and only include your custom metadata.

​
Output

The Text File Loader provides two output formats:
Returns an array of document objects containing the file content and metadata.
[
  {
    "pageContent": "# Getting Started\n\nThis is a markdown file...",
    "metadata": {
      "source": "blob",
      "blobType": "text/plain",
      "loc": {
        "lines": {
          "from": 1,
          "to": 1
        }
      }
    }
  }
]

​
Usage Examples

​
Loading a Single Text File

1

Add Text File Node

Drag the Text File node from the Document Loaders category onto your canvas.
2

Upload File

Click the file input and select your text file (e.g., README.md, config.json, script.py).
3

Optional: Add Text Splitter

For large files, connect a text splitter to chunk the content appropriately.
4

Connect to Workflow

Connect the output to your vector store, LLM, or other processing nodes.

​
Loading Multiple Files

// Upload multiple markdown files:
// - README.md
// - CONTRIBUTING.md
// - CHANGELOG.md

// Each file will be processed as a separate document
// with its own metadata
When you upload multiple files, each file is processed independently and added to the document array.

​
Loading Code Files

# Upload a Python file
# The entire code will be preserved as text

def hello_world():
    print("Hello, World!")

if __name__ == "__main__":
    hello_world()
Code files are loaded as plain text. If you need syntax-aware processing, consider using specialized code analysis tools after loading.

​
With Custom Metadata

{
  "repository": "flowise-ai/flowise",
  "file_path": "src/utils/index.ts",
  "author": "dev-team",
  "version": "1.4.0",
  "tags": ["utility", "helper"]
}
This metadata helps with:
  • Organizing loaded documents
  • Filtering during retrieval
  • Tracking document sources
  • Version control integration

​
Common Use Cases

Code Documentation

Load README files and documentation for AI-assisted code exploration

Knowledge Base

Import markdown documentation into searchable knowledge bases

Configuration Analysis

Load and analyze configuration files (JSON, YAML, XML)

Content Processing

Process text content for summarization, analysis, or transformation

​
Specific Scenarios

Scenario: Load all markdown files from a documentation repositorySteps:
  1. Upload all .md files from your docs folder
  2. Add metadata to tag documentation sections
  3. Use a text splitter to create searchable chunks
  4. Store in a vector database for semantic search
  5. Build a Q&A chatbot over your documentation
Scenario: Index source code for AI-assisted developmentSteps:
  1. Load code files (.js, .ts, .py, etc.)
  2. Add metadata for file paths and modules
  3. Use code-aware text splitters
  4. Enable developers to ask questions about the codebase
Scenario: Load translated documentation in multiple languagesSteps:
  1. Upload files for each language
  2. Add language field to metadata
  3. Use language-specific text splitters if available
  4. Enable language-filtered retrieval

​
File Storage Reference

The Text File Loader supports both direct file uploads and file storage references:
Files are uploaded directly through the UI and converted to base64.
data:text/plain;base64,SGVsbG8gV29ybGQ=

​
Best Practices

Recommended Practices
  1. Use Text Splitters: Always use text splitters for files larger than a few hundred lines
  2. Add Metadata: Include source information for better retrieval and debugging
  3. Encoding: Ensure files are UTF-8 encoded to avoid character issues
  4. Organization: Use consistent naming conventions for uploaded files
Important Considerations
  • Binary Files: This loader is for text files only. Binary files may produce garbled output.
  • Large Files: Files with millions of lines may cause memory issues. Consider splitting large files before upload.
  • Encoding: Non-UTF-8 encoded files may have character encoding issues.

​
Troubleshooting

Cause: File encoding is not UTF-8Solution: Convert your file to UTF-8 encoding before uploading:
iconv -f ISO-8859-1 -t UTF-8 input.txt > output.txt
Cause: File exceeds memory limitsSolutions:
  1. Split the file into smaller chunks before uploading
  2. Use a more aggressive text splitter configuration
  3. Process the file in batches
Cause: File extension not in the supported listSolutions:
  1. Rename the file to a supported extension if it’s text-based
  2. Convert the file to a supported format
  3. Use a different, more appropriate loader (e.g., PDF loader for PDFs)

​
Performance Tips

Optimization Strategies
  • Chunk Size: Set appropriate chunk sizes based on your LLM’s context window
  • Metadata: Keep metadata lightweight to reduce storage overhead
  • Batch Processing: Upload multiple related files together for efficiency
  • Caching: Reuse loaded documents when possible instead of re-loading

File Loader

Generic loader for multiple file types

Vector Stores

Store text content for semantic search

Document Loaders

Explore all document loader options

Build docs developers (and LLMs) love

Get started for freeTalk to us