Skip to main content

​
Overview

NL2FOL is an AI/ML research project that automatically translates natural language arguments into first-order logic (FOL) formulas to detect logical fallacies. The system leverages Large Language Models (LLMs) combined with Satisfiability Modulo Theories (SMT) solvers to formally verify the validity of logical arguments.

​
How It Works

The NL2FOL pipeline consists of three main stages:
1

Natural Language to First-Order Logic

The system uses LLMs to extract claims, implications, referring expressions, and properties from natural language sentences, then converts them into formal first-order logic formulas.
2

FOL to SMT Conversion

First-order logic formulas are converted into SMT-LIB format and processed using CVC4/CVC5 solvers to check satisfiability.
3

Result Interpretation

The SMT solver results are interpreted to classify arguments as logically valid or fallacious based on whether the negated formula is unsatisfiable.

​
Key Features

Multi-LLM Support

Works with both open-source models (Llama) and commercial APIs (GPT-4)

Entity Grounding

Identifies relationships between entities using Natural Language Inference (NLI) models

Property Extraction

Automatically extracts properties and their implications from natural language

Formal Verification

Uses SMT solvers for rigorous logical validation

​
Datasets

The project includes multiple curated datasets for logical fallacy detection:
  • Logic Dataset: 3,227 general logical fallacies from educational sources
  • LogicClimate Dataset: 1,415 climate-related logical fallacies
  • NLI Datasets: Over 170,000 entailment and fallacy examples
  • FOLIO Dataset: Formal logic inference examples
Each dataset contains labeled examples distinguishing valid logical arguments (label=1) from fallacious ones (label=0).

​
Research Context

This work bridges natural language understanding and formal logic verification, enabling automated detection of reasoning errors in arguments. The approach is particularly useful for:
  • Fact-checking and misinformation detection
  • Educational tools for teaching logical reasoning
  • Automated argument quality assessment
  • Research in logical fallacy classification

​
Citation

For academic use, please cite the original paper: 
NL2FOL: Translating Natural Language to First Order Logic for Logical Fallacy Detection
ArXiv: https://arxiv.org/abs/2405.02318

​
Next Steps

Installation

Set up the NL2FOL environment and dependencies

Quick Start

Run your first logical fallacy detection

Build docs developers (and LLMs) love

Get started for freeTalk to us