FAQ

​

How It Works

1. Identifying refusal directions: Computing the difference in hidden states between “harmful” and “harmless” prompts
2. Orthogonalizing weight matrices: Modifying specific transformer components (attention out-projection and MLP down-projection) to inhibit these refusal directions
3. Preserving model capabilities: Making surgical changes that remove censorship while minimizing damage to the model’s general intelligence

​

Why “Abliteration”?

• Ablation: Removing or inhibiting specific model behaviors
• Obliteration: Completely eliminating censorship responses

​

Heretic’s Innovations

• Automatic parameter optimization: Uses TPE (Tree-structured Parzen Estimator) to find optimal ablation parameters
• Flexible weight kernels: Adjusts ablation strength across different layers
• Direction interpolation: Uses fractional direction indices to explore directions beyond individual layers
• Component-specific ablation: Applies different parameters to attention vs. MLP components

​

By Model Size (RTX 3090, default settings)

​

Factors Affecting Speed

• GPU model (faster GPUs = faster processing)
• VRAM capacity (affects batch size)
• CPU speed (for CPU-based operations like PaCMAP)

• Number of trials (--n-trials, default ~20)
• Batch size (auto-determined or manual)
• Quantization (slightly slower but uses less VRAM)

• Number of layers
• Hidden dimension size
• MoE models take longer than dense models

​

Phases of Processing

1. Model loading: 1-5 minutes (depends on model size and quantization)
2. Batch size benchmarking: 1-2 minutes
3. Computing refusal directions: 5-15 minutes
4. Optimization trials: 30 minutes - several hours (bulk of time)
5. Model merging (if saving): 5-20 minutes

​

Without a GPU

[bold yellow]No GPU or other accelerator detected. 
Operations will be slow.[/]

• An 8B model might take 12-24 hours instead of 45 minutes
• Larger models (20B+) could take days
• System RAM requirements are higher

​

Supported Accelerators

• CUDA (NVIDIA GPUs) - Most tested and recommended
• XPU (Intel GPUs)
• MLU (Cambricon)
• SDAA (SD Technology)
• MUSA (Moore Threads)
• NPU (Ascend)
• MPS (Apple Metal on M-series Macs)

​

Recommended Hardware

​

Enabling Quantization

heretic --quantization bnb_4bit your-model-name

quantization = "bnb_4bit"

​

Benefits

• ~4x VRAM reduction: Loads models in 4-bit precision instead of 16-bit
• Enables larger models: Run 13B models on 16GB VRAM, 70B on 48GB
• Quality preservation: Quantization affects loading, but final model can be full precision

​

Important Considerations

​

The Trade-off

• Low VRAM usage
• Slightly slower than full precision
• Fully functional optimization

• Must reload base model at full precision
• Merge LoRA adapter into base model
• Requires substantial system RAM

​

Best Practices

1. Use quantization during processing
2. Test the model using the built-in chat feature
3. Only save/upload if you have sufficient RAM, or:
4. Use a high-RAM instance specifically for the merge step

​

Benchmark: Gemma-3-12B-IT

​

What This Means

• All abliteration methods achieve similar refusal suppression
• Heretic: 3/100 (97% refusal removal)
• Comparable to best manual efforts

• Measures how much the model’s behavior has changed
• Lower KL divergence = better preservation of original capabilities
• Heretic achieves 0.16 vs. 0.45-1.04 for manual methods
• Values above 1.0 typically indicate significant capability damage

​

User Feedback

“I just downloaded GPT-OSS 20B Heretic model and holy shit. It gives properly formatted long responses to sensitive topics, using the exact uncensored words that you would expect from an uncensored model, produces markdown format tables with details and whatnot. Looks like this is the best abliterated version of this model so far…”

“Heretic GPT 20b seems to be the best uncensored model I have tried yet. It doesn’t destroy the model’s intelligence and it is answering prompts normally would be rejected by the base model.”

“Qwen3-4B-Instruct-2507-heretic has been the best unquantized abliterated model that I have been able to run on 16gb vram.”

​

Why Automatic Is Better

• Requires deep understanding of transformer internals
• Trial-and-error parameter tuning
• Time-intensive experimentation
• Hard to find optimal balance between refusal suppression and capability preservation

• Explores thousands of parameter combinations
• Uses sophisticated optimization (TPE algorithm)
• Optimizes for both refusal reduction AND capability preservation simultaneously
• Anyone can achieve expert-level results

​

Quick Summary

• Most dense transformer models
• Many multimodal (vision-language) models
• Several MoE (Mixture of Experts) architectures
• Standard decoder-only transformers

• State Space Models (SSMs) like Mamba
• Hybrid architectures (e.g., Jamba)
• Models with inhomogeneous layers
• Novel attention mechanisms that deviate from standard transformers

​

Finding Models to Use

• Over 1,000 Heretic models on Hugging Face
• Search for models tagged with heretic

• The Bestiary
• Hand-picked high-quality examples

​

Testing Compatibility

heretic model-name

​

During Processing

1. Save the model to a local folder
2. Upload the model to Hugging Face ← Select this
3. Chat with the model
4. Return to trial selection

​

Upload Process

# Option A: Login before running Heretic
huggingface-cli login

# Option B: Provide token when prompted
# Heretic will ask for your access token

• Repository name: Defaults to username/model-name-heretic
• Visibility: Public or Private

• Merging LoRA adapter (if quantized)
• Uploading model weights
• Uploading tokenizer
• Creating/updating model card with:
  • Heretic badge and tags (heretic, uncensored, abliterated)
  • Performance metrics (refusals, KL divergence)
  • Link back to base model
  • Processing details

​

Model Card Tags

• heretic - Identifies it was processed with Heretic
• uncensored - Makes it discoverable
• decensored - Alternative search term
• abliterated - Technique used

​

Best Practices

username/base-model-name-heretic

• p-e-w/gemma-3-12b-it-heretic
• your-name/llama-3.1-8b-heretic

• Base model reference
• Refusal suppression metrics
• KL divergence score
• Heretic version used
• Link to Heretic project

• Your model inherits the base model’s license
• Heretic itself is AGPL-3.0, but this doesn’t affect generated models
• Always respect the original model’s license terms

​

Manual Upload

1. Select “Save the model to a local folder”
2. Edit the model card manually
3. Upload using:

huggingface-cli upload your-name/model-name ./local-model-path

​

How Checkpointing Works

• Completed trials and their results
• Configuration parameters
• Best found parameters so far

~/.cache/heretic/studies/<model-name>.jsonl

heretic --study-checkpoint-dir /your/path model-name

​

Interrupting Safely

• Current trial will be marked as pruned
• All previous completed trials are saved
• You can resume later from the same point

​

Resuming a Run

heretic your-model-name

[yellow]You have already processed this model, but the run was interrupted.[/]

How would you like to proceed?
1. Continue the previous run
2. Ignore the previous run and start from scratch
3. Exit program

• Resumes optimization from where it stopped
• Uses the original configuration (ignores new CLI arguments)
• Completes remaining trials

• Deletes the checkpoint file
• Begins fresh optimization
• Uses current configuration

​

Completed Runs

[green]You have already processed this model.[/]

How would you like to proceed?
1. Show the results from the previous run
2. Ignore the previous run and start from scratch
3. Exit program

• Export models from completed runs
• Run additional trials for better results
• Test different trials from the Pareto front

​

Adding More Trials

Select: "Run additional trials"
Enter: 10  # Run 10 more trials