Unpreallocated file write times
This is the critical metric that prompted the creation of memefs.The benchmark images referenced below are from the source repository at
/benchmarks/ and show real-world performance comparisons.Results
Key findings:
- Original memfs: Write times increase dramatically for unpreallocated files, making it essentially unusable for downloads and dynamic file creation
- memefs: Maintains consistent performance regardless of whether files are preallocated
- Impact: Makes memefs suitable for real-world usage as a RAM disk
Why this matters
Most real-world file operations do not preallocate files:- Web browsers: Download files incrementally
- File copies: Standard copy operations don’t preallocate
- Application writes: Most apps write data as it becomes available
- Document editors: Save files without preallocation
File I/O speeds
Comparison metrics:
- Sequential read speed: How fast data can be read sequentially
- Sequential write speed: How fast data can be written sequentially
- Random read speed: Random access read performance
- Random write speed: Random access write performance
- memefs: Excellent unpreallocated write performance, good sequential performance
- Original memfs: Maximum sequential speed for preallocated files, poor unpreallocated performance
FSBench results
FSBench is a comprehensive file system benchmark that tests:
- File creation/deletion
- Directory operations
- Read/write performance
- Metadata operations
- Mixed workloads
Performance analysis
Why memefs is faster for unpreallocated files
The original memfs used heap allocation for every file write:Memory management differences
| Aspect | Original memfs | memefs |
|---|---|---|
| Storage | Per-sector heap allocation | Vector of sectors |
| Allocation | HeapAlloc per sector | Vector growth + private heap |
| Fragmentation | High (many small allocations) | Low (contiguous vectors) |
| Performance | Degrades with file size | Consistent |
| Memory overhead | Higher (heap metadata) | Lower (vector overhead) |
Threading and concurrency
Both implementations benefit from WinFsp’s multi-threaded dispatcher:- Thread pool: WinFsp creates 8-16 threads typically
- Concurrent operations: Different files accessed simultaneously
- Locking: memefs uses per-file shared mutexes for better concurrency
Real-world scenarios
Scenario 1: Web browser downloads
Test: Download a 1GB file from a web server to the RAM disk- Original memfs: Download slows to a crawl, may timeout
- memefs: Download proceeds at full network speed
Scenario 2: Large file copy (preallocated)
Test: Copy a 10GB file using Windows Explorer- Original memfs: Fast (if preallocated by the copy mechanism)
- memefs: Very fast
Scenario 3: Build directory (many small files)
Test: Compile a large C++ project, writing object files to RAM disk- Original memfs: Very slow due to many unpreallocated writes
- memefs: Fast, consistent performance
Scenario 4: Random I/O (database)
Test: SQLite database with random reads/writes- Original memfs: Good performance
- memefs: Similar or better performance
Optimization impact
Key optimizations in memefs:- Vector-based sectors: 10-100x improvement for unpreallocated writes
- Private heap: Reduces fragmentation, more predictable performance
- Modern C++: Better compiler optimizations, safer code
- Per-file locking: Better concurrent access patterns
- Reference counting: Efficient file handle management
Running your own benchmarks
To benchmark memefs yourself:Using fsbench
The WinFsp project includes fsbench:Using diskspd
Microsoft’s diskspd is excellent for I/O benchmarks:Using CrystalDiskMark
For a GUI option, CrystalDiskMark provides comprehensive benchmarks:- Start memefs with a drive letter
- Launch CrystalDiskMark
- Select the memefs drive
- Run the benchmark suite
Benchmark considerations
System memory impact
RAM disk performance is affected by:- Available RAM: Less free RAM = slower allocation
- Memory pressure: System paging impacts performance
- CPU speed: Memory copy operations are CPU-bound
Cache effects
Windows caches file metadata:- Cold cache: First access slower
- Warm cache: Subsequent accesses faster
- FileInfoTimeout: 15 seconds (configurable)
Measurement accuracy
For accurate benchmarks:- Disable antivirus: Real-time scanning affects results
- Close other apps: Reduce memory and CPU competition
- Run multiple iterations: Average results over several runs
- Use large files: Small files dominated by overhead
- Warm up: Run a test iteration first to warm caches
Benchmark summary
memefs excels at:- Unpreallocated file writes (10-100x faster)
- Mixed workloads (downloads, builds, general use)
- Consistent performance across scenarios
- Real-world usage patterns
- Maximum sequential write speed (if preallocated)
- Specific optimized scenarios