File Storage Configuration

CCDigital stores document files on the local filesystem with metadata tracked in the MySQL database. The FileStorageService manages all file operations with security validation and integrity checks.

Architecture Overview

The file storage system consists of:

Base Directory: Configured via CCDIGITAL_FS_BASE_PATH environment variable
Person Folders: One folder per person, named using normalized identity
File Metadata: SHA-256 hash, size, and relative path stored in database
Security Validation: Path traversal prevention and user access control

/var/ccdigital/storage/          ← Base path (CCDIGITAL_FS_BASE_PATH)
├── GarciaJuan/                  ← Person folder (normalized name)
│   ├── diploma.pdf
│   ├── birth_certificate.pdf
│   └── id_card.pdf
├── MartinezMaria/
│   ├── passport.pdf
│   └── degree.pdf
└── ...

Configuration

Base Path Setup

Set Environment Variable

The CCDIGITAL_FS_BASE_PATH environment variable defines the root directory for all file storage.

export CCDIGITAL_FS_BASE_PATH='/var/ccdigital/storage'

This variable maps to both ccdigital.fs.base-path and app.user-files-base-dir in application.properties.

Create Base Directory

sudo mkdir -p /var/ccdigital/storage
sudo chown ccdigital:ccdigital /var/ccdigital/storage
sudo chmod 750 /var/ccdigital/storage

Verify Permissions

Ensure the application user has read/write access:

ls -ld /var/ccdigital/storage
# Should show: drwxr-x--- ccdigital ccdigital

Properties Mapping

The base path is configured in application.properties:

# FileStorageProperties configuration
ccdigital.fs.base-path=${CCDIGITAL_FS_BASE_PATH}

# UserDocsController validation base
app.user-files-base-dir=${CCDIGITAL_FS_BASE_PATH}

Both properties must point to the same physical directory to prevent “Path not allowed” errors during document access.

FileStorageService

The FileStorageService class (co.edu.unbosque.ccdigital.service.FileStorageService) handles all file operations.

Key Components

FileStorageProperties

Configuration class that loads ccdigital.fs.base-path from environment.Location: co.edu.unbosque.ccdigital.config.FileStorageProperties

@ConfigurationProperties(prefix = "ccdigital.fs")
public class FileStorageProperties {
    private String basePath;
    // getters/setters
}

FileStorageService

Main service for file operations.Location: co.edu.unbosque.ccdigital.service.FileStorageServiceKey Methods:

ensurePersonFolder(Person): Creates person folder if needed
storePersonFile(Person, MultipartFile): Saves file with metadata
resolvePath(FileRecord): Resolves absolute path from relative path
loadAsResource(FileRecord): Loads file as Spring Resource

Folder Structure

Person Folder Naming

Each person gets a dedicated folder named using their identity: Algorithm (from FileStorageService.buildPersonFolderName):

Concatenate Names

Combine last_name + first_name from persons table.Example: “García” + “Juan” → “GarcíaJuan”

Remove Whitespace

Strip all spaces.Example: “García Juan” → “GarcíaJuan”

Normalize to ASCII

Remove accents and diacritics using NFD normalization.Example: “GarcíaJuan” → “GarciaJuan”

Keep Alphanumeric Only

Remove all non-alphanumeric characters.Example: “García-Juan” → “GarciaJuan”

Examples:

Last Name	First Name	Folder Name
García	Juan	GarciaJuan
Martínez	María	MartinezMaria
O’Neill	Patrick	ONeillPatrick
Müller	Hans	MullerHans

Automatic Folder Creation

Folders are created automatically when:

Person is created via PersonService.createPersonAndFolder()
First document is uploaded for a person

The ensurePersonFolder() method ensures idempotent folder creation:

public Path ensurePersonFolder(Person person) {
    String folderName = buildPersonFolderName(person);
    Path personFolder = getBasePath().resolve(folderName);
    Files.createDirectories(personFolder); // Creates if not exists
    return personFolder;
}

File Storage Process

When a document is uploaded:

Validate Upload

Check file is not empty
Validate file extension (PDF required for issuer uploads)
Verify MIME type
Check PDF signature bytes (%PDF) for PDFs

Store File

FileStorageService.storePersonFile() executes:

Ensure person folder exists
Resolve target path: basePath/PersonFolder/filename.pdf
Write file to disk (replaces if exists)
Calculate SHA-256 hash of stored file
Get file size in bytes
Build relative path from base directory

Persist Metadata

Create FileRecord entity with:

storage_path: Relative path (e.g., GarciaJuan/diploma.pdf)
sha256: File integrity hash
size_bytes: File size
version: Auto-incremented by trg_files_autoversion trigger
uploaded_at: Timestamp

Link to person_documents table.

Update Document Status

Set person_documents.review_status to PENDING for admin review.

File Metadata

All file metadata is stored in the files table:

bigint

Primary key.

person_document_id

bigint

Foreign key to person_documents. Links file to document instance.

storage_path

varchar(500)

Relative path from basePath using / separator.Example: GarciaJuan/diploma.pdf

Always uses forward slash / regardless of OS, normalized when stored.

sha256

char(64)

SHA-256 hash of file content (hexadecimal).Purpose: Integrity verification, duplicate detection.

size_bytes

bigint

File size in bytes.

version

int

Auto-incremented version number per person_document_id.Mechanism: trg_files_autoversion trigger automatically sets version.

uploaded_at

timestamp

Upload timestamp.

Path Resolution

When accessing a file, the system resolves absolute paths from stored relative paths:

// FileStorageService.resolvePath()
public Path resolvePath(FileRecord fileRecord) {
    String relative = fileRecord.getStoragePath();
    // Normalize Windows backslashes to forward slashes
    relative = relative.replace("\\", "/");
    
    return getBasePath()
        .resolve(relative)
        .normalize()        // Remove . and .. components
        .toAbsolutePath();  // Convert to absolute path
}

Example:

BasePath: /var/ccdigital/storage
StoragePath: GarciaJuan/diploma.pdf
Resolved: /var/ccdigital/storage/GarciaJuan/diploma.pdf

Security Validation

CCDigital implements multiple security layers for file access:

1. Path Traversal Prevention

The system prevents path traversal attacks (e.g., ../../etc/passwd) by normalizing and validating all paths.

All resolved paths are:

Normalized using Path.normalize() to remove . and .. components
Converted to absolute paths
Validated against allowed base directories

2. User Access Validation

Before serving a file to an end user, UserDocsController validates:

User Authentication

Verify user is authenticated via Spring Security.

Document Ownership

Check that person_documents.person_id matches user’s users.person_id.

Access Request Authorization

For issuer access, verify:

Valid access_requests entry exists
Request status is APPROVED
Access is within valid_from to valid_until window
User has consent via consents table

Path Containment

Verify resolved file path is within app.user-files-base-dir:

Path resolved = fileStorageService.resolvePath(fileRecord);
if (!resolved.startsWith(allowedBasePath)) {
    throw new SecurityException("Path not allowed");
}

3. Document Review Status

Only documents with review_status = 'APPROVED' can be:

Included in access requests
Downloaded by authorized users
Synced to Fabric ledger

Documents with PENDING or REJECTED status are not accessible to non-admin users.

File Versioning

The files table supports automatic versioning:

Trigger: trg_files_autoversion (BEFORE INSERT)
Behavior: Automatically sets version = MAX(version) + 1 for same person_document_id
Use Case: Track document updates/corrections over time

Example:

-- First upload
INSERT INTO files (person_document_id, storage_path, sha256, size_bytes)
VALUES (123, 'GarciaJuan/diploma.pdf', 'abc...', 52000);
-- Trigger sets version = 1

-- Updated document
INSERT INTO files (person_document_id, storage_path, sha256, size_bytes)
VALUES (123, 'GarciaJuan/diploma_v2.pdf', 'def...', 53000);
-- Trigger sets version = 2

Storage Maintenance

Disk Space Monitoring

Monitor storage usage regularly:

# Check overall usage
df -h /var/ccdigital/storage

# Check per-person usage
du -sh /var/ccdigital/storage/*/ | sort -h

# Find large files
find /var/ccdigital/storage -type f -size +10M -ls

Orphan File Detection

Find files on disk not referenced in database:

-- Files in database
SELECT storage_path FROM files;

-- Compare with filesystem
-- Use script to cross-reference

Integrity Verification

Verify file integrity using stored SHA-256 hashes:

#!/bin/bash
# Verify all files match their stored hashes
mysql -u ccdigital_user -p -D ccdigital -e "
  SELECT id, storage_path, sha256 FROM files
" | while read id path hash; do
  full_path="/var/ccdigital/storage/$path"
  if [ -f "$full_path" ]; then
    actual_hash=$(sha256sum "$full_path" | awk '{print $1}')
    if [ "$actual_hash" != "$hash" ]; then
      echo "MISMATCH: File $id ($path)"
    fi
  else
    echo "MISSING: File $id ($path)"
  fi
done

Backup Strategy

File System Backup

Back up the entire storage directory:

# Using rsync
rsync -av --delete /var/ccdigital/storage/ /backup/ccdigital/storage/

# Using tar
tar -czf ccdigital_files_$(date +%Y%m%d).tar.gz /var/ccdigital/storage

Database Backup

Back up file metadata from files table:

mysqldump -u ccdigital_user -p ccdigital files > files_table_backup.sql

Always backup both filesystem and database together. They must remain synchronized.

Performance Considerations

File System Performance

Avoid deep nesting: Person folders are flat (one level deep)
Use SSD storage: Recommended for production deployments
Monitor I/O: Track read/write operations and latency

Database Queries

Index on storage_path: Speeds up file lookups
Index on person_document_id: Optimizes versioning queries
Index on sha256: Enables duplicate detection

Troubleshooting

Path Not Allowed Error

Problem: User gets “Path not allowed” when accessing documents. Cause: Mismatch between ccdigital.fs.base-path and app.user-files-base-dir. Solution:

# Ensure both point to same directory
export CCDIGITAL_FS_BASE_PATH='/var/ccdigital/storage'
# This sets both properties in application.properties

File Not Found

Problem: Database has file record but file missing from disk. Cause: File deleted manually or backup restore incomplete. Solution:

Check file path: SELECT storage_path FROM files WHERE id = ?
Verify physical file: ls -l /var/ccdigital/storage/path/to/file.pdf
Restore from backup if missing

Permission Denied

Problem: Application cannot write to storage directory. Cause: Incorrect file system permissions. Solution:

sudo chown -R ccdigital:ccdigital /var/ccdigital/storage
sudo chmod -R 750 /var/ccdigital/storage

Hash Mismatch

Problem: File SHA-256 doesn’t match stored hash. Cause: File corruption or manual modification. Solution:

Identify affected file from integrity check script
Restore from backup

Update hash in database if file is known good:

UPDATE files SET sha256 = '<new_hash>' WHERE id = ?;

Migration from Legacy Storage

If migrating from an older storage structure:

Configure Legacy Base

Set optional legacy directory:

export APP_USER_FILES_LEGACY_BASE_DIR='/old/storage/path'

Run Migration Script

Create migration script to:

Copy files from legacy to new structure
Update storage_path in files table
Verify SHA-256 hashes match

Validate Migration

Test file access for sample documents across all modules.

Remove Legacy Configuration

Once validated, remove APP_USER_FILES_LEGACY_BASE_DIR.

Setup

Blockchain Integration

File Storage Configuration

Architecture Overview

Configuration

Base Path Setup

Properties Mapping

FileStorageService

Key Components

FileStorageProperties

FileStorageService

Folder Structure

Person Folder Naming

Automatic Folder Creation

File Storage Process

File Metadata

Path Resolution

Security Validation

1. Path Traversal Prevention

2. User Access Validation

3. Document Review Status

File Versioning

Storage Maintenance

Disk Space Monitoring

Orphan File Detection

Integrity Verification

Backup Strategy

File System Backup

Database Backup

Performance Considerations

File System Performance

Database Queries

Troubleshooting

Path Not Allowed Error

File Not Found

Permission Denied

Hash Mismatch

Migration from Legacy Storage

Build docs developers (and LLMs) love

Setup

Blockchain Integration

​Architecture Overview

​Configuration

​Base Path Setup

​Properties Mapping

​FileStorageService

​Key Components

FileStorageProperties

FileStorageService

​Folder Structure

​Person Folder Naming

​Automatic Folder Creation

​File Storage Process

​File Metadata

​Path Resolution

​Security Validation

​1. Path Traversal Prevention

​2. User Access Validation

​3. Document Review Status

​File Versioning

​Storage Maintenance

​Disk Space Monitoring

​Orphan File Detection

​Integrity Verification

​Backup Strategy

File System Backup

Database Backup

​Performance Considerations

​File System Performance

​Database Queries

​Troubleshooting

​Path Not Allowed Error

​File Not Found

​Permission Denied

​Hash Mismatch

​Migration from Legacy Storage

Build docs developers (and LLMs) love

Architecture Overview

Configuration

Base Path Setup

Properties Mapping

FileStorageService

Key Components

Folder Structure

Person Folder Naming

Automatic Folder Creation

File Storage Process

File Metadata

Path Resolution

Security Validation

1. Path Traversal Prevention

2. User Access Validation

3. Document Review Status

File Versioning

Storage Maintenance

Disk Space Monitoring

Orphan File Detection

Integrity Verification

Backup Strategy

Performance Considerations

File System Performance

Database Queries

Troubleshooting

Path Not Allowed Error

File Not Found

Permission Denied

Hash Mismatch

Migration from Legacy Storage