Enterprise Sync

Overview

Enterprise Sync is a core component of FreeTAKServer that provides comprehensive data synchronization, file storage, and content management capabilities. It serves as the backbone for storing and retrieving various types of data including data packages, mission content, ExCheck templates, and other synchronized content across the TAK ecosystem.

Architecture

Component Structure

FreeTAKServer/core/enterprise_sync/
├── enterprise_sync_facade.py      # Public interface
├── controllers/
│   ├── enterprise_sync_general_controller.py
│   ├── enterprise_sync_database_controller.py
│   ├── enterprise_sync_filesystem_controller.py
│   └── enterprise_sync_format_synchronization_controller.py
├── persistence/
│   └── sqlalchemy/
│       ├── enterprise_sync_data_object.py
│       └── enterprise_sync_keyword.py
├── base/
│   └── enterprise_sync_action_mapper.py
└── configuration/
    └── enterprise_sync_constants.py

Key Classes

EnterpriseSync Facade

Location: FreeTAKServer/core/enterprise_sync/enterprise_sync_facade.py:15

class EnterpriseSync(DefaultFacade):
    def __init__(self, sync_action_mapper, request, response, 
                 configuration, tracing_provider_instance=None):
        self.general_controller = EnterpriseSyncGeneralController(
            request, response, sync_action_mapper, configuration
        )

Public Methods:

save_enterprise_sync_data() - Save data and metadata
get_enterprise_sync_data() - Retrieve data by UID or hash
get_multiple_enterprise_sync_data() - Batch retrieval
update_enterprise_sync_data() - Update existing data
delete_enterprise_sync_data() - Remove data
get_enterprise_sync_metadata() - Get metadata only
get_all_enterprise_sync_metadata() - List all metadata

EnterpriseSyncGeneralController

Location: FreeTAKServer/core/enterprise_sync/controllers/enterprise_sync_general_controller.py:28 Coordinates operations across database, filesystem, and format synchronization controllers.

class EnterpriseSyncGeneralController(Controller):
    def __init__(self, request, response, sync_action_mapper, configuration):
        super().__init__(request, response, sync_action_mapper, configuration)
        self.filesystem_controller = EnterpriseSyncFilesystemController(...)
        self.persistence_controller = EnterpriseSyncDatabaseController(...)
        self.format_sync_controller = EnterpriseSyncFormatSynchronizationController(...)

Core Functionality

Saving Data

Method: save_enterprise_sync_data()
Location: enterprise_sync_general_controller.py:113

def save_enterprise_sync_data(
    self, 
    synctype: str,       # Type of sync object
    objectuid: str,      # Unique identifier
    objectdata: str,     # The actual data
    objkeywords: list,   # Search keywords
    tool: str,           # Tool that created it
    mime_type: str,      # MIME type
    logger,
    objecthash: str = None,      # SHA256 hash
    file_name: str = None,       # Display name
    length: int = None,          # Data length
    creator_uid: str = "",       # Creator UID
    privacy: int = 0,            # Privacy level
    *args, **kwargs
):
    # Auto-generate hash if not provided
    if objecthash == None:
        objecthash = str(hashlib.sha256(objectdata).hexdigest())
    
    # Save to filesystem
    self.filesystem_controller.save_file(synctype, objectuid, objectdata)
    
    # Save metadata to database
    data_obj = self.persistence_controller.create_enterprise_sync_data_object(
        synctype, objectuid, objecthash, obj_length, 
        objkeywords, mime_type, tool, creator_uid, file_name, logger, privacy
    )
    
    return data_obj

Usage Example:

# Save a data package
enterprise_sync.save_enterprise_sync_data(
    synctype="content",
    objectuid="package-001",
    objectdata=file_bytes,
    objkeywords=["maps", "terrain"],
    tool="public",
    mime_type="application/zip",
    file_name="terrain-maps.zip",
    creator_uid="user-123",
    logger=logger
)

Retrieving Data

Single Object Retrieval

Method: get_enterprise_sync_data()
Location: enterprise_sync_general_controller.py:179

def get_enterprise_sync_data(
    self,
    logger,
    objecthash: str = None,
    objectuid: str = None,
    use_bytes: bool = False,
    convert_newlines: bool = False,
    *args, **kwargs
):
    # Get metadata from database
    data_obj = self.persistence_controller.get_enterprise_sync_data_object(
        logger, objectuid, objecthash
    )
    
    # Retrieve file from filesystem
    object_data = self.filesystem_controller.get_file(
        data_obj.file_type, data_obj.PrimaryKey, use_bytes=use_bytes
    )
    
    return object_data

Usage Example:

# Retrieve by UID
request.set_value("objectuid", "package-001")
request.set_value("use_bytes", True)
enterprise_sync.get_enterprise_sync_data(logger=logger)
data = response.get_value("objectdata")

# Retrieve by hash
request.set_value("objecthash", "sha256hash...")
enterprise_sync.get_enterprise_sync_data(logger=logger)

Multiple Object Retrieval

Method: get_multiple_enterprise_sync_data()
Location: enterprise_sync_general_controller.py:190

# Batch retrieve by UIDs
enterprise_sync.get_multiple_enterprise_sync_data(
    logger=logger,
    objectuids=["uid-1", "uid-2", "uid-3"],
    use_bytes=True
)
data_list = response.get_value("objectdata")

# Batch retrieve by hashes
enterprise_sync.get_multiple_enterprise_sync_data(
    logger=logger,
    objecthashs=["hash1", "hash2"],
    use_bytes=True
)

Metadata Operations

Get Metadata

Method: get_enterprise_sync_metadata()
Location: enterprise_sync_general_controller.py:257

# Get metadata by UID
enterprise_sync.get_enterprise_sync_metadata(
    logger=logger,
    objectuid="package-001"
)
metadata = response.get_value("objectmetadata")

# Metadata includes:
# - PrimaryKey (UID)
# - hash (SHA256)
# - file_type (synctype)
# - mime_type
# - length
# - keywords
# - tool
# - creator_uid
# - file_name
# - private (privacy level)

Update Metadata

Method: update_enterprise_sync_metadata()
Location: enterprise_sync_general_controller.py:149

# Update metadata fields
enterprise_sync.update_enterprise_sync_metadata(
    logger=logger,
    objectuid="package-001",
    keywords=["updated", "keywords"],
    file_name="new-name.zip",
    privacy=1
)

Get All Metadata

# List all enterprise sync objects
enterprise_sync.get_all_enterprise_sync_metadata(logger=logger)
all_metadata = response.get_value("objectmetadata")

Updating Data

Method: update_enterprise_sync_data()
Location: enterprise_sync_general_controller.py:134

# Update existing data
enterprise_sync.update_enterprise_sync_data(
    objectuid="package-001",
    objecthash="new-sha256-hash",
    objectdata=updated_bytes,
    logger=logger
)

Deleting Data

Method: delete_enterprise_sync_data()
Location: enterprise_sync_general_controller.py:287

# Delete data and metadata
enterprise_sync.delete_enterprise_sync_data(
    logger=logger,
    objectuid="package-001"
)

Data Types (synctype)

Enterprise Sync supports various data types:

Synctype	Description	Used By
`content`	General data packages	Missions, file sharing
`ExCheckTemplate`	ExCheck templates	ExCheck component
`ExCheckChecklist`	Active checklists	ExCheck component
`ExCheckChecklistTask`	Checklist tasks	ExCheck component
Custom types	Application-specific	Various components

Storage Architecture

Dual Storage Model

Enterprise Sync uses a dual storage approach:

Filesystem Storage: Actual file data stored on disk
- Organized by synctype
- Accessed via EnterpriseSyncFilesystemController
- Supports both binary and text data
Database Metadata: Searchable metadata in database
- Indexed by UID and hash
- Supports keyword search
- Tracks ownership and permissions

Data Object Model

class EnterpriseSyncDataObject:
    PrimaryKey: str          # Object UID
    hash: str                # SHA256 hash
    file_type: str           # Synctype
    mime_type: str           # MIME type
    length: int              # Data size
    keywords: List[str]      # Search keywords
    tool: str                # Creating tool
    creator_uid: str         # Creator UID
    file_name: str           # Display name
    private: int             # Privacy level (0=public, 1=private)
    submission_time: datetime

Integration with Components

Mission Component

# Missions use Enterprise Sync for content storage
self.request.set_value("synctype", "content")
self.request.set_value("objectdata", mission_content)
self.request.set_value("objectuid", content_uid)
self.execute_sub_action("SaveEnterpriseSyncData")

ExCheck Component

# ExCheck stores templates and checklists
self.request.set_value("synctype", "ExCheckTemplate")
self.request.set_value("objectdata", template_xml)
self.request.set_value("objkeywords", ["Template"])
self.execute_sub_action("SaveEnterpriseSyncData")

Data Package Service

Enterprise Sync provides the storage backend for the Data Package service, handling file uploads and downloads.

Advanced Features

Format Synchronization

The EnterpriseSyncFormatSynchronizationController handles:

Newline conversion for cross-platform compatibility
Format normalization
Encoding management

Hash-Based Deduplication

Data is identified by SHA256 hash, enabling:

Automatic deduplication
Content-addressable storage
Data integrity verification

Privacy Levels

# Public data (privacy=0)
enterprise_sync.save_enterprise_sync_data(
    privacy=0,  # Accessible to all
    ...
)

# Private data (privacy=1)
enterprise_sync.save_enterprise_sync_data(
    privacy=1,  # Restricted access
    ...
)

Best Practices

Always Use Hashes: Let the system auto-generate hashes for integrity
Meaningful Keywords: Use descriptive keywords for searchability
Proper MIME Types: Set correct MIME types for content handling
Batch Operations: Use multiple get/update methods for efficiency
Cleanup: Delete unused data to conserve storage

Performance Considerations

Batch Retrieval: Use get_multiple_enterprise_sync_data() for multiple objects
Metadata Only: Use metadata methods when full data isn’t needed
Indexing: Keywords and UIDs are indexed for fast lookup
Caching: Consider caching frequently accessed data

Error Handling

try:
    enterprise_sync.get_enterprise_sync_data(
        logger=logger,
        objectuid=uid
    )
    data = response.get_value("objectdata")
    if data is None:
        # Object not found
        handle_missing_object()
except Exception as e:
    logger.error(f"Enterprise sync error: {e}")

Mission - Uses Enterprise Sync for mission content
ExCheck - Stores templates and checklists
Data Package Service - Backend for file operations

Core Components

Extended Components

Overview

Architecture

Component Structure

Key Classes

EnterpriseSync Facade

EnterpriseSyncGeneralController

Core Functionality

Saving Data

Retrieving Data

Single Object Retrieval

Multiple Object Retrieval

Metadata Operations

Get Metadata

Update Metadata

Get All Metadata

Updating Data

Deleting Data

Data Types (synctype)

Storage Architecture

Dual Storage Model

Data Object Model

Integration with Components

Mission Component

ExCheck Component

Data Package Service

Advanced Features

Format Synchronization

Hash-Based Deduplication

Privacy Levels

Best Practices

Performance Considerations

Error Handling

Build docs developers (and LLMs) love

Core Components

Extended Components

​Overview

​Architecture

​Component Structure

​Key Classes

​EnterpriseSync Facade

​EnterpriseSyncGeneralController

​Core Functionality

​Saving Data

​Retrieving Data

​Single Object Retrieval

​Multiple Object Retrieval

​Metadata Operations

​Get Metadata

​Update Metadata

​Get All Metadata

​Updating Data

​Deleting Data

​Data Types (synctype)

​Storage Architecture

​Dual Storage Model

​Data Object Model

​Integration with Components

​Mission Component

​ExCheck Component

​Data Package Service

​Advanced Features

​Format Synchronization

​Hash-Based Deduplication

​Privacy Levels

​Best Practices

​Performance Considerations

​Error Handling

​Related Components

Build docs developers (and LLMs) love

Overview

Architecture

Component Structure

Key Classes

EnterpriseSync Facade

EnterpriseSyncGeneralController

Core Functionality

Saving Data

Retrieving Data

Single Object Retrieval

Multiple Object Retrieval

Metadata Operations

Get Metadata

Update Metadata

Get All Metadata

Updating Data

Deleting Data

Data Types (synctype)

Storage Architecture

Dual Storage Model

Data Object Model

Integration with Components

Mission Component

ExCheck Component

Data Package Service

Advanced Features

Format Synchronization

Hash-Based Deduplication

Privacy Levels

Best Practices

Performance Considerations

Error Handling

Related Components