Skip to main content

​
Overview

The deployment process is an end-to-end workflow that takes a repository URL and transforms it into a live, hosted application. The system handles cloning, uploading, queuing, building, and deploying your project automatically.

​
Architecture Flow

User Submits URL → Upload Service → S3 Storage → Redis Queue → Deploy Service → Build & Host
1

Repository Submission

User submits a repository URL via the /get/url endpoint
POST /get/url
{
  "url": "https://github.com/user/repo.git"
}
2

ID Generation & Cloning

The system generates a unique 10-character random ID and clones the repositoryImplementation: upload-service/src/server.ts:17-26
const randomId = generateRandomId()  // e.g., "aB3xK9mP2q"
const clonePath = path.join(distPath, 'output', randomId)

await git.clone(repoUrl, clonePath)
The generateRandomId() function uses alphanumeric characters (A-Z, a-z, 0-9) to create collision-resistant identifiers.
3

File Collection

All files are collected while respecting .gitignore rulesImplementation: upload-service/src/utils/getAllFiles.ts:5-47
const files = getAllFiles(clonePath)
The getAllFiles function:
  • Reads the project’s .gitignore file
  • Recursively traverses directories
  • Filters out ignored files (node_modules, .env, etc.)
  • Returns full file paths for upload
4

S3 Upload

All project files are uploaded to S3 in parallelImplementation: upload-service/src/server.ts:32-41
const uploadPromises = files.map(async (localFilePath) => {
  const relativePath = path.relative(clonePath, localFilePath)
  const s3Key = path.posix.join('output', randomId, relativePath)
  return uploadFiles(s3Key, localFilePath)
});

await Promise.all(uploadPromises);
S3 Key Structure:
output/{randomId}/src/index.ts
output/{randomId}/package.json
output/{randomId}/public/logo.png
5

Queue Insertion

The deployment ID is pushed to Redis queue for processingImplementation: upload-service/src/utils/buildQueue.ts:9-17
await client.LPUSH('build-queue', randomId)
The upload service immediately returns success to the user after queuing, allowing asynchronous build processing.
6

Response

The upload service returns the deployment ID to the user
{
  "success": true,
  "id": "aB3xK9mP2q"
}
Users can track their deployment using this ID.

​
Deploy Service Flow

While the upload service handles the initial submission, the deploy service processes queued builds:
1

Queue Listening

Deploy service continuously listens for new deploymentsImplementation: deploy-service/src/server.ts:11-27
while (1) {
  const response = await client.blPop(
    commandOptions({ isolated: true }),
    'build-queue',
    0
  )
  // Process deployment...
}
2

Download from S3

Source files are downloaded from S3 to the deploy service
await downloadS3Folder(`output/${response.element}`)
3

Docker Build

Project is built in an isolated Docker container
await buildProject(response.element)
4

Upload Built Assets

Final build output is uploaded back to S3 for hosting
await copyFinalDist(response.element)

​
Complete Deployment Timeline

  1. T+0s: User submits repository URL
  2. T+2s: Repository cloned to dist/output/aB3xK9mP2q
  3. T+3s: 150 files collected (excluding node_modules)
  4. T+8s: All files uploaded to S3 (output/aB3xK9mP2q/*)
  5. T+8.1s: ID pushed to Redis queue
  6. T+8.2s: User receives deployment ID
  7. T+8.3s: Deploy service picks up from queue
  8. T+10s: Source files downloaded from S3
  9. T+15s: Docker container created and build started
  10. T+45s: Build completes, output copied from container
  11. T+50s: Built assets uploaded to dist/aB3xK9mP2q/*
  12. T+50s: Deployment live at https://{id}.yourplatform.com

​
Error Handling

The system includes robust error handling at each stage:
Implementation: upload-service/src/server.ts:46-49
catch (error) {
  console.error('Clone error:', error)
  res.status(500).json({ error: 'Failed to clone repository' })
}
Common scenarios:
  • Invalid repository URL
  • Private repositories without credentials
  • Network timeouts

​
Key Implementation Files

FilePurposeKey Functions
upload-service/src/server.tsMain upload endpointRepository cloning, file upload orchestration
upload-service/src/utils/getAllFiles.tsFile collectionRecursive traversal with .gitignore support
upload-service/src/utils/uploadFiles.tsS3 uploadsStream-based file uploads
upload-service/src/utils/buildQueue.tsQueue operationsRedis LPUSH for queue insertion
deploy-service/src/server.tsQueue consumerContinuous build queue processing
deploy-service/src/utils/downloadS3Folder.tsS3 downloadsBatch file downloads from S3
deploy-service/src/utils/buildProject.tsDocker buildsContainer-based project building

​
Next Steps

Build System

Learn about Docker-based build isolation

Queue Management

Understand Redis queue operations

Build docs developers (and LLMs) love

Get started for freeTalk to us