Client Module Architecture

Module Overview

The client module (app/) is an Android application that provides the user interface and handles video/audio decoding, user input collection, and ADB communication. Package: org.client.scrcpy
Location: app/src/main/java/org/client/scrcpy/

Core Components

MainActivity

The main activity manages the application lifecycle and coordinates between UI and streaming service. File: MainActivity.java (837 lines)

Key Responsibilities

Connection Management

Handles server connection setup, including:

Reading user preferences (resolution, bitrate, delay)
Validating server address
Deploying server JAR via ADB
Establishing socket connection

From MainActivity.java:738-791:

private void connectScrcpyServer(String serverAdr) {
    String[] serverInfo = Util.getServerHostAndPort(serverAdr);
    String serverHost = serverInfo[0];
    int serverPort = Integer.parseInt(serverInfo[1]);
    
    // Extract and write server JAR
    InputStream inputStream = assetManager.open("scrcpy-server.jar");
    byte[] buffer = new byte[inputStream.available()];
    inputStream.read(buffer);
    
    FileOutputStream outputStream = new FileOutputStream(
        new File(context.getExternalFilesDir("scrcpy"), 
        "scrcpy-server.jar"));
    outputStream.write(buffer);
    
    // Deploy via ADB and start streaming
    sendCommands.SendAdbCommands(context, serverHost, 
        serverPort, localForwardPort, videoBitrate, maxSize);
}

Lifecycle Management

Manages activity states with service binding:

private final ServiceConnection serviceConnection = new ServiceConnection() {
    @Override
    public void onServiceConnected(ComponentName name, IBinder binder) {
        scrcpy = ((Scrcpy.MyServiceBinder) binder).getService();
        scrcpy.setServiceCallbacks(MainActivity.this);
        serviceBound = true;
        scrcpy.start(surface, serverAddress, 
            screenHeight, screenWidth, delayControl);
    }
};

Key lifecycle methods:

onCreate(): Initialize UI and restore state
onPause(): Pause streaming and optionally disconnect
onResume(): Resume streaming or reconnect
onSaveInstanceState(): Persist configuration

Display and Touch Handling

Calculates aspect ratio and sets up touch event handling (MainActivity.java:362-433):

public void set_display_nd_touch() {
    int[] rem_res = scrcpy.get_remote_device_resolution();
    int remote_device_height = rem_res[1];
    int remote_device_width = rem_res[0];
    float remote_aspect_ratio = (float) remote_device_height / remote_device_width;
    
    // Calculate padding for aspect ratio matching
    // ...
    
    // Set touch listener
    surfaceView.setOnTouchListener((view, event) -> 
        scrcpy.touchevent(event, landscape, 
            surfaceView.getWidth(), surfaceView.getHeight()));
}

Rotation Handling

Supports dynamic orientation changes:

@Override
public void loadNewRotation() {
    unbindService(serviceConnection);
    landscape = !landscape;
    swapDimensions(); // Swap width and height
    
    if (landscape) {
        setRequestedOrientation(
            ActivityInfo.SCREEN_ORIENTATION_SENSOR_LANDSCAPE);
    } else {
        setRequestedOrientation(
            ActivityInfo.SCREEN_ORIENTATION_SENSOR_PORTRAIT);
    }
}

Scrcpy Service

Background service that manages the streaming connection and coordinates decoders. File: Scrcpy.java (486 lines)

Architecture

Key Methods

Connection Loop (Scrcpy.java:342-470):

private void loop(DataInputStream in, DataOutputStream out, int delay) {
    while (LetServiceRunning.get()) {
        // Send queued events
        byte[] sendevent = event.poll();
        if (sendevent != null) {
            out.write(sendevent, 0, sendevent.length);
        }
        
        // Read incoming packets
        if (in.available() > 0) {
            byte[] packetSize = new byte[4];
            in.readFully(packetSize, 0, 4);
            int size = ByteUtils.bytesToInt(packetSize);
            
            byte[] packet = new byte[size];
            in.readFully(packet, 0, size);
            
            if (MediaPacket.Type.getType(packet[0]) == VIDEO) {
                VideoPacket videoPacket = VideoPacket.readHead(packet);
                videoDecoder.decodeSample(packet, offset, size, 
                    timestamp, flags);
            } else if (MediaPacket.Type.getType(packet[0]) == AUDIO) {
                AudioPacket audioPacket = AudioPacket.readHead(packet);
                audioDecoder.decodeSample(packet, offset, size, 
                    timestamp, flags);
            }
        }
    }
}

Touch Event Processing (Scrcpy.java:136-182):

public boolean touchevent(MotionEvent event, boolean landscape, 
                          int displayW, int displayH) {
    // Calculate scaling between display and remote device
    float remoteW = landscape ? 
        Math.max(remote_dev_resolution[0], remote_dev_resolution[1]) :
        Math.min(remote_dev_resolution[0], remote_dev_resolution[1]);
    float remoteH = landscape ? 
        Math.min(remote_dev_resolution[0], remote_dev_resolution[1]) :
        Math.max(remote_dev_resolution[0], remote_dev_resolution[1]);
    
    switch (event.getAction()) {
        case MotionEvent.ACTION_MOVE:
            // Handle all pointers for multi-touch
            for (int i = 0; i < event.getPointerCount(); i++) {
                int pointerId = event.getPointerId(i);
                int x = (int) (event.getX(i) * realW / displayW);
                int y = (int) (event.getY(i) * realH / displayH);
                sendTouchEvent(action, buttonState, x, y, pointerId);
            }
            break;
        // ... handle other actions
    }
}

VideoDecoder

Decodes H.264 video stream using Android’s MediaCodec API. File: decoder/VideoDecoder.java (131 lines)

Implementation Details

public void configure(Surface surface, int width, int height, 
                      ByteBuffer csd0, ByteBuffer csd1) {
    MediaFormat format = MediaFormat.createVideoFormat(
        "video/avc", width, height);
    format.setByteBuffer("csd-0", csd0); // SPS
    format.setByteBuffer("csd-1", csd1); // PPS
    
    mCodec = MediaCodec.createDecoderByType("video/avc");
    mCodec.configure(format, surface, null, 0);
    mCodec.start();
}

AudioDecoder

Decodes AAC audio stream and plays back using AudioTrack. File: decoder/AudioDecoder.java (172 lines)

Key Features

Format: AAC (audio/mp4a-latm)
Sample Rate: 48 kHz
Channels: Stereo (2 channels)
Bit Rate: 128 kbps

Playback Pipeline

private void configure(byte[] data) {
    MediaFormat format = MediaFormat.createAudioFormat(
        MIMETYPE_AUDIO_AAC, SAMPLE_RATE, 2);
    format.setInteger(MediaFormat.KEY_BIT_RATE, 128000);
    format.setByteBuffer("csd-0", ByteBuffer.wrap(data));
    
    mCodec = MediaCodec.createDecoderByType(MIMETYPE_AUDIO_AAC);
    mCodec.configure(format, null, null, 0);
    mCodec.start();
    
    // Initialize AudioTrack
    initAudioTrack();
    audioTrack.play();
}

// Worker thread reads decoded PCM and writes to AudioTrack
ByteBuffer outputBuffer = mCodec.getOutputBuffer(index);
byte[] data = new byte[info.size];
outputBuffer.get(data);
audioTrack.write(data, 0, info.size);

SendCommands

Handles ADB command execution for server deployment. File: SendCommands.java (117 lines)

Server Deployment Process

Connect to Device

App.adbCmd("connect", ip + ":" + port);

Push Server JAR

App.adbCmd("-s", ip + ":" + port, "push", 
    serverJarPath, "/data/local/tmp/scrcpy-server.jar");

Setup Port Forwarding

App.adbCmd("-s", ip + ":" + port, "forward", 
    "tcp:" + serverport, "tcp:7007");

Launch Server

String[] commands = new String[]{
    "-s", ip + ":" + port,
    "shell",
    "CLASSPATH=/data/local/tmp/scrcpy-server.jar",
    "app_process",
    "/",
    "org.server.scrcpy.Server",
    "/" + localip,
    Long.toString(size),
    Long.toString(bitrate) + ";"
};
App.adbCmd(commands);

Data Models

Packet Structures

VideoPacket and AudioPacket share a common base structure:

public class MediaPacket {
    public enum Type {
        VIDEO(0x00), AUDIO(0x01);
    }
}

public class VideoPacket {
    public enum Flag {
        CONFIG,      // SPS/PPS configuration
        KEY_FRAME,   // I-frame
        FRAME,       // P-frame or B-frame
        END          // End of stream
    }
    
    Type type;
    Flag flag;
    long presentationTimeStamp;
    byte[] data;
}

Threading Model

Main Thread

UI rendering
Activity lifecycle
User input collection

Service Thread

Socket I/O loop
Packet routing
Event queue management

VideoDecoder Worker

MediaCodec input feeding
Output buffer management
Surface rendering

AudioDecoder Worker

MediaCodec audio decoding
AudioTrack playback
PCM buffer management

Error Handling

The client implements retry logic and error recovery:

// Connection retry in MainActivity
private void connectExitExt(boolean userDisconnect) {
    if (!userDisconnect) {
        errorCount += 1;
        if (errorCount >= 3) {
            // Restart ADB server after 3 failures
            App.startAdbServer();
        }
    }
    
    if (headlessMode && !userDisconnect) {
        // Show reconnection dialog
        Dialog.displayDialog(this, 
            getString(R.string.connect_faild),
            getString(R.string.connect_faild_ask), 
            () -> connectScrcpyServer(serverAddress), 
            () -> finishAndRemoveTask());
    }
}

Performance Optimizations

Frame Dropping

The service implements intelligent frame dropping based on delay threshold:

if (System.currentTimeMillis() - 
    (lastVideoOffset + (videoPacket.presentationTimeStamp / 1000)) < delay) {
    videoDecoder.decodeSample(...); // Decode frame
} else {
    videoPassCount++; // Skip frame
}

Multi-touch Support

Supports multiple simultaneous touch points with pointer ID tracking (Scrcpy.java:161-170):

for (int i = 0; i < event.getPointerCount(); i++) {
    int currentPointerId = event.getPointerId(i);
    int x = (int) event.getX(i);
    int y = (int) event.getY(i);
    sendTouchEvent(action, buttonState, x, y, currentPointerId);
}

Server Module

Learn about screen capture and encoding

Architecture Overview

Understand the complete system design

Architecture

API Reference

Contributing

Module Overview

Core Components

MainActivity

Key Responsibilities

Scrcpy Service

Architecture

Key Methods

VideoDecoder

Implementation Details

AudioDecoder

Key Features

Playback Pipeline

SendCommands

Server Deployment Process

Data Models

Packet Structures

Threading Model

Main Thread

Service Thread

VideoDecoder Worker

AudioDecoder Worker

Error Handling

Performance Optimizations

Frame Dropping

Multi-touch Support

Server Module

Architecture Overview

Build docs developers (and LLMs) love

Architecture

API Reference

Contributing

​Module Overview

​Core Components

​MainActivity

​Key Responsibilities

​Scrcpy Service

​Architecture

​Key Methods

​VideoDecoder

​Implementation Details

​AudioDecoder

​Key Features

​Playback Pipeline

​SendCommands

​Server Deployment Process

​Data Models

​Packet Structures

​Threading Model

Main Thread

Service Thread

VideoDecoder Worker

AudioDecoder Worker

​Error Handling

​Performance Optimizations

​Frame Dropping

​Multi-touch Support

​Related Documentation

Server Module

Architecture Overview

Build docs developers (and LLMs) love

Module Overview

Core Components

MainActivity

Key Responsibilities

Scrcpy Service

Architecture

Key Methods

VideoDecoder

Implementation Details

AudioDecoder

Key Features

Playback Pipeline

SendCommands

Server Deployment Process

Data Models

Packet Structures

Threading Model

Error Handling

Performance Optimizations

Frame Dropping

Multi-touch Support

Related Documentation