/monitor_screen endpoint monitors screen captures to automatically detect quiz questions and provide answers. It combines quiz detection with question answering in a single API call.
Endpoint
Authentication
Requires a valid Gemini API key provided via:- Recommended:
X-API-Keyheader - Alternative:
apiKeyin request body
Request parameters
Headers
Your Google Gemini API key
Must be
multipart/form-data for image uploadBody parameters
Screenshot image to analyze for quiz questionsConstraints:
- Maximum size: 5MB (server.js:79)
- Supported formats: PNG, JPEG
- MIME type validation enforced
Your Gemini API key (alternative to header)
Response
Successful detection
true if a quiz question was found in the imageArray of answer strings (only present when
detected is true)No question detected
false when no quiz question is foundExplanation message: “No quiz question detected in the image”
How it works
The endpoint performs a two-step process:Quiz detection
The image is analyzed with a simple prompt: “Is this a quiz question image? Answer only yes/no.”If the AI responds with “yes”, proceed to step 2. Otherwise, return
detected: false.This two-step approach minimizes false positives by first confirming a quiz question exists before attempting to answer it.
Examples
Basic usage
Continuous monitoring
JavaScript - Screen Capture Loop
Response examples
Quiz question detected
No quiz question
Error responses
400 Bad Request
400 Bad Request
No image providedMissing API key
429 Too Many Requests
429 Too Many Requests
Rate limit exceeded (server.js:334-345)This endpoint enforces a 5-second window between requests per client IP.
500 Internal Server Error
500 Internal Server Error
Processing failure
Rate limiting
This endpoint has stricter rate limiting than other endpoints: Additional limits:- Global limit: 100 requests per 15 minutes per IP
- Internal quota: 50 API calls per minute
Implementation details
Detection logic (server.js:234-271)
The detection uses a simplified prompt:File cleanup
Images are automatically deleted after processing using the history tracking system:Best practices
The detection step helps conserve API quota by filtering out non-quiz screens before attempting to answer.
Browser compatibility
Screen capture requiresnavigator.mediaDevices.getDisplayMedia():
| Browser | Support |
|---|---|
| Chrome | ✅ 72+ |
| Firefox | ✅ 66+ |
| Edge | ✅ 79+ |
| Safari | ✅ 13+ |
Related endpoints
- Process Question - Process questions without detection step
- Rate Limiting - Understanding rate limits
- Examples - Complete integration examples