Design decisions

Sequential: [Tweet 1] → [Tweet 2] → [Tweet 3] → [Tweet 4]
            1s         1s         1s         1s = 4s total

Concurrent: [Tweet 1]
            [Tweet 2]
            [Tweet 3]  → All complete in ~1s
            [Tweet 4]

# Sequential error handling
try:
    result = analyze(tweet)
except APIError:
    checkpoint.save(current_index)  # Simple!
    raise

# Concurrent error handling
async with TaskGroup() as group:
    tasks = [group.create_task(analyze(tweet)) for tweet in batch]
    results = await asyncio.gather(*tasks, return_exceptions=True)
    
    # Which tweets succeeded? Which failed?
    # Where should checkpoint be?
    # Should we retry only failed tweets?
    # What if retry succeeds for some but not others?

# Infrastructure layer: Raise exceptions
class JSONParser:
    def parse(self) -> list[Tweet]:
        if not path.exists():
            raise FileNotFoundError(f"File not found: {path}")
        
        try:
            return [Tweet(...) for item in json.load(f)]
        except json.JSONDecodeError as e:
            raise ValueError(f"Invalid JSON: {e}") from e

# Application layer: Return Result
class Application:
    def extract_tweets(self) -> Result:
        try:
            tweets = parser.parse()
            writer.write_tweets(tweets)
            return Result(success=True, count=len(tweets))
        except FileNotFoundError as e:
            return Result(
                success=False,
                error_type="file_not_found",
                error_message=str(e)
            )

# CLI layer: Handle Result
result = app.extract_tweets()
if not result.success:
    print(f"Error: {result.error_message}", file=sys.stderr)
    sys.exit(1)

Processing tweets 0-4999 (5,000 total)

Per-tweet checkpoint:
- Interrupted at tweet 3,427
- Resume from tweet 3,427
- Rework: 0 tweets
- Checkpoint writes: 3,427

Per-batch checkpoint (batch_size=10):
- Interrupted at tweet 3,427 (batch 342, tweet 7/10)
- Resume from tweet 3,420 (batch 342 start)
- Rework: 7 tweets
- Checkpoint writes: 342