Testing Philosophy
Aurora prioritizes practical testing approaches that ensure reliability without hindering development velocity. Before adding new test infrastructure, consult with the team to align on testing strategies.Manual Testing
Manual testing is currently the primary testing method for Aurora. This guide covers how to effectively test your changes.Starting the Test Environment
Start Aurora in development mode to test your changes:- Backend changes auto-reload in Flask
- Frontend changes rebuild instantly with Turbopack
Viewing Logs
Monitor logs to debug issues:Testing Checklist
When testing your changes, verify:Backend Changes
- Service starts successfully: No errors in startup logs
- API endpoints respond: Test with curl or Postman
- Database operations work: Check PostgreSQL logs
- Error handling: Test with invalid inputs
- Logging is appropriate: No excessive debug logs
- Security: Verify authentication/authorization
Frontend Changes
- Page loads without errors: Check browser console
- Responsive design: Test on different screen sizes
- User interactions work: Click buttons, submit forms
- Error states: Test error handling and user feedback
- Accessibility: Keyboard navigation, screen reader support
- Cross-browser compatibility: Test on Chrome, Firefox, Safari
Integration Testing
- End-to-end flows: Test complete user journeys
- Cloud provider integrations: Test with actual credentials
- WebSocket communication: Test chatbot interactions
- Background tasks: Verify Celery tasks complete
- State persistence: Check data saves correctly
Testing Cloud Integrations
AWS Integration Testing
- Configure credentials in Vault or
.env:
- Test resource listing:
- Verify in UI: Navigate to AWS dashboard and check resources display
GCP Integration Testing
- Place service account JSON in
server/connectors/gcp_connector/:
- Set project ID in
.env:
- Test via chatbot: Ask “List my GCP compute instances”
Azure Integration Testing
- Configure Azure credentials:
- Test subscription listing:
Testing the Chatbot
WebSocket Connection Testing
- Open browser console on http://localhost:3000/chat
- Check WebSocket connection: Look for connection established message
- Send test message: “Hello” should receive a response
- Monitor backend logs:
Agent Tool Testing
Test agent tools by asking specific questions:LangGraph Workflow Testing
Monitor the agent workflow in logs:- Agent state transitions
- Tool invocations
- LLM API calls
- Error handling
Testing Database Operations
PostgreSQL Testing
- Connect to database:
- Verify schema:
- Check data:
- Exit:
Weaviate Vector Database Testing
- Access Weaviate console: http://localhost:8080/v1
- Test semantic search:
- Verify embeddings: Check that knowledge base documents are indexed
Testing Vault Integration
Store Test Secret
Retrieve Test Secret
Test in Application
Verify secrets are resolved correctly:Testing Storage (SeaweedFS)
Access Web Interface
- File Browser: http://localhost:8888
- Cluster Status: http://localhost:9333
Test S3 API
Test in Application
Testing Celery Background Tasks
Monitor Task Queue
Trigger Test Task
Trigger a background task and monitor execution:Testing Infrastructure Provisioning
Terraform Workflow Testing
- Request infrastructure via chatbot:
- Review generated Terraform:
- Approve changes: Confirm in UI
- Monitor execution:
- Verify in cloud console: Check resource was created
- Test cleanup:
Performance Testing
Monitor Resource Usage
Database Query Performance
Frontend Performance
- Open Chrome DevTools (F12)
- Network tab: Check request timing
- Performance tab: Record page load
- Lighthouse: Run audit (Performance, Accessibility, Best Practices)
Security Testing
Authentication Testing
Authorization Testing
Test that users can only access their own resources:- Create two test users
- Create projects for each user
- Verify User A cannot access User B’s projects
Input Validation Testing
Test with malicious inputs:Automated Testing (Future)
While Aurora currently focuses on manual testing, future test infrastructure may include:Backend Testing (pytest)
Frontend Testing (Playwright)
CI/CD Testing
Aurora includes CI/CD pipelines that run on every PR:- Environment validation: Checks Docker Compose files are in sync
- Linting: Runs ESLint on frontend code
- Build verification: Ensures containers build successfully
Troubleshooting Test Issues
Services Won’t Start
Database Connection Errors
WebSocket Connection Failures
Frontend Build Errors
Backend Import Errors
Best Practices
- Test early and often: Test changes as you make them
- Test edge cases: Don’t just test the happy path
- Test with real data: Use actual cloud credentials when possible
- Document test scenarios: Record steps for reproducing issues
- Clean up test resources: Delete test cloud resources after testing
- Check logs: Always review logs for warnings/errors
- Test cross-browser: Don’t assume all browsers behave the same
- Test mobile: Verify responsive design on mobile devices
Next Steps
Development Setup
Set up your local development environment
Architecture
Learn about Aurora’s architecture
Contributing
Guidelines for contributing to Aurora