Design Interview

Goal: Identify all systems involved and their connection details.

Questions:

• Where does data come from (source systems)?
• Where does data go (destination systems)?
• What protocols or APIs are used?
• Are there intermediary systems (enrichment, validation)?

Example Exchange:

AI: Where do orders come from?

You: The e-commerce website posts orders to an HTTP endpoint 
     we need to expose.

AI: What is the destination for validated orders?

You: They go to a Kafka topic called "warehouse.orders" 
     that our WMS consumes.

AI: Do you need to connect to any other systems for 
     enrichment or validation?

You: Yes, we need to look up customer credit status in 
     a PostgreSQL database before sending to the warehouse.

MCP Verification: As you mention systems, the AI verifies component names:

AI: (MCP lookup: "http") ✓ camel-rest
AI: (MCP lookup: "kafka") ✓ camel-kafka
AI: (MCP lookup: "postgresql") ✓ camel-sql

If you say “We need to use SuperQueue,” the AI responds:

AI: I couldn't find a component for "SuperQueue" in the 
    Camel catalog. Could you clarify which messaging system? 
    (ActiveMQ, RabbitMQ, Kafka, AWS SQS, etc.)

This is Iron Law #1 in action: no component name enters the design without MCP verification.

Goal: Understand input/output formats and transformation requirements.

Questions:

• What format does input data arrive in? (JSON, XML, CSV, etc.)
• What format should output data be in?
• Are transformations needed between input and output?
• Are there schemas or data contracts?

Example Exchange:

AI: What format do orders arrive in from the website?

You: JSON with fields like orderId, customerId, items array, 
     and totalAmount.

AI: What format does the Kafka message need to be?

You: Also JSON, but we need to add customer credit status 
     and warehouse location code.

AI: This sounds like a field-level transformation. Would you 
     like to use DataMapper for reliable mapping, or simple 
     JSON manipulation?

You: Let's use DataMapper since this needs to be very reliable.

DataMapper Decision: If transformations are complex, the AI offers DataMapper (XSLT-based). For simple transformations, it suggests JSON processors or bean methods.

Goal: Identify business logic, validation rules, and processing steps.

Questions:

• What validations are required?
• What business rules must be enforced?
• Are there calculations or enrichments?
• Any conditional routing (if-then-else)?

Example Exchange:

AI: What validations are needed for incoming orders?

You: The orderId must be unique, totalAmount must be greater 
     than zero, and customerId must exist in our database.

AI: What should happen if validation fails?

You: Invalid orders go to a separate Kafka topic "orders.invalid" 
     for manual review, with the validation error message included.

AI: You mentioned adding credit status. What's the business rule?

You: If creditStatus is "BLOCKED", don't send to warehouse. 
     Route to "orders.pending" topic instead.

Conditional Routing: The AI identifies choice/when patterns and includes them in the design.

Goal: Define how failures are handled across different failure modes.

Questions:

• What should happen if source system is unavailable?
• What about destination system failures?
• Should there be retries? How many?
• Dead letter queues or error topics?
• Alerting or notifications on errors?

Example Exchange:

AI: What should happen if the database lookup for customer 
    credit status fails?

You: Retry up to 3 times with exponential backoff. If all 
     retries fail, send to "orders.error" topic.

AI: What about Kafka publish failures?

You: Kafka should be highly available, but if publish fails, 
     log the error and retry indefinitely until it succeeds.

AI: Should anyone be notified of errors?

You: Yes, send an email to ops@company.com if we get more 
     than 10 database failures in 5 minutes.

Error Classification: The AI categorizes errors:

• Transient errors → retry
• Validation errors → route to dead letter / invalid topic
• System errors → alert

Goal: Capture non-functional requirements and monitoring needs.

Questions:

• What is the expected throughput?
• What are latency requirements?
• How should metrics be exposed?
• What logging is needed?
• Any tracing requirements?

Example Exchange:

AI: What is the expected order volume?

You: Peak hours see about 500 orders per minute. Off-peak 
     is around 50 per minute.

AI: What latency is acceptable?

You: Each order should be processed end-to-end in under 
     2 seconds.

AI: What metrics do you need to track?

You: Total orders received, validation pass rate, database 
     lookup time, Kafka publish time, and error count by type.

Constitution Rule 5: Observability is mandatory. The AI ensures metrics and logging are included in the design.

List of distinct flows that make up the integration.

• Order Reception — REST endpoint for POST requests
• Order Validation — schema and business rules
• Customer Credit Lookup — PostgreSQL query
• Routing Decision — warehouse / pending / invalid
• Warehouse Publisher — Kafka warehouse.orders
• Error Handler — retries and alerting

Detailed connection information for all systems.

• Source: HTTP POST to /api/orders (JSON)
• Enrichment: PostgreSQL at ${db.url}
• Destinations: Kafka topics warehouse.orders, orders.pending, orders.invalid
• Alerts: SMTP email to ${ops.email}

All connection details externalized via application.properties.

Input/output formats and transformation specifications.

Input: JSON with orderId, customerId, items[], totalAmount

Enriched output: Same fields + creditStatus, warehouseLocation

Transformation: DataMapper XSLT for reliable field-level mapping.

Detailed strategy for each failure mode:

• Validation errors → route to orders.invalid (no retry)
• Database errors → retry 3x with exponential backoff, then alert
• Kafka errors → retry indefinitely (wait for recovery)
• Credit blocked → route to orders.pending (not an error)

Alert threshold: > 10 database failures in 5 minutes → email ops.

Components: camel-rest, camel-jackson, camel-sql, camel-kafka, camel-mail, camel-bean

Performance:

• Throughput: 500/min peak
• Latency: < 2s per order
• Concurrency: 10 threads

Metrics: order counts, validation failures, DB lookup time, Kafka publish latency

Logging: INFO (received), WARN (validation fail), ERROR (DB/Kafka fail)

External config: db.url, kafka.brokers, ops.email, rest.port — all via application.properties

Greenfield: /camel-brainstorm conducts a full Socratic interview because there’s no existing implementation to analyze.

Migration: /camel-migrate parses existing artifacts (Mule XML, Camel 2.x XML, etc.) and generates the Design Specification automatically by analyzing existing flows. It still presents a spec for approval, but skips the interview.

The interview is defined in the brainstorm skill. You can customize questions by editing the skill file.

Example: Add security questions to every interview:

### Security Requirements
Ask about:
- Authentication and authorization
- Data encryption (in-transit and at-rest)
- Secrets management
- Compliance (GDPR, HIPAA, etc.)

Now every design interview includes security, and the spec gains “Section 8: Security Requirements.”

Asynchronous fire-and-forget — accept immediately, process in background.

AI: Should the HTTP endpoint wait for Kafka confirmation?

You: Return immediately with HTTP 202 Accepted

AI: Should we use an intermediary queue for reliability?

You: Yes, use SEDA for async processing

Design Output: Two flows: REST → SEDA (return 202), then SEDA → Kafka.

Complex multi-system — parallel writes with distributed transaction handling.

AI: Where does data go?
You: Three places: inventory DB, CRM, and shipping email

AI: Sequence or parallel?
You: Parallel, but rollback all if any fails

AI: This requires Saga pattern. Should we use it?
You: Yes, with compensating transactions

Design Output: Multi-flow with Saga coordination.