Skip to main content

Testing FAQ & Troubleshooting

Trace-Driven Testing is a paradigm shift. It's natural to have questions about how it handles the messy reality of software development.

Core Concepts​

Does BitDive replace Mockito?​

Short Answer: It replaces the manual labor of using Mockito. Long Answer: You no longer need to write verbose when(...).thenReturn(...) chains. BitDive automates this boilerplate. Under the hood, the engine configures standard Mockito definitions based on the captured trace data. You get the reliable isolation of mocks without the maintenance overhead of writing and updating them by hand.

Handling Database Schema Changes​

Since BitDive replays the ResultSet (the data returned to Java) rather than executing the SQL against a live DB, simple schema changes often don't break the test. However, if you change your Java Entity mappings such that they no longer match the recorded data structure, the test will fail. The Fix: Simply capture a new trace with the updated schema and create a new test ID. It takes 2 minutes.

How does BitDive close the AI Reality Gap?​

BitDive closes the gap by providing AI agents with "Real Runtime Data", real runtime context including method arguments, SQL queries, and service dependencies. This allows agents to verify their own code against recorded reality before it ships.

Testing External APIs (Stripe, Salesforce)​

Absolutely. This is one of the strongest use cases. BitDive captures the HTTP response coming back from Stripe (including headers and body). When you replay the test, we serve that response instantly. You don't need internet access, and you won't get billed by Stripe for running your test suite 1,000 times.

Troubleshooting Failures​

"My test failed with a 'Method Call Deviation'."​

This is the most common error and it means: Your code did something different.

  • Deviation: Expected calculateTax(100) but saw calculateTax(105).
  • Cause: Did you change the tax logic? If yes, the test is doing its job (alerting you to a change). If no, look for upstream changes that might have fed bad data into this method.

"Refactoring broke my tests."​

BitDive tests validate internal method calls. If you rename a method or change its signature, the "sheet music" (Replay Plan) no longer matches the instruments (your code).

  • Minor Refactors: BitDive uses fuzzy matching to handle small changes.
  • Major Refactors: If you completely rewrite a service class, the old trace is invalid. Treat this like deleting a legacy feature: you delete the old test and capture a new one against the new architecture.

Best Practices​

"How do I handle random data (UUIDs, Dates)?"​

If your code creates a random UUID every time it runs, it won't match the recorded UUID.

  • Automatic: BitDive attempts to detect and mask these fields automatically.
  • Manual: You can configure the test profile to "Ignore" specific fields or arguments that are inherently non-deterministic.

"How big should a trace be?"​

Don't try to capture the entire lifecycle of the universe in one trace.

  • Good: "User Login" (single operation).
  • Good: "Checkout Cart" (single operation).
  • Bad: "Server startup + 50 user logins + Database migration." Keep traces focused on a specific business transaction for easier debugging.

"How do I prevent flaky tests from external API dependencies?"​

When your code calls an external service (Stripe, Salesforce, a partner API), tests can flake due to network issues, rate limits, or the external service being down. There are three common approaches:

  • Containerize: Use WireMock or MockServer in a Testcontainer to serve canned responses.
  • Stub Server: Use OkHttp MockWebServer to serve responses inline in your test.
  • BitDive Replay: Let BitDive capture the real HTTP responses during a recording, then replay them deterministically. No stub server to maintain, and the data is production-real.

BitDive's approach is the most practical when you have many external integrations, since you don't need to manually maintain JSON fixtures or stub configurations for each one.

Need more help?​

  • Check the Concepts Guide for deep technical details.
  • Contact our support if you hit an edge case not covered here.
Last updated on by turmyshevd