23 posts tagged with "Software Testing"

TL;DR: Trace-based testing is a software testing approach where tests are built from real execution traces captured from a running application. Instead of writing mock data manually or using AI to guess test cases from source code, trace-based testing records actual method calls, SQL queries, and API responses, then replays them as standard JUnit tests.

Teams usually discover trace-based testing when they hit the same wall: green unit tests, green code review, and still a production regression after a harmless-looking change.

TL;DR: In the industry, there is a weird habit: if a tool can generate tests, it is considered automatically useful. If you have 300 new .java files in your repo after recording a scenario, the team assumes they have "more quality." They are wrong. Automated test generation often turns into a source of engineering pain, cluttering repositories and burying real regressions in noise. There is a more mature path: capture real execution traces, store them as data, and replay them dynamically.

TL;DR: AI coding agents are incredibly powerful at reading and writing source code, but when it comes to understanding what code actually does at runtime, they are blind. They can read `repository.findAll()` in the source code, but they cannot see the 40-query N+1 explosion it causes in production. This post explores how AI routinely breaks APIs and how connecting agents to runtime context and before/after trace comparison through BitDive and the Model Context Protocol (MCP) dramatically improves their reliability.

TL;DR: Bumping the Spring Boot version in pom.xml takes 10 seconds. Finding out that the upgrade silently changed your JSON serialization, broke your security filters, or altered your error responses takes days. This post covers seven categories of silent breaking changes that Spring Boot upgrades introduce, and a practical workflow to catch them before production.

TL;DR: Your integration test hits the happy path: 200 OK, correct response body, all good. But what happens when the downstream returns a 404? A 500? A 503 with retry-after? If your service parses the error response body, and the body format changes silently, you will find out at 3 AM when the retry storm starts. Error contracts are just as important as success contracts, and almost nobody tests them.

TL;DR: Your service compiles. Your unit tests pass. Your integration tests are green. But a single line in your ObjectMapper configuration just changed what every outgoing HTTP request looks like. The downstream service cannot parse the payload anymore, and you will find out in production. Here are five Jackson configuration changes that cause this, with exact before/after JSON for each.

TL;DR: The most dangerous bugs in microservices are not inside a service. They are between services. A code change can make a service pass all its local tests while silently altering what it sends to downstream APIs: different payload, missing header, changed error format. These regressions are invisible to unit tests, hard to catch with contract tests, and expensive in production. BitDive detects them by capturing real HTTP exchanges in execution traces and comparing them before and after a code change.

TL;DR: Boot the full Spring context. Stub only what lives outside your service boundary: Feign clients, external HTTP APIs, outbound Kafka. Then hit the service through its real HTTP endpoint and verify the entire chain: controller, validation, service logic, @Transactional, repository, database write, response serialization. This is what Spring calls an integration test. It catches the class of bugs that unit tests structurally miss: broken configs, silent serialization changes, transaction proxy bypass, security filter misconfiguration, and DTO contract drift.

TL;DR: In real-world Spring projects, the "unit vs integration" debate almost always stems from the fact that "integration testing" has become a catch-all term for everything from @SpringBootTest with Testcontainers to full-blown E2E runs on staging environments. To stop arguing and start shipping, we need to draw a clear line in the sand regarding responsibility.

A unit test answers one question: "Is the logic correct in total isolation?" It deliberately cuts infrastructure out of the equation.

A component test answers another: "Does the component work as a system within its own boundaries, including its Spring wiring, configurations, serialization, transactions, and data access?"

If you only have units, you'll inevitably get burned at the seams. If you only have component tests, you'll pay with execution time, flakiness, and painful debugging. The winning strategy is simple: unit tests provide the speed and density of logic verification; component tests provide the confidence that the "real assembly" actually works.

Deterministic Verification: The Safety Layer for AI-Native Development​

TL;DR: As AI models like Claude, GPT-4, and Gemini write more of our code, the bottleneck has shifted from writing to verifying. Traditional mock-heavy tests are too fragile for AI-native workflows. BitDive provides runtime context, before/after trace comparison, and replay-based JUnit tests that enable a safe autonomous development loop.

The Verification Gap in AI-Native Development​

In 2026, the industry has reached a tipping point. AI assistants can now create 1,000 lines of functional code in seconds. However, verifying that this code doesn't break subtle production invariants remains a manual, slow process.

We call this the Verification Gap.