Let AI Write Your Tests - Strategies for Improving Test Coverage

Introduction

"I don't have time to write tests." "I don't know how to write good tests." "Even when I write tests, they break constantly and maintenance becomes overwhelming."

These are struggles that most developers face daily. Despite understanding the importance of testing, the pressure of feature delivery keeps pushing test writing down the priority list, and untested code continues to pile up.

DevLoop Runner offers a structural solution to this problem. Out of Dev Run's 10 phases, three are dedicated to testing. Sumire, the AI QA Engineer, handles everything from test scenario design to test code implementation to test execution and verification.

This article covers how Dev Run's test phases work, a guide to different test types, coverage strategies, and quality checkpoints for AI-generated tests.

Dev Run's Test Phases: Sumire's Three Roles

In Dev Run's 10-phase structure, testing is split into three distinct phases -- all handled by Sumire, the QA Engineer.

Loading diagram...

Phase 4: Test Scenario (What to Test)

This phase runs before the implementation phase. Following a test-first approach, it determines what needs to be tested.

What happens in this phase:

• Identifying test cases (happy paths, error cases, boundary values)
• Detecting edge cases
• Setting test priorities
• Checking consistency with existing tests

Phase 6: Test Implementation (How to Test)

This phase runs after the implementation phase. It writes test code based on the scenarios designed in Phase 4.

What happens in this phase:

• Creating test code
• Following test framework conventions
• Building mocks and stubs
• Preparing test data

Phase 7: Test Execution (Do Tests Pass?)

This phase runs after test implementation. It executes the written tests and verifies the results.

What happens in this phase:

• Running the test suite
• Aggregating and analyzing test results
• Analyzing causes of failed tests
• Checking coverage

Key point: When tests fail, Sumire includes the failure cause and recommended fix in her report. This information is extremely valuable during review.

Why Three Separate Phases?

Separating testing into "Scenario, Implementation, Execution" has clear benefits.

Benefit	Explanation
Test-first approach	Designing tests before implementation prevents drift from requirements
Separation of design and implementation	"What to test" and "how to test" are considered separately
Objective quality verification	Test results are recorded in reports, supporting review decisions
Re-executable	When tests fail, you can roll back to the implementation phase and retry

The Test-First Advantage

Dev Run designs test scenarios before implementation, following the test-first philosophy.

Traditional Flow vs. Dev Run Flow

Traditional development flow:

Implementation -> Write Tests -> Run Tests
(Implementation comes first, so tests tend to follow the implementation)

Dev Run flow:

Test Scenario Design -> Implementation -> Test Implementation -> Test Execution
(Test design comes first, so implementation aligns with requirements)

With test scenarios defined upfront, Riku enters the implementation phase knowing exactly what the code needs to satisfy. The result is code that's more likely to pass tests -- meaning code that meets requirements.

Test Type Guide

Different types of tests serve different purposes and scopes. Here's how to write Issues for each type when using DevLoop Runner.

Unit Tests

Purpose: Verify the behavior of individual functions or classes

Best for:

• Utility function testing
• Business logic verification
• Data transformation validation

Example Issue:

## Overview
Create unit tests for src/utils/validators.ts.

## Target Functions
- validateEmail()
- validatePassword()
- validatePhoneNumber()

## Test Requirements
- Happy path: Return true for valid input
- Error cases: Return error messages for invalid input
- Boundary values: Empty string, max length, special characters

## Test Data
- Valid emails: user@example.com, test+tag@domain.co.jp
- Invalid emails: invalid, @example.com, user@, user@.com
- Password boundaries: 7 chars (fail), 8 chars (pass), 128 chars (pass), 129 chars (fail)

## Framework
Jest (following project's existing configuration)

Tip: Specifying concrete test data helps AI generate appropriate test cases. Boundary values are especially important to make explicit.

Integration Tests

Purpose: Verify interactions between multiple modules or services

Best for:

• API endpoint testing
• Database operations
• Authentication flows

Example Issue:

## Overview
Create integration tests for the user authentication flow.

## Test Scenarios

### Login Flow
1. Login with correct credentials -> Token is returned
2. Login with wrong password -> 401 error
3. Login with non-existent user -> 401 error
4. Login with locked account -> 403 error

### Session Management
1. Request with valid token -> Normal response
2. Request with expired token -> 401 error
3. Request with tampered token -> 401 error

### Logout
1. Token is invalidated after logout

## Technical Requirements
- Use mock server (no actual DB connections)
- No shared state between tests
- Each test must be independently executable

Tip: For integration tests, writing scenarios as concrete steps is crucial. Mentioning test independence also leads to higher-quality generated tests.

E2E Tests

Purpose: Verify complete user operation flows

Best for:

• Critical user flows (purchase, registration, settings changes)
• Form input and validation
• Page transitions and navigation

Example Issue:

## Overview
Create E2E tests for the checkout flow.

## Test Scenarios

### Happy Path
1. Display product list page
2. Add product to cart
3. Verify quantity on cart page
4. Proceed to checkout
5. Enter shipping information
6. Select payment method
7. Confirm order
8. Order confirmation page is displayed

### Error Cases
1. Attempt to purchase out-of-stock item -> Error message displayed
2. Attempt to proceed with empty required fields -> Validation errors

## Technical Requirements
- Use Playwright
- Capture screenshots at each step
- Timeouts: 10 seconds per action, 60 seconds total
- Manage test data with fixtures

Test Type Comparison

Type	Scope	Speed	Maintenance Cost	Recommended Proportion
Unit Tests	Function/class level	Fast	Low	Most (70-80%)
Integration Tests	Cross-module	Medium	Medium	Moderate (15-20%)
E2E Tests	Full user flow	Slow	High	Few (5-10%)

This ratio follows the "test pyramid" concept. By building on a foundation of unit tests, layering integration tests, and topping with E2E tests, you achieve efficient coverage.

Thinking About Test Coverage

Types of Coverage

Test coverage encompasses several metrics.

Metric	Description	Recommended Target
Line Coverage	Percentage of executed lines	80%+
Branch Coverage	Percentage of executed branches	70%+
Function Coverage	Percentage of tested functions	90%+

Including Coverage Goals in Issues

Specifying coverage targets helps AI generate tests with appropriate thoroughness.

## Coverage Goals
- Line coverage: 80% or higher
- Branch coverage: 70% or higher
- Cover all public methods
- Test error handling paths

Should You Aim for 100%?

While 100% coverage sounds ideal, there's a point of diminishing returns in practice.

Prioritize testing:

• Business logic (revenue calculations, permission checks, etc.)
• Data transformation and validation
• Error handling
• Boundary conditions

Lower priority for testing:

• Simple getters/setters
• Functionality guaranteed by the framework
• One-line delegation methods

What matters most is ensuring critical business logic is reliably tested, not chasing a coverage number.

Quality Checkpoints for AI-Generated Tests

Before merging tests generated by Dev Run, review them through several lenses.

Checklist

Test design:

• Do test cases cover the requirements?
• Are error cases tested, not just happy paths?
• Are boundary value tests included?
• Are edge cases considered?

Test code quality:

• Do test names clearly describe what's being tested?
• Does each test verify a single behavior? (single assertion principle)
• Are tests independent of each other? (order-independent)
• Is test data appropriately parameterized rather than excessively hardcoded?

Maintainability:

• Are test helpers and factories used appropriately?
• Are tests verifying behavior rather than implementation details?
• Are mocks used judiciously rather than excessively?

Common Quality Issues

Issue	Symptom	Solution
Implementation coupling	Tests break when internal structure changes	Test public API behavior instead
Excessive mocking	Tests become copies of the implementation	Use integration tests for actual interactions
Flaky tests	Results vary depending on execution timing	Fix non-deterministic elements like timestamps
Missing assertions	Tests pass but verify nothing	Set explicit expectations in each test

Using Rollback When Tests Fail

When tests fail during Dev Run's test execution phase, you have several options.

Failure Cause Patterns

Loading diagram...

Choosing the Right Rollback Target

Failure Cause	Rollback To	Explanation
Implementation bug	Implementation Phase	Test scenarios are correct, but implementation doesn't meet requirements
Test design error	Test Scenario Phase	Test case preconditions are wrong
Design issue	Design Phase	Architecture-level reconsideration needed

When you roll back, all phases after the target are re-executed. Artifacts from already-completed phases are preserved, so you don't need to start from scratch.

Best Practices for Test Generation

1. Match Existing Test Style

When your project has existing tests, matching their style maintains consistency.

## Reference
Follow the style of the existing test file
src/utils/__tests__/formatter.test.ts.

## Specific style points to match
- describe / it nesting structure
- Test data definition approach
- Mock usage patterns
- File placement (__tests__ directory)

2. Specify Test Data Explicitly

Providing concrete test data helps AI generate more appropriate test cases.

## Test Data
- Valid emails: user@example.com, test+tag@domain.co.jp
- Invalid emails: invalid-email, @example.com, test@, user@.com
- Boundary values: empty string, max length (256 chars), Unicode domains

3. Require Test Independence

## Test Requirements
- Each test must be independently executable
- No shared global state between tests
- Use beforeEach / afterEach for setup and cleanup

4. Consider CI/CD Execution

## Execution Environment Requirements
- Must work in CI environment (GitHub Actions)
- No external service dependencies (all mocked)
- Target execution time: Under 30 seconds for the full test suite

Approaching a Codebase with No Tests

Demanding high coverage immediately for a codebase with zero tests isn't realistic. Take a staged approach.

Stage 1: Critical Business Logic

Start with the highest-impact areas -- revenue calculations, permission checks, and other logic where bugs would be most damaging.

Stage 2: Data Transformation and Validation

Target areas with clear inputs and outputs, such as API response transformations and form validation.

Stage 3: Integration Tests

Test key API flows and authentication flows that span multiple modules.

Stage 4: E2E Tests

Add E2E tests for critical user flows like purchases and registrations.

Using DevLoop Runner's Dev Run at each stage lets AI generate test code for you, dramatically reducing the cost of building out your test suite.

Summary

• Dev Run has three test phases: Test Scenario, Test Implementation, Test Execution -- all handled by Sumire, the QA Engineer
• Test-first approach: Test scenarios are designed before implementation, keeping code aligned with requirements
• Use the right test types: Combine unit, integration, and E2E tests following the test pyramid ratio
• Coverage is about content, not numbers: Ensuring critical business logic is tested matters more than the coverage percentage
• AI-generated tests need review too: Check test design, code quality, and maintainability
• Roll back when tests fail: Roll back to the appropriate phase based on the failure cause

Tests serve as both "insurance" and "documentation" for your development. By letting DevLoop Runner's Sumire handle test creation, you lower the barrier to testing and continuously improve coverage over time.