Debugging Guide for Failed Jobs

Introduction

When running Dev Runs with DevLoop Runner, jobs can occasionally fail. During the AI code generation process, issues like test failures, design inconsistencies, and external service connection errors can arise.

When a job fails, the most important thing is to stay calm. DevLoop Runner records detailed information about the failed phase, and provides the tools to identify the cause and re-execute.

This article provides a systematic guide to debugging failed jobs -- from identifying the type of failure to phase-specific troubleshooting, rollback-based re-execution, and Issue improvement strategies.

Types of Job Failures

Job failures fall into three main categories.

Phase Failures

An error occurs during a specific phase of the Dev Run. This is the most common failure pattern.

Phase	Common Failure Causes
Planning	Issue description too vague to create a plan
Requirements	Contradictory requirements
Design	Incompatibility with existing architecture
Test Scenarios	Unclear test targets
Implementation	Compilation errors, syntax errors
Test Implementation	Test framework configuration issues
Test Execution	Test failures (assertion errors)
Documentation	Documentation template mismatches

Timeouts

The job exceeds its time limit.

• Scope is too large (too many files, large codebase)
• External API responses are slow
• Near-infinite processing loops

Authentication Errors

Authentication with GitHub or other external services fails.

• GitHub token expired
• Insufficient repository access permissions
• API key misconfiguration

For authentication issues, see the credentials setup guide and credential validation guide.

Identifying the Failed Phase

Reading the Job Detail Screen

When a job fails, DevLoop Runner's job detail screen displays failure information.

Key information to check:

1. Failed phase - Which phase the failure occurred in
2. Error message - The error content output by the AI
3. Logs - Detailed logs showing what happened before and after the failure
4. Successful phases - How far the job progressed successfully

For basics on monitoring job progress, see the job progress guide.

Reading Logs

Logs record the execution details and results of each phase.

What to look for in logs:

OK  Planning Phase: Complete
OK  Requirements Phase: Complete
OK  Design Phase: Complete
OK  Test Scenarios Phase: Complete
OK  Implementation Phase: Complete
OK  Test Implementation Phase: Complete
NG  Test Execution Phase: Failed
    └─ Error: 3 tests failed
       ├─ test_create_order: AssertionError - expected 201 but got 400
       ├─ test_invalid_quantity: AssertionError - expected error message not found
       └─ test_duplicate_order: TimeoutError - test timed out after 30s

This log reveals that the test execution phase failed with 3 test failures. The specific error messages provide clues for identifying the root cause.

Phase-Specific Troubleshooting

Planning Phase Failures

Common causes:

• Issue description is too vague
• Requirements contradict each other
• Requirements are technically infeasible

Solutions:

1. Review and refine the Issue description
2. Make vague sections more specific
3. Use the Rewrite Issue feature to have AI optimize the Issue
4. Re-run the job

BAD  Vague Issue
"Improve performance"

GOOD  Specific Issue
"Improve product list API response time
 - Current: 3 seconds average
 - Target: under 500ms
 - Approach: Resolve N+1 queries, introduce caching"

Design Phase Failures

Common causes:

• Incompatibility with existing architecture
• Technical constraints not considered
• Requirements too complex

Solutions:

1. Add technical constraints and existing pattern information to the Issue
2. Run in "plan only" mode to review the design approach first
3. Split requirements and address them one at a time

## Technical context to add
- Backend framework: Express.js
- ORM: Prisma
- Auth method: JWT
- Directory structure: src/controllers, src/services, src/models
- Error handling pattern: uses AppError class

Implementation Phase Failures

Common causes:

• Compilation or syntax errors
• Dependency issues (packages not found)
• Type mismatches with existing code

Solutions:

1. Check the error message
2. For dependency issues, specify package versions in the Issue
3. For type mismatches, add existing type definition information to the Issue
4. Use rollback to restart from the design phase

Test Execution Phase Failures

Test execution failures are the most common pattern. They occur when there are inconsistencies between AI-generated code and tests.

Common causes:

• Test expectations do not match the implementation
• Insufficient test environment setup
• Missing mocks for external service dependencies
• Timeouts

Solutions:

DevLoop Runner analyzes test results and often retries automatically. If automatic retries do not resolve the issue:

1. Read the error messages from failed tests
2. Determine whether the problem is in the implementation or the tests
3. If the implementation is wrong, leave a PR comment with correction instructions
4. If the tests are wrong, leave a PR comment requesting test corrections
5. If there is a fundamental design issue, consider rolling back

Documentation Phase Failures

Common causes:

• Documentation template format mismatches
• Inconsistency with existing documentation

Solutions:

Documentation phase failures are relatively rare. If they occur, you can use phase skipping to bypass the documentation phase and update documentation manually.

Re-Execution with Rollback

When design-level issues cause failures, rollback is the most effective approach.

When to Use Rollback

Loading diagram...

Cases where rollback is effective:

• Design approach does not fit the existing architecture
• You want to change the overall approach
• Implementation became too complex and tests will not pass

Rollback steps:

1. Check the failed job's logs to identify the problematic phase
2. Select the phase to roll back to (usually the design phase)
3. Add supplementary information to the Issue if needed
4. Execute the rollback

See the rollback guide for full details.

Improving Issues for Re-Execution

In many cases, job failures can be resolved by improving the Issue description.

Issue Improvement Checklist

When a job fails, review the Issue against these criteria:

• Are requirements described specifically?
• Are technical constraints documented?
• Is there reference to existing patterns and architecture?
• Are validation rules concrete?
• Is the error handling approach specified?
• Are test expectations clear?

Using Rewrite Issue

DevLoop Runner's Issue Run includes a feature to rewrite Issues. Having the AI optimize a vague Issue can significantly improve the Dev Run success rate.

See the Rewrite Issue guide for details.

Changing Execution Modes as a Workaround

When a full-phase run fails, switching execution modes can help work around specific problems.

Use "Plan Only" to Verify Design First

If the design phase is problematic during a full-phase run, run in "plan only" mode first to review the design approach. Once the design looks sound, continue with "implementation only" mode.

Skip Problematic Phases

If a specific phase fails repeatedly, consider skipping it and handling that work manually.

For example, if the test execution phase keeps failing, you can skip it to generate the PR, then manually review and fix the tests.

See the phase skipping guide for details.

Common Error Patterns and Solutions

Pattern 1: Test Expectation Mismatch

Error: AssertionError
Expected: { status: 201, body: { id: "..." } }
Actual: { status: 400, body: { error: "Validation failed" } }

Cause: Test fixture data fails validation

Solution: Leave a PR comment instructing the AI to fix the test fixture data

Pattern 2: Missing Dependencies

Error: Module not found: 'some-package'

Cause: AI used a package that is not installed

Solution: Specify required packages in the Issue, or leave a PR comment requesting package.json updates

Pattern 3: Type Mismatches

Error: Type 'string' is not assignable to type 'number'

Cause: Existing type definitions conflict with the AI's implementation

Solution: Add existing type definition details to the Issue and re-run

Pattern 4: Expired Authentication Token

Error: 401 Unauthorized - Bad credentials

Cause: GitHub token has expired

Solution: Regenerate the token following the credentials setup guide

Pattern 5: Timeout

Error: Job timed out after 30 minutes

Cause: Issue scope is too large

Solution: Split the Issue into smaller, focused Issues to reduce processing per job

Debugging Flowchart

Here is a decision tree for when a job fails:

Loading diagram...

Conclusion

• Job failures fall into three categories: phase failures, timeouts, and authentication errors
• Use the job detail screen and logs to identify the failed phase and error details
• Test execution failures are the most common -- address them with PR comments or rollback
• Design-level issues call for the rollback feature
• Many failures can be resolved by improving the Issue description
• Changing execution modes or skipping phases can work around persistent problems
• Knowing common error patterns helps you resolve issues quickly

Job failures are a natural part of AI-powered development. Do not let them discourage you. Follow this debugging guide to identify root causes and re-execute with the right approach.