Skip to content

fix(validate): display all validation errors instead of just the first one - #49

Merged
unclesp1d3r merged 3 commits into
mainfrom
32-bug-validation-only-shows-first-error-instead-of-all-validation-issues
Aug 4, 2025
Merged

fix(validate): display all validation errors instead of just the first one#49
unclesp1d3r merged 3 commits into
mainfrom
32-bug-validation-only-shows-first-error-instead-of-all-validation-issues

Conversation

@unclesp1d3r

Copy link
Copy Markdown
Member

Description

Fixes issue #32 where the validate command only showed the first validation error instead of all validation issues.

Changes Made

  • Updated AggregatedValidationError.Error() method to display all validation errors in a numbered list format instead of showing only the first error with 'and X more'
  • Modified validate command to properly handle and display all validation issues for each file
  • Updated tests to match the new comprehensive error message format

Before vs After

Before: Only showed first validation error

validation failed with 7 errors: hostname is required \(and 6 more\)

After: Shows all validation errors clearly numbered

validation failed with 7 errors:
  1. validation error at opnsense.system.hostname: hostname is required
  2. validation error at opnsense.system.domain: domain is required
  3. validation error at opnsense.system.timezone: invalid timezone format: Invalid/Timezone
  ...

Testing

  • ✅ Files with multiple validation errors display all issues
  • ✅ Valid files show success messages correctly
  • ✅ Multiple files handled independently
  • ✅ All existing tests pass
  • ✅ CI checks pass

Impact

Network administrators can now see all configuration problems at once to prioritize and batch their fixes effectively, significantly improving the user experience when working with configurations that have multiple issues.

Closes #32

- Introduced a new DTD file for opnsense configuration, defining the structure and elements for XML configuration files.
- Updated the XSD schema to reflect changes in the configuration structure, including the addition of new elements and attributes.
- Removed deprecated elements and adjusted sequences to improve validation accuracy.

Tested with `just test` and `just ci-check`, all checks passed successfully.
…elements

- Removed multiple deprecated optional elements from the opnsense-config XSD schema to streamline the configuration structure.
- Introduced a new `xs:any` element to allow for additional interface names, enhancing flexibility for DHCP configuration.
- Updated the schema to reflect standard/reserved interface names while maintaining support for custom naming.

Tested with `just test` and `just ci-check`, all checks passed successfully.
…t one

- Update AggregatedValidationError.Error() to show all validation errors with numbered list
- Modify validate command to properly display all validation issues for each file
- Update tests to match new error message format
- Fixes issue #32 where only first validation error was shown
Copilot AI review requested due to automatic review settings August 4, 2025 01:26
@unclesp1d3r unclesp1d3r linked an issue Aug 4, 2025 that may be closed by this pull request
5 tasks
@coderabbitai

coderabbitai Bot commented Aug 4, 2025

Copy link
Copy Markdown
Contributor

Caution

Review failed

The pull request is closed.

Summary by CodeRabbit

  • New Features

    • Added a comprehensive XML DTD for OPNsense configuration files, enabling formal validation of configuration structure and content.
  • Enhancements

    • Expanded and restructured the XML schema (XSD) for OPNsense configurations to support more flexible and detailed interface, DHCP, firewall, VPN, and logging options.
    • Improved error reporting for file validation and validation errors, providing clearer and more detailed multi-line messages.
  • Bug Fixes

    • Updated validation error message formatting to enumerate all errors for greater clarity.
  • Tests

    • Refined tests to verify the new, detailed validation error messages.

Walkthrough

The changes update validation error reporting to display all validation issues in a multi-line format, both in the error aggregation logic and the CLI output. Tests are updated to verify the new error message structure. Additionally, comprehensive OPNsense configuration DTD and expanded XSD schema files are introduced or updated in the test data.

Changes

Cohort / File(s) Change Summary
Validation Error Aggregation and Reporting
cmd/validate.go, internal/parser/errors.go, internal/parser/errors_test.go
Refactored error handling to print all validation errors in a detailed, multi-line format. Updated the aggregation logic to enumerate all errors, and enhanced tests to assert the new message structure.
Test Data: OPNsense Config Schemas
testdata/opnsense-config.dtd, testdata/opnsense-config.xsd
Added a comprehensive DTD for OPNsense configuration. Extensively modified the XSD schema to expand, reorganize, and add new elements for more flexible and detailed configuration validation.

Sequence Diagram(s)

sequenceDiagram
    participant User
    participant CLI (validate.go)
    participant Parser (errors.go)
    participant StdErr

    User->>CLI (validate.go): Run validate command
    CLI (validate.go)->>Parser (errors.go): Validate file
    Parser (errors.go)-->>CLI (validate.go): Return AggregatedValidationError (with all errors)
    CLI (validate.go)->>StdErr: Print all validation errors (multi-line)
    CLI (validate.go)-->>User: Validation complete
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~40 minutes

Assessment against linked issues

Objective Addressed Explanation
Display ALL validation issues, not just the first, in the validate command output (#32)
Validation should continue processing after finding errors to build a comprehensive report (#32)
Validation errors should be shown in a multi-line, detailed format for clarity (#32)

Assessment against linked issues: Out-of-scope changes

Code Change Explanation
Addition of full OPNsense DTD schema (testdata/opnsense-config.dtd) The linked issue is about validation error reporting, not about adding schema files.
Major expansion and restructuring of OPNsense XSD schema (testdata/opnsense-config.xsd) These schema changes are unrelated to the validation error display logic described in the issue.

Possibly related PRs

Poem

A rabbit hops through configs deep,
Now lists all errors—not just one heap!
With schemas vast and tests anew,
Validation’s thorough, clear, and true.
No more guessing, no more dread—
All your config woes are fully said!
🐇✨


📜 Recent review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between db4f1c4 and d7a188c.

📒 Files selected for processing (5)
  • cmd/validate.go (1 hunks)
  • internal/parser/errors.go (1 hunks)
  • internal/parser/errors_test.go (1 hunks)
  • testdata/opnsense-config.dtd (1 hunks)
  • testdata/opnsense-config.xsd (15 hunks)
✨ Finishing Touches
  • 📝 Generate Docstrings
🧪 Generate unit tests
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch 32-bug-validation-only-shows-first-error-instead-of-all-validation-issues

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Explain this complex logic.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai explain this code block.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and explain its main purpose.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
  • @coderabbitai generate unit tests to generate unit tests for this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR fixes issue #32 by modifying the validation error display to show all validation errors instead of truncating after the first one. The changes ensure network administrators can see all configuration problems at once, improving their ability to prioritize and batch fixes effectively.

  • Comprehensive error display in validation output
  • Updated error message format to numbered list
  • Maintained existing test coverage with updated assertions

Reviewed Changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
internal/parser/errors.go Modified AggregatedValidationError.Error() to display all validation errors in numbered format
internal/parser/errors_test.go Updated test assertions to match new comprehensive error message format
cmd/validate.go Enhanced validate command to properly display all validation issues per file
testdata/opnsense-config.xsd Updated XML schema definitions (test data changes)
testdata/opnsense-config.dtd Added comprehensive DTD definitions (test data changes)

Comment thread internal/parser/errors.go
return fmt.Sprintf("validation failed with %d errors: %s (and %d more)",
len(r.Errors), r.Errors[0].Message, len(r.Errors)-1)
// Build a comprehensive error message showing all validation errors
var sb strings.Builder

Copilot AI Aug 4, 2025

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The strings package should be imported at the top of the file. Add 'strings' to the import statement to ensure the code compiles correctly.

Copilot uses AI. Check for mistakes.
@unclesp1d3r unclesp1d3r self-assigned this Aug 4, 2025
@unclesp1d3r

Copy link
Copy Markdown
Member Author

@coderabbitai full review

@coderabbitai

coderabbitai Bot commented Aug 4, 2025

Copy link
Copy Markdown
Contributor
✅ Actions performed

Full review triggered.

@unclesp1d3r
unclesp1d3r merged commit 09038d7 into main Aug 4, 2025
11 checks passed
@unclesp1d3r
unclesp1d3r deleted the 32-bug-validation-only-shows-first-error-instead-of-all-validation-issues branch August 4, 2025 01:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: Validation only shows first error instead of all validation issues

2 participants