Skip to content

Conversation

ktyubeshi
Copy link

@ktyubeshi ktyubeshi commented Apr 12, 2025

Description:

This PR adds a --force-include feature to include specific files regardless of ignore patterns.

Key changes:

  • Added functionality to force include files by overriding ignore patterns
  • Implemented command-line option --force-include <patterns>
  • Added forceInclude property to configuration files
  • Updated related tests and documentation

Usage example:

repomix --force-include "coverage/summary.json,test-results/junit.xml"

This feature allows including normally ignored files (e.g., coverage reports or specific log files) in the output when needed.

Documentation

  • Updated README.md with the new option
  • Updated website documentation in both English and Japanese versions
  • Added explanation of ignore pattern priority in configuration guides

Testing

  • Added tests for the new feature
  • All existing tests pass with the new implementation

Checklist

  • Run npm run test
  • Run npm run lint

Summary by CodeRabbit

  • New Features
    • Introduced a new option to forcibly include specific files, overriding ignore patterns, via both CLI (--force-include) and configuration files.
  • Documentation
    • Updated user guides and configuration examples in both English and Japanese to document the new force include option and its usage.
  • Tests
    • Extended test coverage to verify correct handling of the new force include functionality.

@ktyubeshi ktyubeshi requested a review from yamadashy as a code owner April 12, 2025 17:25
@ktyubeshi ktyubeshi force-pushed the feature/forceinclude branch from f8bb98a to ffef120 Compare April 19, 2025 10:42
Copy link
Contributor

coderabbitai bot commented Apr 19, 2025

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Walkthrough

A new "force include" feature was introduced, allowing users to specify files or patterns that should always be included in file searches, overriding any ignore rules. This is implemented via a new forceInclude option in the configuration, CLI, and file search logic. The change updates the CLI, config schemas, merging logic, file search, and related documentation in both English and Japanese. Corresponding tests and utilities were updated to account for the new option, ensuring that the feature is covered throughout the codebase and documentation.

Changes

Files/Paths Change Summary
src/cli/cliRun.ts, src/cli/types.ts, src/cli/actions/defaultAction.ts Added --force-include CLI option, updated CLI options interface, and extended CLI config builder to process the new option.
src/config/configSchema.ts, src/config/configLoad.ts Added forceInclude to config schemas and merging logic, ensuring it is handled like include patterns.
src/core/file/fileSearch.ts Updated file search to process forceInclude patterns, ensuring specified files are included even if ignored elsewhere. Adjusted logging and file combination logic accordingly.
README.md, website/client/src/en/guide/command-line-options.md, website/client/src/en/guide/configuration.md, website/client/src/ja/guide/command-line-options.md, website/client/src/ja/guide/configuration.md Updated documentation (English and Japanese) to describe the new forceInclude option, its usage, configuration, and priority in ignore pattern processing.
tests/cli/actions/defaultAction.test.ts, tests/cli/cliRun.test.ts, tests/config/configSchema.test.ts, tests/core/file/fileSearch.test.ts, tests/core/treeSitter/parseFile.solidity.test.ts, tests/mcp/tools/packCodebaseTool.test.ts, tests/testing/testUtils.ts Updated test setups, mocks, and config utilities to include and validate the new forceInclude property, ensuring test coverage for the new feature.

Sequence Diagram(s)

sequenceDiagram
    participant User
    participant CLI
    participant ConfigLoader
    participant FileSearch

    User->>CLI: Run with --force-include pattern(s)
    CLI->>ConfigLoader: Build config (merge CLI/config file)
    ConfigLoader->>CLI: Return config with forceInclude
    CLI->>FileSearch: searchFiles(config)
    FileSearch->>FileSearch: Search with include/ignore patterns
    FileSearch->>FileSearch: Search with forceInclude patterns (no ignores)
    FileSearch->>CLI: Return combined file list (forceInclude overrides ignores)
Loading

Possibly related issues


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

‼️ IMPORTANT
Auto-reply has been disabled for this repository in the CodeRabbit settings. The CodeRabbit bot will not respond to your replies unless it is explicitly tagged.

  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai explain this code block.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and explain its main purpose.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

📜 Review details

Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 5d0f92d and ffef120.

📒 Files selected for processing (18)
  • README.md (1 hunks)
  • src/cli/actions/defaultAction.ts (1 hunks)
  • src/cli/cliRun.ts (1 hunks)
  • src/cli/types.ts (1 hunks)
  • src/config/configLoad.ts (1 hunks)
  • src/config/configSchema.ts (2 hunks)
  • src/core/file/fileSearch.ts (4 hunks)
  • tests/cli/actions/defaultAction.test.ts (1 hunks)
  • tests/cli/cliRun.test.ts (2 hunks)
  • tests/config/configSchema.test.ts (2 hunks)
  • tests/core/file/fileSearch.test.ts (1 hunks)
  • tests/core/treeSitter/parseFile.solidity.test.ts (1 hunks)
  • tests/mcp/tools/packCodebaseTool.test.ts (1 hunks)
  • tests/testing/testUtils.ts (1 hunks)
  • website/client/src/en/guide/command-line-options.md (2 hunks)
  • website/client/src/en/guide/configuration.md (3 hunks)
  • website/client/src/ja/guide/command-line-options.md (2 hunks)
  • website/client/src/ja/guide/configuration.md (3 hunks)
🧰 Additional context used
🧬 Code Graph Analysis (1)
tests/core/file/fileSearch.test.ts (1)
src/core/file/fileSearch.ts (1)
  • searchFiles (94-218)
🪛 markdownlint-cli2 (0.17.2)
website/client/src/en/guide/configuration.md

62-62: Lists should be surrounded by blank lines
null

(MD032, blanks-around-lists)


82-82: Files should end with a single newline character
null

(MD047, single-trailing-newline)

🔇 Additional comments (29)
src/cli/types.ts (1)

26-26: LGTM: Properly implemented new CLI option.

The forceInclude property has been correctly added to the CliOptions interface with the appropriate type (optional string) and positioned correctly within the "Filter Options" section.

README.md (1)

154-158: Documentation is clear and well-positioned.

The new section explaining the --force-include option provides a clear explanation of its purpose and includes a helpful example. The placement of this section in the README is logical, following the exclude patterns section and preceding the remote repository section.

tests/cli/cliRun.test.ts (2)

78-79: Test mock properly updated for new config property.

The forceInclude array property has been correctly added to the configuration mock object, which ensures tests reflect the updated configuration structure.


122-123: Test mock correctly updated for remote action.

The forceInclude array has been consistently added to the remote action mock configuration, maintaining parity with the default action mock.

tests/mcp/tools/packCodebaseTool.test.ts (1)

77-77: LGTM - Properly includes new forceInclude property in test configuration

The change correctly adds the new forceInclude array to the mock configuration object, which ensures test compatibility with the new feature. This is consistent with how other array properties are initialized in this mock.

website/client/src/en/guide/command-line-options.md (2)

23-23: Clear documentation of the new CLI option

The documentation for the new --force-include option is clear and concise, appropriately placed in the Filter Options section.


63-65: Helpful example demonstrates practical usage

Good job including a practical example that shows how to use the --force-include option to include specific files that would typically be ignored, such as coverage reports and test results.

tests/core/treeSitter/parseFile.solidity.test.ts (1)

28-28: LGTM - Properly includes new forceInclude property in test configuration

The change correctly adds the new forceInclude array to the default config object used in Solidity file parsing tests. This keeps the test configuration aligned with the updated schema.

src/config/configLoad.ts (1)

112-116: LGTM - Properly implements configuration merging for forceInclude

The implementation correctly merges the forceInclude arrays from all three configuration sources (base, file, and CLI), following the same pattern used for the include property. The code handles potential undefined values with appropriate fallbacks.

website/client/src/ja/guide/command-line-options.md (2)

23-23: Documentation accurately describes the new feature.

The Japanese documentation for the new --force-include <patterns> option clearly explains its purpose of overriding exclusion rules.


63-65: Good example of the new feature usage.

The provided example effectively demonstrates how to use the --force-include option with specific file patterns.

src/cli/actions/defaultAction.ts (1)

110-112: Implementation correctly handles the forceInclude option.

The code appropriately splits the comma-separated string, trims each pattern, and assigns it to the CLI configuration, consistent with how the include option is handled.

src/cli/cliRun.ts (1)

68-68: CLI option added with clear description.

The --force-include <patterns> option is correctly positioned alongside other filter options and has a clear description that explains its purpose of overriding ignore patterns.

tests/testing/testUtils.ts (1)

32-32: Test utility correctly updated to support the new feature.

The createMockConfig function now properly handles the forceInclude property, following the same pattern used for the include property, ensuring consistent testing of the new feature.

website/client/src/ja/guide/configuration.md (2)

36-36: Good addition of the forceInclude configuration example.

The example shows appropriate use cases for the force include feature with test-related files that are typically ignored but may need to be included in specific scenarios.


62-66: Clear priority ordering documentation for the new force include patterns.

Excellent update to the exclusion pattern priority documentation. Placing "force include patterns" at the highest precedence clearly communicates to users how this new feature interacts with existing ignore rules.

tests/core/file/fileSearch.test.ts (2)

301-302: Good update to the mock implementation for the new dual globby calls.

The implementation has been properly modified to handle both the initial globby call and the additional call that's made for forceInclude patterns. This ensures the test correctly simulates the behavior of the updated searchFiles function.


306-307: Appropriate change to use Set comparison for order-independent results.

Using Set comparison rather than direct array comparison is a good approach since the order of results may vary when combining files from both normal search and force-included search. This makes the test more robust against implementation details.

tests/config/configSchema.test.ts (2)

81-81: Properly added forceInclude to default config test.

The empty array default for forceInclude matches the pattern used for other similar fields and ensures the default configuration correctly validates with the updated schema.


171-171: Good test coverage for forceInclude in merged config.

Including multiple file patterns in the test ensures that the merged configuration correctly handles various pattern types for the forceInclude option.

src/config/configSchema.ts (2)

42-42: Well-placed addition of forceInclude to the base schema.

The forceInclude property is correctly defined as an optional array of strings, consistent with the include property definition above it. This maintains the schema's structure and type safety.


89-89: Appropriate default value for forceInclude in the default schema.

Setting an empty array as the default value for forceInclude is consistent with how the include property is handled, ensuring that the property always exists with a sensible default.

website/client/src/en/guide/configuration.md (1)

36-36: LGTM: Clear example of the new forceInclude option.

The addition of the forceInclude field with practical examples like coverage reports and test results is helpful and clearly demonstrates the intended use case for this feature.

src/core/file/fileSearch.ts (6)

122-122: LGTM: Good addition of logging for the new feature.

Adding trace logging for force include patterns is consistent with the existing logging approach for include and ignore patterns.


139-141: LGTM: Clear step labeling.

Adding numbered steps in the comments makes the multi-phase filtering approach clear and easier to follow.


159-176: LGTM: Well-implemented force include logic.

The implementation correctly handles force include patterns by:

  1. Escaping special characters in patterns
  2. Running a separate globby search without ignore filters
  3. Adding appropriate logging

This effectively ensures that force include patterns override all ignore rules as intended.


178-180: LGTM: Efficient deduplication approach.

Using a Set to combine and deduplicate file paths is an efficient approach that avoids returning duplicate entries when a file matches both normal include patterns and force include patterns.


196-198: LGTM: Enhanced logging with force include statistics.

The updated log message provides useful information about how many files were included via the force include mechanism, which will help with debugging.


201-201: LGTM: Proper integration with existing sorting logic.

Using the sortPaths function on the combined paths ensures consistent sorting regardless of how files were included.

@ktyubeshi ktyubeshi force-pushed the feature/forceinclude branch from ffef120 to 68be8a0 Compare April 19, 2025 11:35
@yamadashy
Copy link
Owner

yamadashy commented Apr 19, 2025

Hi, @ktyubeshi !
Thanks so much for the PR!

I assume this is intended to address a similar use case as mentioned in this issue:

I totally understand the motivation here — and I really appreciate the effort you've put into the implementation.
That said, as I mentioned in the issue comment, I'm still a bit hesitant about introducing a --force-include option at this point. My current thinking is to support these scenarios through a useDefaultPatterns: false option instead.

Also, the use of the word "force" might create long-term challenges from a design perspective. For example, if users want to "force include" a set of files but then also ignore some of them, it becomes difficult to reason about what should happen.

I’d love to explore a more flexible and conceptually cleaner solution before locking in this approach. Thanks again for opening the PR!

@ktyubeshi
Copy link
Author

Hi, @yamadashy

Thank you for your feedback.
I wasn't aware of #438 when implementing this, but I think our use cases and ideas are essentially identical.

Also, the use of the word "force" might create long-term challenges from a design perspective.

Regarding the choice of the word "force," my intention was to use a strong expression to clearly indicate that this would be the highest priority.
When adding this feature, I was also concerned that carelessly adding options might cause confusion, and I initially tried to find a solution using only the existing ignore settings. However, I added force include because the configuration would become too complicated, and this could be achieved with a simple setting.
I understand the lingering concerns about whether this is truly the best choice. I agree that careful consideration is needed.

For example, if users want to "force include" a set of files but then also ignore some of them, it becomes difficult to reason about what should happen.

Regarding this, while an option like --force-include-ignore could be considered, I thought it might be acceptable to simply decide against adding such functionality.

In my actual use cases, the targets I would specify with --force-include are things like:

  • Test results
  • Error logs
  • Static analysis tool results
    These files are mostly either single files or exist together in specific folders, so I don't think there's a need for complex specifications that would filter them further with ignore rules.

My current thinking is to support these scenarios through a useDefaultPatterns: false option instead.

The reason I added this option was that I wanted to keep repomix's default excluded files as they are.
If we set useDefaultPatterns to false as you suggested, I think we would need to manually reconfigure everything.
I'd be happy if there's a good way to achieve the same thing with minimal configuration.

@ktyubeshi ktyubeshi force-pushed the feature/forceinclude branch from 68be8a0 to d7af036 Compare April 26, 2025 10:46
@ktyubeshi ktyubeshi force-pushed the feature/forceinclude branch 2 times, most recently from 1cc0ca4 to 0d07b04 Compare May 19, 2025 14:43
@ktyubeshi ktyubeshi force-pushed the feature/forceinclude branch from 0d07b04 to 3c29415 Compare May 29, 2025 02:32
@ktyubeshi ktyubeshi force-pushed the feature/forceinclude branch from 3c29415 to aa473b0 Compare June 6, 2025 00:19
@ktyubeshi ktyubeshi force-pushed the feature/forceinclude branch from aa473b0 to c53743f Compare June 8, 2025 07:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants