Skip to content

feat(collections): opt into hidden directories - #1

Closed
lanej wants to merge 3 commits into
review/upstream-main-86d2365from
fix/collection-dot-directories
Closed

lanej wants to merge 3 commits into
review/upstream-main-86d2365from
fix/collection-dot-directories

Conversation

@lanej

@lanej lanej commented Oct 29, 2026

Copy link
Copy Markdown
Owner

Allow collections to opt into hidden instruction directories such as .claude/ and .agents/, retaining collection masks, ignore patterns, and hidden-file exclusion. Configure names through qmd collection include-dot-dirs or the SDK's includeDotDirs; the setting persists in SQLite. Run qmd update after enabling or clearing it.

This temporary draft mirrors tobi#1097 so the fork can request GitHub Copilot review. Review fixes will be applied to both drafts; this companion will be closed without merging.

lanej and others added 2 commits October 29, 2026 09:17
reindexCollection (store.ts, used by `qmd update`) and indexFiles
(cli/qmd.ts, used by `qmd index`) both glob with fast-glob's `dot: false`
and then apply a second filter that drops any path containing a segment
starting with ".". This is correct for `.git`/`.venv`/`.cache`-style
directories, which can be huge and should never be walked, but it also
silently drops directories like `.claude/` that hold real, intentionally
committed project documentation (e.g. CLAUDE.md) meant to be searchable.

Rather than a compiled-in allow-list, this follows the same architecture
already used for `update` and `includeByDefault`: a per-collection YAML
field (`~/.config/qmd/index.yml`), settable via a `qmd collection`
subcommand backed by `updateCollectionSettings()`, read back at index
time via `getCollectionFromYaml(...)` and threaded through the same
options bag `ignorePatterns` already uses.

  qmd collection include-dot-dirs <name> [dir ...]   # e.g. `claude`
  qmd collection include-dot-dirs <name>              # clear it

New `Collection.includeDotDirs?: string[]` field (collections.ts).
Defaults to unset/empty for every collection — hidden directories stay
excluded exactly as before unless a collection explicitly opts in.
`qmd collection show`/`list` now display it next to `Ignore`.

`globCollectionFiles()` (store.ts, shared by both `reindexCollection`
and `indexFiles`) takes `includeDotDirs` as a parameter instead of
reading a hardcoded module constant. The main crawl (`dot: false`)
is unchanged — `.git`/`.venv`/`.cache` exclusion and crawl performance
are unaffected. For each name in `includeDotDirs` it runs one additional
narrowly-scoped glob using an explicit `.<name>` path segment — explicit
dot segments in a glob pattern always match regardless of the `dot`
option, so this doesn't touch the main crawl's traversal behavior at
all. The post-filter still drops any other hidden path, so only the
collection's own configured names are ever included.

Regression tests in test/store.test.ts cover both sides of the config:
a collection with `includeDotDirs: ["claude"]` set (via the same YAML
path `qmd update` reads) indexes `.claude/` at the root and nested,
while `.venv/`/`.git/` stay excluded; a collection with no
`includeDotDirs` configured — the default for every existing
collection — still excludes `.claude/` entirely.
Preserve original masks, validate literal directory names, and retain per-collection opt-in through SDK updates and SQLite reopen. Document opt-in for CLI and MCP clients.

Detected-By: V3b

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues block the per-collection hidden-directory indexing change.

No open issues 🎉

What changed in this PR

Adds per-collection opt-in indexing for hidden directories such as .claude/ and .agents/, extending QMD’s CLI and SDK while preserving masks, ignore patterns, and hidden-file exclusion.

Changes:

  • Adds validated includeDotDirs configuration, SQLite persistence, and migration support.
  • Shares hidden-directory filtering across indexing paths and exposes CLI configuration commands.
  • Adds regression coverage and documents enabling, clearing, and applying the setting with qmd update.
File Description
test/​store.test.ts Tests hidden-directory filtering, masks, exclusions, and reindexing.
test/​sdk.test.ts Tests configuration modes, persistence, migration, validation, and clearing.
test/​mcp.test.ts Updates the test schema and checks opt-in instructions.
test/​cli.test.ts Tests CLI configuration, validation, display, and indexing behavior.
src/​store.ts Persists settings and centralizes directory filtering.
src/​mcp/​server.ts Adds user-authorized opt-in guidance to server instructions.
src/​index.ts Exposes SDK configuration and passes it into updates.
src/​collections.ts Defines, validates, and saves hidden-directory settings.
src/​cli/​qmd.ts Adds configuration commands, display output, and indexing integration.
skills/​qmd/​SKILL.md Documents authorized opt-in and clearing procedures.
README.md Documents CLI, SDK, and YAML usage.
CLAUDE.md Adds collection-management command examples.
CHANGELOG.md Records the feature under unreleased additions.

Review effort: Balanced

Keep addCollection limited to its supported fields when a JavaScript caller passes a wider object.

Detected-By: V3b

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues block approval of the hidden-directory indexing opt-in change.

No open issues 🎉

Review effort: Balanced

@lanej

lanej commented Oct 29, 2026

Copy link
Copy Markdown
Owner Author

note(non-blocking): Acknowledged Copilot’s approval recommendation.

Copilot reports no open issues on a01ec7f. Closing this review-only companion without merging; upstream #1097 remains a draft for human approval.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants