Skip to content

feat(collections): opt into hidden directories - #1097

Open
lanej wants to merge 3 commits into
tobi:mainfrom
lanej:fix/collection-dot-directories
Open

lanej wants to merge 3 commits into
tobi:mainfrom
lanej:fix/collection-dot-directories

Conversation

@lanej

@lanej lanej commented Oct 29, 2026

Copy link
Copy Markdown

Allow each collection to opt into hidden instruction directories such as .claude/ and .agents/, so their documents become searchable while other hidden directories remain excluded. Configure names through qmd collection include-dot-dirs or the SDK's includeDotDirs; the setting persists in SQLite for SDK updates and DB-only reopen.

Run qmd update after enabling or clearing the setting. Collection-relative masks, ignore patterns, built-in exclusions, and hidden-file exclusion continue to apply.

Verification gap: Local Bun runs retain pre-existing SQLite heap-budget failures reproduced on unmodified upstream.

lanej and others added 2 commits October 29, 2026 09:17
reindexCollection (store.ts, used by `qmd update`) and indexFiles
(cli/qmd.ts, used by `qmd index`) both glob with fast-glob's `dot: false`
and then apply a second filter that drops any path containing a segment
starting with ".". This is correct for `.git`/`.venv`/`.cache`-style
directories, which can be huge and should never be walked, but it also
silently drops directories like `.claude/` that hold real, intentionally
committed project documentation (e.g. CLAUDE.md) meant to be searchable.

Rather than a compiled-in allow-list, this follows the same architecture
already used for `update` and `includeByDefault`: a per-collection YAML
field (`~/.config/qmd/index.yml`), settable via a `qmd collection`
subcommand backed by `updateCollectionSettings()`, read back at index
time via `getCollectionFromYaml(...)` and threaded through the same
options bag `ignorePatterns` already uses.

  qmd collection include-dot-dirs <name> [dir ...]   # e.g. `claude`
  qmd collection include-dot-dirs <name>              # clear it

New `Collection.includeDotDirs?: string[]` field (collections.ts).
Defaults to unset/empty for every collection — hidden directories stay
excluded exactly as before unless a collection explicitly opts in.
`qmd collection show`/`list` now display it next to `Ignore`.

`globCollectionFiles()` (store.ts, shared by both `reindexCollection`
and `indexFiles`) takes `includeDotDirs` as a parameter instead of
reading a hardcoded module constant. The main crawl (`dot: false`)
is unchanged — `.git`/`.venv`/`.cache` exclusion and crawl performance
are unaffected. For each name in `includeDotDirs` it runs one additional
narrowly-scoped glob using an explicit `.<name>` path segment — explicit
dot segments in a glob pattern always match regardless of the `dot`
option, so this doesn't touch the main crawl's traversal behavior at
all. The post-filter still drops any other hidden path, so only the
collection's own configured names are ever included.

Regression tests in test/store.test.ts cover both sides of the config:
a collection with `includeDotDirs: ["claude"]` set (via the same YAML
path `qmd update` reads) indexes `.claude/` at the root and nested,
while `.venv/`/`.git/` stay excluded; a collection with no
`includeDotDirs` configured — the default for every existing
collection — still excludes `.claude/` entirely.
Preserve original masks, validate literal directory names, and retain per-collection opt-in through SDK updates and SQLite reopen. Document opt-in for CLI and MCP clients.

Detected-By: V3b
Keep addCollection limited to its supported fields when a JavaScript caller passes a wider object.

Detected-By: V3b
@lanej

lanej commented Oct 29, 2026

Copy link
Copy Markdown
Author

note(non-blocking): Copilot recommends approval of the final revision.

Copilot reviewed a01ec7f with identical base and head commits in a fork companion because reviewer-request permissions blocked the request here. It reports no open issues. This PR remains a draft for human approval.

Upstream CI and Nix require maintainer workflow approval. Local Bun retains two SQLite heap-budget failures reproduced on unmodified upstream.

@lanej
lanej marked this pull request as ready for review October 29, 2026 20:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant