add tool to synchronize add-on repository with latest addonTemplate

[abdel792]

Link to issue number:

Replaces #41.

Summary of the issue:

While upgrading an existing add-on using standard Git commands remains fully supported, some add-on developers prefer an automated and dedicated tool to streamline the process.
Manual step-by-step migration of metadata (from buildVars.py to pyproject.toml) and file merges can require extra care and verification.
Providing an automated script offers a convenient alternative for developers who wish to perform these updates quickly and with built-in AST-based metadata handling.

Description of developer facing changes:

• Introduces the syncAddonTool package at the repository root as an automated alternative to Git-based manual upgrades for add-on developers.
• Requires tomlkit for parsing and merging pyproject.toml configurations (can be run without prior installation via uv run --no-project --with tomlkit).
• Includes a PyInstaller specification file (syncAddonTool/syncAddonTool.spec) to build a standalone executable (syncAddonTool.exe).
• Protects template-specific files, including .github/workflows/tests.yml and tests/, ensuring they are excluded from add-on synchronization.
• Excludes both syncAddonTool and the tests/ directory from static analysis in pyproject.toml (ruff and pyright).
• Adds a comprehensive verification infrastructure including both a unit test suite (tests/unit/template/test_syncAddonTool.py) and an End-to-End (E2E) CLI integration test suite (tests/e2e/template/test_syncAddonTool.py).
• Adds docs/testing.md to provide unified execution guidelines for both unit and End-to-End (E2E) tests.

Description of development approach:

• Synchronization Engine Creation: Implemented syncAddonTool using an AST-aware approach and tomlkit to seamlessly migrate legacy buildVars.py metadata or modern AddonInfo setups into pyproject.toml.
  It supports timestamped backups, custom exclusions via .addonmergeignore, and includes .github/workflows/tests.yml in PROTECTED_ELEMENTS.
• Standalone Binary Support: Added syncAddonTool/syncAddonTool.spec for building a standalone executable using uv run --with pyinstaller pyinstaller syncAddonTool/syncAddonTool.spec, eliminating external Python environment requirements for end users.
• Linter Exclusion Rules: Configured pyproject.toml to exclude syncAddonTool and tests/ from ruff and pyright checks.
• Verification & E2E Suite: Built a dual test infrastructure featuring unit tests (tests/unit/template/test_syncAddonTool.py) for internal AST/synchronization logic, and an E2E test suite (tests/e2e/template/test_syncAddonTool.py) executing isolated subprocess workflows, CLI flags, buildVars migrations, and PyInstaller binary packaging.
• Documentation Integration: Added docs/testing.md to detail CLI usage, available options, and instructions for executing both unit and E2E test suites.

Testing strategy:

Validated the synchronization script and unit test suite locally using uv:

1. Executed syncAddonTool across various CLI modes:

   • Standard Mode (inside add-on repo, -ad is optional):

     uv run --no-project --with tomlkit python syncAddonTool

   • External Directory Mode:

     uv run --no-project --with tomlkit python -m syncAddonTool -ad /path/to/my-nvda-addon

   • Dry Run Simulation:

     uv run --no-project --with tomlkit python syncAddonTool --dry-run

2. Validated Standalone Executable Generation via PyInstaller:

   uv run --with pyinstaller pyinstaller syncAddonTool/syncAddonTool.spec

   Executed dist/syncAddonTool.exe -ad /path/to/my-nvda-addon --dry-run and confirmed expected behavior.

3. Verified that backup folders (<addon>_bak_<timestamp>) were generated, .addonmergeignore rules were respected, and protected elements (including .github/workflows/tests.yml and tests/) were not pushed to target add-on repositories.

4. Executed the complete verification test suite (both unit and E2E tests) via unittest:

   uv run python -m unittest discover -s tests -v

   Confirmed all tests (10 unit tests and 10 E2E tests) passed successfully.

5. Confirmed ruff and pyright ignore syncAddonTool and tests/.

Known issues with pull request:

None.

---

Code Review Checklist

• Testing: Tested manually across diverse execution paths and validated via local unit tests.
  Steps to reproduce are detailed in the testing strategy.
• Compatibility: Developer-facing infrastructure tool; introduces no breaking changes to existing repository behaviors.
• Documentation: Technical developer documentation has been explicitly updated.
• UX of all users considered: Not applicable (Developer-facing infrastructure tool only).
• Security precautions taken: Not applicable (Runs locally in isolated temporary directories during development).

[Copilot]

Pull request overview

Adds a developer-facing synchronization tool to help NVDA add-on authors update their repositories to the latest AddonTemplate structure, including AST-aware metadata handling and TOML merging, with accompanying unit tests and documentation updates.

Changes:

• Introduces syncAddonWithTemplate.py to sync template infrastructure while merging buildVars.py and pyproject.toml.
• Adds unit tests for key merge/formatting behavior and a small template sanity suite.
• Updates dependency groups/docs to support running the sync tool and the unit test suite.

Reviewed changes

Copilot reviewed 8 out of 10 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
syncAddonWithTemplate.py	New sync/merge engine (AST + tomlkit) for updating add-on repos from the template.
tests/unit/test_syncAddonWithTemplate.py	Unit tests for metadata extraction, buildVars merge, TOML indentation, and dependency merging.
tests/unit/template/sanity.py	Minimal sanity tests intended to validate CI test discovery/execution.
docs/managementFromGit/updatingExistingAddons.md	Expanded guidance for updating add-ons (automated tool + manual merge).
docs/unitTesting.md	Updated instructions for running the unit tests and describing the new suites.
pyproject.toml	Adds tomlkit, updates pyright, and excludes the new sync script/tests from ruff/pyright.
uv.lock	Locks tomlkit and bumps pyright to match pyproject.toml.
tests/__init__.py	Updates package docstring.
tests/unit/__init__.py	Updates package docstring.
.addonmergeignore	Present at repo root (empty in this PR context).

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[abdel792]

Hi @seanbudd,
Thanks for your feedback! I have updated the branch accordingly:

• Docstrings adjustment: I moved the test infrastructure docstring from root unit tests to the template submodule and restored the root tests/__init__.py docstring to reflect all test types (system and unit).
• Sanity test removal: I removed the obsolete sanity.py test file.

[abdel792]

@Copilot,
Here are the updates I made regarding your remarks on normalization and .addonmergeignore:

• Package Name Normalization: I replaced underscores with hyphens in getBasePackageName and legacyToolingBases to consistently handle equivalent Python package name formats during dependency merges.
• Dynamic Bootstrapping & Self-Exclusion: Instead of adding .addonmergeignore to protectedElements, I updated syncAddonWithTemplate.setupAddonMergeIgnore to bootstrap .addonmergeignore from the template on the first sync if it is absent locally, ensuring it includes its own self-exclusion. I prefer this approach as it allows the file to self-reference and manage its own persistence dynamically.

[abdel792]

@Copilot,
Regarding your comment on the load_tests implementation in test_syncAddonWithTemplate.py:

• Robust Test Sorting: I updated load_tests to use an orderIndex lookup dictionary with orderIndex.get(a, defaultOrder) instead of methodOrder.index().
• Error Prevention: I added this to prevent potential ValueError exceptions if inherited or dynamic test methods are present during test execution.

[abdel792]

@Copilot,
Regarding your feedback on the documentation:

• Unit Test Path Fix: I fixed the incorrect path for test_syncAddonWithTemplate.py in docs/unitTesting.md.
• CLI Usage & Dependency Clarification: I clarified the tomlkit installation requirement and corrected the no-argument command example in docs/managementFromGit/updatingExistingAddons.md.

[abdel792]

Hi @seanbudd, @Copilot,

Regarding the test coverage for .addonmergeignore:

• Unit Tests: I added testSetupAddonMergeIgnore and testAddonMergeIgnore to test_syncAddonWithTemplate.py to verify both the creation/bootstrapping behavior when missing and the actual file exclusion during synchronization.
• Documentation: I documented both new test methods in docs/unitTesting.md.

[abdel792]

Hi @seanbudd,

When running pyright on version 1.1.407 via uv (e.g., uv run pyright addon), the underlying Python wrapper package (pyright-python) checks for upstream NPM release updates and systematically outputs a warning to stderr before executing the type checker:

WARNING: there is a new pyright version available (v1.1.407 -> v1.1.411).
Please install the new version or set PYRIGHT_PYTHON_FORCE_VERSION to `latest`

I bumped the lock version to 1.1.411 in pyproject.toml to eliminate this noise during local check runs and developer workflows.

Let me know if you would still prefer me to revert it to keep the PR scope strictly minimal!

[abdel792]

Thanks for the review and all these helpful suggestions, @seanbudd!

I have addressed the quick fixes and documentation updates:

• Restored the exact casing for AddonTemplate in docs/managementFromGit/updatingExistingAddons.md.
• Removed the detailed test suite section from docs/unitTesting.md (relying on test docstrings instead).
• Fixed the missing newline at the end of test_syncAddonWithTemplate.py.

Regarding your suggestion to split syncAddonWithTemplate.py into submodules: I completely agree, it's a great idea that will make the code much cleaner and easier to maintain!

I'm currently refactoring the code into submodules, writing the requested additional test cases, and running all necessary tests. I will address all of your remaining feedback and suggestions throughout this process and post a full update over the weekend.

I'll leave the PR in draft mode until everything is ready for review.

[abdel792]

Sean, regarding your comment on legacyToolingBases:

I'm currently implementing a solution for this and will push it in my next commit over the weekend, as I need to finish up some work commitments first.

The previous legacyToolingBases set was indeed handling this, but as you pointed out, maintaining such a hardcoded list long-term is cumbersome.

Replacing it with a dynamic lookup alongside a minimal mapping dictionary for non-supported/replaced dependencies (like pre-commit) makes maintenance much simpler.

The updated logic now dynamically collects all tooling packages directly from the template's dependencies and dependency-groups in pyproject.toml.

Additionally, for deprecated tools that were replaced in the template (such as pre-commit being replaced by prek), I introduced a lightweight dictionary (REPLACED_PACKAGES = {"pre-commit": "prek"}).

This maps replaced tools to their modern template equivalents so they aren't accidentally preserved as custom dependencies.

Here is a snippet of how the dynamic collection works:

# Dynamically collect all template packages (dependencies + dependency-groups)
templateBases: set[str] = {
	getBasePackageName(d) for d in tplDeps if isinstance(d, str)
}
if "dependency-groups" in mergedData and isinstance(mergedData["dependency-groups"], MutableMapping):
	for grp in mergedData["dependency-groups"].values():
		if isinstance(grp, (list, MutableSequence)):
			for grpItem in grp:
				if isinstance(grpItem, str):
					templateBases.add(getBasePackageName(grpItem))

[nvdaes]

Hi @abdel792 . Thanks for considering the suggestion about end to end tests. I think that it will be better to have a PR with end to end test, merge it, and then merge this one, so that we can ensure that test past.
I have an open ADR in NVDA supporting DAISY Consortium to try to add support for dpub roles to NVDA, and Sean suggested to start with tests, and then follow with other pull requests related to code, so I think that this is a good approach

For reference, see

nvaccess/nvda#20877 (comment)

[abdel792]

Hi @nvdaes,

Thanks for the clarification and for sharing the reference.

Following the same approach as my previous Pull Request that introduced the minimal unit test structure under tests/, I completely agree with your suggestion.

I will keep this PR in Draft status, create a separate branch, and submit a dedicated Pull Request for the E2E test infrastructure first.

Once that E2E test PR is merged into master, I will rebase and resume this current PR to have everything merged cleanly in the correct order.

Thanks again for your guidance and support!

[abdel792]

Hi @nvdaes,

Following up on your feedback regarding establishing the testing infrastructure first, I have submitted a dedicated Pull Request to set up the E2E foundation:

👉 PR #59

Summary of changes included in this PR:

• E2E Infrastructure: Added the tests/e2e/ directory structure with an __init__.py file to enable seamless test discovery via unittest.
• Sanity Suite: Added tests/e2e/test_sanity.py featuring both a passing test (verifying temporary directory handling) and an @unittest.expectedFailure test to validate runner behavior.
• Documentation: Consolidated all testing documentation into a single, unified guide in docs/testing.md.
• CI Workflow: Updated the workflow name and job labels to accurately reflect the execution of the full test suite (Unit & E2E).

Once this foundational PR is reviewed and merged into master, I will rebase the syncAddonTool branch and add the specific integration tests for the synchronization tool.

Thanks again for your time and review!

[abdel792]

Hi @nvdaes,

I have pushed the final updates incorporating your feedback, completing the E2E testing infrastructure, and updating the PR description:

• Documentation Update: Replaced "docs/unitTesting.md" with a unified "docs/testing.md" that comprehensively covers execution guidelines for both unit tests and end-to-end (E2E) integration tests.
• E2E Infrastructure & Tests: Added the E2E testing structure under "tests/e2e/template/", containing 10 E2E test cases covering core CLI workflows, flags, options, and PyInstaller standalone packaging.
• Test Migration: Removed the 2 unit tests handling legacy "buildVars.py" dictionary structures and modern "buildVars.py" missing "speechDictionaries" injections, as these have been successfully migrated and consolidated into the E2E test suite (leaving 10 unit tests).
• PR Description: Updated the pull request description to fully reflect the dual test infrastructure and documentation strategy.

The test suite structure now mirrors the unit tests perfectly, with a total of 20 tests (10 E2E and 10 unit tests) passing successfully!

Ready for final review.

[abdel792]

Hi @seanbudd and @nvdaes,

Following a test failure in the CI pipeline regarding the PyInstaller executable build on Linux, I have updated the GitHub Actions workflow (".github/workflows/tests.yml") to run on "windows-latest".

This allows the full test suite—including the PyInstaller standalone packaging and E2E CLI integration tests—to execute successfully in a native Windows environment where building .exe binaries is fully supported.

All 20 tests are now passing successfully!

[Copilot]

🟡 Changes recommended

Unresolved migration and exclusion defects can corrupt or overwrite configuration, and cleanup steps can delete developer files.

12 open findings

• Escape control characters in generated double-quoted literals · New
• Handle annotated addon_info assignments · New
• Preserve or reject nonliteral metadata expressions · New
• Preserve imports and definitions for global expressions · New
• Honor exclusions during configuration merges · New
• Restrict package matching to dependency arrays · New
• Remove existing static versions from merged dynamic metadata · New
• Preserve i18nSources during build variable extraction · New
• Match directory exclusions with trailing separators · New
• Populate repository metadata before merging template defaults · New
• Use temporary paths for packaging tests · New
• Avoid deleting untracked files during synchronization · New

9 resolved since last review

• Report synchronization failures with nonzero exit status
• Abort synchronization on syntax errors before modifying project files
• Preserve or reject non-constant legacy dictionary values
• Avoid altering TOML multiline string contents during substitution
• Use standards-aware requirement parsing and merging
• Map addon_sourceURL to the repository URL
• Implement documented fnmatch and negation ignore patterns
• Protect existing .addonmergeignore files from template overwrites
• Avoid requiring tomlkit when it is unavailable or unnecessary

🧠 Review effort: Balanced

[nvdaes]

@abdel792, there are comments from Copilot. You may want to review them, and I'll review the final result if you need my feedback.
Thanks for this amazing work!

[abdel792]

Hi @seanbudd and @nvdaes,

I have reviewed and processed all 12 Copilot review findings for this pull request.

Each of the 11 code-based corrections has been implemented via a precise, dedicated commit.

• Fixed the escaping of control characters in generated double-quoted literals.
• Handled annotated "addon_info" assignments properly.
• Preserved or rejected non-literal metadata expressions.
• Preserved imports and definitions for global expressions.
• Honored exclusions during configuration merges.
• Restricted package matching to dependency arrays.
• Removed existing static versions from merged dynamic metadata.
• Preserved "i18nSources" during build variable extraction.
• Matched directory exclusions using trailing separators.
• Populated repository metadata before merging template defaults.
• Used temporary paths for packaging tests.
• Addressed the suggestion concerning "git clean -f" during synchronization.

This finding was resolved with a reason.

The documentation referencing "git clean -f" pre-exists on the master branch and is out of scope for the current pull request.

This update must be handled in a separate, dedicated PR.

What are your thoughts on this?

Thanks @nvdaes for offering your help!

Feel free to run some tests on your side whenever you like.

[nvdaes]

Hi @abdel792 , I've performed the test suite, and all test pass! Thanks for this impressive work.

[abdel792]

Hi @seanbudd and @nvdaes,

@seanbudd You are completely right, and I apologize for initially marking this as "won't fix" by mistake.

This section is indeed part of the documentation for syncAddonTool.

Using "git clean -f" can accidentally delete newly generated untracked files (such as a new "pyproject.toml" in legacy add-ons) before they are staged.

I have just pushed a commit to fix this and update the documentation to use "git status --short" instead.

@nvdaes Thank you so much for offering your help, your testing, and your encouragement!

I also wanted to take a moment to express my sincere gratitude to you: you are the one who introduced me to creating NVDA add-ons in the first place, and I am still deeply grateful to you for that.

It is a true honor for me to contribute to the NVDA project with my modest contributions, and I am really proud to be part of this wonderful community!