Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
59 commits
Select commit Hold shift + click to select a range
574bd73
feat(infra): add syncAddonWithTemplate.py script and unit test suite
abdel792 Aug 4, 2026
3553ac3
test(sync): update testFormatAuthorList for PEP 621 compliance
abdel792 Aug 4, 2026
bc4816d
docs(unitTesting): clarify testFormatAuthorList email omission in doc…
abdel792 Aug 4, 2026
241ee9a
test(infra): adjust test package docstrings and remove redundant sani…
abdel792 Aug 5, 2026
81b9779
fix(sync): normalize package names and bootstrap `.addonmergeignore`
abdel792 Aug 5, 2026
f64404c
test(sync): make `load_tests` sorting robust against inherited methods
abdel792 Aug 5, 2026
a8ef694
docs(sync): fix unit test path and CLI usage examples
abdel792 Aug 5, 2026
87372e5
test: add unit tests and documentation for .addonmergeignore
abdel792 Aug 5, 2026
4cbab17
tests(template): add missing trailing newline to test_syncAddonWithTe…
abdel792 Aug 6, 2026
57c1819
docs: restore AddonTemplate URL casing and remove test suite details
abdel792 Aug 6, 2026
f8cd032
refactor(syncAddonTool): modularize architecture and improve unit tests
abdel792 Aug 9, 2026
e073bd9
docs(managementFromGit): update documentation for syncAddonTool and s…
abdel792 Aug 9, 2026
723d869
build(lint): exclude syncAddonTool folder and syncAddonTool.spec file…
abdel792 Aug 9, 2026
9a4afbe
build: ignore PyInstaller output directories
abdel792 Aug 9, 2026
c48885b
fix(sync): remove extra empty lines in generated TOML files
abdel792 Aug 9, 2026
d6a6050
style(sync): add trailing empty line to some files
abdel792 Aug 9, 2026
8e3c1f4
- Remove obsolete `syncAddonWithTemplate.py` top-level script.
abdel792 Aug 11, 2026
7c7b92e
Update docs/managementFromGit/updatingExistingAddons.md to document
abdel792 Aug 11, 2026
9701532
docs: remove invalid -m script syntax in syncAddonTool usage examples
abdel792 Aug 11, 2026
9abf590
docs(cli): document -v/--verbose flag and refine debug logging
abdel792 Aug 12, 2026
8f8694b
fix(logging): include full TOML section path in list merge debug logs
abdel792 Aug 12, 2026
4319d23
Merge branch 'master' into syncAddon
abdel792 Aug 13, 2026
557d209
Merge branch 'master' into syncAddon
abdel792 Aug 25, 2026
bfea2ef
docs: fix formatting and outdated descriptions in updatingExistingAdd…
abdel792 Sep 19, 2026
70b65c1
docs: update unit testing documentation for syncAddonTool
abdel792 Sep 19, 2026
14a8a64
build: remove unused pathspec hidden import from PyInstaller spec
abdel792 Sep 19, 2026
5e530b7
build: revert pyright version update in pyproject.toml
abdel792 Sep 19, 2026
df66e27
Merge branch 'master' into syncAddon
abdel792 Sep 19, 2026
fe32093
Merge branch 'master' into syncAddon
abdel792 Oct 1, 2026
3887d58
fix(astUtils): preserve non-constant unparsed AST expressions in dict…
abdel792 Oct 1, 2026
495969a
fix(buildVarsSync): re-raise SyntaxError on corrupted buildVars.py pa…
abdel792 Oct 1, 2026
cf1dff5
fix(engine): enforce implicit .addonmergeignore protection and fnmatc…
abdel792 Oct 1, 2026
c13f4db
fix(pyproject): improve URL mapping, dependency version parsing, and …
abdel792 Oct 1, 2026
268001f
fix(cli): ensure nonzero exit status on execution or backup failures
abdel792 Oct 1, 2026
e3a42c7
docs(syncAddonTool): explain --no-project flag utility for running ou…
abdel792 Oct 1, 2026
8dc4795
buildVarsSync: preserve quote style during merge and add unit test
abdel792 Oct 1, 2026
c96c565
Merge branch 'master' into syncAddon
abdel792 Oct 9, 2026
27e428a
Docs(updatingExistingAddons.md): replace AddonTemplate with camelCase…
abdel792 Oct 9, 2026
a571d30
Docs(updatingExistingAddons.md): capitalize Chore in example commit m…
abdel792 Oct 9, 2026
1666570
Docs(updatingExistingAddons.md): clarify executable usage instructions
abdel792 Oct 9, 2026
5daf1d7
Docs(updatingExistingAddons.md): replace GitHub callout and clean for…
abdel792 Oct 9, 2026
2ed5002
Docs(unitTesting.md): fix code block language syntax spaces
abdel792 Oct 9, 2026
abc1b28
Style(__main__.py): convert space indentation to tabs
abdel792 Oct 9, 2026
2003150
Merge branch 'master' into syncAddon
abdel792 Oct 14, 2026
0ea3b28
test(e2e): add E2E testing infrastructure, migrate buildVars tests, a…
abdel792 Oct 14, 2026
8dbbe52
ci(workflow): switch test runner from ubuntu-latest to windows-latest
abdel792 Oct 14, 2026
c21773b
fix(buildVarsSync): escape control characters in double-quoted literals
abdel792 Oct 15, 2026
3bdaed6
fix(buildVarsSync): handle annotated addon_info assignments
abdel792 Oct 15, 2026
caff4a7
fix(buildVarsSync): preserve non-literal metadata expressions
abdel792 Oct 15, 2026
d7d0830
fix(buildVarsSync): preserve required imports for global expressions
abdel792 Oct 15, 2026
44107a3
fix(engine): honor custom exclusions during configuration merges
abdel792 Oct 16, 2026
e83a9e2
fix(pyproject): restrict requirement matching to dependency arrays
abdel792 Oct 16, 2026
23053bf
fix(pyproject): purge static fields from merged dynamic metadata
abdel792 Oct 16, 2026
03e6772
fix(buildVarsSync): preserve i18nSources during build variable extrac…
abdel792 Oct 16, 2026
f0586d7
fix(engine): match directory exclusions with trailing separators
abdel792 Oct 16, 2026
5afd6a7
fix(pyproject): populate repository metadata before merging template …
abdel792 Oct 16, 2026
fc1e8f4
test(e2e): use temporary paths for PyInstaller packaging test
abdel792 Oct 16, 2026
e216281
Merge branch 'master' into syncAddon
abdel792 Oct 16, 2026
68c092e
docs(updating-addons): replace git clean -f with safer staging command
abdel792 Oct 17, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .addonmergeignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# Files and directories ignored during template synchronization
.addonmergeignore
2 changes: 1 addition & 1 deletion .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ on:
jobs:
run-tests:
name: Run Test Suite (Unit & E2E)
runs-on: ubuntu-latest
runs-on: windows-latest

steps:
- name: Checkout repository
Expand Down
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,5 @@ manifest.ini
.sconsign.dblite
/[0-9]*.[0-9]*.[0-9]*.json
*.egg-info
dist/
build/
279 changes: 275 additions & 4 deletions docs/managementFromGit/updatingExistingAddons.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,10 +42,10 @@ As AddonTemplate evolves, it receives improvements, bug fixes, new GitHub workfl

You can merge the latest template changes into your repository instead of manually copying updated files.

This document explains the recommended update procedure.
*This document explains the update procedures, including both the recommended automated method using `syncAddonTool` and the manual Git merge workflow.*

> [!NOTE]
> Updating from AddonTemplate only affects your project's infrastructure (build scripts, GitHub workflows, configuration files, etc.). It does **not** modify your add-on's source code.
> **Note:**
> Updating from addonTemplate only affects your project's infrastructure (build scripts, GitHub workflows, configuration files, etc.). It does not modify your add-on's source code.

## Before you begin

Expand Down Expand Up @@ -81,7 +81,278 @@ Then fetch the latest changes:
git fetch template
```

## Merging the latest template
## Recommended Method: Automated Update Using the Companion Tool

To streamline the synchronization process and avoid dealing with syntax errors or manual merge conflicts in infrastructure files, a companion utility script is included in AddonTemplate: `syncAddonTool`.

This tool automatically extracts your legacy project settings (such as the add-on name, summary, authors, and repository URL from `buildVars.py`) and merges them cleanly into the newly generated `pyproject.toml` file, while safely preserving empty values if certain metadata is not set.

This automation ensures a seamless transition to the new template infrastructure without losing your original configuration.

The tool automatically supports updating two types of legacy add-ons:

- **Legacy Structure (Dictionary-based without pyproject.toml):**
For older add-ons where `addon_info` was defined as a standard dictionary, the tool automatically migrates the metadata to the modern `AddonInfo` object structure.
It generates a new, fully populated `pyproject.toml` file matching the latest template standards, and synchronizes all infrastructure files.

- **Modern Structure (AddonInfo-based):**
For newer add-ons that already use the `AddonInfo` object but need upstream template updates, the tool checks for any missing metadata keys in `buildVars.py` to insert them.
It safely updates `pyproject.toml` dependencies and versions while preserving your custom configuration rules for tools like `pyright` and `ruff`.

### Prerequisites

Before running the tool, ensure your system meets the following requirements:

- **Python**:
Version **3.13** or newer must be installed (matching the template's required Python version).

- **Git**:
Git must be installed and available in your system `PATH`.

- **Dependency Management (`tomlkit`)**:
The synchronization engine relies on `tomlkit` to safely parse and update `pyproject.toml`.
You have two options to make this dependency available:

- **On-the-fly via global cache (Recommended):**
Pass `--with tomlkit` along with `--no-project` to `uv run`.
`uv` automatically downloads `tomlkit` into its global cache (`%LocalAppData%\uv\cache` on Windows) and reuses it for all future executions without modifying your local environment.
- **Permanent Global Installation:**
If you prefer to make `tomlkit` permanently available inside `uv`'s managed Python environment, run the following command:
```sh
uv pip install --python 3.13 tomlkit
```
This command installs `tomlkit` directly into the `site-packages` directory of `uv`'s central Python 3.13 installation.
Once installed globally, you can execute the tool using `uv run --no-project python syncAddonTool` without needing to append `--with tomlkit` to every command.

> **Important:**
> Project Structure & Execution Methods:
> The update engine is structured as a Python package that relies on the `syncAddonTool/` directory layout.
> You can copy the `syncAddonTool/` folder directly into any add-on repository and run `uv run --no-project --with tomlkit python syncAddonTool`.
> Alternatively, if you prefer to run the tool from an external directory outside of the target repository, you can specify its location using the `-ad` parameter: `uv run --no-project --with tomlkit python -m syncAddonTool -ad /path/to/my-nvda-addon`.
> Finally, you can use the standalone executable (`syncAddonTool.exe`), which requires no Python dependencies.

### Running the automated tool

The tool is highly flexible and supports two execution modes:

1. **Standard Mode (No arguments):**
Run the tool directly from the root of your repository or from any of its subdirectories.
It will automatically locate the project root by searching for `buildVars.py`.
```sh
uv run --no-project --with tomlkit python syncAddonTool
```

2. **Target Directory Mode (With argument):**
Run the tool from any working directory by supplying the optional `addonDir` path (relative or absolute) pointing to the add-on repository you wish to update.
```sh
uv run --no-project --with tomlkit python syncAddonTool -ad ../MyAddon
```

> **Note:**
> Before applying any modifications, the tool creates an untracked backup directory located next to the add-on folder named `<addon>_bak_<timestamp>`.
> This directory contains a copy of the entire project before the update, allowing you to restore the previous state manually if necessary.

Once the update has completed, verify that the add-on still builds correctly:

```sh
uv sync
uv run scons
```

If everything builds successfully, remove the `<addon>_bak_<timestamp>` directory, stage and commit the updated infrastructure:

```sh
git status --short
git add .
git commit -m "Chore: sync infrastructure with AddonTemplate"
```

### Using the Update Tool via Command Line

The `syncAddonTool` tool provides a non-destructive industrial update engine to align your local add-on repository layout with the latest structure of the official NVDA `AddonTemplate`.

You can execute the tool with various command-line arguments to customize the update workflow.

#### Available Options

| Short Flag | Long Argument | Description | Default Value |
| :--- | :--- | :--- | :--- |
| `-ad` | `--addon-dir` | Path to the root directory of the local add-on you want to update.
If not specified, the script automatically walks up from your current directory to find `buildVars.py`. | Current working directory |
| `-td` | `--template-dir` | Path to a local clone/directory of the NVDA `AddonTemplate`. When provided, the tool skips fetching the template via Git and synchronizes directly using this local reference. | None (clones from GitHub) |
| `-dr` | `--dry-run` | Simulates the execution.
It analyzes structure, logs planned changes, and builds reports without writing or modifying any file on disk. | Disabled |
| `-s` | `--skip-backup` | Disables the automatic creation of a timestamped backup directory (e.g., `addonName_bak_YYYYMMDD_HHMMSS`) before processing updates. | Disabled (Backup is created) |
| `-v` | `--verbose` | Enables detailed debug logging output (`[DEBUG]` level) in the console/log output. | Disabled (`[INFO]` level) |
| `-h` | `--help` | Displays the default automated help menu listing all available parameters. | N/A |

##### 1. Generating the Executable

Since the `syncAddonTool.spec` configuration file is provided inside the `syncAddonTool` directory, you can build the standalone executable using `uv` and PyInstaller:

```sh
uv run --with pyinstaller pyinstaller syncAddonTool/syncAddonTool.spec
```

The compiled executable will be generated inside the `dist/` folder (`dist/syncAddonTool.exe`).

##### 2. Running the Executable

Once compiled, `syncAddonTool.exe` accepts the exact same command-line flags (`-ad`, `-td`, `--dry-run`, `--skip-backup`) as the Python execution modes:

- **Targeting an add-on directory from anywhere**:

```cmd
syncAddonTool.exe -ad C:\path\to\my-nvda-addon
```

- **Performing a dry-run test**:

```cmd
syncAddonTool.exe -ad C:\path\to\my-nvda-addon --dry-run
```

#### Customizing Exclusions with `.addonmergeignore`

Rather than modifying the `syncAddonTool` core source code or changing its internal `PROTECTED_ELEMENTS` array, the update tool includes a robust file-exclusion system driven by a local file named `.addonmergeignore`.

This architectural design allows developers to cleanly decouple their project-specific freeze preferences from the update engine machinery.

##### How to Use `.addonmergeignore`

To declare custom exceptions, create a plain text file named `.addonmergeignore` and place it directly **at the root of your target add-on repository**.

- Inside this file, list the exact relative paths, sub-folder paths, or pattern rules of the files or folders you want the tool to skip during synchronization.
- Patterns are matched using standard `fnmatch` rules (with support for `!` prefix to unprotect specific sub-paths).
- You can write one pattern per line.
Empty lines and lines starting with `#` are automatically treated as comments and ignored.

For instance, if you wish to prevent the synchronization process from overwriting your custom execution scripts or specific workflows, simply add them to the file:

```gitignore
# Preserve local release workflows
.github/workflows/release.yml

# Protect custom localized documentation
addon/doc/fr/custom-extra-help.html
```

##### Crucial Requirements & Design Constraints

1. **Automatic Self-Exclusion:**
The update tool automatically protects `.addonmergeignore` itself from being overwritten during synchronization.
Even if `.addonmergeignore` is present in the template repository, the target add-on's local `.addonmergeignore` file is preserved without needing to explicitly list itself.

2. **Presence Check on Initial Run:**
Before copying or updating any template files, the script explicitly checks whether `.addonmergeignore` is already present or absent at the root of the target add-on repository.
If present, its custom rules are loaded immediately before processing any file transfers.

3. **Case-Insensitivity:**
The update tool evaluates exclusions using a standardized, case-insensitive matching algorithm.
This ensures maximum cross-platform reliability (especially between Windows and Unix-like environments).
Since the tool automatically normalizes all inputs to lowercase during execution, **you can write your rules using any casing you prefer** (e.g., `syncAddonTool` or `syncaddontool` will both work perfectly).

4. **File Location Requirement:**
The update engine always loads custom exclusions from the target add-on's root folder being updated.
Therefore, **the `.addonmergeignore` file must always reside inside the destination add-on directory**, even if you are executing the `syncAddonTool` tool from a completely different directory or an external workspace.

#### Usage Examples

Depending on your workflow, the synchronization tool can be executed directly using `uv` with `syncAddonTool` directory, via Python module flags (`-m`), or using the standalone executable (`syncAddonTool.exe`).

##### 1. Standard Automatic Update

Downloads the latest remote template, creates a safety backup of your repository, and non-destructively synchronizes the machinery files.

- **Syntax A (Directory execution inside the add-on repository)**:

```sh
uv run --no-project --with tomlkit python syncAddonTool
```

- **Syntax B (Targeting an external add-on directory)**:

```sh
uv run --no-project --with tomlkit python syncAddonTool -ad /path/to/my-nvda-addon
```

- **Syntax C (Standalone executable)**:

```cmd
syncAddonTool.exe -ad C:\path\to\my-nvda-addon
```

##### 2. Updating from a Local Template Cache (Offline/Development)

Useful when testing local modifications applied to `AddonTemplate` or when working without an active internet connection.

- **Syntax A (Directory execution inside the add-on repository)**:

```sh
uv run --no-project --with tomlkit python syncAddonTool -td /path/to/local/AddonTemplate
```

- **Syntax B (Targeting an external add-on directory)**:

```sh
uv run --no-project --with tomlkit python syncAddonTool -ad /path/to/my-nvda-addon -td /path/to/local/AddonTemplate
```

- **Syntax C (Standalone executable)**:

```cmd
syncAddonTool.exe -ad C:\path\to\my-nvda-addon -td C:\path\to\local\AddonTemplate
```

##### 3. Simulating Changes Safely (Dry Run)

Analyzes structural layouts, evaluates configurations, reads `.addonmergeignore` directives, and builds reports without writing anything to disk.

- **Syntax A (Directory execution inside the add-on repository)**:

```sh
uv run --no-project --with tomlkit python syncAddonTool --dry-run
```

- **Syntax B (Targeting an external add-on directory)**:

```sh
uv run --no-project --with tomlkit python syncAddonTool --dry-run -ad /path/to/my-nvda-addon
```

- **Syntax C (Standalone executable)**:

```cmd
syncAddonTool.exe --dry-run -ad C:\path\to\my-nvda-addon
```

##### 4. Speeding Up with Backup Omission

Targets a project repository while skipping the automated safety backup creation phase to speed up execution.

- **Syntax A (Directory execution inside the add-on repository)**:

```sh
uv run --no-project --with tomlkit python syncAddonTool --skip-backup
```

- **Syntax B (Targeting an external add-on directory)**:

```sh
uv run --no-project --with tomlkit python syncAddonTool -ad /path/to/my-nvda-addon --skip-backup
```

- **Syntax C (Standalone executable)**:

```cmd
syncAddonTool.exe -ad C:\path\to\my-nvda-addon --skip-backup
```

---

## Alternative Method: Manual Update Using Git Merge

If you prefer not to use the automated tool, you can manually merge the latest version of AddonTemplate into your repository.

Merge the latest version of AddonTemplate:

Expand Down
15 changes: 11 additions & 4 deletions docs/testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,22 +10,29 @@ To run the entire test suite (both unit and E2E tests) locally using `uv`:
uv run python -m unittest discover -s tests -v
```

Here is what each part of the command does:

* `uv`: Executes the command within the virtual environment managed by `uv`.
* `python -m unittest discover`: Automatically finds and runs all test files (matching `test*.py`) within the specified test directory.
* `-s tests`: Sets the start directory for test discovery to the `tests/` folder.
* `-v`: Enables verbose output, displaying the status and description of each test method individually.

## Unit Testing

Unit tests are located under the `tests/unit/` directory.
Unit tests are located under the `tests/unit/` directory to validate internal synchronization logic, AST modifications, and helpers.

To execute tests for a specific unit test file:

```bash
uv run python -m unittest -v tests/unit/template/test_sanity.py
uv run python -m unittest -v tests/unit/template/test_syncAddonTool.py
```

## End-to-End (E2E) Testing

End-to-end tests are located under the `tests/e2e/` directory to validate system-level workflows and CLI integration.
End-to-end tests are located under the `tests/e2e/` directory to validate system-level workflows, CLI interfaces, and real-world packaging.

To execute tests for a specific E2E test file:

```bash
uv run python -m unittest -v tests/e2e/template/test_sanity.py
uv run python -m unittest -v tests/e2e/template/test_syncAddonTool.py
```
8 changes: 7 additions & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -29,10 +29,11 @@ Repository = "https://github.com/nvaccess/addonTemplate"
# PEP 735 dependency groups. `uv sync` installs the `dev` group by default, which
# pulls in every tool needed to build, translate, and lint the add-on.
[dependency-groups]
# Build add-on
# Build add-on & repository synchronization machinery
build = [
"scons==4.10.1",
"Markdown==3.10",
"tomlkit==0.15.0",
]
# Translations management
l10n = [
Expand Down Expand Up @@ -80,6 +81,7 @@ exclude = [
"__pycache__",
".venv",
"buildVars.py",
"syncAddonTool",
Comment thread
abdel792 marked this conversation as resolved.
"tests",
]

Expand Down Expand Up @@ -119,6 +121,7 @@ exclude = [
".venv",
"site_scons",
".github/scripts",
"syncAddonTool",
"tests",
# When excluding concrete paths relative to a directory,
# not matching multiple folders by name e.g. `__pycache__`,
Expand Down Expand Up @@ -228,3 +231,6 @@ reportMissingTypeStubs = false

# Bad rules
# These are sorted alphabetically and should be enabled and moved to compliant rules section when resolved.

[tool.setuptools]
py-modules = []
5 changes: 5 additions & 0 deletions syncAddonTool/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Copyright (C) 2026 NV Access Limited, Abdel
# This file is covered by the GNU General Public License.
# See the file COPYING for more details.

"""Core synchronization logic package for NVDA Add-on update tool."""
Loading