Skip to content

Improve HTML and SVG fidelity and preserve fonts across PAGX conversions. - #3770

Open
YyZz-wy wants to merge 43 commits into
mainfrom
feature/codywwang_html
Open

YyZz-wy wants to merge 43 commits into
mainfrom
feature/codywwang_html

Conversation

@YyZz-wy

@YyZz-wy YyZz-wy commented Oct 16, 2026

Copy link
Copy Markdown
Collaborator

Summary

HTML imports can lose form contents, pseudo-elements, masks and SVG details, paint overlapping layers in the wrong order, or substitute web fonts. This change preserves the browser's measured content and authored font families through HTML and SVG import, PAGX layout, embedding and export.

  • Improve snapshot fidelity for forms and native controls, details, counters and markers, first-letter and line-clamp text, layered backgrounds, masks, SVG patterns and paint order. Preserve authored hidden states and restore DOM nodes replaced while sampling animations.
  • Support referenced TTF, OTF, TTC and WOFF2 fonts in layout and rendering, keep original text independent of glyph runs, preserve mixed glyph-run fonts on export, and improve color emoji and ZWJ handling. Serialize WOFF2 decoding to prevent shared-memory corruption.
  • Extend CLI font registration with authored-family aliases, carry downloaded fonts through the conversion pipeline, support archived HTML imports and HTML deck export, and update referenced-font loading in the web playground.
  • Update public API comments, English and Chinese format specifications, CLI documentation and regression coverage. Remove local diagnostic notes, screenshots and agent-only diagnostic tooling from the final diff.

Validation

  • Rebased onto main at 7cb58c1db; all 42 rebased patches were unchanged and there were no conflicts.
  • ./codeformat.sh: passed with clang-format 14; git diff --check: passed.
  • cmake -G Ninja -DPAG_BUILD_TESTS=ON -DCMAKE_BUILD_TYPE=Debug -B cmake-build-debug: configured with TGFX 35be4ca.
  • cmake --build cmake-build-debug --target PAGFullTest: passed.
  • cd tools/html-snapshot && npm run build: passed.
  • cd tools/html-snapshot && npm test -- --runInBand: 50 suites and 827 tests passed, including a final run after adding the required copyright notices.
  • Full native run: 2,658 of 2,660 tests passed. The two failures are detailed below; the full native suite is not green.

Known validation failures

  1. PAGFrameCompareTest.RenderFiles: error_0.pag, frame 310, differs from the existing baseline by one of 73,080 pixels, with a maximum channel difference of 1/255. The baseline and local cache versions for this frame match (60a88e548). A targeted repeat reproduced the failure; its root cause has not been established.
  2. PAGXHtmlTest.HtmlScreenshotCompare: Chromium Page.captureScreenshot timed out for blend_modes_showcase.html after three attempts; the other 78 of 79 batch captures succeeded. A standalone capture of the same 470x420 HTML with the same Chromium launch arguments subsequently passed in about three seconds. This does not make the original batch test pass, and baseline comparison for that batch did not complete.

The local baseline cache also reports that it is out of date, so some other screenshot comparisons may have been skipped. No baselines were accepted or regenerated, and no test assertions were weakened. Temporary validation scripts, reports and diagnostic images are outside the repository.

Scope and review notes

  • Formal C++ and snapshot regression tests are intentionally included.
  • Deferred HTML font-request metadata and explicit font-mapping work remains on the separate font feature branch and is not part of this final implementation.
  • No local editor-sdk test package or backend deployment configuration is included. Backend SDK integration requires a separate update.

…es and accept dropped folders for relative assets.
…rs on the path.

SVG allows text-anchor on the <textPath> element, where it is inherited by the
referenced run, but the importer only read it from <text>. The anchor written on
<textPath> was therefore dropped: the run started at startOffset instead of being
centred on it, shifting the whole line by about half its length. The textPath's
anchor now takes precedence and maps onto Text::textAnchor, matching the browser.
…amily.

An inline <svg> is serialized standalone and re-imported without the HTML cascade, so a
<text> that omitted font-family inherited nothing and fell back to the importer's default
face (Arial) instead of the document's actual font. The snapshot now stamps font-family on
the wrapper box, and convertInlineSvg rewrites the serialized root's font-family to the
resolved single family (a lone authored family already equals the resolved one, so only CSS
stacks are collapsed). A stack left verbatim would be read as one literal family name by the
SVG importer, which has no stack resolution.
… are not over-wide.

When a viewBox maps onto a different target box, the layer carrying the content transform
had no explicit size and measured to its children's unscaled viewBox extent. A parent that
sizes to content then laid the layer out by that larger extent — an inline SVG at
width:100% inside a flex wrapper came out a scale factor too wide and was clipped to 1200px
instead of 1280px. The transform layer (both the merged single-layer case and the wrapper
layer) now carries the target width/height.
…s survive import.

PAGX can only express a <pattern> fill as an ImagePattern (a tiled bitmap), so a pattern whose
content is vector geometry — a stroked wave, a dotted hatch — has no image to tile and the
importer drops it, leaving the painted shape flat. The browser can rasterize, so a new
pre-snapshot pass draws each vector pattern's tile to a canvas and replaces its content with a
tile <image>; the importer's existing image-pattern path then tiles it. Only
userSpaceOnUse tiles with an explicit pixel size are handled (an objectBoundingBox tile's size
depends on the referencing shape, which the pass does not have); those are left as before.
… it.

marker-start / marker-mid / marker-end are inherited presentation attributes, but the importer
read them only from the shape element itself, so a marker declared on a wrapping <g> — the
common way to mark a whole callout arrow — was dropped and the shape lost its head. The three
references now flow through InheritedStyle, and marker content clears them so a marker that
inherited its own reference cannot re-expand recursively.
…hem.

Browsers paint the check tick, radio dot, range thumb/fill, progress/meter bar,
colour swatch and date text+calendar glyph through the UA shadow DOM, which the
light-DOM walk cannot reach. `classify` therefore dropped the element entirely
(checkbox / radio / range — no text, no authored box visuals) or kept a bare box
(color / date / progress / meter), so an imported form lost its checkboxes,
sliders and bars.

Add a render branch that emits each control's static appearance as inline <svg>
from its measured rect and computed styles (accent-color, value/min/max,
low/high/optimum, colour value, ISO date). Interactivity is intentionally not
modelled — PAGX is a static format — but the visual is the design content, and
the missing controls were the largest diff bands on the forms fixture.

Also mask password values as one bullet per character rather than emitting the
plain value (which leaked the password into the snapshot).
…line.

renderTextInput flattened every input/textarea/select value into a single-line
span. That is right for <input>/<select>, but a <textarea> soft-wraps its value
to the box width (and keeps embedded newlines): forcing nowrap — and, more
importantly, emitting neither the box width nor anything that gives the importer
a wrap boundary — left it with a bare single-line <Text>, so the value ran off
the box as one long line (the "项目描述" value overflowed instead of wrapping to
two lines).

Pin the span to the textarea's content-box width, keep `pre-wrap` (only when the
cascade did not already set white-space), and add `overflow: hidden` — a textarea
clips its content, and it is what makes the importer synthesise a wrapping
TextBox anchored to the width. The content is top-aligned, as a textarea's is.
… the snapshot.

Both pseudos are painted by the browser but have no DOM node, so the walker never
saw them:

  - A drop cap (`.lead::first-letter { float: left; font-size: 96px }`) was simply
    missing. Worse, its tall float box overlaps the paragraph's first three line
    rects, and the line splitter groups rects by vertical overlap — so the cap and
    those lines folded into one run-together span that overflowed the column. The
    pass replaces the pseudo with a real floated <span> carrying the cap character
    and the pseudo's own box/text styles, drops the character from the text node
    (the paragraph then reports clean per-line rects), and removes the
    `::first-letter` rules from the page's stylesheets so the pseudo cannot style
    the stand-in a second time.

  - `<ol>` / `<ul>` markers were dropped outright. Chromium does not expose the
    marker text (`content` computes to `normal`), so the pass derives it from
    `list-style-type` plus the item's index (honouring `<ol start>` and per-item
    `value`) and re-emits it as an absolutely-positioned box pinned to the item's
    start edge (`right: 100%`), which is where an outside marker is drawn. The
    marker box is sized from the `::marker` computed width/height.
…into the subset.

A clamped box hides its overflow behind N line-heights while the DOM keeps every
character, so the walker materialised the overflowing lines too — PAGX painted
them under the cut with no ellipsis. The line emitter now drops everything past
the authored count and rebuilds the last visible line with the browser's "…"
ending: the longest prefix that still leaves room for the ellipsis, measured on
a scratch canvas so the live DOM stays untouched for the nodes still to come.

A `data:` font issues no network request, so nothing carried it into the subset:
the computed `font-family` survived on each text node, but the importer had no
file to turn into a `<Font file>` resource, and icon fonts (Material Icons
ligatures, …) rendered their ligature names as text. Inline faces are now saved
next to the output by default — no --download-fonts needed — and re-declared in
the subset as local `@font-face` rules. A face whose authored CSS name differs
from the file's own family (Playfair → "Playfair Display") stays on the
manifest's `--font-alias` route instead, because a `<Font file>` resource
registers under the internal name and would never resolve through the rule.

Also fixes the family-level de-duplication that dropped the second cut of a
family (roman vs italic) when re-declaring faces, and carries the saved faces
(style/weight included) through `resolveFontFamilies`/`SavedFont`.

Rides along: the `applyLayerName` → `applyLayerAttributes` rename and its test.
Multi-layer url() backgrounds: split two stacked url layers into a
synthesized full-bleed child (border-radius inherited, host keeps the
bottom layer) and resolve ExtractCssUrl to the first url() function
instead of splicing every layer up to the last ')' into one
unresolvable path.

Table rowspan overflow: hoist painted tr backgrounds into standalone
boxes ahead of every row with z-index:-1, replaying the browser's
'row backgrounds paint first' order so a rowspan cell overflowing its
row is no longer covered by the following rows' backgrounds.

<select>: synthesize the UA chevron (two-tone sizing measured off the
reference render) and land the rounded overflow mask on the padding box
-- it used to shift by the authored padding and clipped the trailing
indicator to a sliver.

Date picker glyph: two-tone 12px calendar icon matching the reference
render (dark-grey frame, solid black title bar).

Tests: new hoist-table-row-backgrounds.test.js and selectArrowMarkup
coverage in native-form-controls.test.js; full suite 824 green.
- `form` leaves DROP_TAG_NAMES: a form is a transparent layout container and
  the walker already renders input/textarea/select/button as plain boxes or text
  leaves, so dropping the host silently erased every control it wrapped (the
  Flect landing page's email capture groups vanished from the subset).
- The body-root loop in snapshotMainImpl now replays the same paint order
  renderChildrenInto uses for element children (new shared helper
  resolveChildPaintEntry), so a `position: fixed; z-index: 100` header authored
  before the page content no longer sorts by DOM order alone and buries itself
  under the following opaque sections.
- New AUTO_SCROLL_RESCUE_SCRIPT force-reveals boxes that are fully transparent,
  carry an opacity transition (a reveal wired up but never fired) and hold real
  content; display:none / visibility:hidden stay respected. The finalize pass
  additionally waits (bounded by animWaitMs) for the finite transitions the
  warm-up sweep triggered to settle, so delayed entrance paragraphs are sampled
  at rest instead of pruned at opacity 0 or frozen as blur ghosts.

Verified end-to-end on www.getflect.app (forms, navigation and the HLI copy all
survive into the rendered PAGX); npx jest: 55 suites green.
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 78.97052% with 478 lines in your changes missing coverage. Please review.
✅ Project coverage is 83.47%. Comparing base (7cb58c1) to head (14dbd58).

Files with missing lines Patch % Lines
src/cli/ZipArchive.cpp 57.97% 31 Missing and 27 partials ⚠️
src/pagx/html/importer/HTMLCssCascade.cpp 70.45% 24 Missing and 28 partials ⚠️
src/cli/CommandExport.cpp 20.00% 41 Missing and 3 partials ⚠️
src/renderer/FontEmbedder.cpp 67.46% 25 Missing and 16 partials ⚠️
src/cli/CliUtils.cpp 27.27% 30 Missing and 10 partials ⚠️
src/pagx/html/HTMLExporter.cpp 73.13% 17 Missing and 19 partials ⚠️
src/pagx/html/importer/HTMLTextFragmentBuilder.cpp 83.45% 8 Missing and 15 partials ⚠️
src/pagx/html/importer/HTMLElementEmitter.cpp 79.16% 4 Missing and 11 partials ⚠️
src/pagx/utils/FontFileLoader.cpp 81.01% 5 Missing and 10 partials ⚠️
src/pagx/html/importer/HTMLLayerBuilder.cpp 17.64% 11 Missing and 3 partials ⚠️
... and 27 more
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3770      +/-   ##
==========================================
- Coverage   83.51%   83.47%   -0.04%     
==========================================
  Files         731      737       +6     
  Lines      105532   107571    +2039     
  Branches    29218    29989     +771     
==========================================
+ Hits        88138    89799    +1661     
- Misses      10803    10977     +174     
- Partials     6591     6795     +204     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants