Skip to content

feat: rewrite the KQL compiler as an expression-tree walker (4.0.0) - #57

Open
Egorrko wants to merge 2 commits into
mainfrom
feat/kql-compiler-rewrite
Open

Egorrko wants to merge 2 commits into
mainfrom
feat/kql-compiler-rewrite

Conversation

@Egorrko

@Egorrko Egorrko commented Nov 1, 2026

Copy link
Copy Markdown
Contributor

Release 4.0.0: the KQL compiler (sqlalchemy_kusto/dialect_kql.py) is rewritten as an expression-tree walker instead of regex fixes over compiled SQL text. T-SQL dialect and dbapi are untouched.

Full list of fixes and output changes — CHANGELOG.md, section 4.0.0. Key ones:

  • HAVING and DISTINCT were silently dropped; now compiled.
  • Series limit (join on a grouped subquery) works for physical and virtual datasets.
  • COUNT(col) → countif(isnotnull(col)); LIKE '%x%' → contains_cs; ORDER BY without direction is explicit asc.
  • String literals no longer get rewritten by text-based WHERE conversion.
  • A FROM text that is only a table name renders as database("s").["t"] (Superset SQL Lab table preview).
  • New kustokql+http:// for the Kusto emulator; OFFSET support.

Verification:

  • 202 unit tests pass.
  • All 9 Superset 6.1 probe forms (virtual and physical dataset) compile and run on the Kusto emulator with correct numbers.
  • Commit 92b6134 is pinned in the Dodo Superset fork (6.1-dodo) since 2026-09-24.

Note: open PRs #51 and #52 touch the old dialect_kql.py and will need rework on top of this.

After merge: tag 4.0.0 on the merge commit, make release, GitHub release.

🤖 Generated with Claude Code

Egorrko and others added 2 commits September 21, 2026 18:20
The old KustoKqlCompiler rendered a Select to SQL text and then patched
that text with regular expressions. It dropped HAVING and DISTINCT, broke on
series-limit joins, saved metrics and datasets with comments before let,
and rewrote operators inside string literals (L1-L7 in HANDOFF.md).

The new compiler walks the Select with visitor methods and emits a KQL
pipeline: let for virtual tables (statements hoisted, comments dropped),
join kind=inner (...) on $left.x == $right.x__ for Superset's series
limit, summarize alias = agg() by alias = expr, where after summarize
for HAVING with aggregates replaced by their aliases, distinct, explicit
sort direction, take and serialize | where row_number() > n. Bind values
are always inlined, so statement caching is off.

Every statement shape Superset 6.1 builds for charts, datasets and filter
values is covered by a unit test on the exact KQL and an integration test
on the Kusto emulator that checks the numbers.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Superset builds the SQL Lab table preview as select * from
text(quote_schema(schema) + "." + quote(table)), i.e. ["events"].["T"].
The 3.x compiler converted that to database("events").["T"]; the 4.0.0
rewrite passed FROM text through verbatim, so every KQL table preview in
SQL Lab failed with SEM0139 "Failed to resolve expression". A text that is
nothing but [schema.]table is now rendered like a TableClause; any other
FROM text is still KQL and stays as written.

Checked on the Superset fork against the production Kusto cluster
(/api/v1/database/<id>/select_star) and on the emulator.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant