Skip to content

[ASIM] Fix ITP workflow regarding ASimDnsActivityLogs ASimDataTester with datetime/string columns - #15509

Merged
Derrick Lee (yummyblabla) merged 1 commit into
masterfrom
derricklee/fix-itp-workflow-dns
Oct 30, 2026
Merged

Derrick Lee (yummyblabla) merged 1 commit into
masterfrom
derricklee/fix-itp-workflow-dns

Conversation

@yummyblabla

@yummyblabla Derrick Lee (yummyblabla) commented Oct 30, 2026 •

Copy link
Copy Markdown
Collaborator

Change(s):

  • Normalize three ASimDnsActivityLogs columns to logical ASIM types before invoking ASimDataTester: ThreatFirstReportedTime and ThreatLastReportedTime to datetime, and ThreatOriginalRiskLevel to string.
  • Prefer the corrected _d/_s columns, falling back to casts of the legacy columns. Use column_ifexists to handle missing column variants.
  • Add a code comment linking the documented table-schema discrepancy, update the validator README, and add regression tests for DNS normalization and unchanged behavior for other target tables.
  • Limit normalization to the data-tester input for ASimDnsActivityLogs; ingestion transforms, parser output, and QTP/parity queries are unchanged.

Reason for Change(s):

  • ITP validation run 40631090500, attempt 2 for [ASIM] Create CrowdStrike Falcon DNS ITP #15454 compiled the parser and accepted all 10 fixture rows, but failed ASIM data testing on three column-type mismatches.
  • The validator tests the destination table slice, including optional fields the parser does not emit. The physical table defines the legacy threat timestamps as string and the original threat risk level as int, while the logical ASIM schema expects datetime and string, respectively.
  • Accommodate this discrepancy in the test-query adapter rather than changing the parser or destination table. See the ASimDnsActivityLogs table reference.

Version Updated:

  • Not applicable: no detection or analytic-rule templates changed.

Testing Completed:

  • python -m unittest discover -s '.script\tests\asimParsersTest\ingestTimeParser\tests' -p 'test_asim_ingestion_validation.py' -k QueryConstructionTests - before the fix, failed as expected: the new DNS regression test detected the missing normalization (6 tests, 1 failure).
  • python -m unittest discover -s '.script\tests\asimParsersTest\ingestTimeParser\tests' -p 'test_asim_ingestion_validation.py' - passed after the fix: all 25 tests.
  • git --no-pager diff --check - passed.
  • Live Azure/KQL execution and a rerun of the affected ingestion-time workflow were not performed.

Checked that the validations are passing and have addressed any issues that are present:

  • Local runtime-validation regression tests and patch whitespace checks passed.
  • GitHub Actions validations are still in progress at the time of this update; their final results have not yet been verified.

@yummyblabla
Derrick Lee (yummyblabla) merged commit 4ef9459 into master Oct 30, 2026
24 checks passed
@yummyblabla
Derrick Lee (yummyblabla) deleted the derricklee/fix-itp-workflow-dns branch October 30, 2026 20:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants