[nflexfo]

One line description of pull request

Added a test to check the timeliner attribute mappings and fixed the mappings that do not match event data attributes

Description:

plaso/data/timeliner.yaml defines, per data type, which event data attributes the timeliner turns into events. These attribute names are written a second time in the YAML file, and nothing checks that they still match the event data classes.

When they do not match, the error is silent: the timeliner reads the attribute with getattr(event_data, name, None), so a missing attribute is handled like an empty one, without a warning. With place_holder_event, the event data still ends up in the timeline, but with a "Not a time" placeholder event instead of its date and time. Parser tests do not catch this either, since they check event data values and do not run the timeliner.

Test

The new tests/data/timeliner.py follows the existing tests/data/presets.py. It reads timeliner.yaml with YAMLTimelinerConfigurationFile and checks, for every definition, that:

• the data type is defined by an event data class;
• every mapped attribute is defined by every event data class of that data type.

Some details:

• event data classes are loaded by importing the parsers, like the extraction tools do, so the test sees the event data that can be produced during extraction;
• only Plaso event data classes that define their own DATA_TYPE are considered, to exclude event data classes defined by tests and base classes that inherit the data type of their parent;
• attributes are taken from GetAttributeNames() of a new instance. The timeliner processes event data that was read back from storage, which only contains instance attributes, so a mapped attribute that is only set dynamically by a parser is reported as well;
• event data that cannot be instantiated without arguments is skipped, since it is reported by utils/export_event_data.py. A TODO marks this exception to be removed once all event data can be instantiated without arguments, which is the case with Changed Windows distributed link tracking event data to be constructed without arguments #5016 #5364.

Before the fixes, the test reports:

android:sqlite:app_launch: attribute: launch_time not defined by: AndroidAppLaunch
ios:health:workouts: attribute: date_time not defined by: IOSHealthWorkoutsEventData
macos:user:entry: attribute: last_written_time not defined by: MacOSUserEventData
windows:srum:application_usage: attribute: last_connected_time not defined by: SRUMApplicationResourceUsageEventData
windows:srum:network_usage: attribute: last_connected_time not defined by: SRUMNetworkDataUsageEventData

Fixes

android:sqlite:app_launch: mapped launch_time, while the plugin sets start_time. Changed the mapping to start_time.

ios:health:workouts: mapped date_time, while the plugin sets start_time and end_time. Changed the mapping to start_time and added end_time ("Workout End Time"), like the other iOS Health data types that have both, such as ios:health:steps. Note that this adds events for the end of workouts.

I checked the effect with the timeliner on the test databases:

Data type	Event data	Before	After
android:sqlite:app_launch	434	434 "Not a time" events	434 "App Launch Time" events
ios:health:workouts	6	6 "Not a time" events	6 "Workout Start Time" and 6 "Workout End Time" events

macos:user:entry: removed the last_written_time mapping. MacOSUserEventData only defines last_login_attempt_time, last_login_time and last_password_set_time, and the plugin never set last_written_time. The mapping was added with the entry in #4397 and appears to be copied from windows:registry:sam_users, which has the same 4 mappings, where last_written_time is the last written time of the Registry key.

windows:srum:application_usage and windows:srum:network_usage: removed the last_connected_time mapping. Only SRUMNetworkConnectivityUsageEventData defines this attribute. Note that _ParseGUIDTable also sets last_connected_time dynamically on any SRUM event data whose table has a ConnectStartTime column, but in the test database only the network connectivity table has this column:

Table	Data type	Records	ConnectStartTime
{DD6636C4-8929-4683-974E-22C046A43763}	windows:srum:network_connectivity	260	yes
{973F5D5C-1D90-4944-BE8E-24B94231A174}	windows:srum:network_usage	1840	no
{D10CA2FE-6FCF-4F6D-848E-B2E99266FA89}	windows:srum:application_usage	16183	no

So these mappings do not produce events with known data. If you know of SRUM databases where these tables have a ConnectStartTime column, I will restore them. Restricting ConnectStartTime to the network connectivity table, together with connected_time that is set dynamically but neither initialized nor documented, could be handled in a separate SRUM pull request.

The 3 removals do not change the output.

Not covered by this test

The test checks the mappings against the event data, not the other way around. It does not detect:

• a date and time attribute that is defined by event data but missing from timeliner.yaml;
• a data type that has no entry in timeliner.yaml, which is supposed to be checked by utils/check_unused_timeliner_entries.sh. Currently only syslog:comment has no entry;
• an incorrect description. For example, in ios:health:all_watch_sleep, ios:health:headphone_audio_levels and ios:health:heart_rate, end_time is described as "Start Time".

Related issue (if applicable): #5016

Notes:

All contributions to Plaso undergo code review.
This makes sure that the code has appropriate test coverage and conforms to the
Plaso style guide.

One of the maintainers will examine your code, and may request changes. Check off the items below in
order, and then a maintainer will review your code.

Checklist:

• No new new dependencies are required or l2tdevtools has been updated.
• Test data has a Plaso compatible license. If the test data was not authored by you (the contributor), make sure to mention its original source in ACKNOWLEDGEMENTS.
• Reviewer assigned.
• Automated checks (GitHub Actions, AppVeyor) pass.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 85.61%. Comparing base (3733f4d) to head (5c195bb).

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #5366      +/-   ##
==========================================
+ Coverage   85.59%   85.61%   +0.01%     
==========================================
  Files         458      458              
  Lines       41747    41747              
==========================================
+ Hits        35735    35743       +8     
+ Misses       6012     6004       -8     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.