Input file parsing

[wpbonelli]

We have discussed/prototyped using lark to decode MF6 input files into Python containers and primitives (before passing these to cattrs for structuring).

The prototype is currently (mostly) functional. But we still need to determine how performance compares to manual decoding. This will involve benchmarking/profiling the lark prototype against flopy3. If it becomes evident that flopy3 can be optimized in fairly straightforward ways, maybe it will be useful to add a minimal hand-written decoder prototype here for better comparison.

This is related to the performance testing mentioned in #49 (comment), but the focus areas are slightly different:

• here we are only concerned with load performance, and can ignore write performance
• in the other issue, we are most concerned with IO for data structures (arrays/tables/lists) specifically, where here we consider the full contents of input files, including internal and/or external data

[deltamarnix]

Sometimes I could prefer readability over performance.

• Are we concerned that load times will increase significantly?
• Are we only speaking of MF6 input files, or also MF6 output files?

[wpbonelli]

Are we concerned that load times will increase significantly?

This will need benchmarking to make sure they don't, and rather decrease.

Are we only speaking of MF6 input files, or also MF6 output files?

Lark is more suitable for input files than output files IMO. Input files share the same format, so we only have to write 1 grammar. Output files have a variety of formats. Maybe at some point it's worth considering for output files too, but maybe not yet.

[wpbonelli]

The main benefit provided by lark is maintainability. Instead of hand-writing a parser, a parser is generated from a grammar.

We can move forward with lark, and if performance turns out to be unacceptable, bite the bullet and hand-write something. To maximize performance the grammar should be kept as simple as possible (e.g., injecting component-specific keywords into the grammar rather than defining a grammar that is generic over all components).

Perhaps consider a prospective back-compat policy for flopy4 (e.g. dfns will be supported X years or Y releases). We can be more lenient with retrospective back-compat — upon competing an initial prototype, we can test against historic dfn versions and see how far back we can support.

In the long run, pursue standardizing input files with

• netcdf
• toml/etc

[wpbonelli]

The reason the proof of concept is so slow is the grammar is ambiguous. So lark has to use the Earley parser. For LALR the grammar can't have ambiguities. If we can get them out it should run in linear time.

[wpbonelli]

#74

[wpbonelli]

Both the basic and typed parsers now use LALR

[wpbonelli]

I think we can implement a custom xarray backend to read an input file into a dataset/tree. Internally open_dataset can invoke the lark reader.

[wpbonelli]

Dedicated issue at #368. This is icing on the cake, not immediately relevant to input file loading

[wpbonelli]

We now have a more or less functional input file loading capability with Lark (still some edge cases to work through, but successfully loads most of the test/example models). There are two versions:

• basic: simple grammar, usable for any input file, encodes only the concepts of block, line, etc.
• typed: field-aware grammar per input component, generated from DFNs. still in development.

Related, superseded work

• Reading input files #119
• feat: mf6 input file parser MODFLOW-ORG/modflow-devtools#193

I did some ASCII file benchmarking. The basic flopy4 loader is roughly competitive with flopy3, with some interesting differences.

• flopy4 is slower to cold start due to import time. one time cost, easy to fix
• flopy4 is marginally faster on most models (127/165) due to lower fixed overhead
• flopy4 is significantly slower on large internal arrays (see token cost, described below)
• flopy4 is significantly faster on large external arrays (see token cost, described below)
• performance is mixed on lists, depending on size/regularity (record vs union/keystring)

The findings show that some of my early reasoning was wrong. The original motivation for the typed loader was multiple: better error (more specific) messages, less manual validation in transformation, and better performance. The first two points hold, but the last is probably wrong. Transformation is only taking a small (<15%) portion of the total load time. Lark lexing/parsing/etc dominates, largely due to the fact we're representing each array value as a distinct token. This blows up the size of the parse tree. It'd be faster and use less memory to parse array and list rows as a single chunk of tokens to load directly with numpy/pandas/etc. The difference between internal and external arrays demonstrates the potential for improvement here: flopy4 already bulk loads external arrays like this, but parses individual tokens for internal arrays, while flopy3 takes a sort of "middle road", parsing lines in both cases. With bulk loading everywhere, I'd expect flopy4 to be faster across the board.

This shows that the basic loader probably suffices for now, and the benefit of the typed loader will mostly be clearer error messages on malformed input files.

Benchmarking infra will be checked in soon

[wpbonelli]

improved parse performance in #474

[wpbonelli]

more improvements in #479

[wpbonelli]

some new benchmarks on 13 test models from 5 KB to 84 MB

• load: flopy4 is 1.5–13.5× faster, in most cases 2.5–7×
• write: flopy4 is 1.1–9.8× faster, and 2.4× or more from models about 1 MB up
• memory: flopy4's peak is now lower than flopy3's on 11 of 13 models; it's still higher on the 751x751 grid (307 vs. 279 MB) and secp (360 vs. 345 MB)
• startup: importing flopy4 now takes ~0.75 s, the same as flopy3 (was ~2.2 s)
• output: flopy4 writes ~half the bytes of flopy3 due to a formatting difference: flopy3 pads every value to 8 decimals, and flopy4 currently writes compact values

model	input MB	flopy3 load	flopy4 load	load ×	flopy3 write	flopy4 write	write ×	memory fp3 / fp4	reps
test001a_Tharmonic	0.005	0.010	0.004	2.8×	0.008	0.008	1.1×	174 / 167	5
test006_gwf3_disv_ext	0.015	0.034	0.005	6.4×	0.019	0.012	1.6×	176 / 168	5
test028_sfr	0.38	0.031	0.009	3.4×	0.022	0.019	1.2×	176 / 163	5
test045_lake4ss_dev	0.50	0.032	0.010	3.1×	0.030	0.017	1.7×	179 / 165	5
test120_mv_dis-lgr	1.6	0.066	0.016	4.2×	0.094	0.026	3.7×	178 / 167	3
test016_Keating_dev	3.3	0.353	0.026	13.5×	0.097	0.041	2.4×	186 / 173	3
test033_wtdecay	6.9	0.064	0.014	4.7×	0.171	0.017	9.8×	180 / 173	3
test019_VilhelmsenGF	10.8	0.357	0.123	2.9×	1.78	0.26	7.0×	321 / 304	2
test1000_751x751	33	0.318	0.215	1.5×	1.16	0.21	5.4×	279 / 307	1
test016_Keating_disu_mfusg	55	2.15	0.72	3.0×	1.93	0.73	2.7×	323 / 291	1
test1005_secp	66	2.60	0.81	3.2×	3.91	0.68	5.7×	345 / 360	1
test1002_biscqtg_disv_dev	77	10.57	1.53	6.9×	7.45	2.89	2.6×	597 / 449	1
test1001_Peterson	84	2.84	1.10	2.6×	5.82	1.01	5.8×	457 / 392	1

updated after #483: all models rerun on develop

so the next things to look at are cutting memory usage and import time on startup

[wpbonelli]

note flopy4 is still using the basic loader, still haven't finished the typed one