[PJBrs]

The code that parses embedded font files for encodings was slightly overzealous, which led to corrupted text extraction, see #4343 as reported by @KRF-Technologies . This PR builds on a test case written by @Ak4shh to refactor the code for parsing font files from _parse_tounicode() to _parse_encoding(), where, according to the specification, it better belongs. Thanks to @Ak4shh for the analysis!

First, this PR refactors the code that parses embedded font encodings. The reason that this code exists is that, for simple fonts, the encoding of an embedded font resource itself can be used as an encoding in the PDF. Previously, this code was called in the _parse_to_unicode() method, when an actual "/ToUnicode" CMap was missing from the font resource. This is incorrect in two ways.

First, conceptually, an encoding is not the same as a ToUnicode CMap. The encoding describes the relation between a character code and an actual glyph, while the a ToUnicode CMap describes the relation between PDF byte code and the unicode code points it represents.

Second, because the embedded font was only parsed after the encoding dictionary of a font dictionary was parsed, and applied afterwards to both encoding and character_map / map_dict, it overwrote a differences encoding, where really it should be the other way around - the differences should outrank the embedded font encoding. This is described in #4343.

As a consequence of the refactoring, the code that specifically derived an encoding from an embedded Type1 font was covered by only one test, which tested issue #2290. It appeared as if that issue was actually not covered by the file in that test, so I decided to change the wording of the test, and keep it only for coverage. In so doing, I found out that the previous fix for #2290 actually was a work-around. This PR fixes it by actually getting an encoding from the associated file's embedded font.

Finally, I refectored the code that decided the font handler for the embedded font, after I noticed that we unneccessarily passed around the variables map_dict (now: embedded_font_encoding) and int_entry. As I got going, I tightened the typing for that variable from dict[Any, Any] to dict[str, str] and added various early returns where I could. This simplified the method signatures for three methods.

Fixes:
#4343 and #2290

I consulted AI while writing this PR, which especially helped identify the underlying cause for #2290, but wrote most of the code myself.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.87%. Comparing base (1e3e7e3) to head (e6b517b).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #4392   +/-   ##
=======================================
  Coverage   98.87%   98.87%           
=======================================
  Files          59       59           
  Lines       12299    12311   +12     
  Branches     2342     2344    +2     
=======================================
+ Hits        12161    12173   +12     
  Misses         68       68           
  Partials       70       70           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[PJBrs]

OK so now this should be finished. I noticed a lot of small opportunities for improvement: one small security fix, a lot of early returns and less indentation, less passing around of variables and simpler method signatures, with a net addition of 7 lines of code to _cmap.py, three of which are whitespace to make things more readable, and another four because I factored out the new _correct_encoding() method :-) Anyway, I think this should be good to go!

[PJBrs]

Ugh... While I was trying to find out what one of the original test was covering, I discovered that the code for deriving an encoding from a Type1 font doesn't even always work. If possible, I'll add a small bugfix for that one as well...

Here's the bug:

The characters (, ), <, >, [, ], {, }, /, and % are special. They delimit syntactic entities
such as strings, procedure bodies, name literals, and comments. Any of these
characters terminates the entity preceding it and is not included in the entity.

(P. 27, Chapter 3, of the PostSript Language Reference third edition.)

So, when the character encoding lines are as follows:

'dup 61/equal put',
'dup 62/greater put',
'dup 63/question put',
'dup 64/at put',
'dup 65/A put',
'dup 66/B put',
'dup 67/C put',
'dup 68/D put',
'dup 69/E put',
'dup 70/F put',
'dup 71/G put',
'dup 72/H put',

then the slash should be seen as a starting a new token. However, the current code only splits on space here:

    txt = encoding_part[1]  # To get the encoding part
    lines = txt.replace(b"\r", b"\n").split(b"\n")
    for li in lines:
        if li.startswith(b"dup"):
            words = [_w for _w in li.split(b" ") if _w != b""]  <-----
            if len(words) < 3 or (len(words) > 3 and words[3] != b"put"):
                continue
            try:
                i = int(words[1])
            except ValueError:
                continue

That line needs to be:
words = li.replace(b"/", b" /").split()

iss2290.pdf accidentally is the only file I tested, and the only file that has this encoding syntax.