Skip to content

fix: read and write text files with utf-8 encoding (#170) - #176

Merged
pablocaeg merged 2 commits into
pablocaeg:masterfrom
markankamaransheogarat-sys:fix/utf8-file-encoding-170
Nov 3, 2026
Merged

pablocaeg merged 2 commits into
pablocaeg:masterfrom
markankamaransheogarat-sys:fix/utf8-file-encoding-170

Conversation

@markankamaransheogarat-sys

Copy link
Copy Markdown
Contributor

Summary

Fixes #170.

On Windows, Python opens files with the locale encoding (e.g. cp1252), leading to character decoding errors when reading non-ASCII characters such as em-dashes – in dataset fixtures and calibration files.

Changes

  • Explicitly specified encoding="utf-8" across all text-mode open(), .open(), .read_text(), and .write_text() calls in app/ and tests/.
  • Preserved binary file opens ("wb", .part downloads).
  • Enabled PLW1514 (unspecified-encoding) with preview = true in pyproject.toml so unencoded text file opens are flagged by ruff.

Verification

  • git grep -rnE "read_text\(\)" app tests scripts -> 0 matches.
  • ruff check app tests scripts -> all checks passed.

@sloptotal-bot

sloptotal-bot Bot commented Oct 24, 2026

Copy link
Copy Markdown

Thanks for your first contribution to SlopTotal, Konstantin! 🎉

@pablocaeg reviews every pull request personally, and you'll hear back in this thread.

  • Because this is your first pull request here, GitHub waits for the maintainer to approve the CI run before it starts. That's a GitHub safeguard, not a problem with your change.
  • It's linked to Read text files as UTF-8 so tests pass on Windows #170, which will close when this is merged.
  • To check locally what CI runs: pytest -q and ruff check app tests scripts && ruff format --check app tests scripts.

Automated reply from the SlopTotal assistant. @pablocaeg reads every thread.

explicit-preview-rules enables PLW1514 without opting into every other
preview rule. The analyze.py subprocess output is decoded as UTF-8 too,
which is where the Windows failure showed up (from pablocaeg#184).

Co-authored-by: Priyadharshini537 <Priyadharshini537@users.noreply.github.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@pablocaeg
pablocaeg merged commit 05ced29 into pablocaeg:master Nov 3, 2026
6 checks passed
@pablocaeg

Copy link
Copy Markdown
Owner

Thanks @markankamaransheogarat-sys! This now covers every text-mode open in the repo, and the ruff guard keeps it that way. I pushed one commit: explicit-preview-rules so only PLW1514 comes from preview, the analyze.py subprocess output decoded as UTF-8 (the spot where the Windows failure showed up, from #184), and a format pass. Merged.

@sloptotal-bot

sloptotal-bot Bot commented Nov 3, 2026

Copy link
Copy Markdown

Merged, thanks Konstantin! That's your first contribution to SlopTotal. 🎉 #170 is closed with it.

If SlopTotal is useful to you, a ⭐ on the repository helps other people find it.

Automated reply from the SlopTotal assistant. @pablocaeg reads every thread.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Read text files as UTF-8 so tests pass on Windows

3 participants