Skip to content

Retry package-set heap exhaustion with clean compiler output #815

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
thomashoneyman wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Retry package-set heap exhaustion with clean compiler output #815

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 8 additions & 2 deletions app/src/App/CLI/Purs.purs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -191,10 +191,16 @@ callCompiler compilerArgs = do
compilerRaceConditionError :: String
compilerRaceConditionError = "Unexpected end of JSON input"

-- | Heap exhaustion can leave stdout empty, so JSON decoding also reports EOF.
isCompilerHeapExhaustion :: CompilerFailure -> Boolean
isCompilerHeapExhaustion = case _ of
UnknownError err -> String.contains (String.Pattern "purs: Heap exhausted") err
_ -> false

@thomashoneyman thomashoneyman Oct 18, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm intentionally classifying the unknown text rather than expanding the compiler failure sum type, as this information is really only needed in niche scenarios and not something every consumer of the compiler failure enum should need to think about

@f-f f-f Oct 18, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

makes sense. can we replace the (not very useful) docstring with a comment that contains this note?


-- | Check if a compiler failure is due to the known race condition bug.
isCompilerRaceCondition :: CompilerFailure -> Boolean
isCompilerRaceCondition = case _ of
UnknownError err -> String.contains (String.Pattern compilerRaceConditionError) err
isCompilerRaceCondition failure = case failure of
UnknownError err -> not (isCompilerHeapExhaustion failure) && String.contains (String.Pattern compilerRaceConditionError) err
_ -> false

-- | Compile PureScript source files with automatic retry for transient errors.
Expand Down
21 changes: 15 additions & 6 deletions app/src/App/Effect/PackageSets.purs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -263,12 +263,21 @@ handle env = case _ of
compileInstalledPackages :: Version -> Run _ (Either CompilerFailure String)
compileInstalledPackages compiler = do
Log.debug "Compiling installed packages..."
Purs.compile
{ globs: [ Path.concat [ Path.basename packagesWorkDir, "**/*.purs" ] ]
, profile: Purs.PackageSetBuild
, version: Just compiler
, cwd: Just env.workdir
}
let
compile = Purs.compile
{ globs: [ Path.concat [ Path.basename packagesWorkDir, "**/*.purs" ] ]
, profile: Purs.PackageSetBuild
, version: Just compiler
, cwd: Just env.workdir
}
result <- compile
case result of
Left err | Purs.isCompilerHeapExhaustion err -> do
Log.warn "Compiler heap exhausted; clearing package-set output and retrying once."
-- Keep the backup intact so a failed retry can still roll back.

@f-f f-f Oct 18, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this comment is above a line that removes a folder. why do we talk about backups?

FS.Extra.remove outputWorkDir
compile
_ -> pure result

attemptChanges :: Version -> PackageSet -> ChangeSet -> Run _ (Either CompilerFailure PackageSet)
attemptChanges compiler (PackageSet set) changes = do
Expand Down
11 changes: 11 additions & 0 deletions app/test/App/CLI/Purs.purs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,17 @@ spec = do
let err = UnknownError "Some prefix text\nJSON: Unexpected end of JSON input\nSome suffix"
Purs.isCompilerRaceCondition err `Assertions.shouldEqual` true

Spec.it "Recognizes heap exhaustion without treating its empty JSON output as a race" do
let
err = UnknownError $ String.joinWith "\n"
[ "purs: Heap exhausted;"
, "purs: Current maximum heap size is 3221225472 bytes (3072 MB)."
, "JSON: Unexpected end of JSON input"
]
Purs.isCompilerHeapExhaustion err `Assertions.shouldEqual` true
Purs.isCompilerRaceCondition err `Assertions.shouldEqual` false
Purs.isCompilerHeapExhaustion (UnknownError "JSON: Unexpected end of JSON input") `Assertions.shouldEqual` false

Comment on lines +57 to +67

@f-f f-f Oct 18, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this test is very useful. We are checking that a function that matches a certain string can in fact match the same string.
A more useful test would be to have the compiler actually exhaust the heap and see if we can match that, but that is more setup and may be not worth it.

Spec.it "Does not detect race condition for normal UnknownError" do
let err = UnknownError "Some other compiler error message"
Purs.isCompilerRaceCondition err `Assertions.shouldEqual` false
Expand Down
Loading