Skip to content

Make the scheduler trim-safe and persist plain job records - #12

Merged
quinnj-bot[bot] merged 2 commits into
mainfrom
claude/workshop-trim-scheduler
Oct 30, 2026
Merged

quinnj-bot[bot] merged 2 commits into
mainfrom
claude/workshop-trim-scheduler

Conversation

@quinnj-bot

@quinnj-bot quinnj-bot Bot commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

What was wrong

Tempus' scheduler could not be compiled with juliac --trim=safe, and a JSON-encoded store did not work even in normal Julia:

  • JSON stores failed at run time. Tempus persisted the runtime Job object (its ReentrantLock and action Function) and whole JobExecutions. With FileStore(dir; codec=JSONCodec()), reading a job back fails with MethodError: no method matching Base.IntrusiveLinkedList{...}(::Nothing, ::Nothing) while StructUtils rebuilds the lock.
  • The scheduler path was dynamic. Scheduler.store::Store was an abstract field, so every store call from the dispatch loop was dynamically dispatched. Job actions were stored as action::Function and called through invokelatest. parseCron(::String) produced a Cron{S,M,H,D,Mo,DoW} whose type depends on the string, so Job.schedule was abstract. _function_ref(f::Function) was not specialized and sliced a variable-length fullname tuple. JobOptions called repr(::Symbol) (about 200 verifier errors in Base's expression printer) and held retry_check as Any. Timezone support (validating the name, computing a zoned schedule) reaches TimeZones' database loading, which is not trim-safe (it reads tz files and can download tzdata), and because timezone is a runtime value that code was compiled into every scheduler program. errormonitor pulls in Base's error display.
  • Restoring actions by name is reflection. Stores held actions through Julia Serialization, which re-resolves functions by name; that cannot exist in a trimmed binary.

What changed

  • Stores hold plain records. JobRecord (definition and disabled state; schedule as its cron string, overlap_policy as a string) and JobExecutionRecord (id, times, succeeded::Bool). These round-trip through JSONCodec, SerializedCodec, and memory. JSON.jl cannot decode Symbol fields without dynamic dispatch, so records use strings and a Bool instead.
  • Actions are in-process. addJob!(store, job) (and push!(scheduler, job)) persists the record and keeps the job's action in that Store object. A scheduler runs a stored job only when a job of that name was added in this process; any function works, including closures over application state. restoreJobs!(store) is the explicit opt-in that resolves stored named functions through resolve_function; it uses reflection and is documented as unavailable in trimmed binaries.
  • Type-erased callbacks. Job actions, retry checks, and timezone schedule lookups are stored as Callback{C}: a @cfunction pointer specialized on the callable's type, called with a fixed-signature ccall that passes a concrete call record (the approach of Servo's handlerfn.jl, without a dependency). Calls through the pointer run in the latest world, so invokelatest is gone. The type is mutable so a callback built during precompilation (whose pointer Julia saves as C_NULL) throws an ArgumentError instead of calling a stale pointer.
  • Concrete types on the scheduler path. Scheduler{S<:Store}; Cron is one concrete type whose fields are a five-type union, dispatched through an explicit cronsplit because inference splits at most four types; Cron == compares expressions (so separately parsed list schedules are equal). Store reads are asserted to the view's element type, so an untyped MemoryStore (the default InMemoryStore()) stays static too.
  • Smaller fixes. _function_ref specializes on the function and walks parentmodule. JobOptions validates without repr(::Symbol), forces specialization on retry_check, and builds the timezone callback only when a timezone is given, so TimeZones' database code is only compiled into programs that use timezones. Execution ids format seconds without milliseconds (Julia 1.13's trim verifier rejects the millisecond padding). Background-task failures are reported with @error and rethrown instead of errormonitor.
  • Rebased onto 3.1.4. The check from Reject cron schedules that can never fire #13 that a schedule can fire (so parseCron rejects "0 0 30 2 *") now goes through cronsplit like the rest of the cron code, because a cron field is now a five-type union.

Closing a scheduler held in an abstractly typed field

The trim workload closed the scheduler through an abstractly typed field, but inference could see the concrete scheduler there, so the check did not cover that case; behind a function barrier, close(::Scheduler) was a dynamic call. close is now unspecialized, reads and writes fields with the getfield/setfield! builtins, and polls for the stop without a closure over the scheduler, so one compiled method serves every store type. The workload now closes the scheduler behind a @noinline barrier; it fails verification without this change.

Breaking changes (version 4.0.0)

  • Stores written by Tempus 3 are not readable; jobs must be added again.
  • A reopened store no longer brings back actions on its own: add jobs at every startup, or call Tempus.restoreJobs!(store).
  • getNMostRecentJobExecutions returns JobExecutionRecords (succeeded instead of status; no result/exception, which stay on the in-process JobExecution).
  • Store(jobs, executions) takes AbstractStore{JobRecord} and AbstractStore{Vector{JobExecutionRecord}}.
  • Job.action is a callback wrapper (still called as job.action()), or nothing for a stored job without an action in this process.
  • Scheduler and Cron changed type parameters; JobOptions gained an internal zone field.

How this was verified

  • test/tempus_trim_state.jl now mirrors an application: a JSONCodec FileStore, Tempus.Store(backend; prefix="tempus/"), jobs whose actions are closures over app state added at every startup, getJobs/schedule comparison, a Scheduler with a retry_check, run!, a one-shot job that fails once and is retried, a restart that reads the persisted records back, close through an abstractly typed field, history, disableJob!, and purgeJob!.
  • juliac --trim=safe (JuliaC 0.3.12) on Julia 1.14.0-alpha2: 0 verifier errors, and the binary runs the jobs. On Julia 1.13.3 the same workload with a MemoryStore backend: 0 errors, and the binary runs. On 1.13 a JSONCodec store cannot verify for a reason outside Tempus: typed JSON.parse of any nested struct (here AbstractStores.Entry{JobRecord}) leaves an "unresolved invoke" of StructUtils' inner make. A standalone program that only calls JSON.parse(bytes, Outer{Inner}) on two plain structs reproduces it on 1.13.3, and the same program verifies on 1.14. The workload therefore uses memory on Julia < 1.14.
  • New tests: closures over app state in a JSONCodec FileStore across two starts with persisted history; a JSON round trip of a job's options (including timezone) and restoreJobs!; precompilation-pointer failure; list-schedule equality. Existing tests were updated for records and restoreJobs!.

🤖 Generated with Claude Code

quinnj and others added 2 commits October 30, 2026 10:58
Stores now hold JobRecord and JobExecutionRecord values instead of runtime
Job and JobExecution objects, so JSON-encoded stores round-trip (reading a
persisted Job back failed while rebuilding its ReentrantLock). Job actions
live in the Store object they were added to; restoreJobs! is the explicit,
reflection-based way to resolve stored named functions.

Actions, retry checks, and timezone schedule lookups are type-erased behind
a @cfunction callback, and the scheduler path uses concrete types:
Scheduler{S<:Store}, a non-parametric Cron with explicit field dispatch,
typed store reads, and no repr(::Symbol), invokelatest, or errormonitor.
The trim workload now covers an application's use (JSON FileStore, closures
over app state, run!, retries, restart) and executes jobs in the binary.

Breaking: version 4.0.0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The trim workload closed the scheduler through an abstractly typed field,
but inference could see the concrete scheduler there, so it did not test
that case. Behind a function barrier, close(::Scheduler) is a dynamic
call that juliac --trim=safe rejects. close is now unspecialized, uses
the getfield and setfield! builtins, and polls without a closure over the
scheduler, so one compiled method serves every store type.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@quinnj-bot
quinnj-bot Bot force-pushed the claude/workshop-trim-scheduler branch from a3162e9 to e60bc8b Compare October 30, 2026 17:07
@quinnj-bot
quinnj-bot Bot merged commit 098ddd9 into main Oct 30, 2026
12 checks passed
@quinnj-bot
quinnj-bot Bot deleted the claude/workshop-trim-scheduler branch October 30, 2026 17:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant