[irvingouj@Devolutions (irvingoujAtDevolution)]

AI analysis of a recording, started by DVLS with a plain HTTP request and run as a provisioner Task on the existing job queue.

ProvisionerTaskRunner is the one place that keeps a Task's record (#2041) and its job in step. Each kind of work is a ProvisionerTask: what one attempt does, its time limits and payloads, and what to finish once the Task ends. The record is the source of truth and the job only wakes it up, so a stale or duplicate job does nothing. At startup, an unfinished Task left without a job gets a new one.

API (stable, no enable_unstable)

• POST /jet/jrec/{session_id}/ai-analysis, scope gateway.tasks.recording-ai-analysis.start. Body: taskId, provider, model, baseUrl (needed for openai-compatible), maxOutputTokens, language, apiKey.
  • 202 + Task when created, 200 + Task for the same request again
  • 400 bad request, 404 no recording
  • 409 task_id_conflict / recording_active / analysis_in_progress (+ its taskId)
• GET /jet/jrec/{session_id}/ai-analysis/{task_id}, scope gateway.tasks.recording-ai-analysis.read: the Task, or 404.

The two scopes in Devolutions.Gateway.Utils and the regenerated OpenAPI clients are in #2054, stacked on this PR.

How it behaves

• The API key and the base URL stay in memory, encrypted, in the SecretStore (feat(dgw): add an encrypted in-memory secret store #2058): never in the job, gateway.db or our logs. A restart loses them, so an unfinished Task fails with keyLost.
• An attempt fails with a closed kind and no text, recoverable or permanent. Recoverable ones (rate limit, 5xx, network, request timeout) are retried, up to 5 attempts. Permanent ones (401, spent quota, an answer cut short or refused) fail the Task at once. A failed payload is { error, usage }.
• Every payload but queued carries usage: the tokens used so far over all attempts, plus unreportedRequests for requests the provider didn't count, a request still in flight included.
• A Task fails if it doesn't start within 2 h. Once it runs, it fails only after 30 min without progress, so a long recording that keeps going is never cut off. Waiting for a remux isn't progress. Each AI request times out after 10 min.
• The new log stays in the session, unlisted, until the success is recorded in gateway.db. Then the manifest lists it as ai-analysis-0.slog and drops the earlier analysis. If that last step fails, a new job does it again later, after a restart too. A new Task that finds such a log first finishes it: it lists the log of a succeeded Task, and removes any other.
• A failed Task removes everything it made and leaves the earlier analysis in place.
• The manifest is written to a temporary file, synced, then renamed over the old one.
• language is 1 to 10 characters, such as fr-CA; without it, English. It's a Task parameter, so the same taskId with another language is a task_id_conflict.

Where to look hardest

• provisioner_tasks/runner/mod.rs: run_attempt (deadline watchdog, retry vs. fail, end_later) and resume
• provisioner_tasks/recording_ai_analysis/mod.rs: run, on_end, lock_session
• recording.rs: handle_replace_artifacts

Known, accepted

• At VerbosityProfile All, reqwest/hyper's own TRACE lines show the provider's address (not the key). Our code never logs it.
• If recording the success and then the failure both fail, the Task fails at its deadline as deadlinePassed instead of internal, and a job queued for then removes its log.
• A store error while a new Task finishes the logs earlier Tasks left unlisted fails it as internal, before anything is paid.
• An end that keeps failing, such as on a manifest that can't be parsed, is tried again every 5 minutes, with no limit.

Tests

• cargo test -p devolutions-gateway --lib: provisioner tasks, recording and secret store tests pass.
• cargo test -p devolutions-gateway-ai --lib recording_analysis: 45 passed.
• cargo test -p testsuite --test integration_tests ai_analysis: 10 passed, against the Gateway binary with a mock AI provider; the key and the base URL reach neither the logs nor gateway.db.
• clippy (--features openapi --tests) and nightly rustfmt are clean.
• Not run against a real AI provider at this head.

🤖 Generated with Claude Code

[github-actions]

Let maintainers know that an action is required on their side

• Add the label release-required Please cut a new release (Devolutions Gateway, Devolutions Agent, Jetsocat, PowerShell module) when you request a maintainer to cut a new release (Devolutions Gateway, Devolutions Agent, Jetsocat, PowerShell module)

• Add the label release-blocker Follow-up is required before cutting a new release if a follow-up is required before cutting a new release

• Add the label publish-required Please publish libraries (`Devolutions.Gateway.Utils`, OpenAPI clients, etc) when you request a maintainer to publish libraries (Devolutions.Gateway.Utils, OpenAPI clients, etc.)

• Add the label publish-blocker Follow-up is required before publishing libraries if a follow-up is required before publishing libraries

[irvingouj@Devolutions (irvingoujAtDevolution)]

Intent drafts: handoff

The interview read this branch at a4e8a98, before the rebase onto master. The intent it produced is in f231c48.

Provenance

Levels: 1 = my words verbatim, 2 = my words tidied, 3 = agent wording I adopted, 4 = agent fill-in, not intent.

#	Commitment as written	My words (if tidied)	Level
1	provisioner_tasks/INTENT.md: Background, Naming, the first five Management rules, the last Secrets line, and the first three AI analysis rules. jrec.intent.md: Background and the first three AI analysis lines.	written by hand in 4400ea0, 18572ef, 3cc6fe9 and d2c11a0, typos kept	1
2	An attempt error is recoverable or permanent: a recoverable one is retried with the job up to a limit, and a permanent one fails the Task at once. A failed Task's payload is that permanent error, giving up included, as a closed set of kinds without text for people, because Gateway does not translate: the caller words it.	"make your error Error = Permernat | Recoverable"; "why not reuse the same permernant error type?"; "let's not include message, if we include message we need to provide translation, but we don;t wwant handle translation in gateway"; "we good, use it"	3
3	When Gateway starts, it gives a new job to every unfinished Task that has none. This can happen if Gateway stopped after saving a Task but before saving its job. Gateway starts the job without checking whether the Task can still succeed: each kind of Task decides that for itself. For example, an AI analysis fails because its key is gone. If the new job can't be saved, the Task fails.	"save"; "the idea is, let job fail per tasks defination, task start job regardless"; "go ahead, this is good"	3
4	Secrets a Task needs, such as an API key or a base URL that may carry credentials, never reach the disk: not gateway.db, not the job, not the logs.	"url need to be secrte as well"	3
5	They are kept encrypted in memory, in the SecretStore, until the Task ends or reaches its deadline.	"name the standalone store" (translated); "good"	3
6	An analysis keeps working files next to the recording, so a retry can pick up where it stopped. When a Task ends, it deletes its own working files, whether it succeeded or failed. A new Task first deletes any working files an earlier analysis left behind. A Task never deletes another Task's working files.	"hard to read, use plain english"; "good, go ahead"	3
7	A session keeps one AI analysis: a new Task replaces the earlier one atomically once it succeeds, and a Task that fails leaves the earlier one in place.	"let's update to replace automically after success"	2
8	Keeping one analysis per language is a future extension that must stay possible.	"I agree"	3
9	A Task succeeds only once its success is recorded in gateway.db. Until then, its new log sits in the session unlisted and the earlier analysis stays. After that, the manifest lists the new log and drops the earlier one; if this last step is cut short, Gateway does it again when the Task ends and when Gateway starts.	"who said it's a success? this is a failure, no?"; "I want this 1 2 3 to be automic, or best effort automic"; "sure, go ahead"	3
10	A recoverable failure, such as a rate limit (429), is retried with the job, and each retry continues from the progress saved so far; only a new Task starts over.	"transiant, we save progress and contile"	2
11	A Task that fails keeps its record, but removes everything it produced, leaving no file on disk, even if that wastes the money already spent, because an analysis missing a large part would mislead whoever searches it, more than the misses of a nondeterministic AI already do.	"task fail, delete completely the artifact projduced this session, and leaves notion on disk, even if that means wasted money"; "missed search, even though missed search is already tehre with undeterminsitic AI, but missed search wtih a large chunk would be misleading to user"; "keep task record, but remove the produced artifact completely"	2
12	A Gateway restart loses the API key, so an unfinished Task fails with keyLost at its next attempt, without a retry.	"when gateway restart, it should fail permerantly with keylost, no?"	2
13	A rate limit, a server error, a lost connection or a request timeout is recoverable; a refused API key (401), a spent quota, or an answer the model cut short or refused is permanent.	"we good" (the error types); "can we tread http timeout as a transient error?"	3
14	An AI analysis fails if it hasn't started within 2 hours. Once it runs, it fails only after 30 minutes without progress, so a long recording that keeps making progress is never cut off. Each request to the AI times out after 10 minutes and is retried like any other recoverable failure.	"queded to start, 2 hour"; "each http request, 10 minutes"; "3 is good, we already track progress with task"; "30 min"; "go ahead"	3
15	An analysis waits until Gateway has finished remuxing the session's videos, because remuxing replaces the file the analysis reads. Waiting is not progress, since remuxing should not take that long.	"C, remux should not take that long"; "go ahead"	3
16	Each Task has a deadline. While the Task runs, its deadline may move forward each time it makes progress.	"go ahead"	3
17	The caller chooses the language of the analysis in the request, under the rules of crates/devolutions-gateway-ai/INTENT.md. The language is part of the Task like every other parameter, so the same Task ID sent again with another language, model or provider is a conflict.	"A"	3
18	A request sent again is compared on its parameters only: the API key and the base URL are never stored, so another key or base URL gets the same Task, which keeps the first ones.	"ok, I agree, go ahead"	3
19	A new Task answers 202 with its path in Location, and the same request again answers 200 with the same Task. A conflict answers 409 with a code: task_id_conflict, recording_active, or analysis_in_progress with the running taskId. A session without recording, or an unknown Task, answers 404. An invalid request answers 400 and records no Task. Unknown fields are ignored, so a newer provisioner works with an older Gateway. The provider is one of openai, anthropic, mistral, gemini or openai-compatible.	"good, go ahead"	3
20	Every Task reports the tokens it used so far, whether it is running, succeeded or failed, so the provisioner can add up what all analyses cost, failed ones included. Requests whose usage the provider didn't report are counted apart, so a total is never shown as complete when it isn't. A succeeded Task also shows the log's file name, the model the provider reported, and the prompt versions.	"why usage can be optional? I don't think it should be optional"; "how many token have we ever used for all the recording analysis? failed or not"; "I agree"	3
21	Gateway replaces the manifest in one step and never rewrites it in place, so neither a reader nor a crash ever leaves half of it.	"I want this 1 2 3 to be automic, or best effort automic"; "sure, go ahead"	3

Rows 9, 13, 20 and 21 were worded after the interview and shown to me together before the code work started.

Agent fill-in (level 4, not in the intent files): 5 attempts as the retry limit, a failed end retried every 5 minutes with no cap, the unlisted log name .ai-analysis-<taskId>.slog, and deadlines cut to whole seconds.

Left out on purpose

• Starting an attempt only if it can finish before the deadline, and a 30-minute limit per attempt: rejected.
• Deleting the earlier analysis when a new Task starts: replaced by row 7.
• Answering 409, or swapping in the new key, when a request comes again with another key: rejected for row 18.
• Rejecting an analysis while the remux runs, or not counting the wait against the deadline: rejected for row 15.
• The list of permanent error kinds lives in the code and the OpenAPI document, not in the intent.

Placement

• devolutions-gateway/src/provisioner_tasks/INTENT.md (new): rows 2 to 15.
• crates/provisioner-task/INTENT.md: row 16.
• devolutions-gateway/src/api/jrec.intent.md (new): rows 17 to 20. Since feat(dgw): search slog artifacts across recordings #2112 moved jrec.rs to jrec/mod.rs, this file sits beside the jrec/ folder. It stays there for now: as api/jrec/INTENT.md it would also cover search.rs, where search.intent.md rejects unknown fields and row 19 ignores them.
• devolutions-gateway/src/recording.intent.md: row 21.

Pending experiments

None.

Note

Human-tuned, LLM-assisted content.

[Benoît Cortier (CBenoit)]

Suggested change

	Each Task has a deadline. While the Task runs, its deadline may move forward each time it makes progress.
	Each Task has a deadline.
	While the Task runs, its deadline may move forward each time it makes progress.

I would like a rationale added to this new line ^