Skip to content

Latest commit

 

History

History
175 lines (135 loc) · 20.3 KB

Jev.md

File metadata and controls

175 lines (135 loc) · 20.3 KB

Collection

Tutor

  • flaviocopes A deep dive into Jev, TypeSafe's System One model

Start here

  • Introduction — What Jev is, how System One models differ from text-generation models, and the Choice, Score, and Noul primitives.
  • Quick start — The shortest path from an API key to a typed decision in Python or JavaScript.
  • How to build with TypeSafe — Design guidance for decomposing a workflow into narrow judgments while keeping policy and side effects in code.
  • TypeSafe Console — Create keys and inspect live Jev requests.

Official resources

Product and documentation

  • TypeSafe AI — Official product site for System One models and Jev.
  • Documentation — Guides, SDK references, patterns, cookbooks, and the HTTP API.
  • HTTP API reference — Request and response contract for direct API integrations.
  • Interactive demos — Official hands-on examples, including the smart-home assistant.
  • Workflow evals — TypeSafe's published workflows, model comparisons, methodology, and example queries.

SDKs and developer tools

  • JavaScript SDK — Official JavaScript and TypeScript client with inferred answer types.
  • Python SDK — Official synchronous and asynchronous Python client.
  • System One Adapter — Drop-in Python adapter for running the same typed interface over OpenAI, Anthropic, and OpenAI-compatible LLM APIs.
  • TypeSafe Agent Skills — Official agent skill for designing TypeSafe workflows from Claude Code, Codex, and other skill-compatible agents.
  • TypeSafe GitHub organization — Source repositories maintained by TypeSafe.

Concepts, patterns, and cookbooksCompute

  • Primitives — Choice, Score, and Noul, including their result shapes and when to use each one.
  • Confidence — How confidence differs from answer probability and how to use it as an architectural control.
  • Patterns — Confidence-gated routing, composite scoring, speculative fan-out, and intent routing.
  • Example use cases — A map from real-world workflows to typed judgments.
  • Cookbooks — Reproducible implementations for parallel questions, reranking, semantic search, guardrails, extraction, classification, and more.
  • Agent skill guide — Installation and usage instructions for the official TypeSafe skill.
  • jev-cookbook ⚡ 适合中国宝宝的 Jev 入门教程|手把手带你了解关于 Jev 的一切——Jupyter Notebook 轻松实验,从三种问题原语到 18 篇实战配方、语音智能家居、模型评测与本地微调,全面掌握 System One 判断模型的开发范式

Research and writing

Community and updates

  • Discord — Official community server for builders, support, and discussion.
  • Show and Tell — Builder demos and work in progress; joining the Discord server is required.
  • X — Product and research updates.
  • LinkedIn — Company announcements and hiring updates.

Community projects

Community projects are independent unless their repository says otherwise. Read the code, licenses, data-handling notes, and evaluation caveats before using them in a consequential system.

Tutor

  • flaviocopes Learn how Jev turns text into typed choices, scores, and probabilities, with JavaScript examples, practical patterns, limits, and real use cases

Client libraries and integrations

  • Advocaat — Small TypeScript client with ergonomic tagged helpers for typed chances, choices, and scores.
  • HA-Jev — Home Assistant integration that turns typed questions about entity state into sensors and automation actions, with entities reporting daily calls, tokens, and estimated cost and a token budget that halts evaluation; answers carry no explanation, so it is not suitable for safety decisions.
  • pi-typesafe — Pi extension and library that gives the agent and other extensions one consented, key-managed TypeSafe client with a batched typesafe_evaluate tool and offline-testable transport; requests are billable and opt-in per user.
  • RubyLLM TypeSafe — TypeSafe provider for RubyLLM 2 with offline model metadata and typed responses.
  • s1-rs — Rust derive layer for Choice, Score, Noul, typed question sets, confidence gates, and network-free testing.
  • TypeSafe AI for Rust — Rust client with asynchronous and blocking transports, typed responses, observable retries, and inspectable errors.
  • typesafe-ai-rails — Community Rails integration for TypeSafe's System One API, built on typesafe-sdk, with Rails configuration, persisted usage and cost telemetry, and opt-in confidence policies for Choice and Score answers.
  • Rust typesafe-rs — Latency-focused Rust transport SDK designed around behavioral parity with the official clients.
  • Elixir typesafe_sdk — Elixir SDK for TypeSafe AI and Jev with typed Choice, Score, and Noul structs, configurable retries, and upstream API parity.
  • Ruby typesafe-sdk — Community Ruby client for TypeSafe's System One API with typed Noul, Choice, and Score questions, retries, model listing, and thread-safe pooled HTTP connections; requires Ruby 3.1 or newer and has no async client.
  • TypeSafeAI.Net — .NET client for TypeSafe's API with Noul, Choice, and Score question sets, HttpClientFactory and dependency injection support, plus Microsoft.Extensions.AI guardrail, routing, tool, and evaluator adapters.
  • Vercel AI Gateway — Third-party hosted gateway entry for calling Jev through Vercel's AI SDK and gateway.

Agent, developer tooling and harness

  • https://github.com/nouli-ai/nouli

  • hermes-jev-skills Jev-powered model routing, memory, compaction, skill selection, computer and browser use for Hermes agents (also Claude Code and Codex)

  • Bicameral — Pi coding harness where an LLM writes while Jev supplies typed reflexes for policy, loop detection, and review; explicitly not a sandbox.

  • Every — Semantic code search CLI that asks a yes/no question of every function and ranks the resulting probabilities.

  • Jev MCP — Python MCP server exposing classify, score, check, match, and screen tools to MCP-compatible agents.

  • jev-axi — Agent-ergonomic CLI following the AXI conventions that gives coding agents Jev judgments for blocking risky tool calls, screening fetched content for prompt injection, triaging build logs, flagging risky diffs, and filtering or ranking many items; its own benchmark found agents using it read fewer files but cost the same, so it is meant for judgments rather than as a substitute for reading code.

  • jev-mobile — Experimental Android agent that uses Jev for bounded, per-step choices over prevalidated UI actions, with confidence gates, pagination, escalation, and optional LLM planning; currently a proof of concept tested mainly against Android Settings.

  • jevcal — CLI that fits a per-question confidence threshold to a target accuracy on your own labeled data, verifies it on a held-out split, estimates how much traffic still needs a fallback model, and re-checks the locked thresholds in CI; publishes no Jev results of its own, and thresholds fitted on fewer than about 100 labeled rows should not be trusted.

  • pi-jev — Pi extension with a shadow-mode tool-call gate, output judge, and a general typed jev_ask tool.

  • pi-warden — Pi guardrails built on pi-typesafe that return Jev's verdict to the agent as a held tool result or a short steer instead of a dialog, check writes against a project rules file, and grade their own holds against the user's next message on recorded sessions; the action guard is calibrated on one user's 17k calls, the other guards on synthetic cases only.

  • Supercov — Code quality for coding agents: Jev scores each source file so the agent knows what to fix first.

  • TypeSafe MCP — Go CLI and single-binary MCP server with setup for Claude Desktop, Claude Code, and Codex.

  • Jev-mcp Proof of concept MCP for Typesafe's new Jev AI model

  • foreman Software Factory Foreman based on TypeSafe Jev model

  • jev-codex-router Per-turn model & reasoning routing for Codex, driven by Jev (TypeSafe System One): picks the model, thinking depth and speed mode for every turn.

  • safer-with-jev Neon Function proxy for the Neon AI Gateway with TypeSafe Jev routing.

  • CC A calibrated context sieve for Claude Code: every tool result is judged by a System One model before it enters context.

  • blink Codebase search powered by Jev from

  • fast-jev-compaction Claude Code plugin that replaces the compaction summary with Jev decisions: every tool call and result is scored in one fast request, stale ones are dropped or truncated, everything kept stays verbatim.

  • stanley-code Bounded TypeSafe Jev workflows for coding agents.

Code-Review

  • Jev Review — Staged code-review workflow and local dashboard that follows structured signals through focused Jev calls.
  • jev-review Local-first MCP plugin for continuous software-quality review by AI coding agents, powered by Jev.

How-To-Use-Jev-Skill

  • building-with-jev-skill This repo holds an agent skill for writing and improving programs that call Jev, TypeSafe's System One model
  • jev-skill Jev 负责选择、分类和评分,Agent 负责提供证据和执行。 这里有 57 个项目与资料入口、5 个技能、108 个场景,以及 14 组已记录的输入输出。

Browser agents

  • Jev Browser — Agent skill and runtime that lets Jev select browser actions inside a continuous observation-action-verification loop.
  • Jev Ultrafast — Browser Use agent with a dynamic indexed action space, batched operation and target decisions, traces, and a measured Google Flights demo.

Games, robotics, and interactive demos

  • Crowdcheck — Live demo that tests a 144-character post on 10,000 persistent synthetic personas: code decides who sees it, and batched Jev calls return read, like/dislike, agreement, repost, follow, and block probabilities per persona group; posting requires Google sign-in, post text is sent to Jev through Vercel AI Gateway, and the simulated reactions are not a forecast of real audience behavior.
  • HEIST//ONE — Observable browser stealth game where Jev supplies batched typed judgments for six guards while deterministic code owns the simulation and validates every proposal; includes a Decision Lens, scripted offline mode, evidence traces, tests, and one documented live sandbox extraction.
  • Jev Drone — MuJoCo quadrotor stack that keeps control and safety in code while using Jev for slower tactical judgments.
  • Jev Plays Pokémon — FireRed and Showdown harness where Jev battles the Elite Four by picking a move or switch from the current Pokémon, moves, stats, and opponent HP.
  • Jev Plays StarCraft — Structured-state harness, verified run, probability trace, and evidence bundle for the original StarCraft shareware campaign.
  • Jev Search — Web search demo using Jev's typed Choice and Noul judgments to select sources, time ranges, and query candidates, then rank results retrieved through Search1API; relevance scores are model judgments, not verified accuracy.
  • TypeSafe Mario — NES controller experiment that turns emulator telemetry into structured state and has Jev choose legal actions.
  • TypeSafe Typewriter — Live Val Town demo that updates 16 typed judgments as text changes.

Evaluations and independent research

  • calibre — Independent calibration measurement of Jev on two labelled datasets, Banking77 and Web of Science, with a Jev to frontier cascade priced per row from measured tokens; the protocol was frozen before any result and the raw JSONL and figures are committed, and no routing parameter transferred between the two datasets, as the optimal threshold, the sign of the accuracy gap between the two models, and whether routing paid for itself all changed; the Web of Science labels come from publication metadata rather than per-document annotation, so part of the error measured there is label ambiguity.
  • Jev Judge vs Dimension Scores — Independent measurement on three classification tasks: one direct Jev question per row against 12–14 Jev-scored dimensions with locally fitted weights, 5,477 test rows and 34.1M input tokens for $1.43; decomposition reached 0.9076 against 0.8373 on Japanese NLI but flagged about 25× more hard benign rows as attacks, and four repair attempts failed, on dimensions the author wrote himself.
  • Jev Rerank Bench — Reranking comparison with raw provider responses, scoring code, dataset-level results, uncertainty intervals, and documented limitations.
  • Jev Spam Eval — Exploratory zero-shot spam study against trained TF-IDF baselines, including results and explicit post-hoc-tuning caveats.

Showcases and field notes

Jev-Like-Decisions-Engine

  • nimble Local typed decisions, contrastive data curation, and model evaluation.
  • laya Multilingual, non-autoregressive System 1 decision model
  • NanoJev A nano replica of Jev: parallel decisions, dynamic candidates, and an end-to-end training pipeline.
  • OpenJev — Independent open-model research baseline for direct typed option scoring; it reproduces the interface pattern, not Jev's undisclosed model or training.
  • openvons a decision layer that answers finite choices with probabilities
  • openjev localjev Jev uses a typed decision API rather than an OpenAI chat API. OpenJev implements the Jev wire protocol and obtains probabilities with a special one-step DiffusionGemma structured read.
  • kev Jev-inspired decision model. Typed questions in, calibrated probabilities out, one forward pass.

Benchmark

  • jev-models JevBench is Benchmark Heaven's own benchmark for Jev-class decision models: state and a bounded rubric in, a typed answer out.
  • screensaver-acquisitions
  • TypeSafe AI Benchmark — Side-by-side Jev and Qwen-on-Cerebras comparison with raw exports, cost accounting, methodology, and task-specific limitations.

Compute-Use and Mobile-Use

  • mobile-jev One goal. A real Android phone. Jev makes the decisions.
  • agent-desktop Agent Desktop gives any agent reliable computer use on the desktop. Built with Rust, it sees any app's real UI structure through OS accessibility trees and operates it — refs stay stable and actions stay safe to retry, instead of guessing from pixels.
  • typesafe-computer-use Computer use for about $0.0002 a step: OCR the screen, classify the next action with TypeSafe, click. macOS.
  • cua Give AI agents computers they can use. Cua provides open-source desktop automation, isolated cloud desktops, local macOS VMs, and benchmarks for evaluating computer-use agents.

Video

  • jevmeter Put a live Jev (TypeSafe) meter on any video: every sentence scored, rendered as a 16:9 edit

Orchestrator