Microscope Command Server

Socket-based command server for remote microscope control and QuPath integration.

Part of the QPSC (QuPath Scope Control) system. For complete installation and setup instructions, see the QPSC Installation Guide.

Features

• Socket Server: TCP/IP server for remote microscope control
• QuPath Integration: Designed for QuPath annotation-driven acquisition
• Client Library: Python functions for stage control and acquisition
• Acquisition Workflows: Multi-tile, multi-modality acquisition orchestration
• Real-time Monitoring: Progress tracking and cancellation support
• Multi-Channel Widefield IF / BF+IF: Vendor-agnostic channel library driven by Micro-Manager ConfigGroup presets and device property writes
• Demo mode: --demo runs the whole server against a simulated microscope, with no Micro-Manager and no hardware

Demo Mode (--demo)

microscope-command-server --demo

starts the server without Micro-Manager. Run it in the environment that holds the QPSC packages (activate it first, or call the script by path, for example C:\Users\<you>\venv_qpsc\Scripts\microscope-command-server.exe --demo). Within a second or two the console prints DEMO MODE: no Micro-Manager, driving a simulated microscope and Micro-Manager core: SIMULATED (demo mode, no hardware), then the server listens on port 5000 on all interfaces. Stop it with Ctrl+C. ERROR: Another server instance is already running on port 5000! means a server is already up. QPSC's demo connect reuses it only if it reports one of the demo cameras (DCam or DMonoCam), and refuses one with a real camera.

Use --demo to develop or test a handler, an acquisition workflow or autofocus logic on any machine: every handler runs unchanged, and the slide pattern is seeded, so a stage position always returns the same content. The Pycro-Manager Core and Studio are replaced by microscope_command_server.demo.virtual_microscope.DemoCore and DemoStudio; everything above them -- the camera and stage classes in microscope_control, every socket handler, the acquisition workflow, autofocus -- is the production code. The core answers the subset of the MMCore API this code base calls: device properties with allowed values and limits, an XY stage and a focus stage with simulated travel time (device_busy is true while a move is under way), two cameras that snap and stream through a circular buffer (switched with Core-Camera, as a profile switch does), ROI and binning per camera, config groups. Unknown devices and properties raise, as MMCore does.

The camera looks at a procedurally generated H&E-like slide (bilinearly interpolated lattice value noise summed over two to three octaves), 40 x 20 mm centered on the stage origin, with tissue islands, texture and nuclei on a focal surface that tilts and undulates between about -22 and +20 um of Z = 0. Moving the stage pans across it; moving Z defocuses it in proportion to the distance from that surface, so an autofocus sweep scores one clear peak. The slide pattern is deterministic (VirtualSlide.seed); each frame adds Gaussian read noise of sigma 2 gray levels from a generator seeded once per server start, so runs repeat in content, not bit for bit. Bare glass renders near-white (about 243 of 255) at 10 ms exposure and lamp intensity 25, just under clipping.

The same slide carries two more layers, seen through a second, monochrome camera DMonoCam, so the bundled config offers three modalities that switch camera the way a two-camera scope does:

• Fluorescence: a four-line epi LED DLED (385, 475, 550, 621 nm) excites four fluorophore maps laid over the same tissue: nuclei (a DAPI stand-in), weak autofluorescence, eosin (which does fluoresce in a TRITC set) and sparse puncta in part of the tissue (a Cy5 stand-in).
• LC-PolScope: on the transmitted path the camera sits behind a liquid-crystal universal compensator MeadowlarkLC and a circular analyser, modelled as polscope_library.calibration models them (polarizer at 0 deg, LC-A at 45 deg, LC-B at 0 deg). The specimen has a birefringence map: fibrous stroma at 3 to 18 nm with a slowly wandering slow axis, isotropic nuclei and glass. Five states reconstruct to that map with either polscope_library method (tested: median error under 0.1 nm, orientation under 1 deg, with a background). The instrument is deliberately imperfect in the ways a real one is: each crystal lands 0.004 / -0.012 waves from what it is told, the optics add a field-fixed residual retardance of about 0.6 to 2.7 nm (so uncorrected tiles disagree at their seams and a background collected on glass fixes it), 99.25% of the light stays polarized (extinction ratio about 265), and the adapter's Wavelength scales every retardance by Wavelength / 549 -- set it to 546 to watch the readiness check and the acquisition log complain. The bundled palette is the demo compensator's calibrated one; Calibrate Liquid Crystals finds it again.

All of these are DemoCore and VirtualSlide constructor defaults; no command-line flag or YAML key changes them.

Quantity	Value
Pixel size	0.65 um/px (a 6.5 um sensor pixel at 10x); binning 1, 2 or 4 scales it
Sensor	512 x 512 px, a 332.8 x 332.8 um field
Pixel types	32bitRGB (default, 8-bit RGB) and 8bit (mono)
Exposure	0.1 to 10000 ms; brightness is linear in (exposure / 10 ms) x (intensity / 25)
Lamp DLamp	State 0/1, Intensity 0 to 100
Gain	accepted, -5 to 8, no effect on the image
Defocus blur	Gaussian, sigma 0.35 px per um from the focal surface, capped at 30 px (about 86 um out)
Stage speeds	XY 12000 um/s (any move on the 50 mm stage finishes inside the 5 s wait timeout), Z 800 um/s, minimum move 20 ms; device_busy is true while moving
Live frames	one per max(exposure, 10 ms); 32-frame circular buffer
Mono camera DMonoCam	640 x 512 px at 0.65 um/px, 12-bit in 16bit (or 8bit), offset 100 counts, read noise 3 counts plus shot noise; Gain 0 to 24 dB; Gain Auto Off/Continuous and Gamma behave as their names say, so the LC-PolScope camera checks have something to catch
Epi LED DLED	State 0/1; Enable-<line> On/Off and Intensity-<line> 0 to 100 for 385nm, 475nm, 550nm, 621nm; 96 counts per ms at full intensity for a fluorophore density of 1
LC MeadowlarkLC	Retardance LC-A [in waves], Retardance LC-B [in waves] 0 to 1; read-only ... [in nm]; Wavelength 400 to 800 nm (549 by default, the lamp's filter)
Transmitted, mono	2300 counts per ms at lamp intensity 100 through clear glass before the analyser; at the shipped palette and 50 ms the swing states read about 1530 of 4095 on glass (extinction about 530) and the brightest birefringent stroma about 70% of full scale

Not simulated: a shutter device, a rotation stage, Z sequencing, a second objective, vignetting or illumination non-uniformity, liquid-crystal settling time, stage backlash or drift. wait_for_device times out after TimeoutMs (5000 ms by default).

Device labels follow Micro-Manager's Demo adapter where it has an obvious counterpart (DCam, DStage) and the Meadowlark adapter for MeadowlarkLC and its property names; XYStage, DLamp (State, Intensity), DMonoCam and DLED are this core's own labels. Running config_Demo.yml against a real Micro-Manager Demo configuration has not been tried. That config (with its autofocus, imageprocessing and resources sidecars) ships inside the QPSC extension, which extracts it and starts this server when you choose Extensions > QP Scope > Connect to a Demo Microscope...; --demo is what that menu item passes. start_server.bat forwards its arguments, so start_server.bat --demo works too and its banner then says so instead of asking for Micro-Manager. A SNAP against a scope with no rotation stage (the demo, or any brightfield-only scope) ignores --angle instead of failing.

Multi-Channel Acquisition (Widefield IF, BF+IF)

The BGACQUIRE command now supports a vendor-agnostic channel-based acquisition branch used by widefield immunofluorescence (IF) and combined brightfield + IF (BF+IF) workflows. When a command carries --channels and --channel-exposures in place of --angles / --exposures, the single-image tile loop iterates the resolved channel plan once per tile position, writing one TIFF per channel per tile. The Python server does not know anything about specific illuminators; it drives the hardware entirely through core.setConfig(group, preset) and core.setProperty(device, property, value), so the same code path serves CoolLED, Lumencor, DLED, Colibri, and custom builds without modification.

See the cross-repo overview at QPSC/docs/multichannel-if-overview.md for the full pipeline description, YAML schema, and end-to-end BF+IF example. This section covers only the Python server's slice of the pipeline.

New BGACQUIRE flags

The BGACQUIRE (and ACQUIRE) acquisition message parser accepts optional flags on top of the existing angle-based flags:

For multi-channel acquisition (widefield IF, BF+IF):

• --channels "(id1,id2,...)" -- ordered list of channel ids to acquire at every tile position. Ids must match entries in the modality's channel library declared in the microscope YAML.
• --channel-exposures "(exp1,exp2,...)" -- per-channel exposures in milliseconds. Must be the same length and order as --channels; missing or non-positive entries fall back to the channel library's default exposure_ms.

For background collection:

• --profile <key> -- acquisition profile key to apply for background collection (e.g., profile=Brightfield_10x or profile=IF_40x). If omitted, the profile is resolved automatically from modality and objective. The profile determines which camera, illumination intensity and device properties are applied during background collection.
• --channels "(id1,id2,...)" -- when used with BGACQUIRE, enables per-channel background collection: the server collects one background image per channel instead of per-angle, saving as {channel_id}.tif under the output folder. Useful for widefield illumination systems where per-channel backgrounds (with channel-specific optics and illumination) better represent the correction needed during acquisition.

When --channels is present for acquisition, the server takes the channel acquisition branch in acquisition/workflow.py. --channels is mutually exclusive with --angles. If both are supplied (for example, a stale angle field from an older client), the server logs a warning and clears the angles so the channel path is the single source of truth.

How it works

Three helpers in microscope_command_server/acquisition/workflow.py implement the channel path:

• resolve_channel_plan(ppm_settings, scan_type, channel_ids, channel_exposures) -- resolves the profile (acquisition_profiles.<scan_type>), looks up its modality, then reads modalities.<modality>.channels from the YAML and filters / reorders to the requested ids. For each channel it merges in the profile's channel_overrides.<id>.device_properties and returns an ordered list of channel plan dicts containing id, display_name, exposure_ms, mm_setup_presets, device_properties, and optional settle_ms.
• _merge_device_property_overrides(library_props, override_props) -- private helper mirroring the Java-side merge rule (MicroscopeConfigManager.mergeDevicePropertyOverrides) exactly: match by (device, property) tuple, replace the value in place when matched, append to the end of the list otherwise. This lets a profile tune one property on one channel with a single YAML line without redeclaring the whole channel.
• apply_channel_hardware_state(hardware, channel_plan_entry, logger_) -- applies mm_setup_presets via core.setConfig(group, preset) followed by core.waitForConfig, then applies device_properties via core.setProperty(device, property, value) and calls core.waitForDevice on every touched device. This is the critical settle pass that stops back-to-back channel transitions from racing the camera snap on serial LED controllers. An optional settle_ms field on the channel entry adds a dumb-sleep fallback for hardware whose isBusy() reports complete too early (some filter turrets, reflector wheels, serial LED controllers).

Inside the "Single image acquisition: no rotation angles" block of the tile loop, the server checks for a non-empty channel plan. If present, it iterates the plan for the current tile position: apply channel state, set the channel exposure, snap, (optionally) flat-field correct, saturation-check, and write the per-channel TIFF. The tile loop then continues past the default single-snap path. If no channel plan is resolved, the tile loop falls back to the existing single-snap behavior -- see "Backward compatibility" below.

File layout on disk

Per-tile the channel branch writes one TIFF per channel into a per-channel subdirectory under the existing annotation output folder:

{projectsFolder}/{sample}/{scan_type}/{annotation}/
    {channel_id_1}/tile_0_0.tif
    {channel_id_1}/tile_0_1.tif
    {channel_id_2}/tile_0_0.tif
    {channel_id_2}/tile_0_1.tif
    ...
    TileConfiguration.txt

This mirrors the PPM per-angle layout exactly -- channel ids double as subdirectory names. The stitcher (qupath-extension-tiles-to-pyramid) can then isolate each channel at stitch time by pointing its existing per-axis stitching helper at each channel subdirectory, without any channel-aware logic in the stitcher itself.

Per-channel background correction (opt-in)

When the tile loop loads background images for an acquisition, the channel branch additionally looks for per-channel flat-field images under the background directory:

{background_dir}/{channel_id}/background.tif

Any channel whose file is present is flat-field corrected via BackgroundCorrectionUtils.apply_flat_field_correction using the configured method (divide by default). Channels whose file is missing are skipped silently -- they are acquired without correction. The loader also accepts the flat alternates {background_dir}/{channel_id}.tif and {channel_id}.tiff for convenience.

This is the channel-axis analog of the PPM per-angle background path: the key is the channel id rather than the rotation angle, but the correction call and the missing-file behavior are the same.

Backward compatibility

BGACQUIRE commands that do not pass --channels fall through unchanged:

• Commands with --angles take the multi-angle branch (PPM and similar).
• Commands with neither angles nor channels take the default single-snap branch (brightfield, single-snap fluorescence, laser scanning).

Modalities whose YAML has no channels: library never enter the channel branch, so existing profiles keep working without any YAML edits.

Acquisition Loop Ordering (--inner-axis)

The BGACQUIRE (and ACQUIRE) acquisition message parser accepts an optional --inner-axis flag to control the nesting order of hardware sweeps. This affects performance by changing the frequency of expensive hardware transitions (rotation moves for PPM, filter-cube changes for widefield).

Flag syntax

--inner-axis <value>

Allowed values:

• z -- Z-position is the innermost loop
• channel -- Channel is the innermost loop
• angle -- Rotation angle is the innermost loop

PPM (Multi-Angle) Acquisition

Default behavior (--inner-axis angle or omitted): z-outer / angle-inner

• Outer loop: z-planes
• Inner loop: angles
• Effect: At each z-plane, every angle is re-acquired before z advances. Tight per-z registration across angles, but a 5-z x 4-angle field pays 20 rotation-stage moves per tile. This is the historical PPM ordering.

Alternative mode (--inner-axis z): angle-outer / z-inner

• Outer loop: angles
• Inner loop: z-planes
• Effect: Each angle sweeps its full z-stack before rotating to the next angle. Fewer rotation moves (one per angle per tile instead of one per angle-z pair). The per-angle WB / JAI-calibration / exposure block also hoists outside the inner z loop. Faster for thicker tissue z-stacks (especially at 40x).

Widefield (Multi-Channel) Acquisition

Default behavior (--inner-axis z or omitted): channel-outer / z-inner

• Outer loop: channels
• Inner loop: z-planes
• Effect: Each channel sweeps its full z-stack before switching to the next channel. Fewer filter-cube changes (one per channel per tile instead of one per channel-z pair). Optimized for fixed slides where focus drift is not a concern.

Alternative mode (--inner-axis channel): z-outer / channel-inner

• Outer loop: z-planes
• Inner loop: channels
• Effect: At each z-plane, every channel is re-acquired before z advances. Tighter per-channel z registration -- right for live-cell or drifting samples where minutes between channel acquisitions could cause axial decorrelation. Costs channels x z_planes filter switches per tile.

Backward compatibility

Omitting --inner-axis preserves byte-identical behavior to pre-flag acquisitions:

• PPM defaults to z-outer / angle-inner (the historical ordering)
• Widefield defaults to channel-outer / z-inner (fewer filter changes)

Z-Stack Acquisition (--z-stack, --z-projection)

The BGACQUIRE (and ACQUIRE) acquisition message parser accepts optional flags to control Z-stack (axial) image acquisition and output format.

Flag syntax

Z-stack enable and range:

--z-stack true
--z-start <micrometers>
--z-end <micrometers>
--z-step <micrometers>

Projection and output format:

--z-projection <method>

Z-stack parameters

• --z-stack true|false -- Enable Z-stack acquisition (acquire multiple planes at different focal depths). Default: false. When enabled, requires --z-start, --z-end, and --z-step.
• --z-start <um> -- Starting Z position (micrometers). Often a negative value (e.g., -5.0) to acquire above the focal plane.
• --z-end <um> -- Ending Z position (micrometers). Often a positive value (e.g., +5.0) to acquire below the focal plane.
• --z-step <um> -- Distance between successive planes (micrometers). Typical values: 0.5 to 2.0 depending on sample refractive index and objective.

The server computes the number of planes as ceil(|z_end - z_start| / z_step).

Z-projection methods

• max -- Maximum intensity projection (default). Each tile output is a single 2D image (the brightest pixel at each xy position across the Z-stack).
• mean -- Mean intensity projection. Single 2D output per tile.
• edf -- Extended Depth of Field (EDF) projection. Computes a single 2D image by selecting pixels from the Z-stack according to a sharpness metric (see "EDF tuning parameters" below). Each tile output is a single 2D image.
• none -- Preserve individual Z-planes without projection. Output structure changes to a per-plane layout, enabling 5D stitching (x, y, z, channel/angle, time). See "File layout on disk" below.

EDF tuning parameters

When using --z-projection edf, three optional parameters control the sharpness metric and filtering behavior:

• --edf-metric <name> -- Per-pixel sharpness map (default: tenengrad). Valid values are exactly tenengrad, modified_laplacian and variance -- the names in microscope_imageprocessing.focus.sharpness_maps. Anything else raises rather than silently substituting, so a typo fails before the acquisition starts. tenengrad matches the autofocus metric of the same name; modified_laplacian peaks more sharply in Z but is noisier; variance is the most forgiving of noise. Only meaningful with edf.
• --edf-window <pixels> -- Averaging window for the sharpness map, in pixels (integer, >= 1; default 9). Raw per-pixel sharpness is too noisy to choose a plane from, so it is averaged over this window first -- the setting that matters most. Raise it if the fused output looks blocky or speckled; lower it if boundaries between in-focus regions look smeared. Scales with pixel size. Only meaningful with edf.
• --edf-index-smooth <pixels> -- Median-filter size applied to the map of which plane each pixel selected, in pixels (integer, >= 0; default 5; 0 disables). Real focal surfaces are smooth, so this removes pixels that chose an odd plane for no physical reason. Raise it for a tilted but flat sample; lower it where focus genuinely steps (a fold, a torn section), because a large median bridges the step and picks a plane sharp on neither side. Only meaningful with edf.

The defaults are reasoned starting points, not measured optima.

Example:

--z-projection edf --edf-metric tenengrad --edf-window 9 --edf-index-smooth 5

File layout on disk

When --z-projection max|mean (default projection behavior): Single 2D image per tile per channel/angle:

{projectsFolder}/{sample}/{scan_type}/{annotation}/
    {angle_or_channel}/tile_000.tif
    {angle_or_channel}/tile_001.tif
    ...

When --z-projection none (preserve individual planes): Each Z-plane is written to a per-Z-plane subdirectory. For single-timepoint acquisitions, the layout is {group}/z{zz}/{filename}:

{projectsFolder}/{sample}/{scan_type}/{annotation}/
    {angle_or_channel}/z000/tile_000.tif
    {angle_or_channel}/z000/tile_001.tif
    {angle_or_channel}/z001/tile_000.tif
    {angle_or_channel}/z001/tile_001.tif
    ...

For time-lapse acquisitions (with --timepoints > 1), a t{tt}/ segment is added to track each timepoint, enabling reassembly into a true 5D mosaic:

{projectsFolder}/{sample}/{scan_type}/{annotation}/
    {angle_or_channel}/t000/z000/tile_000.tif
    {angle_or_channel}/t000/z001/tile_000.tif
    ...
    {angle_or_channel}/t001/z000/tile_000.tif
    {angle_or_channel}/t001/z001/tile_000.tif
    ...

The stitcher (qupath-extension-tiles-to-pyramid) can then assemble all planes (and timepoints) into a unified 5D (x, y, z, channel/angle, time) mosaic by reading these directory structures.

Use case: 5D confocal / light-sheet stacks

Set --z-projection none when acquiring thick (volumetric) samples that require a full Z-stack for analysis downstream (e.g., volume rendering, 3D reconstruction, confocal microscopy). Single-timepoint 3D stacks produce {group}/z{zz}/ layouts byte-identical to the existing --save-raw behavior. Time-lapse 3D stacks add t{tt}/ segments for temporal registration and 5D reassembly.

Backward compatibility

Omitting --z-stack or setting it to false disables Z-stack acquisition entirely. Omitting --z-projection defaults to max (maximum intensity projection), preserving the original single-image-per-tile output.

LC-PolScope Acquisition (Automatic Birefringence Reconstruction)

LC-PolScope acquisitions automatically reconstruct birefringence (retardance and orientation) maps from polarization states when properly configured. The server queues reconstruction for each tile after all state images are acquired.

Configuration Requirements

Reconstruction requires three settings in the microscope YAML under modalities.lcpolscope.reconstruction:

modalities:
  lcpolscope:
    reconstruction:
      swing_waves: 0.03          # Calibration swing amplitude (required)
      wavelength_nm: 546         # Light wavelength in nanometers (required)
      scheme: "5-State"          # Polarization scheme; default "5-State"
      state_order:               # Optional: state reordering before reconstruction
        - State0
        - State1
        - State2
        - State4
        - State3

• swing_waves: The calibration swing amplitude used during system calibration. A fixed value per microscope.
• wavelength_nm: Illumination wavelength in nanometers. Must match the wavelength used during acquisition and calibration.
• scheme: Reconstruction scheme (default "5-State"). The only accepted values are "5-State" and "4-State"; the scheme is fixed by how the system was calibrated and is not a free choice at acquisition time.
• state_order (optional): A permutation of the acquired state ids that reorders them before inversion. Used when the acquisition order differs from the calibration order (e.g., when using OpenPolScope, which acquires in the Oldenbourg order with pairs (1,4) and (2,3), requiring State3 and State4 to swap). Omitting this field means states are consumed in acquisition order, which is correct for recOrder and similar tools. Must list every acquired state exactly once; specifying a partial list or unknown state ids will raise an error and skip reconstruction.

If this block is missing or incomplete, the server logs a warning and skips reconstruction; raw state images are still saved and can be reconstructed offline.

State Image Acquisition

Acquire 4 or 5 polarization states (depending on your scheme) as separate channels using the standard multi-channel acquisition flags:

--channels "(State0,State1,State2,State3,State4)"
--channel-exposures "(100,100,100,100,100)"

The state ids must match exactly (case-sensitive); Micro-Manager presets should map to these state names in your configuration.

One critical invariant that fails silently: The Stokes inversion treats the state intensities as samples of a single radiometric scale, so a per-state difference biases the result.

• All states must share one exposure and gain. Several layers cooperate to hold this: the QuPath extension equalises the exposures before sending them, every channel in config_LCPolScope.yml carries the same exposure_ms, and the LC-PolScope acquisition profiles deliberately carry no channel_overrides. Do not add per-channel exposure tuning to any of them. The server also checks the plan before anything moves: an acquisition that leaves out one of the configured states, adds a non-state channel, or gives the states unequal exposures (more than 0.01 ms apart) is refused at the start with an error naming the problem. Requests that omit --channels are not checked here.

State order is configurable. The state_order parameter in the YAML reorders states before inversion to match the calibration scheme. When omitted, states are consumed in acquisition order (correct for recOrder). A wrong permutation rotates or mirrors the orientation map without raising an error. If you did not run the calibration yourself, identify it from the data with polscope-scheme-check (shipped with polscope-library) before trusting any orientation output. The correct order is a property of the software that ran the calibration -- not of the microscope -- so it can change without any hardware changing, and must be re-checked after a software switch.

Automatic Reconstruction Behavior

After all state images for a tile are acquired and saved:

1. The server validates the state set (4 or 5 images, in calibration order)
2. If valid, reconstruction is queued in the background write pool
3. Retardance and orientation maps are computed and written as OME-TIFF files
4. A refusal to reconstruct does not abort the acquisition — raw states remain saved and can be reconstructed offline

Constraints and Validation

Background Correction: QLIPP applies the background in Stokes space, using background intensities passed to the reconstruction library. Flat-field division of the raw state tiles is a different operation and biases retardance and orientation with no visible symptom -- the images still look right.

The server routes this correctly on its own: for LC-PolScope the acquisition loop skips the flat-field divide, so raw states reach the reconstruction, which applies the background itself. No special flag selects this behaviour; it follows from the modality.

You do still have to ask for backgrounds. Per-channel background images are only loaded when the acquisition is sent with --bg-correction true -- placing files in the background folder without that flag has no effect.

A partial background set is refused. The correction is not a per-state subtraction. The background states are inverted to Stokes parameters, an attenuating-depolarizing-retarder Mueller matrix is estimated from them, and its inverse is applied to the sample's Stokes vector. All states feed that one matrix, so a missing state corrupts the entire correction rather than degrading a single channel. Supply a background for every state, or none. Reconstructing with no background at all is valid, just lower quality.

Refusals do not abort the acquisition. Raw state images are still saved, so the run can be reconstructed offline afterwards, and a refusal is logged once per acquisition rather than once per tile.

Output File Layout

Derived retardance and orientation maps are written to sibling directories alongside the state images:

{projectsFolder}/{sample}/{scan_type}/{annotation}/
    State0/tile_0_0.tif
    State1/tile_0_0.tif
    State2/tile_0_0.tif
    State3/tile_0_0.tif
    State4/tile_0_0.tif
    retardance/tile_0_0.tif          # Retardance (uint16, hundredths of nm)
    orientation/tile_0_0.tif         # Orientation (uint16, hundredths of degrees, [0, 18000) = [0, 180) degrees)
    TileConfiguration.txt

Each is a tile directory in its own right, with a TileConfiguration.txt copied from the state directories, so the stitcher (qupath-extension-tiles-to-pyramid) stitches each into its own mosaic.

Each derived tile file carries OME-XML metadata recording the reconstruction parameters (wavelength, swing waves, scheme, state order), whether background correction was applied, and critical handling rules for downstream processing. This metadata travels with the file, so a reader can identify both the calibration used to produce it and the processing constraints that must be respected.

Orientation is axial data and must not be resampled as an ordinary scalar. 0 and pi are the same physical orientation, so the mean of 179 degrees and 1 degree is 90 degrees -- perpendicular to the truth, and entirely plausible-looking. Anything that averages these pixels (blending in a stitch seam, pyramid downsampling) has to go through sin(2theta)/cos(2theta), or encode the angle as hue, rather than averaging the angle directly. Retardance is an ordinary scalar and has no such constraint. The metadata in the OME-XML flags which channel is axial and how it may be resampled.

Backward Compatibility

Omitting the modalities.lcpolscope.reconstruction block disables reconstruction. Raw state images are still acquired and saved normally, and reconstruction can be performed offline using the polscope-library directly.

LC-PolScope readiness (LCREADY)

No payload. Replies with a 4-byte big-endian length and UTF-8 JSON: camera (the summary camera_snapshot takes), camera_text, camera_fatal (auto gain or exposure on), camera_advisory (gamma, 8-bit), adapter_wavelength_nm and reconstruction_wavelength_nm. Read-only. QPSC shows it in the LC-PolScope acquisition panel before a run; the same camera checks also run at the start of the calibration and on the first state of every acquisition.

LC-PolScope state signal (LCSIGNAL)

Payload: 32-byte profile name, then two big-endian doubles, exposure_ms (0 or less: the camera's current exposure) and the target fraction of full scale (0 or less: 0.75). Applies every state of the profile's channel library, each at that ONE exposure, snaps it, and replies like LCREADY with states[] (id, mean, p999, max, saturated_fraction, extinction), full_scale, brightest_state, brightest_fraction, clipped, suggested_exposure_ms (the exposure that puts the brightest swing state's 99.9th percentile at the target, above reconstruction.black_level; null when clipped or dark) and the camera problems. QPSC's Live Viewer Camera tab uses it to choose the one exposure all states share. Implementation: calibration/lc_state_signal.py.

Micro-Manager preset (MMPRESET)

Request and reply are a 4-byte big-endian length plus UTF-8 JSON. {"action": "get", "group", "preset"} reports the group's current preset, whether the group and preset exist, and the names that do (presets, or groups when the group is missing); "set" applies the preset and adds applied, previous and the resulting pixel_size_um. A set is refused (an error field) while an acquisition thread runs or before CONFIG, because a preset can turn a motorized turret. QPSC uses it to keep Micro-Manager's objective preset in step with the objective chosen in QPSC (mm_preset in the microscope YAML). Implementation: server/handlers/presets.py.

When Micro-Manager reports pixel size 0, GETPXSZ also logs (once per state) every pixel-size preset with the properties it needs and their current values, e.g. '20x 0.4 ORYX' needs DObjective.Label=20x (now 4x) <- differs.

LC-PolScope Calibration (LCCALIB)

The LCCALIB command calibrates the liquid crystals on an LC-PolScope system, finding the extinction point and swing states. It produces a calibration palette and metadata file that guides all subsequent LC-PolScope acquisitions.

When to calibrate

• First setup: After installing or aligning new liquid crystals
• Configuration change: When wavelength or scheme changes
• Whenever the extinction ratio drops: it is the health metric, so a falling value is the signal to recalibrate. How often that happens on this rig is not yet known.

Calibration does not need the microscope left in the right state. Before it searches, it applies the first state's hardware state (its lamp, light-path presets and crystal settings) and, on a two-camera scope whose modality uses one detector, switches to that camera. Starting it straight after a fluorescence or brightfield profile therefore measures the LC-PolScope field, not a dark one.

Command syntax

LCCALIB --yaml config.yml --output /path/to/output [--modality lcpolscope] \
        [--swing 0.03] [--scheme "5-State"] [--wavelength 546.0] \
        [--black-level 100.0] [--exposure-ms 50] [--strategy single_pass] ENDOFSTR

Parameters

• --yaml config.yml (required) -- Microscope configuration file. The server reads modalities.<modality>.reconstruction for default swing, wavelength, scheme, and settle time if not overridden on the command line.
• --output /path/to/output (required) -- Folder where the calibration palette and metadata file are written. The server creates this folder if it doesn't exist.
• --modality lcpolscope (optional) -- Which modality to calibrate. Default: lcpolscope. Only relevant if the YAML defines multiple LC-PolScope configurations.
• --swing 0.03 (optional) -- Calibration swing amplitude in waves. If omitted, taken from modalities.<modality>.reconstruction.swing_waves in the YAML.
• --scheme "5-State" (optional) -- Polarization scheme to calibrate. Accepted values: "5-State" or "4-State". If omitted, taken from the YAML; defaults to "5-State". The scheme is fixed by your hardware and calibration procedure; it is not a free choice at runtime.
• --wavelength 546.0 (optional) -- Illumination wavelength in nanometers. If omitted, taken from the YAML; defaults to 546.0 nm with a warning. The default is used only as a placeholder when no wavelength is configured; it does not apply silently, so calibration and acquisition can be compared for agreement.
• --black-level 100.0 (optional) -- Dark-frame intensity used to correct the raw measurements. If omitted, the calibration falls through its black-level chain, in this order: (1) an explicit value here or in the YAML, (2) an averaged dark frame if lamp control is configured, (3) zero, with a warning. The value matters: a 50-count error moves the extinction ratio by roughly 10%.
• --exposure-ms 50 (optional) -- Camera exposure for the calibration frames. Set for the run and restored afterwards, on every exit path. If omitted the calibration snaps at whatever Micro-Manager is set to. Either way the exposure used is recorded in the reply (camera.exposure_ms, camera.exposure_source request or micro-manager). The palette does not depend on exposure, so this never changes the acquisition's exposure in the YAML; it only has to be long enough for the field to register -- see the too-dark refusal below.
• --strategy single_pass (optional) -- Search strategy. single_pass (default) makes one pass per state; iterative repeats, re-centring on the crystals, until the residual is small enough. The second costs roughly three times the exposures, so it is a trade rather than an upgrade.

The black level is also measured on every run. After the search, the crystals are held at extinction and four frames-averaged snaps are taken at 25, 50, 75 and 100% of the calibration exposure. Light scales with exposure and the camera offset does not, so the intercept of a straight-line fit is the offset: the black level without blocking the lamp, which this scope cannot do. The reply carries it as black_level_estimate (black_level, slope_per_ms, r_squared, residual_rms, the exposures and means); a configured value more than 2 counts (or 2%) away is warned about, and QPSC's Apply offers to write the measured one. The fit doubles as a linearity check: R^2 under 0.999 means the camera is not linear (gamma, or gain changing between frames), adds camera response not linear to reliability_issues, and makes the result unreliable.

The black level must be in the counts of the camera's current pixel format. OpenPolScope's stored 103 for this instrument was recorded in its own format; the same camera in 8-bit, or with a different offset setting, has a different one. A black level above the darkest state measured is reported as a warning, with the camera state, and makes the extinction ratio unmeasurable.

Response sequence

The command sends multiple responses as it progresses:

1. STARTED:<output_folder> -- Calibration has begun; acknowledges the output path the operator requested.

2. PROGRESS:<current>:<total>:<message> -- Updates during the search, in the same four-field shape PPMBIREF uses.

3. SUCCESS:<json> -- Calibration succeeded. The JSON payload always contains:

   {
     "success": true,
     "scheme": "5-State",
     "swing_waves": 0.03,
     "wavelength_nm": 546.0,
     "wavelength_source": "request|config|fallback",
     "lc_control_mode": "MM-Retardance",
     "strategy": "single_pass",
     "black_level": 100.0,
     "black_level_source": "request|config: modalities.lcpolscope.reconstruction.black_level|measured|assumed",
     "extinction_ratio": 150.5,
     "assessment": "good|acceptable|poor|unmeasurable|unreliable",
     "assessment_by_ratio": "good|acceptable|poor|unmeasurable",
     "reliability_issues": [],
     "palette_waves": {
       "State0": [0.2500, 0.5000],
       "State1": [0.2200, 0.5000],
       "...": "[LC-A, LC-B] in waves, one entry per state"
     },
     "lc_device": "MeadowlarkLC",
     "lc_retardance_properties_waves": {"LCA": "Retardance LC-A [in waves]", "LCB": "Retardance LC-B [in waves]"},
     "lc_adapter_wavelength_nm": 546.0,
     "calibrated_at": "2026-10-12T14:03:00",
     "state_intensities": {...},
     "exposures": 42,
     "elapsed_s": 15.3,
     "camera": {"device": "Oryx ORX-10G-51S5M", "exposure_ms": 50.0, "exposure_source": "request|micro-manager",
                "bit_depth": 12, "significant_bits": 12, "bytes_per_pixel": 2, "width": 2448, "height": 2048,
                "binning": "1", "pixel_type": "Mono12", "gain": "0"},
     "signal": {"extinction": 410.2, "brightest_state": 1890.5, "grid_peak": 3950.0,
                "full_scale": 4095.0, "grid_peak_fraction": 0.9646,
                "extinction_above_black": 307.2, "swing_contrast": 1480.3, "swing_contrast_fraction": 0.3615,
                "state_saturated_fraction": {"State0": 0.0, "...": 0.0}, "state_brightest_pixel": {"State0": 612.0, "...": 0.0},
                "acquisition_exposure_ms": 50.0, "projected_swing_contrast": 1480.3, "projected_brightest_pixel": 2210.0,
                "usable_exposure_ms": [1.4, 83.4], "histogram_path": "/path/to/output/lc_calibration_YYYYMMDD_HHMMSS_histograms.png",
                "frame_dtype": "uint16", "frame_max_seen": 4095.0},
     "warnings": [],
     "output_folder": "/path/to/output",
     "metadata_path": "/path/to/output/lc_calibration_YYYYMMDD_HHMMSS.json"
   }

   Key fields:

   • success: true always indicates a palette was produced
   • wavelength_nm is the wavelength used for reconstruction (in nanometers)
   • wavelength_source indicates where the wavelength came from: "request" (command-line parameter), "config" (microscope YAML), or "fallback" (default 546.0 nm when neither is set; a warning is logged in this case)
   • extinction_ratio is the quality metric; higher is better (>100 is good)
   • assessment is the headline. It is the ratio's band (assessment_by_ratio) unless something about the frames means the ratio cannot be trusted, in which case it is unreliable and reliability_issues names why: a clipped state, a swing contrast under 1% of range, a black level above the darkest state (wrong units), a camera gamma curve, 8-bit frames. The 2026-10-16 rig run reported good at ER 133.8 with State1 at 99.9% saturation under auto gain; that is now unreliable, and QPSC asks before applying it
   • assessment_by_ratio is the band the extinction ratio alone gives. reliability_issues is [] when nothing overrode it; otherwise it lists each reason, as short strings such as "State1 clipped", "swing contrast under 1% of range", "black level above the darkest state", "camera gamma curve" or "8-bit frames"
   • palette_waves contains the retardance values (in waves) for each state and LC axis, keyed by the channel the state becomes
   • lc_device and lc_retardance_properties_waves name the Micro-Manager device and properties QPSC writes the palette to, so the property names live in one place (the server)
   • The calibration sets the LC adapter's Wavelength property to wavelength_nm before it measures, and fails if the adapter does not read it back. The adapter converts the palette's waves to volts at that property; Micro-Manager restarts and OpenPolScope both reset it (on the LC-PolScope to 546, against a 549/15 lamp filter), so it is set every run. lc_adapter_wavelength_before_nm is what it was; lc_wavelength_property names it, and QPSC's Apply writes lc_adapter_wavelength_nm ahead of every state's retardances so the acquisition replays the palette at the conversion it was measured with
   • lc_adapter_wavelength_nm is the LC adapter's own wavelength setting, if it exposes one. The palette is in the adapter's waves, so a value that differs from wavelength_nm scales every retardance by the ratio; the server adds a warning saying so. The full adapter property snapshot is in the metadata file (lc_device_properties), not the reply
   • camera is the camera as the frames were taken: device, exposure and where it came from, bit depth, significant_bits (the smaller of Micro-Manager's bit depth and the number in the pixel format name, so a Mono12p unpacked into a 16-bit buffer is 12 and full scale is 4095; every range check uses it), bytes per pixel, frame size, and binning / pixel type / gain when the adapter exposes them (matched loosely by property name; a camera without one does not report it). Every camera property as a string is in the metadata file (camera_properties), not the reply
   • signal is what the frames held, in raw counts: the extinction state, the brightest swing state, the brightest frame of the extinction grid, and that peak as a fraction of the camera's full scale (2^significant_bits - 1). This is the one-line answer to "was the field bright enough" -- the first rig runs peaked at 16.7 counts. It also carries the two numbers a palette stands or falls on, and three checks on them:
     • extinction_above_black is what the extinction ratio divides by; swing_contrast (brightest swing state minus extinction) is what the swing states were resolved from. By design the contrast is small, sin^2(pi * 0.03) = 0.9% of the open intensity, so a field can pass the grid check and still leave the states a few counts apart. Under 1% of full scale a warning says the palette was resolved from noise.
     • state_saturated_fraction and state_brightest_pixel, per state, from the final frame of each. The far grid points are allowed to clip; the states are not. Over 0.1% of pixels at the top of the range in any state is warned about (a clipped mean is biased low and the search converges on the wrong setting).
     • acquisition_exposure_ms is the one exposure the YAML's channels share (channels that disagree are reported and nothing is projected). projected_swing_contrast and projected_brightest_pixel scale the measured counts to it, and usable_exposure_ms is the range at this illumination that keeps the contrast over 1% and the brightest pixel under 90% of range. A warning names the exposure to set when the acquisition would be dark or would clip. The palette does not depend on exposure; the acquisition does, so a calibration taken at 500 ms that leaves a 50 ms acquisition dark is reported here, not discovered later.
     • histogram_path: a PNG beside the metadata file, one log-scale pixel histogram per state (final frame), with the mean and the black level marked. For the human check a frame mean cannot make: a dim lamp, a closed diaphragm and a slide left in the beam all give a low mean and different histograms
   • warnings lists any issues that don't prevent use (e.g., a poor extinction ratio, or a missing wavelength configuration)
   • metadata_path points to a JSON file with the full trace (per-exposure details)

   Camera settings are checked before the first frame (camera_properties, matched loosely by name). Auto gain or auto exposure on (any *Auto* gain/exposure property not Off) is refused: the camera steers every state to its own brightness target and the search settles wherever that is (2026-10-16: five states at 173 counts, Gain Auto: Continuous). A gamma curve (Gamma Enabled on, or Gamma not 1.0) and an 8-bit pixel format are warned about and mark the result unreliable: the palette is findable, the acquisition is not quantitative. The same check runs on the first state of every LC-PolScope acquisition (modality/lcpolscope.py): fatal cases log an ERROR, the rest a WARNING, and every derived tile carries polscope.camera_exposure_ms, polscope.camera_bit_depth, polscope.camera_significant_bits, polscope.camera_pixel_type, polscope.camera_gain, polscope.camera_binning, polscope.camera_device and, when anything was wrong, polscope.camera_problems.

   The reply is written with success: false and an error instead when the field is too dark to calibrate: the brightest frame of the extinction grid averaged less than a tenth of full scale. A search on a dark field does not fail, it converges on noise and returns a palette that looks like any other (2026-10-13: frame means of 1 to 17 counts, a swing state 0.002 waves from extinction), so the run is stopped after the grid, about 25 exposures in, and the message carries the counts and the camera state. Raise the exposure (--exposure-ms) or the illumination. When the camera reports neither a bit depth nor a bit count in its pixel format name (significant_bits is unknown), the check is skipped and a warning says so.

4. FAILED:<reason> -- Calibration could not complete. The reason string describes the error (e.g., "could not reach the liquid crystals", "polscope_library is not installed").

Extinction ratio interpretation

The extinction ratio is the primary quality metric, computed from the measured intensities at the calibration swing. Higher is better.

• 100 or above: Good. These are recOrder's bands; this rig reached 267.
• 80-100: Acceptable. Usable, but check alignment and optical cleanliness.
• Below 80: Poor. Recorded with a warning rather than rejected.
• Unmeasurable: The search failed to find a valid extinction point. The hardware or configuration may need attention.

A poor result is still returned (not rejected), so the operator can inspect the data and decide whether to retry or investigate the hardware.

Using the calibration result

QPSC applies it: the result dialog's Apply to acquisition writes the palette into the microscope YAML through its comment-preserving editor. For every state, the channel's device_properties become the two retardance writes (lc_retardance_properties_waves, values in waves), its mm_setup_presets are removed (they hold the previous calibration as volts and would be applied before the retardance on every state), and under modalities.lcpolscope.reconstruction the state_order is reset to the palette's order and lc_calibration_file, lc_calibrated_at, lc_extinction_ratio are recorded. QPSC then reloads its configuration and sends RECONFG so this server does too.

The server does not write the YAML itself (PyYAML would drop every comment in it), and nothing here writes Micro-Manager presets: after an apply, the Channel ConfigGroup in Micro-Manager still holds the old calibration and is simply no longer used by QPSC.

Where each value goes -- the full trace, so a value can be followed from measurement to the metadata of a derived tile:

Value	Calibration (LCCALIB reply / file)	Microscope YAML after Apply	Acquisition	Derived-tile metadata
Palette, waves	palette_waves[State]	channels[State].device_properties (retardance in waves)	written per state; read back on the first tile, compared within 0.001 waves	polscope.lc_readback, polscope.lc_readback_agrees
State roles	palette key order (State0 ext, State1 I0, State2 I45, State3 I90, State4 I135)	reconstruction.state_order (identity after Apply)	--	polscope.state_order
Swing, waves	swing_waves (read from the YAML, not chosen here)	reconstruction.swing_waves (unchanged)	--	polscope.swing_waves
Wavelength, nm	wavelength_nm + wavelength_source	reconstruction.wavelength_nm (unchanged)	--	polscope.wavelength_nm
LC adapter wavelength, nm	set to wavelength_nm before measuring (the run fails if the adapter will not take it); lc_adapter_wavelength_before_nm, lc_adapter_wavelength_nm (what the palette was measured at), lc_wavelength_property	written FIRST in each state's device_properties, at the measured value	applied with the state, read back on the first visit; the first state logs it against reconstruction.wavelength_nm (WARNING on a mismatch)	polscope.lc_adapter_wavelength_nm
Black level, counts	black_level + black_level_source (--black-level, else reconstruction.black_level, else a dark frame, else assumed 0)	reconstruction.black_level (set by hand)	--	polscope.black_level
Provenance	calibrated_at, extinction_ratio, metadata_path	reconstruction.lc_calibrated_at, lc_extinction_ratio, lc_calibration_file	logged at the first state of every run, or a WARNING that no calibration was applied	polscope.lc_calibrated_at, polscope.lc_extinction_ratio, polscope.lc_calibration_file
LC adapter state	lc_device_properties (file only)	--	--	--
Camera state	camera (exposure + source, bit depth, size, binning, pixel type, gain); camera_properties (file only); auto gain/exposure refused, gamma / 8-bit -> unreliable	channels[State].exposure_ms (set by hand, identical across states)	snapshot + the same checks on the first state; ERROR / WARNING	polscope.camera_*, polscope.camera_problems
Frame counts	signal (extinction, leakage above black, swing contrast, grid peak, per-state saturation, projection to the acquisition exposure, usable_exposure_ms); *_histograms.png beside the file	channels[State].exposure_ms is what the projection is made for	--	--

LC-PolScope outputs per tile

Each tile's five states are reconstructed during acquisition into three folders beside the state folders, each with its own TileConfiguration.txt so it stitches on the same grid:

• retardance/: uint16, nanometres x 100.

• orientation/: uint16, slow-axis degrees x 100 over [0, 180), axial (declared in the OME metadata, so downsampling and blending average it on the doubled angle).

• orientation_color/: display only, 8-bit RGB. Hue is the orientation, one colour cycle over 0..180 deg, so 0 and 180 are the same colour; brightness is the retardance, full at reconstruction.display_retardance_ceiling_nm (default 20 nm). QuPath cannot put a cyclic colour map on an image channel, so the server renders it. Measure on the other two. reconstruction.display_colormap picks the colours (modality/orientation_colormap.py):

  • uniform (default): a constant-lightness OKLCH wheel (L 0.75, C 0.125), so every orientation looks equally light at equal retardance and brightness means retardance only.
  • hsv: OpenPolScope's look; its yellows and cyans look much lighter than its blues.
  • ppm: the hue a PPM camera image shows for the angle, from display_hue_calibration: {source, slope, intercept, hue_offset} copied from a PPM sunburst .npz (slope forced to one cycle per 180 deg so the map stays cyclic).

  display_orientation_sign (+1/-1) and display_orientation_offset_deg register this instrument's angles to the borrowed frame before colouring (default identity). Each colour tile records the map in polscope.colormap. An unknown map, a bad sign, or ppm without a usable calibration is refused at acquisition start. --display-colormap <name> on the acquisition command (QPSC's Orientation colour image dropdown) overrides display_colormap for one run.

LC-PolScope background correction

The background is a set of the five states taken on a specimen-free, slightly defocused field (10-20 um), as OpenPolScope does it. It is applied per tile inside the reconstruction, in Stokes space (QLIPP) or in OpenPolScope's normalised difference space (reconstruction.method: openpolscope), never by dividing the raw states, which the channel acquisition paths skip for this modality.

• Collect it with QPSC's background collection for the LC-PolScope profile, after the LC calibration has been applied. The collector applies each state exactly as the acquisition does, at the profile exposure, averages reconstruction.background_frames frames per state (default 10; a single frame's noise would be copied into every tile), refuses a camera on auto gain or auto exposure, warns on clipped pixels, and writes <State>.tif (float32) plus background_provenance.json: the LC calibration time and file, exposures, frames, adapter wavelength, black level, camera summary, per-state means, swing_state_spread (how far apart the swing states' background means lie, as a fraction of their mean) and swing_field_variation (how much their balance changes across the field, from 64 px blocks; a uniform imbalance from an uncalibrated palette stays low, specimen or debris pushes it up; the demo reads 0.13 on glass and 2.0 on tissue). Above 10% the collector logs a WARNING. If the balance also varies across the field (above 25%), or a calibration was applied, the warning says the field probably had specimen or debris in it: move to an empty area and collect again. With no calibration applied and a flat field, the warning says the spread is expected, since the states are not balanced for this light path, and asks you to calibrate and collect again.
• At acquisition the five states are loaded from the profile's background folder and checked against the current calibration. A background with no provenance, or one collected under a different LC calibration, is not used: an ERROR says why and the run reconstructs without background correction, because an uncorrected map is honest and one corrected against the wrong states is wrong everywhere. A different exposure or adapter wavelength is a WARNING. Collect a new background after every calibration.
• Derived tiles record polscope.background_corrected and, when corrected, polscope.background_collected_at, polscope.background_lc_calibrated_at, polscope.background_frames and polscope.background_folder.

Estimated background (opt-in, per tile)

A residual the measured background missed (strain elsewhere on the slide, a different focus, drift) is a smooth ramp of a few nm that differs from tile to tile and steps at the seams. reconstruction.estimated_background fits a low-order polynomial surface to it in each tile and removes it, after the measured background (recOrder's "estimated background", ported in polscope_library.background_fit):

estimated_background:
  enabled: true                  # off when absent or false
  order: 2                       # polynomial degree; 1 = plane
  block_size: 32                 # px; block medians are what is fitted
  max_block_retardance_nm: 5     # blocks above this are specimen, left out; null keeps all
  min_background_fraction: 0.25  # fewer background blocks than this: tile left uncorrected

It cannot tell a smooth residual from smooth specimen retardance. A tile that is mostly specimen is therefore skipped, not fitted, and weakly birefringent tissue under the threshold can still be removed, so it is off unless enabled. Unknown keys or bad values refuse the run at the first tile. Every derived tile records polscope.estimated_background (the reason line), ..._applied, ..._order, ..._block_size, ..._max_block_retardance_nm, ..._blocks_used and, when applied, ..._peak_removed_nm / ..._mean_removed_nm, which compared across tiles is the size of the residual the measured background missed. The server logs one INFO line per tile.

Black-level options

The --black-level parameter controls the dark-frame correction, which improves extinction ratio by accounting for detector noise and dark current:

• Omitted or 0: Falls through the library's chain: measure if a lamp is configured, use the YAML value if present, otherwise assume zero.
• Positive value: Use the supplied dark level directly.

A measured dark frame (via a lamp) is preferred when available, as it captures the actual hardware behavior on the day.

LC control mode

The server currently supports MM-Retardance mode: commands the liquid crystals in retardance (wave) units, and the Micro-Manager device adapter converts to voltage. Voltage mode (MM-Voltage) is not yet wired up and is refused with an error message directing the operator to use retardance mode.

PPM Acquisition Options (--ppm-high-bit-depth)

The BGACQUIRE (and ACQUIRE) acquisition message parser accepts optional flags specific to PPM (polarized-light microscopy) acquisitions.

High-bit-depth angle capture

--ppm-high-bit-depth true|false

Parameters:

• --ppm-high-bit-depth true|false -- Enable high-bit-depth capture for PPM angle frames (opt-in). Default: false.
  • When true: PPM angle frames are captured at the camera's native high-bit PixelFormat (e.g., 12-bit on JAI cameras) instead of the standard 8-bit. This provides higher precision inputs to the birefringence calculation, potentially improving measurement accuracy for low-intensity samples.
  • When false or omitted: Uses the standard 8-bit capture path; behavior is byte-identical to prior releases.

Constraints:

• This flag is only honored on cameras that implement set_high_bit_mode() (currently the JAI camera interface).
• If specified on unsupported camera hardware, the server logs a warning and silently falls back to 8-bit capture.
• Scoped to angle acquisition only: autofocus and non-PPM imaging always use standard 8-bit capture regardless of this flag.

Use case: When acquiring PPM images from weakly birefringent samples or at high zoom factors, enabling high-bit-depth capture can improve the signal-to-noise ratio of birefringence measurements by working with full camera precision instead of quantized 8-bit data.

Backward compatibility

Omitting --ppm-high-bit-depth or setting it to false preserves the standard 8-bit capture behavior, ensuring acquisitions are unchanged from prior releases.

Property Limits (GETPROPL)

GETPROPL queries Micro-Manager's accepted range for a numeric device property. A UI that bounds an intensity knob or other control needs to know what the hardware allows — on OWS3 a DLED wavelength range is 0–100 while the DiaLamp runs to 2100. Without this, the UI either guesses a range or lets the user enter a value the hardware rejects.

Protocol

Request: 64-byte payload

• Bytes 0–31: Device name (UTF-8, null-padded to 32 bytes)
• Bytes 32–63: Property name (UTF-8, null-padded to 32 bytes)

Response: Always 9 bytes

• Byte 0: Availability flag
  • 0x01: Numeric limits exist; bytes 1–8 contain valid floats (see below)
  • 0x00: No numeric limits, unknown device/property, or Core could not answer
• Bytes 1–4: Lower bound (IEEE 754 big-endian float32; 0.0 if unavailable)
• Bytes 5–8: Upper bound (IEEE 754 big-endian float32; 0.0 if unavailable)

When the availability flag is 0x00, the caller should keep its own default range instead of trusting a fabricated one.

Example

Request:  DLED (device, 32 bytes, null-padded) + Intensity-385nm (property, 32 bytes)
Response: 0x01 + 0.0 (float) + 100.0 (float)  =>  The DLED intensity property accepts 0.0–100.0

Finding tissue before autofocus (FINDTISS)

FINDTISS moves the stage in XY until the camera is looking at tissue. It never changes Z, exposure, or any camera setting -- the caller orders the pair itself:

MOVE -> FINDTISS -> STRMAFZ

Why it exists

A multi-slide batch predicts each slide's first alignment landmark from the base transform. Measured over 8 slides (2026-08-24) that prediction lands a median 613 um from its target, worst case 1507 um, which frequently puts the camera over blank glass. A focus scan there commits to coverslip contrast or walks its whole attempt budget and gives up.

The second landmark, corrected by the first's translation, lands within 26 um. So the base transform's error is very nearly a constant per-slide offset, and only the first landmark of a slide needs this -- which is the only place QPSC sends it.

Alignment reach is not what is being fixed. The search only has to put tissue -- any tissue -- in view so the focus scan has something real to find.

FOUND leaves the stage at the tissue, and the caller is expected to leave again. What it keeps is the Z. QPSC focuses at the found position and then drives back to the position it predicted, because the alignment step that follows matches against a WSI region anchored on the intended TILE: a camera several hundred micrometres away has no overlap with it. Treating the found position as a new place to align from would trade a focus scan over glass for an alignment that cannot match.

Parameters

FINDTISS --yaml config_PPM.yml [--modality <name>] [--objective <id>] [--dir <dx>,<dy>] \
         [--step <um>] [--max-attempts <n>] ENDOFSTR

• --modality selects the modalities.<name> binding in autofocus_<scope>.yml, and with it the strategy whose validity_params decide what counts as tissue. Optional in the parser but not in practice -- see How tissue is decided below.
• --objective is the fallback when no modality binding matches: the flat texture_threshold / tissue_area_threshold on that objective's autofocus_settings entry.
• --dir is a stage-space hint toward where tissue is believed to be; only its bearing is used. QPSC computes it as the vector from the predicted position toward the centre of the tile grid. Malformed input is ignored with a warning rather than failing the command -- the search still works unhinted.
• --step defaults to one camera FOV diagonal. Note this is deliberately COARSE and does leave gaps: stepping 446 um along X with a 357 x 267 um field skips 89 um. The largest step that could not skip anything on an arbitrary bearing is the field's SHORT side (267 um), which would need far more positions for the same reach. Gaps are acceptable here because the target is a tissue mass many fields across, not a specific field -- the same trade the acquisition path's own first-tile search makes.
• --max-attempts counts the starting position. Default is two complete rings, 17 either way -- a hint reorders a ring, it does not shrink one. Capped at 33 (four rings, what the measured 1507 um worst case needs at a 446 um step). Derived from the bearing count rather than written as a literal, because a budget stopping mid-ring biases the search toward whichever bearings are enumerated first -- and with a hint, that is exactly where the hint is least trustworthy.

Search pattern

server/tissue_search.py is pure geometry and unit-tested (tests/test_tissue_search.py). The first position is always where the caller already is. After that, positions lie on rings at whole multiples of --step, eight bearings to a ring. The radius swept in every direction is step * ((max_attempts - 1) // 8), so an attempt budget converts directly into a distance -- which is how the default was sized against the measurement above.

A hint orders a ring; it never trims one. --dir sorts the ring's bearings nearest-first (down the hint, then +/-45, out to 180), so a good hint returns on the first or second position. It deliberately does NOT restrict the search to a fan. The hint is the vector from the predicted position to the tile-grid centre, measured in the transform's own frame -- and that transform is off by the very offset the search exists to defeat, which displaces both ends of the vector. Its angular error is about asin(offset / separation): negligible when the predicted point is far from the grid centre, unbounded when it is close. Close is the common case, because the caller's tile picker favours interior, high-texture tiles, which on a compact section sit near the middle. A +/-45 deg fan would then march two rings the wrong way and report NOTFOUND with tissue directly behind it.

How tissue is decided

By the same strategy validity check the acquisition path uses, resolved through the same chain: modalities.<modality> binding -> strategies.<name>.validity_check + validity_params, with the binding's overrides.validity_params merged on top. The resolver is server/focus_validity.py -- pure, unit-tested (tests/test_focus_validity.py), and deliberately outside server.handlers so a test needs no pycromanager.

Going through the modality is not a nicety. autofocus_<scope>.yml states thresholds in two places, and they disagree:

Where	Scope
autofocus_settings[].texture_threshold etc.	Per OBJECTIVE, flat keys
strategies.<n>.validity_params + modalities.<m>.overrides.validity_params	Per MODALITY -- where the real tuning lives

LC-PolScope's binding sets tissue_area_threshold: 0.1 where the objective entry still says 0.2, with the comment that the 20% floor "rejects valid fields"; PPM and LC-PolScope both widen tissue_mask_range to [0.05, 0.95], which the flat keys cannot express at all. A search reading only the flat keys on those scopes reports "no tissue" while looking straight at some.

A modality bound to a manual-only strategy resolves to always_false, which no search can satisfy. That returns FAILED:no-automatic-tissue-check-for-this-modality immediately, without moving the stage, instead of walking the pattern to a foregone NOTFOUND.

Frames come from the live stream when there is one

With the Live Viewer streaming -- which is the state the caller has just put the rig in so SIFT can work -- the camera holds a buffer of frames from before and during the move just made. Reading one of those judges the PREVIOUS position, which is how a search reports tissue at a place that has none. So after each move the buffer is drained and the next frame to arrive is used. It is also ~400 ms per position cheaper than a blocking JAI snap (17 positions: seven seconds a slide), and it carries the LIVE exposure, which is the state SIFT will match against. Falls back to snap_image() whenever no sequence is running or no stream frame arrives within 250 ms.

Exposure is deliberately not adjusted

The caller has just put the modality into its alignment reference state -- for PPM, the calibrated uncrossed angle and exposure -- and SIFT is about to match against that state. A brightness-chasing loop here would silently change what the next step depends on. This differs from the acquisition path's first-tile tissue search, which does double exposure: that one owns the camera state, this one borrows it.

Responses

• FOUND:<x>:<y>:<attempt>:<of> -- the stage is standing at (x, y).
• NOTFOUND:<x>:<y>:<of> -- everything searched was background, and the stage has been put back where the search started. A search that found nothing has no reason to prefer its last guess over its first, and leaving the stage elsewhere would silently invalidate the caller's own prediction.
• ABORTED:<x>:<y>:<completed> -- the operator cancelled via ABORTAF, which FINDTISS honours deliberately: the Live Viewer turns its Autofocus button into a Cancel toggle before the search starts, so one Cancel has to stop whichever half of "search then focus" is running. The stage is returned to the starting point. Polled BETWEEN positions, never mid-move.
• FAILED:<reason> -- could not run at all. Nothing moved.

Objective retraction when an acquisition finishes

At the end of every acquisition -- success, cancel or failure -- the objective is retracted to the declared safe Z before any further stage motion. An acquisition covers one annotation, and no annotation spans two slides, so finishing one is the point at which the next motion is unconstrained: the next region, a slide change, or an operator on the joystick.

Configuration

stage:
  safe_z_um: 0                      # the declared retracted Z (um)
  focus:
    retract_sign: positive          # which Z direction moves AWAY from the sample

Both keys are optional, and the feature is off without them -- an acquisition that cannot retract safely proceeds exactly as before.

stage.safe_z_um -- a Z at which the objective is clear of the sample. On PPM this is 0 while focus sits near -420 um, so retracting buys about 420 um of clearance.

stage.focus.retract_sign -- positive or negative (case-insensitive; 1/+1 and -1 also work). On PPM it is positive because, measured 2026-08-26, increasing Z moves the stage DOWN, away from the objective.

This is a safety gate, not a convenience. A "safe" Z on the wrong side drives the objective into the slide, and config_PPM.yml carries the scar of exactly that: an earlier value of -500 was inferred from "focus is near -400, so retract further" without establishing the sign. Measure it; do not derive it.

What it does

1. Resolve stage.safe_z_um and stage.focus.retract_sign. If either is missing or malformed, log and do nothing.
2. Read the current Z.
3. If it is already within 1 um of the safe Z, do nothing.
4. Direction check: if the safe Z lies on the sample side of the current Z, given the declared sign, refuse and log an error naming both positions. This is what catches a wrong-side declaration.
5. Otherwise move, and log how much clearance was gained.

It runs BEFORE the end-of-region return move, so that move is itself made retracted.

The step never raises. A retraction that cannot be made must not turn a completed acquisition into a failed one, so a config problem, an unreadable Z, a stage error or a target outside the configured Z limits are all logged and stepped over.

Why

On the 2026-10-02 four-slide run, every long XY move -- all fourteen, including an 85.9 mm traverse across the whole holder -- ran at focus height, between -306 and -459 um, while the declared safe Z was 0. The end-of-region return move was among them, up to 12.7 mm, and its log line used to read "(preserving Z)"; it now reads "(at the retracted Z)".

Clearance is not the only stake. The Prior XY stage is open loop, so an objective touching a slide or a holder rib mid-traverse is mechanical load, and lost steps cannot be reported: the controller tells you the count it was given, so arrivals read exact, and a slip during a move leaves no position discontinuity to find. The MicroManager CoreLog for that run shows a provably clean counter -- 293,227 position reads across the acquire pass with no unexplained change -- while three of the four slides came out offset in Y.

Per-insert safe Z is not resolved here

stage.inserts.configurations.<id>.safe_z_um overrides are not applied, because the acquisition message does not say which insert is fitted. Declare a scope-level safe_z_um that is on the retracted side for every insert you use and inside stage.limits.z_um. The direction check is the backstop: a target on the sample side is refused rather than driven to.

Autofocus / Streaming Focus (--safe-z, --approach-max, --tissue-gate)

The STRMAFZ command supports multiple autofocus strategies. The default edge-retry walk is always available; an alternative approach-from-safe-Z strategy can be enabled via parameters measured during a separate validation run.

Refused while an acquisition is running

The focus stage has one owner at a time. While any client's acquisition is in state RUNNING or CANCELLING -- including one whose client has timed out without cancelling it -- the STRMAFZ streaming-autofocus command (written STREAMING_FOCUS in the examples below) does not move the stage. It replies at once with

FAILED:busy: an acquisition is running for <addr>; refusing to move the focus stage under it

and nothing moves. Wait for the acquisition to finish, or send a cancel and wait for it to complete, then re-issue the autofocus command.

Focus Surface Fitting (--focus-surface, --focus-survey, --focus-survey-tiles)

The BGACQUIRE (and ACQUIRE) acquisition message parser accepts optional flags to fit a robust plane through autofocus measurements and optionally validate or override per-tile autofocus results against that plane.

Why a focus surface

Measured autofocus results occasionally land far from the true sample plane -- wide-search autofocus lands more than 5 um off 25% of the time and more than 15 um off 21% of the time, worst case 73 um. A single bad measurement adopted at the start of a region is then handed to every tile around it, putting the whole region out of focus with nothing in the log to say so.

Nothing in the system could notice, because a single measurement has nothing to be wrong against. A plane fitted to the region's other measurements does: one region had 12 of 12 wide-search results a median 43 um away from a plane its 111 drift sweeps defined to 0.27 um RMS. Full measurement and method: claude-reports/design/autofocus-empirical/summaries/focus_surface.md.

Flag syntax

--focus-surface <mode>          # off | observe | enforce (default: off)
--focus-survey <points>         # Number of points to measure before the tile loop (0 = none)
--focus-survey-tiles <indices>  # Comma-separated tile indices to survey (optional; server chooses if omitted)

Focus surface modes

• off (default) -- No focus surface; behavior is identical to prior releases. The surface is not fitted or consulted.

• observe -- Fit a robust plane and report what enforcement would do, but change nothing. Every autofocus result is checked against the surface, and the log reports disagreements. This mode is a dry run: use it to measure what enforcement would have done over a real region before committing to automatic overrides.

• enforce -- Fit a robust plane and replace autofocus results that disagree with it beyond a gate. When an autofocus result lands far from the fitted plane (residual > max of a 5 um floor or 4 sigma of the fit's RMS), that result is rejected and the tile uses the plane's prediction instead, with the stage moved there.

Focus surface licensing

The surface refuses to speak unless:

• It has at least 6 points (three points fit a plane exactly; six is the minimum that leaves enough redundancy that one bad point cannot silently define the surface).
• The fit's residual is below max_rms_um (default 3.0 um). Above that threshold the sample is not planar -- a folded section, a cytology smear -- and the surface declines to predict.
• At least 60% of the points agree with the fit (measured inlier fraction). RANSAC over a scattered set always finds some flat-looking subset, and the inlier RMS over that subset looks excellent; this gate detects that a surface is fitted to a minority.
• The points span enough in their weaker direction to constrain a tilt. The absolute floor is 0.5 mm, but for narrow regions (e.g., a strip of tiles only 0.4 mm wide), the requirement scales down to 60% of the region's narrower dimension. An unconstrained tilt can only do harm over the distance it is extrapolated, and in a thin strip that is almost nothing. Six points down one raster column are six points on a line; they constrain the tilt along the column and say nothing across it. The server checks this and refuses to extrapolate a tilt in an unconstrained axis.

When any check fails, the mode is logged as "NOT licensed: ..." and the acquisition falls back to the nearest-neighbour seeding behavior from before this feature.

Pre-acquisition focus survey

--focus-survey N measures N additional focus points before the tile loop starts, instead of discovering the plane one tile at a time:

• Timing: Runs after Phase 8 (pre-acquisition autofocus at the first tissue position), which sets the autofocus exposure and rotation angle. The survey uses those settings.

• Point selection: If --focus-survey-tiles is omitted, the server spreads its candidate tiles using farthest-point sampling: it picks positions that span the region extremes first, which constrains both tilt terms. To avoid edge artifacts (boundary tiles are often half off the tissue), the server keeps candidates inset from the region edge by approximately one camera FOV. Tissue checks reject empty fields automatically, and the survey moves to the next candidate.

• Explicit tiles: If --focus-survey-tiles is provided (comma-separated indices like "2,7,15,23"), the survey measures those positions in order. The indices are from the tile grid; the client typically scores them for tissue content on the macro image and supplies the well-covered ones.

• Cannot fail the run: A point that cannot be focused, has no tissue, or fails to focus is skipped. If too few points survive the surface stays unlicensed and the tile loop behaves as it always did. A survey that produces no usable surface is safe: it is a time investment that did not pay off.

• Cost: Each point is one wide autofocus scan, roughly 10 s, so nine points cost a couple of minutes.

• Sizing: replayed over 45 real regions, a 9-point survey licenses a surface in 91% of them and a 12-point survey in 91% with a slightly better median error; past 12 the differences are noise. A 5-point survey licenses nothing at all -- the surface requires six mutually agreeing points. Scored against every autofocus result in the region (not just the ones that agree), a 12-point survey predicts them to a median 1.05 um with a p90 near 6 um; the gap is the outliers, which are what the surface exists to catch.

• What the survey does and does not save: it does not change how often the tile loop autofocuses. That is still n_tiles in autofocus_<scope>.yml, and raising it is a separate, deliberate decision. What the survey buys is a licensed surface from the first tile rather than after a region's worth of measurements have accumulated -- which is what makes raising n_tiles safe. The prize is large: the 2026-09-24 session spent 8.95 hours in per-tile autofocus (2,084 sweeps at a mean 15.5 s), and 21% of those sweeps returned no measurement at all because the metric never bracketed focus.

Example: Observe mode dry-run

--focus-surface observe --focus-survey 9

Runs one survey with nine well-spread points, fits a plane, and logs every time the surface would have overruled a tile's autofocus result. Nothing acts on it: no tile's Z changes and the stage is never moved on the surface's say-so, so the acquisition is the one it would have produced anyway. The output is the per-tile disagreement lines plus the end-of-region === FOCUS SURFACE === summary, which reports how many results would have been overruled and out of how many. Run this before enforce.

Example: Enforce mode with client-selected points

--focus-surface enforce --focus-survey 6 --focus-survey-tiles "5,12,19,31,44,51"

Measures the six client-specified tiles, fits a plane, and replaces autofocus results that land far from it. The client (e.g., QuPath) scores tiles for tissue content and supplies the well-covered ones.

Backward compatibility

Omitting --focus-surface or setting it to off preserves byte-identical behavior to pre-feature acquisitions. The focus surface is off by default.

Two autofocus strategies

Edge-Retry Walk (default): Starts from wherever the stage currently is and scans a window; if the peak appears to lie past an edge of that window, it shifts the window in that direction and scans again, up to --max-attempts times. Needs no operator measurement and works from any starting position, but each continuation is an inference that moving further -- possibly toward the sample -- is correct.

Approach-from-Safe-Z (new): A bounded single-pass scan that retracts to a known-safe position first, then approaches the sample once. The operator measures the safe Z (clear of sample) and the maximum distance to travel during a separate validation run. This strategy:

• Guarantees a clear retraction on failure
• Bounds travel distance (no open-ended walk)
• Optionally validates tissue before committing to a focus peak

Dedicated Focus Channel (--af-channel)

For multi-channel modalities (widefield IF, BF+IF), autofocus normally runs on whatever channel the previous tile happened to leave applied. This makes focus quality unpredictable and dependent on acquisition order. The --af-channel flag lets you name a single channel to use for all autofocus attempts, optionally at a dedicated exposure and intensity.

Motivation: Focus frames need good contrast, but not publication quality, so a shorter exposure keeps every focus attempt cheap. A well-covered channel (e.g., DAPI where nuclear signal is dense) provides consistent, robust focus metrics regardless of which angle or timepoint is being acquired.

Flag syntax:

--af-channel <channel_id>              # Required: channel library id for focusing
--af-channel-exposure <milliseconds>   # Optional: override channel's default exposure
--af-channel-intensity <value>         # Optional: override channel's default intensity

Parameters:

• --af-channel <channel_id> (required) -- The channel library id to use for autofocus. Must match an entry in the modality's channel library declared in the microscope YAML. When --af-channel is set, the server applies that channel's hardware state (Micro-Manager presets and device properties) before every autofocus scan, overriding whatever channel the previous tile left on the light path.

• --af-channel-exposure <ms> (optional) -- Exposure time in milliseconds for autofocus frames on this channel. If omitted, the channel's library default (exposure_ms) is used. Typically shorter than the imaging exposure to reduce focus latency (e.g., 20 ms for focusing vs. 100 ms for acquisition).

• --af-channel-intensity <value> (optional) -- Illumination intensity override for autofocus frames on this channel. If omitted, the channel's library default is used. Allows independent tuning of focus lighting vs. imaging lighting.

Behavior:

• Nothing is restored afterwards: the per-tile acquisition loop applies each channel's own hardware state before it snaps, so the focus channel never leaks into an acquired image.
• If the named channel cannot be resolved (not in the channel library), the server logs a warning and autofocus falls back to the last-applied channel.
• Hardware failures during channel setup are logged and do not abort the acquisition; focus proceeds on the currently-applied channel.

Example (widefield IF):

--channels "(DAPI,TRITC,Alexa647)" --channel-exposures "(100,150,120)" \
--af-channel DAPI --af-channel-exposure 20.0

Acquires three channels at their specified exposures, but autofocus always runs on DAPI at 20 ms (faster) instead of reusing whatever channel the previous tile left applied.

Backward compatibility:

Omitting --af-channel preserves the original behavior: autofocus runs on the currently-applied channel. The exception is --focus-channel <channel_id>, the autofocus reference channel the QPSC dialog sends with every acquisition. Without --af-channel, that channel's hardware state is applied before each autofocus scan, at its own library exposure and intensity (the --af-channel-exposure and --af-channel-intensity overrides apply only to --af-channel). --af-channel wins when both are given.

Flag syntax

--safe-z <micrometers>         # Retraction Z (where the sample is definitely not)
--approach-max <micrometers>   # Distance to scan from safe Z toward the sample
--tissue-gate 1                # Require tissue validation before committing (omit to disable)

Approach-from-safe-Z parameters

• --safe-z <um> -- the RETRACTED Z, where the objective is clearly clear of the sample for the insert and objective in use. Not the coverslip and not the sample plane -- those are what it must stay away from. The stage retracts here before scanning, and returns here if the scan fails. Required to activate approach mode; no safe default, because a guessed retraction could be on the wrong side of the sample.

• --approach-max <um> -- Maximum distance to travel from safe Z toward the sample (positive or negative, depending on Z direction). The actual scan span is clamped by stage limits. Required to activate approach mode; no safe default. Typically measured from a validation run as the distance from safe Z to the auto-detected focus peak, with a safety margin.

• --tissue-gate 1 -- If set, each focus peak candidate is tested for tissue presence before committing. Peaks that fail the tissue check (e.g., coverslip reflections producing metric peaks but no tissue texture) are rejected and the scan continues to the next peak. Omitted means no tissue check: commit to the first prominent peak. QPSC sets this when its validation run found surfaces BEFORE focus, which is exactly when committing to the first peak would land on glass.

Activation rule: Approach-from-safe-Z is enabled ONLY when BOTH --safe-z AND --approach-max are provided. If only one is given, the server logs a warning and falls back to the edge-retry walk.

Tissue gate validation

When --tissue-gate true, each focus peak undergoes a texture-based tissue presence check before commitment. The check uses the same validity criteria applied during acquisition (via resolve_validity_check("texture_and_area")), configured by the microscope YAML:

autofocus:
  texture_threshold: 0.010        # Texture variance threshold (default 0.010)
  tissue_area_threshold: 0.200    # Tissue coverage % (default 20%)
  rgb_brightness_threshold: 240.0 # Max RGB value for coverslip rejection (default 240)

A peak passes the gate if the image at that Z shows sufficient texture (cell structure, not uniform glass) and has tissue in the required fraction of pixels. Coverslip reflections are typically rejected because they produce uniform, bright pixels rather than textured features.

Autofocus Configuration (YAML)

Per-objective autofocus settings live in autofocus_<scope>.yml under autofocus_settings, keyed by objective id.

Colour frame reduction (channel_reduction)

How a colour frame is collapsed to one plane before the focus metric scores it. Valid values: equal_mean (default) or green.

autofocus_settings:
- objective: LOCI_OBJECTIVE_OLYMPUS_20X_POL_001
  score_metric: brenner_gradient
  channel_reduction: green        # score the green plane only

• equal_mean (default) -- an equal-weighted mean of the three colour planes. This is the historical behaviour, and it is also what the standard (non-streaming) autofocus path uses, so leaving it alone keeps the two paths agreeing.

• green -- the green plane alone. Use it where achromatic debris competes with stained tissue. Dust and fibres absorb across the spectrum, so they carry full contrast into every channel; stained tissue carries most of its contrast where the stain absorbs. Averaging the planes therefore preserves all of the debris signal and dilutes the tissue's, and on a field holding both, the debris wins.

  Measured on four PPM 20x traverses, 2026-09-04: over a field holding a textile fibre and tissue, equal_mean peaked at Z=-219.2 (the fibre) while green peaked at -327.6 against a hand focus of -332. On a dust-only field green scored lower amplitude (6.9% vs 9.8%, i.e. closer to being correctly rejected as flat), and on clean tissue it scored higher (66.1% vs 58.6%) at the same Z.

Unknown values fall back to equal_mean with a warning rather than failing, so a YAML typo does not take autofocus offline mid-run. The resolved value is logged on every run as STREAM_AF:channel reduction = ....

Example: Validating and using approach-from-safe-Z

Step 1: Validation run (operator measurement)

STREAMING_FOCUS --yaml config.yml --metric tenengrad --range 100
# Server scans across the sample, prints the detected focus peak Z.
# Operator measures the safe Z (e.g., at the coverslip) = 2.00 mm.
# Difference: peak_Z - safe_Z = 0.042 mm -> measure approach_max as 0.050 mm (with margin).

Step 2: Subsequent acquisitions

STREAMING_FOCUS --yaml config.yml --metric tenengrad \
  --safe-z 2.00 --approach-max 0.050 --tissue-gate true
# Server retracts to 2.00 mm, scans 0.050 mm toward the sample,
# finds the first prominent peak, validates tissue, and commits.
# On failure, stage returns to 2.00 mm (safe).

Backward compatibility

Omitting --safe-z and --approach-max (or providing only one) preserves the default edge-retry walk, ensuring autofocus behavior is unchanged when these parameters are not supplied. The --tissue-gate parameter has no effect when approach mode is not active.

Autofocus Profiling (--z-start, --z-end)

The STRMAFZ command supports an explicit profiling mode to acquire the focus metric over a named interval without committing to any focus result. This is used for validation workflows that need the raw metric profile over a specific region to analyze focus behavior and measure safe-Z distances.

Profiling mode parameters

--z-start <micrometers>   # Start Z position for the profiling scan
--z-end <micrometers>     # End Z position for the profiling scan

Activation rule: Profiling mode is enabled ONLY when BOTH --z-start AND --z-end are provided. If only one is given, the server logs a warning and ignores both. Profiling mode is mutually exclusive with approach-from-safe-Z mode (--safe-z / --approach-max).

Profiling behavior

• Single pass: The stage scans exactly once from z_start to z_end, in whichever Z direction that requires.
• No focus decision: The metric profile is recorded and returned, but no peak is committed. The focus is not updated.
• Return to start: After the scan, the stage returns to z_start, so the acquisition leaves the Z position unchanged from before the profiling run.
• Direction-agnostic: Unlike the edge-retry walk (which always increments Z), profiling supports scans in either direction: z_end can be greater than or less than z_start.

Use case: Focus approach validation

Profiling is used during Focus Approach Validation to measure safe-Z distances and validate that the sample is reachable:

1. Step 1: Profiling run - Scan across the expected sample region and capture the metric profile.
2. Step 2: Analyze profile - Identify the focus peak, the safe-Z (clear of sample), and the approach distance.
3. Step 3: Subsequent acquisitions - Use the measured values with --safe-z and --approach-max for bounded, repeatable focusing.

Example: Profiling scan

STREAMING_FOCUS --yaml config.yml --metric tenengrad \
  --z-start -0.500 --z-end 0.500
# Server scans from -0.500 mm to +0.500 mm, records the metric profile,
# and returns the stage to -0.500 mm without committing to any focus.

Backward compatibility

Omitting --z-start and --z-end (or providing only one) disables profiling mode, ensuring autofocus behavior is unchanged when these parameters are not supplied.

Autofocus diagnostics (--dump, --dump-label, --dump-frames)

STRMAFZ can write the scan out to disk for offline analysis.

--dump 1              enable the dump (CSV traces + manifest)
--dump-label <text>   append a label to the folder name
--dump-frames 1       ALSO write one TIF per sample

--dump 1 writes the traces and the manifest. Frames are not written unless --dump-frames is also set: they are several hundred TIFs, often ~750 MB per scan, whereas the traces are a few kB and are what the analysis actually reads.

--dump-label exists because a focus-approach validation performs two scans -- one over tissue and one over blank slide -- and both would otherwise land as streaming_af_<timestamp>, indistinguishable without opening them. Anything outside [A-Za-z0-9_-] is replaced with -, and the result is truncated to 40 characters.

Layout

<yaml_dir>/autofocus_tests/streaming_af_<TIMESTAMP>[_<LABEL>]/
    samples.csv       idx, wall_ms, z_assumed_um, z_actual_um, metric,
                      chroma_median, chroma_p95, chroma_frac
    z_poll.csv        wall_ms, z_actual_um   -- the raw stage-position poll
    blocks.csv        per-block focus energy profile (spatial distribution of sharpness)
    manifest.json     scan parameters, summary, and focus-plane analysis result
    frames/           ONLY when --dump-frames is set
        frame_0000_t000093ms_zass-001.072.tif
        ...

samples.csv carries both Z columns on purpose. z_actual_um is interpolated from the poll trace and is what the focus fit uses; z_assumed_um is the old wall_ms * velocity model, kept so the two can be compared. They diverge whenever slow_speed_um_per_s is mis-calibrated, and the divergence grows with distance travelled -- see Where a sample's Z comes from in the QPSC autofocus documentation.

Block-wise focus analysis

blocks.csv contains per-block focus energy for each sample, revealing where in the field the sharpness is concentrated. This spatial distribution separates the actual sample plane from false peaks caused by dust specks or fibres on the coverslip:

• A sample plane shows sharpness distributed across many blocks, scattered over the frame.
• A dust speck or fibre lights up only the few blocks covering it, and those blocks are contiguous -- which is what the dispersion figure measures.

This matters because a whole-frame metric cannot make that distinction, and neither can chroma: chroma is defocus-invariant, so two candidate planes inside ONE field report essentially the same chroma by construction.

The file has one row per sample frame with columns: idx, z_actual_um, b00, b01, ... b77. The grid is a fixed 8x8 (64 blocks), not detector-dependent -- on a 772x1024 frame that is ~96x128 px per block, large enough that a block of blank glass still has a meaningful gradient statistic and small enough that a fibre spanning a corner lights up only a few.

The manifest includes a focus_plane field with the block-wise analysis result:

• z: The Z identified as the sample plane, or null if no plane was accepted.
• n_blocks: How many blocks agree on this plane. Must reach 10 of 64.
• dispersion: How SPREAD OUT those agreeing blocks are -- mean pairwise block distance over the grid diagonal. Higher is better: a scatter across the frame scores ~0.42 and is what tissue looks like, while a blob on one object scores ~0.2. Must reach 0.30.
• n_live_blocks: How many blocks have any focus peak at all (>= 1.3x their own baseline).
• reason: Why, in words -- "accepted", or e.g. "only 4 of 64 blocks have any focus peak".
• rejected: Candidate planes that were ruled out, each with its block count, dispersion, and which bar it missed.

Measured values behind those bars, from the four positions that failed a 4-slide run (PPM 20x, 2026-09-04): a fibre scored 8 blocks / 0.247, dust specks 4 / 0.199, and real tissue 14 / 0.366 and 63 / 0.424. The bars sit in those gaps, which are narrow -- four planes is not a distribution, so re-measure before trusting them on a different stain or objective.

Chroma columns for tissue detection

Three chroma columns in samples.csv measure tissue-vs-glass separation from a focus-approach validation pair, without dumping frames (a 300-sample traverse is ~0.7 GB of frames per scan; these columns are ~21 KB):

• chroma_median -- median chroma, max(RGB) - min(RGB), over the frame.
• chroma_p95 -- 95th-percentile chroma. Useful for setting the stain threshold: place the threshold above the blank's p95 and below the tissue's median to separate them cleanly. The median alone does not pin the tail; the p95 answers what that tail reaches.
• chroma_frac -- fraction of pixels at or above the stain threshold (28.0, the same default chroma_deviation uses, so the numbers mean what that check would mean by them).

All three are blank when the frame is monochrome, rather than reporting a number that would look like an answer.

They are surveyed across the WHOLE traverse on purpose. Chroma is expected to be defocus-invariant -- that is the property that would make it useful in a tissue gate -- and the first measured pair (PPM, 10x, H&E, 2026-08-28) bore that out: median chroma held at 36 across a 256 um tissue traverse over which the focus metric swung 2.4x, varying by under 5% and showing no peak at best focus.

That pair also showed why the columns are worth having rather than assuming a threshold. Blank glass was NOT near zero -- it held a median of 11-13, because the lamp and the debayer both put a cast on it -- so the separation is 36 against 12, not 36 against 0, and a threshold picked by eye at 12 would have passed bare glass as tissue. Run the pair on any rig whose stain or illumination differs before trusting the defaults on it.

manifest.json records z_start, z_end, velocity_um_s_configured, motion_duration_ms, metric_name, n_kept_samples, n_z_poll_samples, frames_written, and the observed average velocity from the poll trace.

The retry walk gives each attempt its own attempt_<N>/ subfolder; a single-pass profiling or approach scan writes straight into the dump root. Readers should handle both.

Example

A profiling traverse from a retracted safe Z of 0 um in to -267 um, over tissue, keeping only the traces:

STRMAFZ --yaml config_PPM.yml --modality ppm --range 267.0 \
        --z-start 0.0 --z-end -267.0 \
        --max-attempts 1 --dump 1 --dump-label tissue

Repeat it over a bare part of the same slide with --dump-label blank. Any peak present in both is a surface rather than the sample.

Installation

Part of QPSC (QuPath Scope Control)

Requirements:

• Python 3.10 or later
• pip (Python package installer)
• Git (for pip install git+https://... commands)

Important: This package depends on microscope-imageprocessing (required) and microscope-control (required). Optional dependencies:

• ppm-library -- only needed for PPM (polarized light) modality support
• polscope-library -- only needed for LC-PolScope birefringence reconstruction

See the QPSC Installation Guide for complete setup instructions.

Quick Install (from GitHub)

Install dependencies first:

# 1. Install microscope-imageprocessing (required - background correction, OME-TIFF I/O)
pip install git+https://github.com/uw-loci/microscope_imageprocessing.git

# 2. Install microscope-control (required - hardware abstraction)
pip install git+https://github.com/uw-loci/microscope_control.git

# 3. (Optional) Install ppm-library for PPM modality support
pip install git+https://github.com/uw-loci/ppm_library.git

# 4. (Optional) Install polscope-library for LC-PolScope birefringence reconstruction
pip install git+https://github.com/uw-loci/polscope_library.git

# 5. Then install microscope_command_server
pip install git+https://github.com/uw-loci/microscope_command_server.git

Development Install (editable mode)

git clone https://github.com/uw-loci/microscope_command_server.git
cd microscope_command_server
pip install -e .

For automated setup, use the QPSC setup script.

Troubleshooting Installation

Problem: ModuleNotFoundError: No module named 'microscope_command_server'

Cause: Package not installed correctly or virtual environment not activated.

Solution:

1. Ensure the environment is activated. Not every rig uses a venv -- the LC-PolScope machine has no system Python and runs from conda -- so check which kind this one is before assuming:

   # Windows, venv
   path\to\venv_qpsc\Scripts\Activate.ps1
   
   # Windows, conda
   conda activate <env name>        # conda env list, if you do not know it
   
   # Linux/macOS
   source path/to/venv_qpsc/bin/activate

   start_server.bat handles both without being told, and prints the interpreter it settled on. If it cannot find the right one, copy server_env.bat.example to server_env.bat and set the environment up there explicitly -- that file is git-ignored, so per-rig settings survive a pull.

2. Reinstall the package:

   pip install -e . --force-reinstall

3. Verify installation:

   pip show microscope-command-server

Problem: Entry point microscope-command-server command not found

Cause: Entry points not registered or PATH not updated.

Solution:

Try running the server directly:

# Using Python module
python -m microscope_command_server.server.qp_server

# Or with PYTHONPATH set (if needed). "parent directory" is the folder that
# directly contains the microscope_command_server PACKAGE, i.e. this repo's
# root -- NOT the folder that holds all the checkouts (see the next problem).
export PYTHONPATH="/path/to/parent/directory:$PYTHONPATH"
microscope-command-server

Problem: cannot import name '...' from 'polscope_library' (unknown location)

Symptom: the server starts, the version block prints the library's version, but its git line reads unknown (namespace package ...) and an ERROR follows naming the shadowing folder. The failure itself surfaces later, when something first does from polscope_library import ... -- on the LC-PolScope rig it was twenty seconds into an LC calibration (2026-10-12). Any of the lab packages can be affected; polscope_library was the one whose install mode allowed it.

Cause: a bare folder named after the package, with no __init__.py, sits on sys.path ahead of the real install -- normally the package's own repo root, because Python was started in the folder that contains all the checkouts (C:\qpsc-extension\python_server), or that folder was put on PYTHONPATH. Python builds a namespace package from it, so the package's __init__.py never runs and its top-level names do not exist.

Solution:

1. Pull polscope_library (its install now registers the project root as a path entry, which beats the bare folder) and re-run update_env.bat, or pip install -e "<checkout>" for each package.
2. Start the server from this repo's folder (start_server.bat here), not from the folder above it, and keep that folder off PYTHONPATH.
3. Check: python -c "import polscope_library as p; print(p.__file__)" must print a path ending in __init__.py, not None.

Problem: Port 5000 already in use

Symptom: OSError: [Errno 48] Address already in use

Cause: Another server instance or application is using port 5000.

Solution:

# Find process using port 5000
# Windows:
netstat -ano | findstr :5000
# macOS/Linux:
lsof -i :5000

# Kill the process if safe

For more troubleshooting, see the QPSC Installation Guide.

Quick Start

Server Side

from microscope_command_server.server.qp_server import main

# Start the server. It listens on port 5000 (TCP_PORT in server/protocol.py).
main()

Or run from command line:

# Option 1: Entry point command (NOTE: uses hyphens, not underscores!)
microscope-command-server

# Option 2: Python module syntax
python -m microscope_command_server.server.qp_server

Common mistake: The command is microscope-command-server (with hyphens), not microscope_command_server (with underscores).

Client Side

from microscope_command_server.client import get_stageXY, move_stageXY

# Get current position
x, y = get_stageXY()

# Move stage
move_stageXY(x + 1000, y + 1000)

Architecture

The server coordinates between QuPath (Java) and the microscope hardware (Python/Micro-Manager):

QuPath Extension -> Socket Client -> Microscope Server
                                          |
                  +-------+---------------+-------+-------+
                  |       |               |       |       |
          Microscope Microscope      PPM Lib  PolScope   ...
           Control ImageProcessing (opt)     Lib(opt)
              |        |                      |
              v        v                      v
        Micro-Manager Debayering,      Birefringence
          Hardware    Background,     reconstruction
                    OME-TIFF I/O,     per-tile
                    Z-stack projections

Server Configuration

The microscope command server uses a dynamic configuration approach:

Startup

• Server loads a minimal generic configuration (config_generic.yml)
• Connects to Micro-Manager (hardware must be available)
• Waits for client connections

During Acquisition

• Client sends ACQUIRE command with --yaml /path/to/config.yml parameter
• Server loads microscope-specific config from the provided path
• Hardware settings are updated dynamically
• Microscope-specific methods (e.g., PPM rotation) are initialized

Exploratory Commands

Commands like GETXY, MOVE, GETZ use the most recently loaded config:

• Before first ACQUIRE: Uses generic startup config with permissive stage limits
• After ACQUIRE: Uses the microscope-specific config from that acquisition

Note: Always provide the --yaml parameter in ACQUIRE commands to ensure correct microscope configuration.

Testing

This package includes automated unit tests for components that can be tested without hardware.

Automated Unit Tests

Pytest-compatible unit tests are located in the tests/ directory:

• tests/test_tiles.py - Tests for TileConfiguration.txt parsing and generation

These tests:

• Run without hardware (use synthetic test data and temp files)
• Can be integrated into CI/CD pipelines
• Test protocol handling, tile configuration, and utility functions

Running Unit Tests:

# Install dev dependencies
pip install -e ".[dev]"

# Run all tests
pytest

# Run specific test file
pytest tests/test_tiles.py

# Run with coverage report
pytest --cov=microscope_command_server --cov-report=html

# View coverage report
open htmlcov/index.html  # or xdg-open on Linux

Test Coverage:

Current automated tests achieve ~60-70% coverage for testable components:

• ✅ TileConfiguration parsing (coordinates extraction)
• ✅ TileConfiguration generation (2D pixel coordinates and 3D stage coordinates)
• ⏸️ Socket protocol (future test expansion)
• ⏸️ Server communication (requires integration testing)

Hardware Diagnostic Tools:

This package does not include standalone diagnostic tools. Hardware testing is performed via:

• The TESTAF and TESTADAF server commands (call diagnostic functions from microscope_control)
• The PPMSENS and PPMBIREF server commands (call diagnostic functions from ppm_library)

See the microscope_control and ppm_library documentation for details on these diagnostic tools.

License

MIT License - see LICENSE for details.

AI-Assisted Development

This project was developed with assistance from Claude (Anthropic). Claude was used as a development tool for code generation, architecture design, debugging, and documentation throughout the project.