dsh-llm-finish-reason-tolerance

A DeepSeek Harness host plugin that makes the agent tolerate OpenAI-compatible providers whose streaming responses end without a finish_reason.

The canonical case is the Snowflake Cortex gateway (*.snowflakecomputing.com/api/v2/cortex/v1): its Chat Completions SSE streams content deltas and a terminal data: [DONE] but never sends a finish_reason — not for plain text, not for tool calls (non-streaming responses return finish_reason: ""). The harness's model client (pi-ai) treats that as a truncated stream (Stream ended without finish_reason), and the harness maps it to a TRANSPORT error finish. Every request delivers content and then fails the turn.

This plugin rewrites only that specific terminal error into the successful finish the content already deserves.

How it works

The plugin listens on the harness's llm/stream waterfall (registered globally and prepended, so its returned iterable is the one consumers iterate) and wraps every model stream:

• a tool-call block was delivered → the terminal finish becomes { kind: 'tool-calls' }
• text content was delivered → the terminal finish becomes { kind: 'stop' }
• nothing was delivered, or the error is anything else → passed through untouched

The rewrite is gated on both the exact pi-ai message (Stream ended without finish_reason) and on delivered content, so genuine mid-stream truncations (which surface as different pi-ai errors) are never masked.

No changes to pi-ai, dsh-llm, or dsh-llm-pi-ai are required.

Requirements

• DeepSeek Harness (any recent deployment; @deepseek-ai/cordis ≥ 4 and @deepseek-ai/schemastery ≥ 3 are already present)
• Node ≥ 20
• The plugin must be installed into a profile (e.g. web) and mounted in the host composition — it is host-level and single-instance. Do not mount it inside an agent preset.

Install

The package is shipped as a profile bundle: it carries its own cordis.patch.yml layer (declared via dsh.bundle.patch in package.json), so installing it mounts the plugin with no manual composition editing.

Bundle install (recommended)

dsh plugin --profile web add /path/to/dsh-llm-finish-reason-tolerance-0.2.0.tgz

What this does:

1. pnpm installs the package into the profile's dependency tree.

2. The dsh plugin CLI detects dsh.bundle.patch and automatically appends the package to the profile's dsh.profile.bundles in package.json:

   "dsh": { "profile": { "bundles": ["@deepseek-ai/dsh-base", "@deepseek-ai/dsh-web-app", "dsh-llm-finish-reason-tolerance"] } }
   

3. At the next boot, the bundle's own patch layer (cordis.patch.yml inside the package) inserts the llm-finish-reason-tolerance row into the host composition.

That's the whole install. Use <profile> instead of web for any other profile (e.g. dsh plugin --profile tui add …).

Plain install (alternative)

If you prefer a plain dependency (e.g. the package is only published to a registry without bundle semantics), install it and add the row yourself. The profile patch layer (~/.dsh/profiles/<profile>/cordis.patch.yml) is a list of patch operations, so a new row must be wrapped in insert: — a bare id: row is an id-targeted override of an existing entry and is silently skipped when none matches:

# from the profile directory (e.g. ~/.dsh/profiles/web)
pnpm add /path/to/dsh-llm-finish-reason-tolerance-0.2.0.tgz

# ~/.dsh/profiles/<profile>/cordis.patch.yml
- insert:
    - id: llm-finish-reason-tolerance
      name: dsh-llm-finish-reason-tolerance
      config:
        # Provider route keys from the `llm-pi-ai` settings section.
        # Empty (or omitted) applies to every provider.
        providers:
          - snowflake-cortex

Restart

A restart of the harness is required after install — the plugin is a new module and is only mounted at boot. After restart, llm-finish-reason-tolerance appears in the plugins list.

Do not combine both install methods (that would insert the row twice), and do not also mount the row inside an agent preset.

Configuration

Scoping the tolerance to specific routes

The bundle ships with providers: [] — the tolerance applies to every provider route. To restrict it to specific routes, override the row's config in your own profile patch layer. This works because bundle layers apply first and their inserted rows can be targeted by later layers (the config value replaces the bundle's wholesale, so state the full config):

# ~/.dsh/profiles/<profile>/cordis.patch.yml
- id: llm-finish-reason-tolerance
  config:
    providers:
      - snowflake-cortex

With the plain install, scoping is simply the providers value inside the insert: row above.

Using it with Snowflake Cortex

The Snowflake provider must be configured in the llm-pi-ai settings section with a route key that matches the plugin's providers list. With the default (empty) providers list the plugin applies to every route, so no matching is required; if you scope the plugin, the route key must match exactly (e.g. snowflake-cortex).

# ~/.dsh/settings.yaml
llm-pi-ai:
  providers:
    snowflake-cortex:
      displayName: Snowflake Cortex (SG)
      apiKeyEnv: SNOWFLAKE_CORTEX_API_KEY
      api: openai-completions
      baseURL: https://<account>.snowflakecomputing.com/api/v2/cortex/v1
      models:
        - id: claude-sonnet-5
          name: Claude Sonnet 5
        # e.g. a DeepSeek model your account serves (deepseek-r1 is deprecated)
        # - id: deepseek-r1
        #   name: DeepSeek R1

Notes:

• Use api: openai-completions with the full …/api/v2/cortex/v1 base URL. The OpenAI SDK authenticates with Authorization: Bearer from apiKeyEnv automatically — no headers needed.
• The plugin fixes the missing finish_reason; deepseek-r1 is deprecated at the endpoint and cannot be used regardless of the plugin.

Safety

• The rewrite fires only for pi-ai's Stream ended without finish_reason terminal error and only when content was delivered. All other errors — auth, rate limit, quota, genuine mid-stream truncation, empty responses — pass through unchanged.
• A response that ends without finish_reason and without content keeps its original error finish.
• Scope with providers to keep the behavior local to the routes that need it.

Development

npm test        # node --test — exercises the wrapper with synthetic chunk streams
npm pack        # build the distributable tarball

Layout:

• lib/index.js — the plugin (plain ESM, no build step; the harness loads it directly)
• lib/index.d.ts — TypeScript declarations
• test/finish-reason.test.mjs — self-contained wrapper tests
• cordis.patch.yml — the bundle's own patch layer (auto-applied on bundle install)
• cordis.example.yml — the mount row for the plain-install path

License

MIT