Migrate Speechmatics STT plugin to Agent STT by Smiljanic19A · Pull Request #7028 · livekit/agents

Smiljanic19A · 2026-08-28T14:50:44Z

The plugin is a translation layer between LiveKit's STT interface and the Speechmatics Agent STT endpoint (/v2/agent).
It used to talk to Speechmatics' older "voice" endpoint through the speechmatics-voice SDK. This PR moves it to the dedicated Agent STT endpoint (/v2/agent) and its SDK, speechmatics-agent-stt - and makes all the necessary consequential changes:

Changelog

Non-breaking:

Diarization settings (max_speakers, speaker_sensitivity, prefer_current_speaker,
known_speakers) are now sent to the server
Server errors surface as APIError on the stream.

Breaking:

OperatingPoint (enhanced/standard) is removed. Use model — the Model enum or a
string like "linden-1". operating_point still works but is deprecated, and only linden-1
is valid.
TurnDetectionMode is now just DEFAULT (server-side VAD) and EXTERNAL; the old
FIXED/ADAPTIVE/SMART_TURN values are gone. The default is DEFAULT :)
Speaker focus is removed: focus_speakers, ignore_speakers, focus_mode,
SpeakerFocusMode, update_speakers(). Agent STT has no equivalent.
Removed unsupported config fields: max_delay, end_of_utterance_silence_trigger, end_of_utterance_max_delay,
punctuation_overrides, speaker_passive_format.
Default base_url is now wss://global.rt.speechmatics.com/v2/agent. This might require some discussion.
Silero VAD is no longer auto-loaded for external mode.

What's left

Update docs
Use master Agent STT SDK once released

Context: problems in the previous plugin

Sessions transcribed nothing. Agent STT sends one segment per message with the text in a
transcript field; the old SDK expected a list of segments with a text field, so every
transcript was parsed against a shape that wasn't there and dropped. Now Segment.from_message
reads the singular segment the server actually sends.
The handshake was rejected. The old SDK appended fields Agent STT forbids (max_delay,
enable_entities, vad_config, ...), so StartRecognition was rejected and the socket closed
at startup. The plugin now sends only allowed fields.
Sessions hung forever. The default turn_detection_mode=EXTERNAL turns off server-side VAD,
but Agent STT needs that VAD to finalize a turn — so nothing ever finalized. The default is now
server-side VAD, which endpoints on its own; EXTERNAL stays for callers who drive turns via
finalize().
Wrong operating point. Agent STT only accepts linden-1, but the enum offered
enhanced/standard. Resolution now runs through _resolve_model(model, operating_point) —
model preferred, operating_point deprecated, conflict is a fail-fast error, default
linden-1.
Errors vanished. A server rejection was logged and swallowed, leaving the caller with a
timeout and no reason. Server errors now raise an APIError on the stream (non-retryable for a
bad config, retryable for a dropped connection).
Dead config surface. Voice-endpoint knobs Agent STT has no equivalent for — speaker focus,
punctuation overrides, end-of-utterance tuning — were exposed and either ignored or rejected.
They're removed.
Diarization was ignored. max_speakers and friends were accepted from the caller but never
sent; they're now put on the wire.
Hidden Silero dependency. External mode auto-loaded livekit.plugins.silero, an ImportError
waiting to happen on a package the plugin never declared. Nothing is auto-loaded now; pass a VAD
explicitly if you want one.

--- Import agent STT from pseudo-version + add constraints for model<->op

--- Revert everything except using agent STT SDK as the driver

--- model<->op sync

--- Rip out tdm handling

--- Added new tdm handling - depends on agent stt sdk

--- Completely remove Voice SDK

--- url handling

--- Error surfacing

--- Remove dead knobs

--- Wire dz fields to sdk

…t-migration # Conflicts: # livekit-plugins/livekit-plugins-speechmatics/livekit/plugins/speechmatics/stt.py # livekit-plugins/livekit-plugins-speechmatics/pyproject.toml

CLAassistant · 2026-08-28T14:50:57Z

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

devin-ai-integration

Note

This report is out of date. Scroll down for Devin Review's latest report on this PR.

Devin Review found 2 potential issues.

devin-ai-integration · 2026-08-28T14:53:12Z

        # Log all other messages
        else:
-            logger.debug(f"unhandled message: {event}", extra={"lk.pii.message": message})
+            logger.debug(f"{event} -> {message}")


🟡 Unhandled provider messages logged without PII marker

The catch-all branch interpolates the full provider message into the debug log body via f-string, dropping the prior lk.pii.message structured marker, so payloads that can contain customer content are logged unredacted.

Suggested change

logger.debug(f"{event} -> {message}")

logger.debug("unhandled speechmatics message", extra={"event": str(event), "lk.pii.message": message})

Was this helpful? React with 👍 or 👎 to provide feedback.

--- fix capability obs issue with dz

devin-ai-integration

Devin Review found 2 new potential issues.

2 flags not posted on this PR by your GitHub settings — view them in Devin Review. (Configure)

devin-ai-integration · 2026-08-28T15:14:40Z

MultiSpeakerAdapter

                streaming=True,
                interim_results=True,
-                diarization=enable_diarization if is_given(enable_diarization) else True,
+                diarization=True,


🟡 Diarization capability advertised even when disabled

The diarization capability is hardcoded to True regardless of enable_diarization=False, whereas it previously mirrored the flag. MultiSpeakerAdapter gates on this capability, so it accepts an STT that will never diarize.

Suggested change

diarization=True,

diarization=enable_diarization if is_given(enable_diarization) else True,

Was this helpful? React with 👍 or 👎 to provide feedback.

--- dz capability resolution

--- Added finalize gate if self.vad is not none

Smiljanic19A added 11 commits August 25, 2026 14:39

---LiveKit Migration to Agent STT

bbf1666

--- Import agent STT from pseudo-version + add constraints for model<->op

---LiveKit Migration to Agent STT

5c7bb2b

--- Revert everything except using agent STT SDK as the driver

---LiveKit Migration to Agent STT

bf9fd61

--- model<->op sync

---LiveKit Migration to Agent STT

7710029

--- Rip out tdm handling

---LiveKit Migration to Agent STT

1699004

--- Added new tdm handling - depends on agent stt sdk

---LiveKit Migration to Agent STT

0b72567

--- Completely remove Voice SDK

---LiveKit Migration to Agent STT

2063228

--- url handling

---LiveKit Migration to Agent STT

ee2652e

--- Error surfacing

---LiveKit Migration to Agent STT

96ff24e

--- Remove dead knobs

---LiveKit Migration to Agent STT

7b6ab32

--- Wire dz fields to sdk

Merge remote-tracking branch 'origin/main' into speechmatics-agent-st…

e68cf02

…t-migration # Conflicts: # livekit-plugins/livekit-plugins-speechmatics/livekit/plugins/speechmatics/stt.py # livekit-plugins/livekit-plugins-speechmatics/pyproject.toml

Smiljanic19A requested a review from a team as a code owner August 28, 2026 14:50

devin-ai-integration Bot reviewed Aug 28, 2026

View reviewed changes

---LiveKit Migration to Agent STT

142723f

--- fix capability obs issue with dz

devin-ai-integration Bot reviewed Aug 28, 2026

View reviewed changes

Smiljanic19A marked this pull request as draft August 28, 2026 15:16

Smiljanic19A added 2 commits August 28, 2026 17:21

---LiveKit Migration to Agent STT

23fe8b3

--- dz capability resolution

---LiveKit Migration to Agent STT

8667391

--- Added finalize gate if self.vad is not none

Navigation Menu

Uh oh!

Migrate Speechmatics STT plugin to Agent STT - #7028

Migrate Speechmatics STT plugin to Agent STT#7028
Smiljanic19A wants to merge 14 commits into
livekit:mainfrom
speechmatics:speechmatics-agent-stt-migration

Smiljanic19A commented Aug 28, 2026

Uh oh!

CLAassistant commented Aug 28, 2026

Uh oh!

devin-ai-integration Bot left a comment •

edited

Loading

Uh oh!

devin-ai-integration Bot Aug 28, 2026

Uh oh!

Uh oh!

devin-ai-integration Bot left a comment

Uh oh!

devin-ai-integration Bot Aug 28, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	logger.debug(f"{event} -> {message}")
	logger.debug("unhandled speechmatics message", extra={"event": str(event), "lk.pii.message": message})

	diarization=True,
	diarization=enable_diarization if is_given(enable_diarization) else True,

Uh oh!

Conversation

Smiljanic19A commented Aug 28, 2026

Changelog

What's left

Context: problems in the previous plugin

Uh oh!

CLAassistant commented Aug 28, 2026

Uh oh!

devin-ai-integration Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

devin-ai-integration Bot Aug 28, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

devin-ai-integration Bot left a comment

Choose a reason for hiding this comment

Uh oh!

devin-ai-integration Bot Aug 28, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

devin-ai-integration Bot left a comment •

edited

Loading