Fix SAM streaming visual tracking when prompts arrive mid-stream, add… - #2891
Fix SAM streaming visual tracking when prompts arrive mid-stream, add…#2891Mike-Medvedev wants to merge 1 commit into
Conversation
… prompt_frame_number to SAM3 video block. The HF streaming session numbers frames internally from 0, but the video blocks registered visual prompts at the pipeline frame number. Whenever the two disagreed (pipeline numbering starts at 1), every streamed frame ran as an unprompted conditioning frame and the tracker emitted empty masks. Anchor prompt registration to the index the session assigns to the prompted image and stop forwarding the stream frame number from the blocks. prompt_frame_number lets visual mode hold its prompts until the stream reaches a chosen frame: earlier frames emit no detections and leave the session untouched, so a track correction can prompt mid-clip without a trimmed upload. Co-authored-by: Cursor <cursoragent@cursor.com>
|
👋 Thanks for the pull request! Here is how automated Claude review works here, so you spend credits (and reviewer time) wisely. 🚧 Right now this is a draft, so automated Claude review is paused — nothing is being spent yet. Mark it Ready for review to trigger it. Warning 💸 The Claude reviewer bills in credits, not vibesAutomated review spins up a real agent that reads real code and spends real credits on every pass. It is glad to help — but it is not a rubber duck, a linter you poke in a loop, or a substitute for reading the contributing guide. Treat it like an expensive senior reviewer whose time you booked, and show up prepared. Draft when unsure, Ready when you mean it:
However you get there, arrive prepared:
Reviews are not free. A draft costs nothing to review; a Ready PR is a promise that it is worth reviewing.
|
|
|
… prompt_frame_number to SAM3 video block.
The HF streaming session numbers frames internally from 0, but the video blocks registered visual prompts at the pipeline frame number. Whenever the two disagreed (pipeline numbering starts at 1), every streamed frame ran as an unprompted conditioning frame and the tracker emitted empty masks. Anchor prompt registration to the index the session assigns to the prompted image and stop forwarding the stream frame number from the blocks.
prompt_frame_number lets visual mode hold its prompts until the stream reaches a chosen frame: earlier frames emit no detections and leave the session untouched, so a track correction can prompt mid-clip without a trimmed upload.
What does this PR do?
Related Issue(s):
Type of Change
Testing
Test details:
Checklist
Additional Context