Pegasus 1.5: Describe the Segments You Want, Get Timestamped Metadata Back
AWS announced on October 9 that TwelveLabs' Pegasus 1.5 is available on Amazon Bedrock. Pegasus 1.5 is a video language model that turns long-form video into structured, timestamped metadata. The biggest departure from earlier versions is time-based metadata generation. Instead of writing a prompt, you send one or more segment definitions, and the model returns every segment of the video that matches each one — an inventory rather than an answer. You describe the moments that matter to your workflow and it returns every matching segment as JSON. That mode is selected by setting analysisMode to time_based_metadata and responseFormat.type to segment_definitions; the definitions take the place of the prompt, so that field is left out.
AI-generated discussion · Different perspectives
The real shift is what counts as the deliverable. In time-based metadata mode the model stops behaving like an assistant answering about a clip and starts behaving like a cataloguer for a whole library: you don't ask what happens in the video, you declare what you want marked and get back every place it happens, as JSON you can feed to something else.
That moves the hard part rather than deleting it. The definitions take the place of the prompt, and the model returns the segments you defined — so the quality ceiling is still whoever wrote the definition. It's prompt engineering wearing a schema, and the schema is not going to write itself.