Multimodal models

Models that work with more than text — images, scans, audio or video.

Basics path · 7 of 7

What it is

Multimodal models accept, and sometimes produce, several kinds of input: photos, scans, screenshots, charts, speech or video. They can read a scanned invoice or describe what is in a photo from the production line.

Example

A screenshot of an error message is enough for the model to explain what went wrong.

What can go wrong

Images and files can carry hidden instructions or metadata, and a photo may show people, documents or screens you didn't mean to share.

How to secure it

Remove metadata, check what is visible before sending, and apply the same injection defenses to files as to text.

Related service: Applied AI