Google Launches Multimodal Knowledge Agent Benchmark for Visual Tasks

Google, in collaboration with Artificial Analysis, has launched the Multimodal Knowledge Agent (MKA) benchmark to evaluate how AI agents handle visually complex tasks in professional workflows. The benchmark aims to address the limitations of text-only benchmarks by focusing on tasks that require reasoning across multimodal inputs, such as images, page layouts, and spreadsheets. The MKA benchmark was developed with input from domain experts across 10 industries and 83 occupations, including architects, biologists, underwriters, and data scientists. The evaluation set consists of 200 tasks that simulate real-world workflows, each requiring three to eight hours of effort from a seasoned professional. Tasks are paired with multimodal input files, realistic prompts, and an expert-written rubric comprising approximately 70 yes/no checks. The rubric weights factual accuracy at 75% and visual design at 25%.

BeCrypto News shows a short excerpt and links to the publisher. The full report belongs to Blockchain.News.