What's new?

📝 New task: Document Question Answering

Document Question Answering is the task of answering questions based on an image of a document. Document Question Answering models take a (document, question) pair as input and return an answer in natural language. Check out the docs for more info!

// npm i @xenova/transformers
import { pipeline } from '@xenova/transformers';

let image = 'https://huggingface.co/datasets/Xenova/transformers.js-docs/resolve/main/invoice.png';
let question = 'What is the invoice number?';

// Create document question answering pipeline
let qa_pipeline = await pipeline('document-question-answering', 'Xenova/donut-base-finetuned-docvqa');

// Run the pipeline
let output = await qa_pipeline(image, question);
// [{ answer: 'us-001' }]

🤖 New models

• Add support for DonutSwin models in https://github.com/xenova/transformers.js/pull/320
• Add support for Blenderbot and BlenderbotSmall in https://github.com/xenova/transformers.js/pull/292
• Add support for LongT5 models https://github.com/xenova/transformers.js/pull/316

💻 New example application

• In-browser semantic image search in https://github.com/xenova/transformers.js/pull/326 (demo, code, tweet)

  https://github.com/xenova/transformers.js/assets/26504141/c2ea6e69-2344-401e-8745-fdea3a0613ad

🐛 Misc. improvements

• Fixing more _call LSP errors + extra typings by @kungfooman in https://github.com/xenova/transformers.js/pull/304
• Remove CustomCache requirement for example browser extension project in https://github.com/xenova/transformers.js/pull/325

Full Changelog: https://github.com/xenova/transformers.js/compare/2.6.1...2.6.2