Automate Document Processing end-to-end with AI Builder

At Microsoft Ignite 2022, we were pleased to share advances in Intelligent Document Processing as well as new AI capabilities that will allow you to automate more scenarios with better performance, using Power Automate and AI Builder.  

Be sure to check out our breakout session, “Automate your document processes with AI Builder” for live demos of the latest capabilities being introduced. 

New model capabilities

Unstructured Document Processing

Most documents used in the enterprise are based off free-form layouts that don’t have a specific content structure. This is true for documents like contracts, statements of work, letters or resumes. Visualizing and extracting content from these documents is even more difficult than with a structured form, as the fields aren’t properly labeled and can be anywhere in the document. AI Builder helps to solve this problem with the Unstructured Document Processing capability based off Azure Form Recognizer’s Custom Document Neural Model.

With Unstructured Document Processing generally available, you can train a custom AI model to identify and extract specific fields that will help you automate how to process information from all types of documents.

Improved accuracy and language support with Azure Form Recognizer 3.0

AI Builder’s document processing models now leverage Azure Form Recognizer v3.0, improving detection accuracy and extending language support. Highlights of improvements include:

• Custom Document Processing adds support for 42 new languages including Arabic, Hindi, and other languages using Arabic and Devanagari scripts to expand the coverage to 164 languages. Handwritten language support expands to Japanese and Korean.
• Pre-built invoice processing model now supports Spanish, French, Dutch, German, Italian and Portuguese.
• Pre-built receipt processing model now supports Spanish, French, German, Italian and Portuguese.
• Pre-built Identity Reader now supports US state ID, social security and green cards, as well as passport visa information.
  

  Multi-page Tables

  Documents across the enterprise like invoices often include large tables of data that can span across multiple pages, making it difficult to accurately extract information. AI Builder has re-built its multi-page table capability, allowing makers to tag tables across pages ensuring that the table data is extracted reliably.

  

  The multi-page table functionality is currently being rolled out and will be available in all regions by the end of October.

  Multi-Line Tagging

  When fields to extract span across two lines in a document, AI models struggle to extract data correctly. With AI Builder’s new continue tagging capability, we make it easy to train a model to identify and extract fields like an address that can be split across multiple lines in a document.

  

  Managing Model Performance

  Feedback Loop

  What happens when you try to process documents that are too different from the original samples you used to train AI Builder? When new documents are processed with specific layouts that differ too much from what you trained for, a low accuracy score can be returned. To help users manage this use case and improve model performance, we’ve created the Feedback Loop capability. Using the AI Builder feedback loop action in Power Automate, makers can automate the process of monitoring documents that weren’t processed correctly and send them back into a queue for re-training. Makers can now easily track new documents with low quality processing and improve their model’s performance.

  

  The Feedback loop functionality is currently being rolled out and it will be available in all regions by the end of October.

  Governing and sharing models

  With increased adoption of AI models to automate content processing, teams of makers want to collaborate on models to speed up the training cycle and transfer ownership rights. This is now possible with Model co-ownership. From the Model Details Page, users can share model ownership rights with other users, granting them the ability to re-train and re-publish models.

  Model co-ownership will be available in all regions by the end of 2022.

  To improve model governance, AI Builder models are now also visible in the Power Platform Center of Excellence (CoE) Starter Kit. Admins can monitor which AI models were created across environments and how they are being used in their tenant, giving the ability to better govern and foster AI innovation within their organization.

  Getting Started

  Head over to https://aka.ms/tryaibuilder to access all the latest capabilities, video tutorials and learning material and get started deploying AI models to make your organization more productive!