Exciting new capabilities for AI Builder

At Microsoft Build 2022, we are excited to announce new and upcoming capabilities for AI Builder that will allow citizen developers to create more intelligent apps and automation with AI.

Unstructured document processing

We are expanding Intelligent Document Processing (IDP) capabilities for Power Automate by adding support for unstructured document processing. AI Builder and Power Automate already enable everyone to build end-to-end workflows that automatically extract and process content from structured (for example, surveys and questionnaires) and semi-structured documents (for example, invoices and purchase orders). AI Builder now supports the ability to train a custom Document Processing AI model that can extract content from unstructured documents like contracts or letters.

Figure 1. Extracting key-value pairs from a contractual agreement

The new document processing capability leverages Azure Form Recognizer’s custom neural model and comes complete with a no code UI training wizard and model governance capabilities.

Model performance metrics for structured documents

To enable admins and makers to better measure the performance of their models, we’ve re-designed the structured document model page to include a global accuracy score. The page also features a breakdown of the training set as well as recommendations on how to improve the model. More information: Improve the performance of your processing model

Figure 2. Model accuracy score and evaluation for structured document processing models

General availability of Image Classification

Image Classification with Lobe is becoming generally available.  Lobe is a free, easy-to-use app from Microsoft that helps you build image classification models on your own computer. All you need to do is add and label images, and Lobe will automatically train a custom machine learning model for you. The Lobe model can be exported to AI Builder and used in Power Apps and Power Automate. More information: Image classification model by Lobe

Figure 3. Lobe app for AI-powered image recognition which can be used with AI Builder

Additions and enhancements to existing capabilities

• To improve transparency of AI Builder credits and facilitate capacity management, the Power Platform Licensing guide has been updated to include individual credit consumption rates for each AI Builder scenario.
• AI Builder learning and training materials have been updated in the AI Builder in a day Lab. New training courses come complete with sample code and data sets to get you started building your first AI models easily.
• Text Recognition has been updated to support 164 languages. The pre-trained model allows you to extract printed or handwritten text from images, such as photos or street signs and products, as well as from documents. The model supports extracting printed text for English, French, German, Italian, Portuguese, Spanish, Chinese, Japanese, Korean, Russian, Arabic, Hindi, and other international languages that use Latin, Cyrillic, Arabic, and Devanagari scripts. The model now also supports extracting handwritten text in 9 languages: English, Chinese (Simplified), French, German, Italian, Japanese, Korean, Portuguese and Spanish.
• AI Builder is now available in South Korea.

Getting started

Get hands on with all the latest updates, discover which model best fits your scenario, and get inspiration from customer stories. Try AI Builder https://aka.ms/tryaibuilder

We are always working to improve AI Builder to meet your needs. Feel free to share your feedback with us in the AI Builder forum, raise an idea in the Power Automate Ideas, or email us directly at aihelpen@microsoft.com.