DEV Community

Cover image for Day 5— Azure Open AI Challenge: Document Intelligence
Sukhpinder Singh
Sukhpinder Singh

Posted on • Originally published at singhsukhpinder.Medium

Day 5— Azure Open AI Challenge: Document Intelligence

Designed for developers familiar with Azure and generative AI, the guide walks you through the process of harnessing the power of the prebuilt model for document intelligence.

Introduction

The Microsoft team has recently launched an Open AI challenge, in which a developer can learn how to build Azure AI solutions and apps.

Prerequisite

  • Experience working with Azure and Azure portals.

  • An understanding of generative AI.

  • Experience in one high-level programming language like C# or Python

Getting Started

The article demonstrates the use of a Prebuilt Model — Invoice in the document intelligence service.

What is a Prebuilt Model — Invoice?

Use a predefined model to identify and collect standard elements from particular types of documents. In this instance, we employ the built-in invoice model to process and extract information from an invoice.

The guide will cover the following features

  • The steps to configure the document intelligence service on the Azure Portal.

  • A console application in C# to process a sample document.

Reference: Sample document

Step 1: Navigate to the Azure Portal

Search for Azure Document Intelligence and fill out the following details

Step 2: Choose Network

Step 3: Assign Identity

Enable system-assigned identity to grant the resource access to other existing resources. For this demonstration, we don't require identity.

Step 4: Add Tags

For this exercise tag names are not required. But in a production environment, it should be added as it’s a best practice.

Step 5: Review & Create

Post validation checks by Azure Cloud, and proceed with creating the resource. Make sure you review the details entered in the previous steps.

Let’s switch to code

Please find below the sample document highlighting the fields that will be processed as part of this exercise using C# code.

The expected output should be as follows along with the confidence score

    Vendor Name: 'CONTOSO LTD.', with confidence 0.93.
    Customer Name: 'MICROSOFT CORPORATION', with confidence 0.915.
    Invoice Total: '$110', with confidence 0.97.
Enter fullscreen mode Exit fullscreen mode

Step 1: Create a console application

To test image generation, create a console application in Visual Studio or Visual Studio Code.

    dotnet new console
Enter fullscreen mode Exit fullscreen mode

Step 2: Read the configuration

Read the configuration from appsettings.json file

    IConfiguration config = new ConfigurationBuilder()
        .AddJsonFile("appsettings.json")
        .Build();
    string? endpoint = config["AzureEndpoint"];
    string? apiKey = config["AzureKey"];
    string? documentUrl = config["DocumentUrl"];
Enter fullscreen mode Exit fullscreen mode

Step 3: Create a client

    var cred = new AzureKeyCredential(apiKey);
    var client = new DocumentAnalysisClient(new Uri(endpoint), cred);
Enter fullscreen mode Exit fullscreen mode

Step 4: Analyze the Invoice

Install the NuGet package using the following command

    dotnet add package Azure.AI.FormRecognizer
Enter fullscreen mode Exit fullscreen mode

and add the following code

    AnalyzeDocumentOperation operation = await client.AnalyzeDocumentFromUriAsync(WaitUntil.Completed, "prebuilt-invoice", fileUri);
    await operation.WaitForCompletionAsync();

    AnalyzeResult result = operation.Value;
Enter fullscreen mode Exit fullscreen mode

Step 5: Display Invoice Information

The final step is to display the invoice information to the user using the following code snippet.

To keep things straightforward, we’re not displaying every key-value pair returned by the service here. For a comprehensive list of all fields supported and their respective types, please refer to the Invoice concept page.

    foreach (AnalyzedDocument invoice in result.Documents)
    {
        if (invoice.Fields.TryGetValue("VendorName", out DocumentField? vendorNameField))
        {
            if (vendorNameField.FieldType == DocumentFieldType.String)
            {
                string vendorName = vendorNameField.Value.AsString();
                Console.WriteLine($"Vendor Name: '{vendorName}', with confidence {vendorNameField.Confidence}.");
            }
        }



        if (invoice.Fields.TryGetValue("CustomerName", out DocumentField? customerNameField))
        {
            if (customerNameField.FieldType == DocumentFieldType.String)
            {
                string customerName = customerNameField.Value.AsString();
                Console.WriteLine($"Customer Name: '{customerName}', with confidence {customerNameField.Confidence}.");
            }
        }

        if (invoice.Fields.TryGetValue("InvoiceTotal", out DocumentField? invoiceTotalField))
        {
            if (invoiceTotalField.FieldType == DocumentFieldType.Currency)
            {
                CurrencyValue invoiceTotal = invoiceTotalField.Value.AsCurrency();
                Console.WriteLine($"Invoice Total: '{invoiceTotal.Symbol}{invoiceTotal.Amount}', with confidence {invoiceTotalField.Confidence}.");
            }
        }
    }
    Console.WriteLine("\nAnalysis complete.\n");
Enter fullscreen mode Exit fullscreen mode

Step 6: Run the code

Make sure the document URL is added to the appsettings.json file.

The URL for the example document is provided below.

https://github.com/MicrosoftLearning/mslearn-ai-document-intelligence/blob/main/Labfiles/01-prebuild-models/sample-invoice/sample-invoice.pdf?raw=true

Complete Program.cs file

    // See https://aka.ms/new-console-template for more information


    // Build a config object and retrieve user settings.
    using Azure;
    using Azure.AI.FormRecognizer.DocumentAnalysis;
    using Microsoft.Extensions.Configuration;

    IConfiguration config = new ConfigurationBuilder()
        .AddJsonFile("appsettings.json")
        .Build();
    string? endpoint = config["AzureEndpoint"];
    string? apiKey = config["AzureKey"];
    string? documentUrl = config["DocumentUrl"];

    Uri fileUri = new Uri(uriString: documentUrl);

    Console.WriteLine("\nConnecting to Forms Recognizer at: {0}", endpoint);
    Console.WriteLine("Analyzing invoice at: {0}\n", fileUri.ToString());

    // Create the client

    var cred = new AzureKeyCredential(apiKey);
    var client = new DocumentAnalysisClient(new Uri(endpoint), cred);

    // Analyze the invoice
    AnalyzeDocumentOperation operation = await client.AnalyzeDocumentFromUriAsync(WaitUntil.Completed, "prebuilt-invoice", fileUri);
    await operation.WaitForCompletionAsync();

    AnalyzeResult result = operation.Value;

    // Display invoice information to the user
    foreach (AnalyzedDocument invoice in result.Documents)
    {
        if (invoice.Fields.TryGetValue("VendorName", out DocumentField? vendorNameField))
        {
            if (vendorNameField.FieldType == DocumentFieldType.String)
            {
                string vendorName = vendorNameField.Value.AsString();
                Console.WriteLine($"Vendor Name: '{vendorName}', with confidence {vendorNameField.Confidence}.");
            }
        }



        if (invoice.Fields.TryGetValue("CustomerName", out DocumentField? customerNameField))
        {
            if (customerNameField.FieldType == DocumentFieldType.String)
            {
                string customerName = customerNameField.Value.AsString();
                Console.WriteLine($"Customer Name: '{customerName}', with confidence {customerNameField.Confidence}.");
            }
        }

        if (invoice.Fields.TryGetValue("InvoiceTotal", out DocumentField? invoiceTotalField))
        {
            if (invoiceTotalField.FieldType == DocumentFieldType.Currency)
            {
                CurrencyValue invoiceTotal = invoiceTotalField.Value.AsCurrency();
                Console.WriteLine($"Invoice Total: '{invoiceTotal.Symbol}{invoiceTotal.Amount}', with confidence {invoiceTotalField.Confidence}.");
            }
        }
    }
    Console.WriteLine("\nAnalysis complete.\n");
Enter fullscreen mode Exit fullscreen mode

Complete code on GitHub

Make sure to give it a star on GitHub and provide feedback on how to improve the tool further..!!

To access the document intelligence sample, move to the directory samples > Azure.OpenAI.DocumentIntelligence

GitHub - ssukhpinder/AzureOpenAI: Sample for testing Azure Open AI GPT3 Turbo Model

C# Programming🚀

Thank you for being a part of the C# community! Before you leave:

If you’ve made it this far, please show your appreciation with a clap and follow the author! 👏️️

Follow us: X | LinkedIn | Dev.to | Hashnode | Newsletter | Tumblr

Visit our other platforms: GitHub | Instagram | Tiktok | Quora | Daily.dev

More content at C# Programming

Top comments (0)