DEV Community

Xiao Ling
Xiao Ling

Posted on • Originally published at dynamsoft.com

Developing .NET MAUI iOS Apps to Scan Barcode, Document and MRZ

#dotnet #maui #ios #csharp

In the previous article, we demonstrated how to incorporate iOS MRZ detection frameworks into the MrzScannerSDK NuGet package. Having mastered the integration of iOS frameworks for a standalone NuGet package, we now turn our attention to integrating iOS SDKs for barcode, MRZ, and document detection into the comprehensive .NET MAUI library: Capture.Vision.Maui. Our ultimate goal is to enable the development of cross-platform .NET MAUI applications for Windows, Android and iOS using a single codebase.

Integrating iOS Frameworks into Document Detection NuGet Package

Based on our experience with integrating the MRZ detection SDK, we can apply similar steps to incorporate the iOS Document Detection SDK into the DocumentScannerSDK NuGet package.

Step 1: Add iOS Frameworks to the NuGet Package

The Dynamsoft Document Normalizer SDK offers a suite of frameworks for iOS:

  • DynamsoftCore
  • DynamsoftDocumentNormalizer
  • DynamsoftImageProcessing
  • DynamsoftIntermediateResult

First, we utilize Objective Sharpie to create the C# bindings for these frameworks. Following this, we add all iOS frameworks and generated files to an iOS binding project.

iOS binding project structure

The csproj file for the binding project is structured as follows:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>net7.0-ios</TargetFramework>
    <Nullable>enable</Nullable>
    <ImplicitUsings>true</ImplicitUsings>
    <AssemblyName>DocumentScannerSDK</AssemblyName>
    <IsBindingProject>true</IsBindingProject>
    <GeneratePackageOnBuild>true</GeneratePackageOnBuild>
  </PropertyGroup>

  <ItemGroup>
    <ObjcBindingApiDefinition Include="DynamsoftCore/ApiDefinitions.cs" />
    <ObjcBindingCoreSource Include="DynamsoftCore/StructsAndEnums.cs" />
    <NativeReference Include="DynamsoftCore.framework">
      <Kind>Framework</Kind>
      <Frameworks></Frameworks>
    </NativeReference>
  </ItemGroup>

  <ItemGroup>
    <ObjcBindingApiDefinition Include="DynamsoftDocumentNormalizer/ApiDefinitions.cs" />
    <ObjcBindingCoreSource Include="DynamsoftDocumentNormalizer/StructsAndEnums.cs" />
    <NativeReference Include="DynamsoftDocumentNormalizer.framework">
      <Kind>Framework</Kind>
      <Frameworks></Frameworks>
    </NativeReference>
  </ItemGroup>

  <ItemGroup>
    <ObjcBindingApiDefinition Include="DynamsoftImageProcessing/ApiDefinitions.cs" />
    <NativeReference Include="DynamsoftImageProcessing.framework">
      <Kind>Framework</Kind>
      <Frameworks></Frameworks>
    </NativeReference>
  </ItemGroup>

  <ItemGroup>
    <ObjcBindingApiDefinition Include="DynamsoftIntermediateResult/ApiDefinitions.cs" />
    <NativeReference Include="DynamsoftIntermediateResult.framework">
      <Kind>Framework</Kind>
      <Frameworks></Frameworks>
    </NativeReference>
  </ItemGroup>

</Project>
Enter fullscreen mode Exit fullscreen mode

Step 2: Align the C# API with Android's Implementation

Copy the DocumentScanner.cs file from the Android binding project into the iOS binding project. Modify the code as necessary to ensure it functions correctly on iOS.

  • InitLicense:

    public class LicenseVerification : LicenseVerificationListener
{
    public override void LicenseVerificationCallback(bool isSuccess, NSError error)
    {
        if (!isSuccess)
        {
            System.Console.WriteLine(error.UserInfo);
        }
    }
}

public static void InitLicense(string license, object? context = null)
{
    DynamsoftLicenseManager.InitLicense(license, new LicenseVerification());
}

  • SetParameters：

    public void SetParameters(string parameters)
{
    try
    {
        NSError error;
        normalizer.InitRuntimeSettingsFromString(parameters, out error);
    }
    catch (Exception ex)
    {
        Console.WriteLine(ex.ToString());
    }
}

  • DetectFile and DetectBuffer:

    public Result[]? DetectFile(string filename)
{
    NSError error;
    iDetectedQuadResult[]? results = normalizer.DetectQuadFromFile(filename, out error);
    return GetResults(results);
}

private Result[]? GetResults(iDetectedQuadResult[]? results)
{
    if (results == null) return null;

    var result = new Result[results.Length];
    for (int i = 0; i < results.Length; i++)
    {
        iDetectedQuadResult tmp = results[i];
        iQuadrilateral quad = tmp.Location;
        result[i] = new Result()
        {
            Confidence = (int)tmp.ConfidenceAsDocumentBoundary,
            Points = new int[8]
            {
                (int)((NSValue)quad.Points[0]).CGPointValue.X, (int)((NSValue)quad.Points[0]).CGPointValue.Y,
                (int)((NSValue)quad.Points[1]).CGPointValue.X, (int)((NSValue)quad.Points[1]).CGPointValue.Y,
                (int)((NSValue)quad.Points[2]).CGPointValue.X, (int)((NSValue)quad.Points[2]).CGPointValue.Y,
                (int)((NSValue)quad.Points[3]).CGPointValue.X, (int)((NSValue)quad.Points[3]).CGPointValue.Y,
            }
        };
    }

    return result;
}

public Result[]? DetectBuffer(byte[] buffer, int width, int height, int stride, ImagePixelFormat format)
{
    NSData converted = NSData.FromArray(buffer);

    iImageData imageData = new iImageData()
    {
        Bytes = converted,
        Width = width,
        Height = height,
        Stride = stride,
        Format = (EnumImagePixelFormat)format,
    };
    NSError error;
    iDetectedQuadResult[]? results = normalizer.DetectQuadFromBuffer(imageData, out error);
    return GetResults(results);
}

  • NormalizeFile and NormalizeBuffer:

    public NormalizedImage NormalizeFile(string filename, int[] points)
{
    iQuadrilateral quad = new iQuadrilateral();
    quad.Points = new NSObject[4];
    quad.Points[0] = NSValue.FromCGPoint(new CGPoint(points[0], points[1]));
    quad.Points[1] = NSValue.FromCGPoint(new CGPoint(points[2], points[3]));
    quad.Points[2] = NSValue.FromCGPoint(new CGPoint(points[4], points[5]));
    quad.Points[3] = NSValue.FromCGPoint(new CGPoint(points[6], points[7]));

    NSError error;
    iNormalizedImageResult? result = normalizer.NormalizeFile(filename, quad, out error);
    return GetNormalizedImage(result);
}

public NormalizedImage NormalizeBuffer(byte[] buffer, int width, int height, int stride, ImagePixelFormat format, int[] points)
{
    iImageData imageData = new iImageData()
    {
        Bytes = NSData.FromArray(buffer),
        Width = width,
        Height = height,
        Stride = stride,
        Format = (EnumImagePixelFormat)format,
    };

    iQuadrilateral quad = new iQuadrilateral();
    quad.Points = new NSObject[4];
    quad.Points[0] = NSValue.FromCGPoint(new CGPoint(points[0], points[1]));
    quad.Points[1] = NSValue.FromCGPoint(new CGPoint(points[2], points[3]));
    quad.Points[2] = NSValue.FromCGPoint(new CGPoint(points[4], points[5]));
    quad.Points[3] = NSValue.FromCGPoint(new CGPoint(points[6], points[7]));
    NSError error;
    iNormalizedImageResult? result = normalizer.NormalizeBuffer(imageData, quad, out error);
    return GetNormalizedImage(result);
}

private NormalizedImage GetNormalizedImage(iNormalizedImageResult? result)
{
    NormalizedImage normalizedImage = new NormalizedImage();
    if (result != null)
    {
        iImageData imageData = result.Image;
        normalizedImage.Width = (int)imageData.Width;
        normalizedImage.Height = (int)imageData.Height;
        normalizedImage.Stride = (int)imageData.Stride;
        normalizedImage.Format = (ImagePixelFormat)imageData.Format;
        normalizedImage.Data = imageData.Bytes.ToArray();
    }
    return normalizedImage;
}

Step 3: Resolve Build Errors

The C# binding code generated may lead to build errors. These errors must be addressed and corrected manually.

iOS binding project build errors

Step 4: Compile the NuGet Package 

# build dll for desktop
cd desktop
dotnet build --configuration Release

# build dll for android
cd android
dotnet build --configuration Release

# build dll for iOS
cd ios
dotnet build --configuration Release

# build nuget package
nuget pack .\DocumentScannerSDK.nuspec
Enter fullscreen mode Exit fullscreen mode

Merging DocumentScannerSDK and MrzScannerSDK into a Single NuGet Package

The DynamsoftCore framework serves as a crucial dependency for both the MrzScannerSDK and DocumentScannerSDK NuGet packages. Utilizing both packages within a .NET MAUI project leads to a build conflict due to this shared dependency. To overcome this challenge, we introduced a new NuGet package named CaptureVision, which consolidates all frameworks and C# files from both MrzScannerSDK and DocumentScannerSDK for compilation.

capture vision iOS binding project

Resolving Build Conflicts between BarcodeQRCodeSDK and CaptureVision

The BarcodeQRCodeSDK NuGet package and the CaptureVision NuGet package are compatible within the same .NET MAUI project for Windows and Android. However, a build conflict arises when attempting to compile our demo application for iOS. This conflict is due to certain data structures, such as iImageData and iQuadrilateral, being defined in both the DynamsoftCore and BarcodeQRCodeSDK. To address this issue, a viable solution involves removing the redundant data structures from the BarcodeQRCodeSDK NuGet package.

Completing the Capture.Vision.Maui NuGet Package with iOS Support

Utilizing CaptureVision 2.0.1 and BarcodeQRCodeSDK 2.3.7, we are now able to finalize the Capture.Vision.Maui NuGet package by incorporating iOS-specific code within Platforms/iOS/NativeCameraView.cs. The code snippet below illustrates the process of detecting documents and MRZ from camera images on iOS.

if (cameraView.EnableDocumentDetect)
{
    DocumentScanner.Result[] results = documentScanner.DetectBuffer(bytearray,
                            (int)width,
                            (int)height,
                            (int)bpr,
                            DocumentScanner.ImagePixelFormat.IPF_ARGB_8888);
    DocumentResult documentResults = new DocumentResult();
    if (results != null && results.Length > 0)
    {
        documentResults = new DocumentResult
        {
            Confidence = results[0].Confidence,
            Points = results[0].Points
        };

        if (cameraView.EnableDocumentRectify)
        {
            NormalizedImage normalizedImage = documentScanner.NormalizeBuffer(bytearray,
                            (int)width,
                            (int)height,
                            (int)bpr,
                            DocumentScanner.ImagePixelFormat.IPF_ARGB_8888, documentResults.Points);
            documentResults.Width = normalizedImage.Width;
            documentResults.Height = normalizedImage.Height;
            documentResults.Stride = normalizedImage.Stride;
            documentResults.Format = normalizedImage.Format;
            documentResults.Data = normalizedImage.Data;
        }
    }

    cameraView.NotifyResultReady(documentResults, (int)width, (int)height);

}

if (cameraView.EnableMrz)
{
    MrzResult mrzResults = new MrzResult();
    try
    {
        MrzScanner.Result[] results = mrzScanner.DetectBuffer(bytearray,
                            (int)width,
                            (int)height,
                            (int)bpr,
                            MrzScanner.ImagePixelFormat.IPF_ARGB_8888);


        if (results != null && results.Length > 0)
        {
            Line[] rawData = new Line[results.Length];
            string[] lines = new string[results.Length];

            for (int i = 0; i < results.Length; i++)
            {
                rawData[i] = new Line()
                {
                    Confidence = results[i].Confidence,
                    Text = results[i].Text,
                    Points = results[i].Points,
                };
                lines[i] = results[i].Text;
            }


            Dynamsoft.MrzResult info = MrzParser.Parse(lines);
            mrzResults = new MrzResult()
            {
                RawData = rawData,
                Type = info.Type,
                Nationality = info.Nationality,
                Surname = info.Surname,
                GivenName = info.GivenName,
                PassportNumber = info.PassportNumber,
                IssuingCountry = info.IssuingCountry,
                BirthDate = info.BirthDate,
                Gender = info.Gender,
                Expiration = info.Expiration,
                Lines = info.Lines
            };

        }
    }

    catch (Exception ex)
    {
        System.Diagnostics.Debug.WriteLine(ex.Message);
    }

    cameraView.NotifyResultReady(mrzResults, (int)width, (int)height);
}
Enter fullscreen mode Exit fullscreen mode

Developing a .NET MAUI iOS App for Barcode, Document, and MRZ Detection

To adapt the existing demo app for iOS, simply replicate the license initialization code from the Android section and incorporate it into Platforms/iOS/Program.cs:

using ObjCRuntime;
using UIKit;
using Dynamsoft;

namespace Capture.Vision.Maui.Example
{
    public class Program
    {
        static void Main(string[] args)
        {
            DocumentScanner.InitLicense("LICENSE-KEY");
            BarcodeQRCodeReader.InitLicense("LICENSE-KEY");
            MrzScanner.InitLicense("LICENSE-KEY");

            UIApplication.Main(args, null, typeof(AppDelegate));
        }
    }
}
Enter fullscreen mode Exit fullscreen mode

.NET MAUI iOS: scan barcode, document and mrz

Source Code

https://github.com/yushulx/Capture-Vision-Maui

Top comments (1)

Collapse
 
jhonwills72 profile image
jhonwills •

Exciting series! Looking forward to learning more about developing .NET MAUI iOS apps with Prince Of persia barcode scanning and document features. Thanks for sharing your expertise in this area.

Read next

pvsdev profile image

WolvenKit code analysis: things to know before modding Cyberpunk 2077

Alexandra Kochetkova -

empiree profile image

My Journey in Open-Source Library Development

Oleg Dubovoi -

marcos_placona profile image

How to add Retrieval-Augmented Generation (RAG) to your app using generated SDKs

Marcos Placona -

islomali99 profile image

Input/Output Amallari

islomAli99 -