Building an AR-Enhanced Pharma Lookup App with Flutter, Dynamsoft Barcode SDK and Database

An AR-integrated pharmaceutical lookup application is a specialized software tool that combines Augmented Reality (AR) technology with a pharmaceutical information database. This integration provides an interactive and enhanced method for accessing drug-related data. Users can scan pharmacy barcodes using their mobile devices to retrieve information about the medication, such as its ingredients, dosage, potential side effects, and other relevant details. In this article, we will develop a Flutter application that demonstrates the pharmaceutical lookup scenario, utilizing the Dynamsoft Barcode SDK and a synthetic dataset.

Try the Demo App Built with Flutter and Dynamsoft Barcode SDK

https://yushulx.me/pharma-lookup/

Synthetic Dataset Generation and Deployment

We use ChatGPT to generate a synthetic dataset of pharmaceutical information and and upload it to a Google Sheets document. The dataset contains 10 rows of data, each including the lot number, drug name, manufacture date, expiration date, batch size, and quality check status.

Navigate to the Extensions menu and select Apps Script to access the App Script editor. Once there, create a new script.

Copy and paste the following code into a new file or the existing default file within the script editor in Google Sheets.

function doGet(e) {
  var records = [];
  var sheet = SpreadsheetApp.getActiveSpreadsheet().getActiveSheet();
  var rows = sheet.getDataRange().getValues();

  rows.forEach(function(row, index) {
    if (index !== 0) { 
      var record = {}; 
      record.LotNumber = row[0];
      record.MedicationName = row[1];
      record.ManufactureDate = row[2];
      record.ExpirationDate = row[3];
      record.BatchSize = row[4];
      record.QualityCheckStatus = row[5];
      records.push(record);
    }
  });

  return ContentService.createTextOutput(JSON.stringify(records))
    .setMimeType(ContentService.MimeType.JSON);
}

The code above will convert the data in the Google Sheets document to a JSON object and return it as a response to the HTTP request.

Click on the Deploy button in the script editor, then select New Deployment. For the deployment type, choose Web App, and under the Who has access option, select Anyone. Finally, click Deploy to complete the deployment process.

You can now access the synthetic dataset through the provided URL to retrieve the JSON data. Try using the data source: https://script.google.com/macros/s/AKfycbyPEx3THAbcLTNaJNOkQ1O3puTmQKXXOE_gkOGyKMzfIEUTr484qS8Dsi7-kTKpD333/exec.

With the lot numbers from the synthetic dataset, we can print out barcodes and label them on medical vials.

Real-time Pharmaceutical Lookup with Flutter Barcode Scanner

Since we have already implemented a Flutter barcode scanner app compatible with Windows, Android, iOS and web, we can repurpose the existing code and integrate new features. The source code is available on GitHub at: https://github.com/yushulx/flutter-barcode-scanner

To implement the AR-enhanced pharmaceutical lookup app, follow these two steps:

1. Fetch the pharmaceutical information from Google Sheets and store it in a global variable.
2. Match the scanned barcode with the corresponding lot number and display the pharmaceutical information.

Fetch the Dataset from Google Sheets

1. Add the http package to the pubspec.yaml file for handling HTTP requests.
   

   dependencies:
     flutter:
       sdk: flutter
     http: ^1.1.2
   

2. In the main.dart file, create a _fetchData() function that makes a GET HTTP request to fetch the JSON data.
   

   Future<void> _fetchData() async {
       final response = await http.get(Uri.parse(
           'https://script.google.com/macros/s/AKfycbyPEx3THAbcLTNaJNOkQ1O3puTmQKXXOE_gkOGyKMzfIEUTr484qS8Dsi7-kTKpD333/exec'));
   
       if (response.statusCode == 200) {
         List<dynamic> list = json.decode(response.body);
         database = {
           for (var item in list)
             item['LotNumber'] as String: Pharma.fromJson(item)
         };
       } else {
         throw Exception('Failed to load data');
       }
     }
   

   The Pharma class is defined as follows:
   

   class Pharma {
     final String lotNumber;
     final String medicationName;
     final DateTime manufactureDate;
     final DateTime expirationDate;
     final int batchSize;
     final String qualityCheckStatus;
   
     Pharma({
       required this.lotNumber,
       required this.medicationName,
       required this.manufactureDate,
       required this.expirationDate,
       required this.batchSize,
       required this.qualityCheckStatus,
     });
   
     factory Pharma.fromJson(Map<String, dynamic> json) {
       return Pharma(
         lotNumber: json['LotNumber'] as String,
         medicationName: json['MedicationName'] as String,
         manufactureDate: DateTime.parse(json['ManufactureDate']),
         expirationDate: DateTime.parse(json['ExpirationDate']),
         batchSize: int.parse(json['BatchSize'].toString()),
         qualityCheckStatus: json['QualityCheckStatus'] as String,
       );
     }
   }
   

   The JSON data will be converted to a Map object, with the lot number as the key and the Pharma object as the value.
   

   Map<String, Pharma> database = {};
   

Match the Barcode with the Lot Number and Display the Pharmaceutical Information

In the global.dart file, locate the paint() function, and then add the provided code within this function to render the pharmaceutical information on the screen.

if (database.containsKey(result.text)) {
  Pharma pharma = database[result.text]!;
    // Draw the pharmaceutical information
    // Background
    // Medication Name
    // Manufacture Date
    // Expiration Date
    // Batch Size
    // Quality Check Status
}

• Background:
  

  final bgPaint1 = Paint()
          ..color = Colors.green.withOpacity(0.6)
          ..style = PaintingStyle.fill;
  const double bgWidth = 220, bgHeight = 110;
  
  minY = minY - bgHeight - 10;
  var backgroundRect = Rect.fromLTWH(minX, minY + bgHeight, bgWidth, 5);
  canvas.drawRect(backgroundRect, bgPaint1);
  
  final bgPaint2 = Paint()
    ..color = Colors.black.withOpacity(0.6)
    ..style = PaintingStyle.fill;
  backgroundRect = Rect.fromLTWH(minX, minY, bgWidth, bgHeight);
  canvas.drawRect(backgroundRect, bgPaint2);
  

• Medication Name:
  

  const int xSpacing = 100, ySpacing = 20, padding = 10;
  const nameSpan = TextSpan(
    style: TextStyle(
        color: Colors.green, fontSize: 14, fontWeight: FontWeight.bold),
    text: 'Name',
  );
  final namePainter = TextPainter(
    text: nameSpan,
    textDirection: TextDirection.ltr,
  );
  namePainter.layout(minWidth: 0, maxWidth: size.width);
  namePainter.paint(canvas, Offset(padding + minX, padding + minY));
  
  var nameValue = TextSpan(
    style: const TextStyle(
        color: Colors.white, fontSize: 14, fontWeight: FontWeight.bold),
    text: pharma.medicationName,
  );
  final nameValuePainter = TextPainter(
    text: nameValue,
    textDirection: TextDirection.ltr,
  );
  nameValuePainter.layout(minWidth: 0, maxWidth: size.width);
  nameValuePainter.paint(
      canvas,
      Offset(padding + minX + xSpacing,
          padding + minY));
  

• Manufacture Date:
  

  const manufactureDateSpan = TextSpan(
          style: TextStyle(color: Colors.green, fontSize: 12),
          text: 'Manufacture Date',
        );
  final manufactureDatePainter = TextPainter(
    text: manufactureDateSpan,
    textDirection: TextDirection.ltr,
  );
  manufactureDatePainter.layout(minWidth: 0, maxWidth: size.width);
  manufactureDatePainter.paint(
      canvas,
      Offset(padding + minX,
          minY + ySpacing + padding)); 
  
  var manufactureDateValueSpan = TextSpan(
    style: const TextStyle(color: Colors.white, fontSize: 12),
    text:
        '${pharma.manufactureDate.day}/${pharma.manufactureDate.month}/${pharma.manufactureDate.year}',
  );
  final manufactureDateValuePainter = TextPainter(
    text: manufactureDateValueSpan,
    textDirection: TextDirection.ltr,
  );
  manufactureDateValuePainter.layout(minWidth: 0, maxWidth: size.width);
  manufactureDateValuePainter.paint(canvas,
      Offset(padding + minX + xSpacing, minY + ySpacing + padding));
  

• Expiration Date:
  

  const expirationDateSpan = TextSpan(
          style: TextStyle(color: Colors.green, fontSize: 12),
          text: 'Expiration Date',
        );
  
  final expirationDatePainter = TextPainter(
    text: expirationDateSpan,
    textDirection: TextDirection.ltr,
  );
  
  expirationDatePainter.layout(minWidth: 0, maxWidth: size.width);
  expirationDatePainter.paint(
      canvas,
      Offset(
          padding + minX,
          minY +
              2 * ySpacing +
              padding)); 
  
  var expirationDateValueSpan = TextSpan(
      style: const TextStyle(color: Colors.white, fontSize: 12),
      text:
          '${pharma.expirationDate.day}/${pharma.expirationDate.month}/${pharma.expirationDate.year}');
  
  final expirationDateValuePainter = TextPainter(
    text: expirationDateValueSpan,
    textDirection: TextDirection.ltr,
  );
  
  expirationDateValuePainter.layout(minWidth: 0, maxWidth: size.width);
  expirationDateValuePainter.paint(canvas,
      Offset(padding + minX + xSpacing, minY + 2 * ySpacing + padding));
  
  

• Batch Size:
  

  const batchSizeSpan = TextSpan(
          style: TextStyle(color: Colors.green, fontSize: 12),
          text: 'Batch Size',
        );
  
  final batchSizePainter = TextPainter(
    text: batchSizeSpan,
    textDirection: TextDirection.ltr,
  );
  
  batchSizePainter.layout(minWidth: 0, maxWidth: size.width);
  batchSizePainter.paint(
      canvas,
      Offset(
          padding + minX,
          minY +
              3 * ySpacing +
              padding)); 
  
  var batchSizeValueSpan = TextSpan(
      style: const TextStyle(color: Colors.white, fontSize: 12),
      text: pharma.batchSize.toString());
  
  final batchSizeValuePainter = TextPainter(
    text: batchSizeValueSpan,
    textDirection: TextDirection.ltr,
  );
  
  batchSizeValuePainter.layout(minWidth: 0, maxWidth: size.width);
  batchSizeValuePainter.paint(canvas,
      Offset(padding + minX + xSpacing, minY + 3 * ySpacing + padding));
  

• Quality Check Status:
  

  const qualityCheckStatusSpan = TextSpan(
          style: TextStyle(color: Colors.green, fontSize: 12),
          text: 'Quality Status',
        );
  
  final qualityCheckStatusPainter = TextPainter(
    text: qualityCheckStatusSpan,
    textDirection: TextDirection.ltr,
  );
  
  qualityCheckStatusPainter.layout(minWidth: 0, maxWidth: size.width);
  qualityCheckStatusPainter.paint(
      canvas,
      Offset(
          padding + minX,
          minY +
              4 * ySpacing +
              padding)); 
  
  var qualityCheckStatusValueSpan = TextSpan(
      style: pharma.qualityCheckStatus == 'Passed'
          ? const TextStyle(color: Colors.green, fontSize: 12)
          : const TextStyle(color: Colors.red, fontSize: 12),
      text: pharma.qualityCheckStatus);
  
  final qualityCheckStatusValuePainter = TextPainter(
    text: qualityCheckStatusValueSpan,
    textDirection: TextDirection.ltr,
  );
  
  qualityCheckStatusValuePainter.layout(
      minWidth: 0, maxWidth: size.width);
  qualityCheckStatusValuePainter.paint(canvas,
      Offset(padding + minX + xSpacing, minY + 4 * ySpacing + padding));
  

Run the App and Scan the Barcode on the Medical Vial

flutter run
# flutter run -d windows
# flutter run -d edge

Source Code

https://github.com/yushulx/pharma-lookup