Use 2D barcodes for aligning image.

Articles about functionality of VintaSoft Imaging .NET SDK.

Moderator: Alex

Post Reply
Alex
Site Admin
Posts: 1474
Joined: Thu Jul 10, 2008 2:21 pm

Use 2D barcodes for aligning image.

Post by Alex » Tue Oct 04, 2016 6:03 pm

Forms processing and optical marks recognition (OMR) technology is widely used for classification, routing and recognition by templates
of forms of electronic documents like invoices, waybills, etc; recognition of optical marks filled into a form of test, questionnaire, ballots, etc.

VintaSoft Forms Processing .NET Plug-in can be used both for recognition of forms and for aligning image by template. This article describes how to use barcodes instead of lines for image identification and so improve the accuracy and reliability of recognition.

The image aligning process may be divided into 2 steps.

STEP 1: Identifying image by template.
The identification of image is performed using the TemplateMatchingCommand class. The class compares an input image with a template image and determines if images are similar. By default, the class uses template, which consists from the straight lines. We can override the logic of class and use a template, which consists from barcodes. The following 2 classes are necessary for overriding the image identification algorithm: As barcode example let us take the compact two-dimensional barcode Aztec Rune. Aztec Rune perfectly fits our task, because it proposes:
  • fast recognition algorithm
  • compactness (11x11 cells)
  • resistance to all types of distortions
  • high accuracy in determining the center of barcode
  • don't require a quite zone around
Aztec Rune stores one byte of information and this is more than enough for encoding a number of the point to which the barcode corresponds to.

The result of successful identification is a matrix of transforms, which defines the offsets, squeezing and rotation of image relatively to the template.


STEP 2: Aligning image by template.
The aligning of image is performed using the TemplateAligningCommand class. The class uses the matrix of transforms achieved in step 1.


EXAMPLE: For example let us use the template containing 3 barcodes:
Image

3 barcodes, which are not collinear (i.e. 3 reference points), this is a minimum allowable number of points necessary for correct work
of algorithm. It is allowed to use more barcodes for the case when they could be damaged or truncated.
The barcodes may be placed into any area of page, but the best accuracy in identification of distortions is reached when
barcodes are located in the edges of document.

Let us use the following images for test:
Image
Image
Image


Here is an example of console application, which performs the process of image comparison and aligning:

Code: Select all

public static void Test()
{
    AlignImages(
        new string[] { "BarcodeTemplateMatching_template1.png" },
        new string[] { 
            "BarcodeTemplateMatching_1_rotate90.png", 
            "BarcodeTemplateMatching_1_scale-rotate150.png",
            "BarcodeTemplateMatching_1_scale-rotate180.png"});
}

public static void AlignImages(string[] templates, string[] images)
{
    // create barcode key zone recognizer
    BarcodeKeyZoneRecognizerCommand barcodeRecognizer = new BarcodeKeyZoneRecognizerCommand();
    // set barcode reader settings
    barcodeRecognizer.Settings.ScanBarcodeTypes = BarcodeType.Aztec;
    barcodeRecognizer.Settings.ScanDirection = ScanDirection.Horizontal | ScanDirection.Vertical;
    barcodeRecognizer.Settings.AutomaticRecognition = true;
    // specify that image contains 3 barcodes
    barcodeRecognizer.Settings.ExpectedBarcodes = 3;

    // create template matching command
    TemplateMatchingCommand templateMatching = new TemplateMatchingCommand();
    templateMatching.ImageImprintGenerator = new ImageImprintGeneratorCommand(barcodeRecognizer);

    // add templates
    foreach (string template in templates)
        templateMatching.TemplateImages.Add(template);

    // process images
    foreach (string imageFilename in images)
    {
        using (VintasoftImage image = new VintasoftImage(imageFilename))
        {
            templateMatching.ExecuteInPlace(image);
            ImageImprintCompareResult compareResult = templateMatching.Result.ImageCompareResult;
            Console.WriteLine(imageFilename);
            Console.WriteLine(string.Format(" IsReliable ={0}", compareResult.IsReliable));
            if (compareResult.IsReliable)
            {
                Console.WriteLine(string.Format(" Confidence ={0}%", compareResult.Confidence * 100));
                Console.WriteLine(string.Format(" TransformMatrix ={0}", compareResult.TransformMatrix));
                TemplateAligningCommand templateAligning = new TemplateAligningCommand();
                templateAligning.CompareResult = compareResult;
                templateAligning.ExecuteInPlace(image);
                image.Save(string.Format("{0}_result.png", Path.GetFileNameWithoutExtension(imageFilename)));
            }
            Console.WriteLine();
        }
    }
}
As a result, we got the following images:
Image
Image
Image



Also the identification using barcodes can be used for recognition of forms (instead of default identification by lines).
This would allow to create a reliable forms recognition system, for case the forms are scanned and acquired from rather large number of different sources with different characteristics.

Here is an example that demonstrates how to identify a form using recognition of barcodes on the form image:

Code: Select all

BarcodeKeyZoneRecognizerCommand barcodeRecognizer = new BarcodeKeyZoneRecognizerCommand();
// set barcode reader settings
barcodeRecognizer.Settings.ScanBarcodeTypes = BarcodeType.Aztec;
barcodeRecognizer.Settings.ScanDirection = ScanDirection.Horizontal | ScanDirection.Vertical;
barcodeRecognizer.Settings.AutomaticRecognition = true;
// specify that image contains 3 barcodes
barcodeRecognizer.Settings.ExpectedBarcodes = 3;

FormRecognitionManager.TemplateMatching.ImageImprintGenerator = new ImageImprintGeneratorCommand(barcodeRecognizer);

Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest