Forms processing: Forms identification and recognition

In This Topic

'Form identification' is a process that compares the form image with one or more images of "form templates" and determines the "form template" that matches the "form". During the identification SDK calculates the transformation matrix that defines the translation, rotation and scale of "form" relative to the "form template". Also during the identification SDK determines the confidence to the results of "form" recognition.

The SDK would identify a "form", even if image of form is rotated or scaled comparing to the "form template". The SDK would NOT identify the "form", if image of form is dimensionally distorted or warped comparing to the "form template".

'Form recognition' is a process of data recognition from "form fields". During the "form" recognition SDK creates a list of recognized "form fields". For each recognized field SDK returns the field value and confidence to the recognition result.

FormRecognitionManager class is intended for identification and recognition of form. The process of identification and recognition can be performed for a single form or for several forms at once, that process can be synchronous or asynchronous. The class can provide information about progress of recognition. Also class allows to apply the processing commands to the template form image and completed form image before form identification.

For synchronous recognition of form is necessary to use FormRecognitionManager.Recognize method, for asynchronous recognition of form is necessary to use FormRecognitionManager.RecognizeAsync method.

The result of recognition is returned as an instance of FormRecognitionResult class. The class FormRecognitionResult contains the result of form identification - FormRecognitionResult.TemplateMatchingResult, and the result of form recognition - FormRecognitionResult.RecognizedPage.

A "form field" is represented by FormField class, which determines the field name, field value and confidence to the result of form recognition. Also class FormField contains reference to a template of form field. Here is a list of standard classes, which define form fields:

FormField - a base class for any "form field"
- OmrField - a "form field" that defines an abstract optical mark
  - OmrEllipticalField - a "form field" that contains recognized optical mark bounded by ellipse
  - OmrRectangularField - a "form field" that contains recognized optical mark bounded by rectangle
  - OmrFieldTable - a table that constructed from optical mark fields
- OcrField - a "form field" that contains recognized text
- FormFieldGroup - a group of "form fields"
  - FormFieldTable - a table of "form fields"

Architecture of FormField class is open, so user may define its own custom form fields if necessary. "Forms Processing Demo" application contains an example of form field that defines a barcode.

Here are articles, which describe how to recognize completed forms in different situations:

Process template form image and completed form image before form identification and recognition

Sometimes template form image and completed form images are not "ideal" and SDK allows to specify a list of image processing commands, which must be applied to the template form image and completed form image before form identification.

Here is C#/VB.NET code that demonstrates how to process form images before form identification (remove noise and apply Dilate command), identify and recognize the form:

/// <summary>
/// Recognizes single image synchronously.
/// </summary>
/// <param name="templateManager">The template manager.</param>
/// <param name="image">The image.</param>
public static void RecognizeSingleImageSync(
    Vintasoft.Imaging.FormsProcessing.FormRecognition.FormTemplateManager templateManager, 
    Vintasoft.Imaging.VintasoftImage image)
{
    // create template matching command
    Vintasoft.Imaging.FormsProcessing.TemplateMatching.TemplateMatchingCommand templateMatching = 
        new Vintasoft.Imaging.FormsProcessing.TemplateMatching.TemplateMatchingCommand();
    // set minimal confidence
    templateMatching.MinConfidence = 0.5f;
    // set template images
    templateMatching.TemplateImages = templateManager.TemplateImages;
    // create recognition manager
    Vintasoft.Imaging.FormsProcessing.FormRecognitionManager recognitionManager = 
        new Vintasoft.Imaging.FormsProcessing.FormRecognitionManager(
            templateMatching, templateManager);

    // if your form template contains OCR fields,
    // make sure OCR engine manager is initialized before recognition
    // (otherwise recognition will return null (Nothing)),
    // see OCR field recognition examples

    // subscribe to ImageRecognitionError event to output recognition errors
    recognitionManager.ImageRecognitionError += recognitionManager_ImageRecognitionError;

    // recognize the image
    Vintasoft.Imaging.FormsProcessing.FormRecognitionResult recognitionResult = recognitionManager.Recognize(image);

    // unsubscribe from ImageRecognitionError event
    recognitionManager.ImageRecognitionError -= recognitionManager_ImageRecognitionError;

    // if recognition failed with error (see ImageRecognitionError event handler output)
    if (recognitionResult == null)
        return;

    // get the result of image comparison
    Vintasoft.Imaging.FormsProcessing.TemplateMatching.ImageImprintCompareResult imageCompareResult =
        recognitionResult.TemplateMatchingResult.ImageCompareResult;
    // if result is not reliable
    if (!imageCompareResult.IsReliable)
    {
        // matching template is not found
        System.Console.WriteLine("Matching template is not found.");
    }
    else
    {
        // get recognized page
        Vintasoft.Imaging.FormsProcessing.FormRecognition.FormPage recognizedPage = recognitionResult.RecognizedPage;
        // write page info
        System.Console.WriteLine(string.Format(
            "Matching template: {0}; confidence: {1:F1}%.",
            recognizedPage.Name,
            imageCompareResult.Confidence * 100));

        // get form field count
        if (recognizedPage.Items.Count == 0)
        {
            System.Console.WriteLine("No form fields were recognized.");
        }
        else
        {
            System.Console.WriteLine(string.Format(
                "Recognized form field count: {0}",
                recognizedPage.Items.Count));
            // for each recognized form field
            foreach (Vintasoft.Imaging.FormsProcessing.FormRecognition.FormField recognizedField in recognizedPage.Items)
            {
                // write field info
                System.Console.WriteLine(string.Format(
                    "  Name: {0}; value: {1}; confidence: {2:F1}%",
                    recognizedField.Name,
                    recognizedField.Value,
                    recognizedField.Confidence * 100));
            }
        }
    }
}

/// <summary>
/// Handles the ImageRecognitionError event of the FormRecognitionManager.
/// </summary>
static void recognitionManager_ImageRecognitionError(object sender, 
    Vintasoft.Imaging.FormsProcessing.FormRecognitionErrorEventArgs e)
{
    System.Console.WriteLine(e.Exception.Message);
}

VB.NET

''' <summary>
''' Recognizes single image synchronously.
''' </summary>
''' <param name="templateManager">The template manager.</param>
''' <param name="image">The image.</param>
Public Shared Sub RecognizeSingleImageSync(templateManager As Vintasoft.Imaging.FormsProcessing.FormRecognition.FormTemplateManager, image As Vintasoft.Imaging.VintasoftImage)
    ' create template matching command
    Dim templateMatching As New Vintasoft.Imaging.FormsProcessing.TemplateMatching.TemplateMatchingCommand()
    ' set minimal confidence
    templateMatching.MinConfidence = 0.5F
    ' set template images
    templateMatching.TemplateImages = templateManager.TemplateImages
    ' create recognition manager
    Dim recognitionManager As New Vintasoft.Imaging.FormsProcessing.FormRecognitionManager(templateMatching, templateManager)

    ' if your form template contains OCR fields,
    ' make sure OCR engine manager is initialized before recognition
    ' (otherwise recognition will return null (Nothing)),
    ' see OCR field recognition examples

    ' subscribe to ImageRecognitionError event to output recognition errors
    AddHandler recognitionManager.ImageRecognitionError, AddressOf recognitionManager_ImageRecognitionError

    ' recognize the image
    Dim recognitionResult As Vintasoft.Imaging.FormsProcessing.FormRecognitionResult = recognitionManager.Recognize(image)

    ' unsubscribe from ImageRecognitionError event
    RemoveHandler recognitionManager.ImageRecognitionError, AddressOf recognitionManager_ImageRecognitionError

    ' if recognition failed with error (see ImageRecognitionError event handler output)
    If recognitionResult Is Nothing Then
        Return
    End If

    ' get the result of image comparison
    Dim imageCompareResult As Vintasoft.Imaging.FormsProcessing.TemplateMatching.ImageImprintCompareResult = recognitionResult.TemplateMatchingResult.ImageCompareResult
    ' if result is not reliable
    If Not imageCompareResult.IsReliable Then
        ' matching template is not found
        System.Console.WriteLine("Matching template is not found.")
    Else
        ' get recognized page
        Dim recognizedPage As Vintasoft.Imaging.FormsProcessing.FormRecognition.FormPage = recognitionResult.RecognizedPage
        ' write page info
        System.Console.WriteLine(String.Format("Matching template: {0}; confidence: {1:F1}%.", recognizedPage.Name, imageCompareResult.Confidence * 100))

        ' get form field count
        If recognizedPage.Items.Count = 0 Then
            System.Console.WriteLine("No form fields were recognized.")
        Else
            System.Console.WriteLine(String.Format("Recognized form field count: {0}", recognizedPage.Items.Count))
            ' for each recognized form field
            For Each recognizedField As Vintasoft.Imaging.FormsProcessing.FormRecognition.FormField In recognizedPage.Items
                ' write field info
                System.Console.WriteLine(String.Format("  Name: {0}; value: {1}; confidence: {2:F1}%", recognizedField.Name, recognizedField.Value, recognizedField.Confidence * 100))
            Next
        End If
    End If
End Sub

''' <summary>
''' Handles the ImageRecognitionError event of the FormRecognitionManager.
''' </summary>
Private Shared Sub recognitionManager_ImageRecognitionError(sender As Object, e As Vintasoft.Imaging.FormsProcessing.FormRecognitionErrorEventArgs)
    System.Console.WriteLine(e.Exception.Message)
End Sub

Display the recognized completed form on image in WinForms image viewer

FormFieldViewerTool class is intended for displaying the recognized completed form on image in WinForms image viewer.
Here is C#/VB.NET code that demonstrates how to recognize a form and display the results in WinForms image viewer.

/// <summary>
/// Recognizes the current image of specified viewer
/// and shows recognition results in viewer.
/// </summary>
/// <param name="viewer">The viewer.</param>
/// <param name="formRecognitionManager">The form recognition manager.</param>
public static bool RecognizeAndShowInViewer(
    Vintasoft.Imaging.UI.ImageViewer viewer,
    Vintasoft.Imaging.FormsProcessing.FormRecognitionManager formRecognitionManager)
{
    // recognize the image
    Vintasoft.Imaging.FormsProcessing.FormRecognitionResult recognitionResult = 
        formRecognitionManager.Recognize(viewer.Image);

    // get the result of image comparison
    Vintasoft.Imaging.FormsProcessing.TemplateMatching.ImageImprintCompareResult imageCompareResult =
        recognitionResult.TemplateMatchingResult.ImageCompareResult;
    // if image is recognized
    if (imageCompareResult.IsReliable)
    {
        // create a visual tool
        Vintasoft.Imaging.FormsProcessing.FormRecognition.UI.VisualTools.FormFieldViewerTool formFieldViewerTool = 
            new Vintasoft.Imaging.FormsProcessing.FormRecognition.UI.VisualTools.FormFieldViewerTool();
        // create a view for the recognized page
        Vintasoft.Imaging.FormsProcessing.FormRecognition.UI.FormFieldView pageView = 
            Vintasoft.Imaging.FormsProcessing.FormRecognition.UI.FormFieldViewFactory.CreateView(recognitionResult.RecognizedPage);
        // add the view to the collection of the visual tool
        formFieldViewerTool.FieldViewCollection.Add(pageView);
        // set the visual tool as current tool of the viewer
        viewer.VisualTool = formFieldViewerTool;
        return true;
    }
    else
    {
        return false;
    }
}

VB.NET

''' <summary>
''' Recognizes the current image of specified viewer
''' and shows recognition results in viewer.
''' </summary>
''' <param name="viewer">The viewer.</param>
''' <param name="formRecognitionManager">The form recognition manager.</param>
Public Shared Function RecognizeAndShowInViewer(viewer As Vintasoft.Imaging.UI.ImageViewer, formRecognitionManager As Vintasoft.Imaging.FormsProcessing.FormRecognitionManager) As Boolean
    ' recognize the image
    Dim recognitionResult As Vintasoft.Imaging.FormsProcessing.FormRecognitionResult = formRecognitionManager.Recognize(viewer.Image)

    ' get the result of image comparison
    Dim imageCompareResult As Vintasoft.Imaging.FormsProcessing.TemplateMatching.ImageImprintCompareResult = recognitionResult.TemplateMatchingResult.ImageCompareResult
    ' if image is recognized
    If imageCompareResult.IsReliable Then
        ' create a visual tool
        Dim formFieldViewerTool As New Vintasoft.Imaging.FormsProcessing.FormRecognition.UI.VisualTools.FormFieldViewerTool()
        ' create a view for the recognized page
        Dim pageView As Vintasoft.Imaging.FormsProcessing.FormRecognition.UI.FormFieldView = Vintasoft.Imaging.FormsProcessing.FormRecognition.UI.FormFieldViewFactory.CreateView(recognitionResult.RecognizedPage)
        ' add the view to the collection of the visual tool
        formFieldViewerTool.FieldViewCollection.Add(pageView)
        ' set the visual tool as current tool of the viewer
        viewer.VisualTool = formFieldViewerTool
        Return True
    Else
        Return False
    End If
End Function

Display the recognized completed form on image in WPF image viewer

WpfFormFieldViewerTool class is intended for displaying the recognized completed form on image in WPF image viewer.

Here is C#/VB.NET code that demonstrates how to recognize a form and display the results in WPF image viewer.

/// <summary>
/// Recognizes the current image of specified viewer
/// and shows recognition results in viewer.
/// </summary>
/// <param name="viewer">The viewer.</param>
/// <param name="formRecognitionManager">The form recognition manager.</param>
public static bool RecognizeAndShowInViewer(
    Vintasoft.Imaging.Wpf.UI.WpfImageViewer viewer,
    Vintasoft.Imaging.FormsProcessing.FormRecognitionManager formRecognitionManager)
{
    // recognize the image
    Vintasoft.Imaging.FormsProcessing.FormRecognitionResult recognitionResult = 
        formRecognitionManager.Recognize(viewer.Image);

    // get the result of image comparison
    Vintasoft.Imaging.FormsProcessing.TemplateMatching.ImageImprintCompareResult imageCompareResult =
        recognitionResult.TemplateMatchingResult.ImageCompareResult;
    // if image is recognized
    if (imageCompareResult.IsReliable)
    {
        // create a visual tool
        Vintasoft.Imaging.FormsProcessing.FormRecognition.Wpf.UI.VisualTools.WpfFormFieldViewerTool formFieldViewerTool = 
            new Vintasoft.Imaging.FormsProcessing.FormRecognition.Wpf.UI.VisualTools.WpfFormFieldViewerTool();
        // create a view for the recognized page
        Vintasoft.Imaging.FormsProcessing.FormRecognition.Wpf.UI.WpfFormFieldView pageView = 
            Vintasoft.Imaging.FormsProcessing.FormRecognition.Wpf.UI.WpfFormFieldViewFactory.CreateView(recognitionResult.RecognizedPage);
        // add the view to the collection of the visual tool
        formFieldViewerTool.FieldViewCollection.Add(pageView);
        // set transform matrix and resolution of template image
        formFieldViewerTool.SetTransform(
            imageCompareResult.TransformMatrix,
            imageCompareResult.SourceImprint.ImageSize.Resolution);
        // set the visual tool as current tool of the viewer
        viewer.VisualTool = formFieldViewerTool;
        return true;
    }
    else
    {
        return false;
    }
}

VB.NET

''' <summary>
''' Recognizes the current image of specified viewer
''' and shows recognition results in viewer.
''' </summary>
''' <param name="viewer">The viewer.</param>
''' <param name="formRecognitionManager">The form recognition manager.</param>
Public Shared Function RecognizeAndShowInViewer(viewer As Vintasoft.Imaging.Wpf.UI.WpfImageViewer, formRecognitionManager As Vintasoft.Imaging.FormsProcessing.FormRecognitionManager) As Boolean
    ' recognize the image
    Dim recognitionResult As Vintasoft.Imaging.FormsProcessing.FormRecognitionResult = formRecognitionManager.Recognize(viewer.Image)

    ' get the result of image comparison
    Dim imageCompareResult As Vintasoft.Imaging.FormsProcessing.TemplateMatching.ImageImprintCompareResult = recognitionResult.TemplateMatchingResult.ImageCompareResult
    ' if image is recognized
    If imageCompareResult.IsReliable Then
        ' create a visual tool
        Dim formFieldViewerTool As New Vintasoft.Imaging.FormsProcessing.FormRecognition.Wpf.UI.VisualTools.WpfFormFieldViewerTool()
        ' create a view for the recognized page
        Dim pageView As Vintasoft.Imaging.FormsProcessing.FormRecognition.Wpf.UI.WpfFormFieldView = Vintasoft.Imaging.FormsProcessing.FormRecognition.Wpf.UI.WpfFormFieldViewFactory.CreateView(recognitionResult.RecognizedPage)
        ' add the view to the collection of the visual tool
        formFieldViewerTool.FieldViewCollection.Add(pageView)
        ' set transform matrix and resolution of template image
        formFieldViewerTool.SetTransform(imageCompareResult.TransformMatrix, imageCompareResult.SourceImprint.ImageSize.Resolution)
        ' set the visual tool as current tool of the viewer
        viewer.VisualTool = formFieldViewerTool
        Return True
    Else
        Return False
    End If
End Function

Send Feedback