在 .NET/C# 中使用二维条形码对齐图像

博客类别:条形码表单处理.NET

2020/05/15

表单处理和光学标记识别 (OMR) 技术广泛应用于电子文档(如发票、运单等)的模板分类、路由和识别;以及识别填写在测试、问卷、选票等表单中的光学标记。

VintaSoft Forms Processing .NET Plug-in 既可用于表单识别,也可用于按模板对齐图像。本文介绍了如何使用条形码代替线条进行图像识别,从而提高识别的准确性和可靠性。

图像对齐过程包括通过模板识别图像并获取变换矩阵,该矩阵定义了图像相对于模板的偏移、压缩和旋转。接下来,使用该变换矩阵进行图像对齐。

通过基于条形码的关键区域使用模板识别图像。


图像识别是使用TemplateMatchingCommand类完成的。识别成功后,会得到一个变换矩阵,该矩阵定义了图像相对于模板的偏移、压缩和旋转。

TemplateMatchingCommand 类会创建输入图像和模板图像的印迹,比较印迹并判断图像是否相似。默认情况下,该类会基于直线生成图像印迹。

在本文中,我们将更改生成图像印记的算法,并使用紧凑型二维条形码 Aztec Rune 生成图像印记。 Aztec Rune 非常适合我们的任务,因为:

要使用基于条形码的关键区域进行图像识别,我们需要执行以下步骤:


按模板对齐图像,使用基于条形码的关键区域进行图像对齐


图像对齐是通过TemplateAligningCommand类实现的。该类使用图像识别过程中获得的变换矩阵。

要使用基于条形码的关键区域进行图像对齐,我们需要执行以下步骤:


图像对齐示例


以下图像将用作模板图像:
Document template image with key zones based on barcodes

它包含 3 个条形码,这些条形码不共线(即 3 个参考点),这是算法正确运行所需的最小参考点数量。如果条形码可能损坏或截断,则允许使用更多条形码。
条形码可以放置在页面的任何区域,但当条形码位于文档边缘时,识别失真的精度最高。

让我们使用 3 张测试文档图像。

这是一张旋转了 90 度的文档图像,关键区域基于条形码:
90-degree rotated document image with key zones based on barcodes

这是一张旋转并缩放了 150 度的文档图像,关键区域基于条形码:
150-degree rotated and scaled document image with key zones based on barcodes

这是一张旋转并缩放了 180 度的文档图像,关键区域基于条形码:
180-degree rotated and scaled document image with key zones based on barcodes


这是一个 C# 控制台应用程序示例,它执行图像比较和对齐过程:
public static void Test()
{
    AlignImages(
        new string[] { "BarcodeTemplateMatching_template1.png" },
        new string[] {
    "BarcodeTemplateMatching_1_rotate90.png",
    "BarcodeTemplateMatching_1_scale-rotate150.png",
    "BarcodeTemplateMatching_1_scale-rotate180.png"});
}

public static void AlignImages(string[] templates, string[] images)
{
    // create barcode key zone recognizer
    BarcodeKeyZoneRecognizerCommand barcodeRecognizer = new BarcodeKeyZoneRecognizerCommand();
    // set barcode reader settings
    barcodeRecognizer.Settings.ScanBarcodeTypes = Vintasoft.Barcode.BarcodeType.Aztec;
    barcodeRecognizer.Settings.ScanDirection = Vintasoft.Barcode.ScanDirection.Horizontal | Vintasoft.Barcode.ScanDirection.Vertical;
    barcodeRecognizer.Settings.AutomaticRecognition = true;
    // specify that image contains 3 barcodes
    barcodeRecognizer.Settings.ExpectedBarcodes = 3;

    // create an imprint generator, which is based on barcode key zones
    Vintasoft.Imaging.FormsProcessing.TemplateMatching.ImageImprintGeneratorCommand barcodeImprintGenerator =
        new Vintasoft.Imaging.FormsProcessing.TemplateMatching.ImageImprintGeneratorCommand(barcodeRecognizer);

    // create template matching command
    Vintasoft.Imaging.FormsProcessing.TemplateMatching.TemplateMatchingCommand templateMatchingCommand =
        new Vintasoft.Imaging.FormsProcessing.TemplateMatching.TemplateMatchingCommand();
    // specify that the template matching command must use imprint generator, which is based on barcode key zones
    templateMatchingCommand.ImageImprintGenerator = barcodeImprintGenerator;

    // for each template image file
    foreach (string template in templates)
    {
        // add template image file to the templates of template matching command
        templateMatchingCommand.TemplateImages.Add(template);
    }

    // for each input image file
    foreach (string imageFilename in images)
    {
        // create input image
        using (Vintasoft.Imaging.VintasoftImage image = new Vintasoft.Imaging.VintasoftImage(imageFilename))
        {
            // find a template for the image
            templateMatchingCommand.ExecuteInPlace(image);
            // get the image compare result
            Vintasoft.Imaging.FormsProcessing.TemplateMatching.ImageImprintCompareResult compareResult = templateMatchingCommand.Result.ImageCompareResult;

            // print image file name
            System.Console.WriteLine(imageFilename);
            // print the image comparison result
            System.Console.WriteLine(string.Format(" IsReliable ={0}", compareResult.IsReliable));

            // if template for image is found
            if (compareResult.IsReliable)
            {
                // print the image comparison confidence
                System.Console.WriteLine(string.Format(" Confidence ={0}%", compareResult.Confidence * 100));
                // print information about matrix, which defines image transformation relative to the template image
                System.Console.WriteLine(string.Format(" TransformMatrix ={0}", compareResult.TransformMatrix));

                // create the template aligning command
                Vintasoft.Imaging.FormsProcessing.TemplateMatching.TemplateAligningCommand templateAligningCommand =
                    new Vintasoft.Imaging.FormsProcessing.TemplateMatching.TemplateAligningCommand();
                // specify that template aligning command must use matrix, which defines image transformation relative to the template image
                templateAligningCommand.CompareResult = compareResult;
                // apply the aligning command to the image
                templateAligningCommand.ExecuteInPlace(image);
                // save aligned image to a file
                image.Save(string.Format("{0}_result.png", System.IO.Path.GetFileNameWithoutExtension(imageFilename)));
            }
            System.Console.WriteLine();
        }
    }
}

应用程序执行后,我们将获得未旋转和未缩放的文档图像。

以下是基于条形码关键区域的90度旋转文档图像的表单识别结果:
The form recognition result for 90-degrees rotated document image with keyzones based on barcodes

以下是基于条形码关键区域的150度旋转缩放文档图像的表单识别结果:
The form recognition result for 150-degree rotated and scaled document image with keyzones based on barcodes

以下是基于条形码关键区域的180度旋转缩放文档图像的表单识别结果:
The form recognition result for 180-degree rotated and scaled document image with keyzones based on barcodes



本文使用的源代码


以下是BarcodeKeyZone类的C#代码,用于确定包含条形码的关键区域:
/// <summary>
/// Represents key zone based on recognized barcode.
/// </summary>
public class BarcodeKeyZone : Vintasoft.Imaging.FormsProcessing.TemplateMatching.KeyZone
{

    #region Constructors

    /// <summary>
    /// Initializes a new instance of the <see cref="BarcodeKeyZone"/> class.
    /// </summary>
    /// <param name="barcodeInfo">The barcode information.</param>
    public BarcodeKeyZone(Vintasoft.Barcode.IBarcodeInfo barcodeInfo)
        : base()
    {
        _barcodeInfo = barcodeInfo;
        if (barcodeInfo is Vintasoft.Barcode.BarcodeInfo.AztecInfo)
        {
            // get the barcode center from barcode info
            _location = ((Vintasoft.Barcode.BarcodeInfo.AztecInfo)barcodeInfo).BulleyeCenter;
        }
        else
        {
            // calculate the barcode center

            _location = System.Drawing.PointF.Empty;
            System.Drawing.Point[] points = barcodeInfo.Region.GetPoints();
            for (int i = 0; i < points.Length; i++)
            {
                _location.X += points[i].X;
                _location.Y += points[i].Y;
            }
            _location.X /= points.Length;
            _location.Y /= points.Length;
        }
    }

    /// <summary>
    /// Initializes a new instance of the <see cref="BarcodeKeyZone"/> class.
    /// </summary>
    /// <param name="barcodeInfo">The barcode information.</param>
    /// <param name="location">The location.</param>
    private BarcodeKeyZone(Vintasoft.Barcode.IBarcodeInfo barcodeInfo, System.Drawing.PointF location)
        : base()
    {
        _barcodeInfo = barcodeInfo;
        _location = location;
    }

    #endregion



    #region Properties

    System.Drawing.PointF _location;
    /// <summary>
    /// Gets the location of the key zone on the image.
    /// </summary>
    public override System.Drawing.PointF Location
    {
        get
        {
            return _location;
        }
    }

    Vintasoft.Barcode.IBarcodeInfo _barcodeInfo;
    /// <summary>
    /// Gets the barcode information.
    /// </summary>
    public Vintasoft.Barcode.IBarcodeInfo BarcodeInfo
    {
        get
        {
            return _barcodeInfo;
        }
    }

    #endregion



    #region Methods

    /// <summary>
    /// Applies a transformation to the key zone.
    /// </summary>
    /// <param name="m">The <see cref="Vintasoft.Imaging.AffineMatrix" />
    /// that specifies the transformation to apply.</param>
    public override void Transform(Vintasoft.Imaging.AffineMatrix m)
    {
        if (m == null)
            return;
        _location = Vintasoft.Imaging.PointFAffineTransform.TransformPoint(m, _location);
    }

    /// <summary>
    /// Returns the similarity of specified <see cref="Vintasoft.Imaging.FormsProcessing.TemplateMatching.KeyZone" /> and the current key zone.
    /// </summary>
    /// <param name="zone">The <see cref="Vintasoft.Imaging.FormsProcessing.TemplateMatching.KeyZone" /> to compare with.</param>
    /// <returns>
    /// The similarity of specified keyzone and the current key zone in range from 0 to 1.<br />
    /// 0 means that zones are absolutely NOT similar;
    /// 1 means that zones are perfectly similar.
    /// </returns>
    public override double CalculateSimilarity(Vintasoft.Imaging.FormsProcessing.TemplateMatching.KeyZone zone)
    {
        BarcodeKeyZone barcodeZone = zone as BarcodeKeyZone;
        if (barcodeZone == null)
            return 0;

        if (barcodeZone.BarcodeInfo.Value == BarcodeInfo.Value &&
            barcodeZone.BarcodeInfo.BarcodeType == BarcodeInfo.BarcodeType)
            return 1;

        return 0;
    }

    /// <summary>
    /// Creates a new object that is a copy of the current instance.
    /// </summary>
    /// <returns>A new object that is a copy of this instance.</returns>
    public override object Clone()
    {
        return new BarcodeKeyZone(BarcodeInfo, Location);
    }

    #endregion

}


以下是BarcodeKeyZoneRecognizerCommand类的C#代码,用于定义搜索包含条形码的关键区域的命令:
/// <summary>
/// Recognizes key zones, based on barcodes, on an image.
/// </summary>
public class BarcodeKeyZoneRecognizerCommand : Vintasoft.Imaging.FormsProcessing.TemplateMatching.KeyZoneRecognizerCommand
{

    #region Constructors

    /// <summary>
    /// Initializes a new instance of the <see cref="BarcodeKeyZoneRecognizerCommand"/> class.
    /// </summary>
    public BarcodeKeyZoneRecognizerCommand()
        : base()
    {
    }

    #endregion



    #region Properties

    static System.Collections.ObjectModel.ReadOnlyCollection<Vintasoft.Imaging.PixelFormat> _supportedNativePixelFormats;
    /// <summary>
    /// Gets a list of supported pixel formats for this processing command.
    /// </summary>
    public override System.Collections.ObjectModel.ReadOnlyCollection<Vintasoft.Imaging.PixelFormat> SupportedNativePixelFormats
    {
        get
        {
            if (_supportedNativePixelFormats == null)
            {
                System.Collections.Generic.List<Vintasoft.Imaging.PixelFormat> supportedPixelFormats = new System.Collections.Generic.List<Vintasoft.Imaging.PixelFormat>();
                supportedPixelFormats.Add(Vintasoft.Imaging.PixelFormat.BlackWhite);
                supportedPixelFormats.Add(Vintasoft.Imaging.PixelFormat.Bgr24);
                supportedPixelFormats.Add(Vintasoft.Imaging.PixelFormat.Bgr32);
                supportedPixelFormats.Add(Vintasoft.Imaging.PixelFormat.Indexed1);
                supportedPixelFormats.Add(Vintasoft.Imaging.PixelFormat.Indexed8);
                supportedPixelFormats.Add(Vintasoft.Imaging.PixelFormat.Gray8);
                _supportedNativePixelFormats = supportedPixelFormats.AsReadOnly();
            }
            return _supportedNativePixelFormats;
        }
    }

    Vintasoft.Barcode.ReaderSettings _settings = new Vintasoft.Barcode.ReaderSettings();
    /// <summary>
    /// Gets or sets the barcode reader settings.
    /// </summary>
    /// <value>
    /// The reader settings.
    /// </value>
    public Vintasoft.Barcode.ReaderSettings Settings
    {
        get
        {
            return _settings;
        }
        set
        {
            if (value == null)
                throw new System.ArgumentNullException();
            _settings = value;
        }
    }

    #endregion



    #region Methods

    /// <summary>
    /// Creates a new <see cref="BarcodeKeyZoneRecognizerCommand"/> that is a copy of the current
    /// instance.
    /// </summary>
    /// <returns>A new <see cref="BarcodeKeyZoneRecognizerCommand"/> that is a copy of this
    /// instance.</returns>
    public override object Clone()
    {
        BarcodeKeyZoneRecognizerCommand recognizer = new BarcodeKeyZoneRecognizerCommand();
        recognizer.IsNested = IsNested;
        recognizer.RegionOfInterest = RegionOfInterest;
        recognizer.Settings = Settings.Clone();
        return recognizer;
    }


    /// <summary>
    /// Recognizes key zones in the specified rectangle of the specified image.
    /// </summary>
    /// <param name="image">The image where key zones must be searched.</param>
    /// <param name="rect">The region of interest on image.</param>
    /// <returns>An array of recognized key zones.</returns>
    protected override Vintasoft.Imaging.FormsProcessing.TemplateMatching.KeyZone[] Recognize(Vintasoft.Imaging.VintasoftImage image, System.Drawing.Rectangle rect)
    {
        Vintasoft.Barcode.BarcodeReader reader = new Vintasoft.Barcode.BarcodeReader();
        reader.Settings = Settings.Clone();
        reader.Settings.ScanRectangle = rect;
        using (System.Drawing.Bitmap bitmap = image.GetAsBitmap())
        {
            Vintasoft.Barcode.IBarcodeInfo[] infos = reader.ReadBarcodes(bitmap);
            Vintasoft.Imaging.FormsProcessing.TemplateMatching.KeyZone[] result = new Vintasoft.Imaging.FormsProcessing.TemplateMatching.KeyZone[infos.Length];
            for (int i = 0; i < infos.Length; i++)
                result[i] = new BarcodeKeyZone(infos[i]);
            return result;
        }
    }

    #endregion

}