OpenCV3.2 单目摄像头的标定与矫正

  1. 云栖社区>
  2. 博客>
  3. 正文

OpenCV3.2 单目摄像头的标定与矫正

jacoby 2018-08-01 16:10:00 浏览744
展开阅读全文

最近打省电赛,与双目立体视觉相关。要实现双目测距首先要进行摄像头的标定,单目标定主要是为了测定摄像头的内参矩阵和畸变矩阵。这方面有大量的博客和论文可以参考,以下贴一下《opencv计算机视觉编程攻略(第三版)》一书中的标定程序。


好几个月过去,没想到也有几百人看过这篇文章了。现在看来感觉自己的认识还是很粗浅的,只是单纯地调用下API,很多概念只是糊弄过去了。最近我看了《Learning OpenCV3》一书,上面关于标定和立体视觉方面的原理讲得很好,所以原理性的东西可以看看我的这篇读书笔记,当然不打算深入原理也无所谓了。

—— Jacob杨帮帮 10/29/2018


img_68ac507fdd1e939410e82555e0e52c1e.png
标定用的棋盘格

1.获取标定板

淘宝上有高精度的标定板,但是价格感人。精确度要求不高完全可以自己打印。这里我使用的是一位前辈的python程序自制一个棋盘格标定板 - 简书

"""
Created on Fri Jan  5 10:57:34 2018
@author: 晚晴风
"""
import cv2 
import numpy as np

width = 450
height = 350
length = 50

image = np.zeros((width,height),dtype = np.uint8)
print(image.shape[0],image.shape[1])

for j in range(height):
    for i in range(width):
        if((int)(i/length) + (int)(j/length))%2:
            image[i,j] = 255;
cv2.imwrite("pic/chess.jpg",image)
cv2.imshow("chess",image)
cv2.waitKey(0)

打印时根据需要缩放大小即可

2.opencv3标定程序

使用的是书中的程序。作者 Robert Laganiere 将一些关键操作封装起来,如寻找棋盘内角。

bool findChessboardCorners( InputArray image, Size patternSize, OutputArray corners,
                            int flags = CALIB_CB_ADAPTIVE_THRESH 
                            + CALIB_CB_NORMALIZE_IMAGE );

精确角点到亚像素(sub-pixel)级。

void cornerSubPix( InputArray image, InputOutputArray corners,
                   Size winSize, Size zeroZone,
                   TermCriteria criteria );

根据自己生成的 objectPoints 二维向量和由图片测得的 imagePoints 二维向量来进行标定,并生成内参矩阵(cameraMatrix),畸变矩阵(distCoeffs),旋转矢量(rvecs),平移矢量(tvecs)等参数。

double calibrateCamera( InputArrayOfArrays objectPoints,
                        InputArrayOfArrays imagePoints, Size imageSize,
                        InputOutputArray cameraMatrix, InputOutputArray distCoeffs,
                        OutputArrayOfArrays rvecs, OutputArrayOfArrays tvecs,
                        int flags = 0, TermCriteria criteria = TermCriteria(
                        TermCriteria::COUNT + TermCriteria::EPS, 30, DBL_EPSILON) );

用内参矩阵和畸变矩阵生成用于矫正图片的map,分x轴和y轴

void initUndistortRectifyMap( InputArray cameraMatrix, InputArray distCoeffs,
                              InputArray R, InputArray newCameraMatrix,
                              Size size, int m1type, OutputArray map1, 
                              OutputArray map2 );

使用上面的map1和map2进行矫正

void remap( InputArray src, OutputArray dst,
            InputArray map1, InputArray map2,
            int interpolation, int borderMode = BORDER_CONSTANT,
            const Scalar& borderValue = Scalar());

头文件如下。虽然不是很懂设计模式,但可以看到作者的c++功底还是可以的

/*------------------------------------------------------------------------------------------*\
This file contains material supporting chapter 11 of the book:
OpenCV3 Computer Vision Application Programming Cookbook
Third Edition
by Robert Laganiere, Packt Publishing, 2016.

This program is free software; permission is hereby granted to use, copy, modify,
and distribute this source code, or portions thereof, for any purpose, without fee,
subject to the restriction that the copyright notice may not be removed
or altered from any source or altered source distribution.
The software is released on an as-is basis and without any warranties of any kind.
In particular, the software is not guaranteed to be fault-tolerant or free from failure.
The author disclaims all warranties with regard to this software, any use,
and any consequent failure, is purely the responsibility of the user.

Copyright (C) 2016 Robert Laganiere, www.laganiere.name
\*------------------------------------------------------------------------------------------*/


#ifndef CAMERACALIBRATOR_H
#define CAMERACALIBRATOR_H

#include <vector>
#include <iostream>

#include <opencv2/core.hpp>
#include "opencv2/imgproc.hpp"
#include "opencv2/calib3d.hpp"
#include <opencv2/highgui.hpp>

class CameraCalibrator {

    // input points:
    // the points in world coordinates
    // (each square is one unit)
    std::vector<std::vector<cv::Point3f> > objectPoints;
    // the image point positions in pixels
    std::vector<std::vector<cv::Point2f> > imagePoints;
    // output Matrices
    cv::Mat cameraMatrix;
    cv::Mat distCoeffs;
    // flag to specify how calibration is done
    int flag;
    // used in image undistortion 
    cv::Mat map1,map2; 
    bool mustInitUndistort;

  public:
    CameraCalibrator() : flag(0), mustInitUndistort(true) {}

    // Open the chessboard images and extract corner points
    int addChessboardPoints(const std::vector<std::string>& filelist, cv::Size & boardSize, std::string windowName="");
    // Add scene points and corresponding image points
    void addPoints(const std::vector<cv::Point2f>& imageCorners, const std::vector<cv::Point3f>& objectCorners);
    // Calibrate the camera
    double calibrate(const cv::Size imageSize);
    // Set the calibration flag
    void setCalibrationFlag(bool radial8CoeffEnabled=false, bool tangentialParamEnabled=false);
    // Remove distortion in an image (after calibration)
    cv::Mat remap(const cv::Mat &image, cv::Size &outputSize );

    // Getters
    cv::Mat getCameraMatrix() { return cameraMatrix; }
    cv::Mat getDistCoeffs()   { return distCoeffs; }
};

#endif // CAMERACALIBRATOR_H

具体实现如下

/*------------------------------------------------------------------------------------------*\
This file contains material supporting chapter 11 of the book:
OpenCV3 Computer Vision Application Programming Cookbook
Third Edition
by Robert Laganiere, Packt Publishing, 2016.

This program is free software; permission is hereby granted to use, copy, modify,
and distribute this source code, or portions thereof, for any purpose, without fee,
subject to the restriction that the copyright notice may not be removed
or altered from any source or altered source distribution.
The software is released on an as-is basis and without any warranties of any kind.
In particular, the software is not guaranteed to be fault-tolerant or free from failure.
The author disclaims all warranties with regard to this software, any use,
and any consequent failure, is purely the responsibility of the user.

Copyright (C) 2016 Robert Laganiere, www.laganiere.name
\*------------------------------------------------------------------------------------------*/


#include "CameraCalibrator.h"

// Open chessboard images and extract corner points
int CameraCalibrator::addChessboardPoints(
         const std::vector<std::string>& filelist, // list of filenames containing board images
         cv::Size & boardSize,                     // size of the board
         std::string windowName) {                 // name of window to display results
                                                   // if null, no display shown
    // the points on the chessboard
    std::vector<cv::Point2f> imageCorners;
    std::vector<cv::Point3f> objectCorners;

    // 3D Scene Points:
    // Initialize the chessboard corners 
    // in the chessboard reference frame
    // The corners are at 3D location (X,Y,Z)= (i,j,0)
    for (int i=0; i<boardSize.height; i++) {
        for (int j=0; j<boardSize.width; j++) {

            objectCorners.push_back(cv::Point3f(i, j, 0.0f));
        }
    }

    // 2D Image points:
    cv::Mat image; // to contain chessboard image
    int successes = 0;
    // for all viewpoints
    for (int i=0; i<filelist.size(); i++) {

        // Open the image
        image = cv::imread(filelist[i],0);

        // Get the chessboard corners
        bool found = cv::findChessboardCorners(image,         // image of chessboard pattern 
                                               boardSize,     // size of pattern
                                               imageCorners); // list of detected corners

        // Get subpixel accuracy on the corners
        if (found) {
            cv::cornerSubPix(image, imageCorners,
                cv::Size(5, 5), // half size of serach window
                cv::Size(-1, -1),
                cv::TermCriteria(cv::TermCriteria::MAX_ITER +
                    cv::TermCriteria::EPS,
                    30,     // max number of iterations 
                    0.1));  // min accuracy

            // If we have a good board, add it to our data
            if (imageCorners.size() == boardSize.area()) {

                // Add image and scene points from one view
                addPoints(imageCorners, objectCorners);
                successes++;
            }
        }

        if (windowName.length()>0 && imageCorners.size() == boardSize.area()) {
        
            //Draw the corners
            cv::drawChessboardCorners(image, boardSize, imageCorners, found);
            cv::imshow(windowName, image);
            cv::waitKey(500);
        }
    }

    return successes;
}

// Add scene points and corresponding image points
void CameraCalibrator::addPoints(const std::vector<cv::Point2f>& imageCorners, const std::vector<cv::Point3f>& objectCorners) {

    // 2D image points from one view
    imagePoints.push_back(imageCorners);          
    // corresponding 3D scene points
    objectPoints.push_back(objectCorners);
}

// Calibrate the camera
// returns the re-projection error
double CameraCalibrator::calibrate(const cv::Size imageSize)
{
    // undistorter must be reinitialized
    mustInitUndistort= true;

    //Output rotations and translations
    std::vector<cv::Mat> rvecs, tvecs;

    // start calibration
    return 
     cv::calibrateCamera(objectPoints, // the 3D points
                    imagePoints,  // the image points
                    imageSize,    // image size
                    cameraMatrix, // output camera matrix
                    distCoeffs,   // output distortion matrix
                    rvecs, tvecs, // Rs, Ts
                    CV_CALIB_USE_INTRINSIC_GUESS  );        // set options
//                  ,CV_CALIB_USE_INTRINSIC_GUESS);



}

// remove distortion in an image (after calibration)
cv::Mat CameraCalibrator::remap(const cv::Mat &image, cv::Size &outputSize) {

    cv::Mat undistorted;

    if (outputSize.height == -1)
        outputSize = image.size();

    if (mustInitUndistort) { // called once per calibration
    

   // cv::Mat dist = (cv::Mat_<double>(1, 14) << -107.8067046233813, 3606.394522865697, -0.003208480233032964, 0.00212257161649465, -706.3301131023582, -107.3590793091425, 3559.830030448713, 855.5629043718579, 0, 0, 0, 0, 0, 0);
   // dist = distCoeffs;

        cv::initUndistortRectifyMap(
            cameraMatrix,  // computed camera matrix
            distCoeffs,    // computed distortion matrix
            cv::Mat(),     // optional rectification (none) 
            cv::Mat(),     // camera matrix to generate undistorted
            outputSize,    // size of undistorted
            CV_32FC1,      // type of output map
            map1, map2);   // the x and y mapping functions

        mustInitUndistort= false;
    }

    // Apply mapping functions
    cv::remap(image, undistorted, map1, map2, 
        cv::INTER_LINEAR); // interpolation type

    return undistorted;
}


// Set the calibration options
// 8radialCoeffEnabled should be true if 8 radial coefficients are required (5 is default)
// tangentialParamEnabled should be true if tangeantial distortion is present
void CameraCalibrator::setCalibrationFlag(bool radial8CoeffEnabled, bool tangentialParamEnabled) {

    // Set the flag used in cv::calibrateCamera()
    flag = 0;
    if (!tangentialParamEnabled) flag += CV_CALIB_ZERO_TANGENT_DIST;
    if (radial8CoeffEnabled) flag += CV_CALIB_RATIONAL_MODEL;
}

main函数如下。将PATH宏改为自己的路径,PICS_NUM改为照片数量即可

#include <iostream>
#include <iomanip>
#include <vector>
#include <opencv2/core.hpp>
#include <opencv2/imgproc.hpp>
#include <opencv2/highgui.hpp>
#include <opencv2/features2d.hpp>
#include "CameraCalibrator.h"

#define PATH "F:/QtProjects/stero/stereo4/right"
#define PICS_NUM 17

int main()
{
    cv::Mat image;
    std::vector<std::string> filelist;

    // generate list of chessboard image filename
    // named chessboard01 to chessboard27 in chessboard sub-dir
    for (int i=1; i<=PICS_NUM; i++) {

        std::stringstream str;
        str << PATH << std::setw(2) << std::setfill('0') << i << ".png";
        std::cout << str.str() << std::endl;

        filelist.push_back(str.str());
        image= cv::imread(str.str(),0);

      //   cv::imshow("Board Image",image);
       //  cv::waitKey(500);
    }

    // Create calibrator object
    CameraCalibrator cameraCalibrator;
    // add the corners from the chessboard
    cv::Size boardSize(8,6);
    cameraCalibrator.addChessboardPoints(
        filelist,   // filenames of chessboard image
        boardSize, "Detected points");  // size of chessboard

    // calibrate the camera
    cameraCalibrator.setCalibrationFlag(true,true);
    cameraCalibrator.calibrate(image.size());

    // Exampple of Image Undistortion
    std::cout << filelist[5] << std::endl;
    image = cv::imread(filelist[5],0);
    cv::Size newSize(static_cast<int>(image.cols*1.5), static_cast<int>(image.rows*1.5));
    cv::Mat uImage= cameraCalibrator.remap(image, newSize);

    // display camera matrix
    cv::Mat cameraMatrix= cameraCalibrator.getCameraMatrix();
    cv::Mat distCoeffs=cameraCalibrator.getDistCoeffs();
    std::cout << " Camera intrinsic: " << cameraMatrix.rows << "x" << cameraMatrix.cols <<  std::endl;
    std::cout << cameraMatrix.at<double>(0,0) << " " << cameraMatrix.at<double>(0,1) << " " << cameraMatrix.at<double>(0,2) << std::endl;
    std::cout << cameraMatrix.at<double>(1,0) << " " << cameraMatrix.at<double>(1,1) << " " << cameraMatrix.at<double>(1,2) << std::endl;
    std::cout << cameraMatrix.at<double>(2,0) << " " << cameraMatrix.at<double>(2,1) << " " << cameraMatrix.at<double>(2,2) << std::endl;
    std::cout << distCoeffs.rows << "x" <<distCoeffs.cols << std::endl;
    std::cout << distCoeffs << std::endl;
    for(int i = 0;i < distCoeffs.cols;i++)
    {
        std::cout << distCoeffs.at<double>(0,i) << " " ;
    }
    std::cout <<std::endl;

    cv::namedWindow("Original Image");
    cv::imshow("Original Image", image);
    cv::namedWindow("Undistorted Image");
    cv::imshow("Undistorted Image", uImage);

    // Store everything in a xml file
    cv::FileStorage fs("calib.xml", cv::FileStorage::WRITE);
    fs << "Intrinsic" << cameraMatrix;
    fs << "Distortion" << cameraCalibrator.getDistCoeffs();

    cv::waitKey();
    return 0;
}

效果如下


img_26dd8ef907d0bb24b75dd6939dabac45.png
image.png

3.经验总结

  • 照片要不同角度和距离拍摄,最好15张以上,不要偷懒。
  • 出现矫正效果不好的情况,可以改变一下函数calibrateCamera中的参数flag。使用CV_CALIB_FIX_K3可以使畸变矩阵输出四个参数。根据网上博客的经验对于低端摄像头来说需要的参数数量一般使用5个或者4个比较合适。

网友评论

登录后评论
0/500
评论
jacoby
+ 关注