JP4224260B2 - Calibration apparatus, method, result diagnosis apparatus, and calibration chart - Google Patents

Description

ここで、Ｘ0、Ｙ0、Ｚ0は、前述したように、図９（Ａ）に示すような投影中心点Ｏｃの地上座標とする。
【００６２】
ここで、式（３）で求めたカメラの傾き（ω、φ、κ）を、式（４）中に代入し、回転行列の計算をして、回転行列の要素ａ11〜ａ33を求める。
【００６３】
次に、求めた回転行列の要素ａ11〜ａ33と式（３）で求めたカメラの位置（Ｘ0、Ｙ0、Ｚ0）、及びタ−ゲットの基準点座標（Ｘ，Ｙ，Ｚ）を共線条件式｛式（５）｝に代入し、タ−ゲットの画像座標（ｘ、ｙ）を求める。ここで、共線条件式とは、投影中心、写真像及び地上の対象物が一直線上にある場合に成立する関係式である。これにより、レンズ収差がない場合の第２マークの位置が算出されるので、レンズ収差のある現実のカメラで撮影した画像におけるタ−ゲットの概略の画像座標が求める。
ｘ＝−Ｃ・｛ａ11（Ｘ−Ｘ0）＋ａ12（Ｙ−Ｙ 0）＋ａ13（Ｚ−Ｚ0）｝／ ｛ａ31（Ｘ−Ｘ0）＋ａ32（Ｙ−Ｙ 0）＋ａ33（Ｚ−Ｚ0）｝
ｙ＝−Ｃ・｛ａ21（Ｘ−Ｘ0）＋ａ22（Ｙ−Ｙ 0）＋ａ23（Ｚ−Ｚ0）｝／ ｛ａ31（Ｘ−Ｘ0）＋ａ32（Ｙ−Ｙ 0）＋ａ33（Ｚ−Ｚ0）｝ （５）
【００６４】
ところで、式（３）中のｔａｎ^-1の演算では解が二つ求めるため、カメラの傾き（ω、φ、κ）はそれぞれ２つ解をもち全通りの計算を行う。そして、第１マーク抽出処理で計測した第１マーク１ａ、１ｂ、１ｃ、１ｄ４点の画像座標と、式（５）で求めた対応する４点の画像座標との残差の比較により、正解となるω、φ、κを算出する。
なお、ここでは射影変換式として二次の射影変換式を用いて説明したが、本発明はこれに限定されるものではなく、三次の射影変換式等の他の射影変換式を利用しても良い。
【００６５】
また、概略マーク位置測定部５では、例えばマーク座標記憶部１０に格納されている基準点座標ファイルに付加してある第２マークの管理番号を、各第１マークのタ−ゲット（第２マーク）に割り振ることにより、第２マークの対応づけを行う。
【００６６】
図７に戻り、精密マーク位置測定部６によって第２マークの精密位置測定を行う（Ｓ６０）。以下、図８を用いて第２マークの精密位置測定の処理手順を詳細に説明する。まず、精密マーク位置測定部６は、第２マークとしてのタ−ゲットを認識する（Ｓ６２）。このターゲット認識には、例えば正規化相関を用いたテンプレ−トマッチングを用いる。以下、ターゲット認識の詳細について説明する。
【００６７】
（III）ターゲット認識
図９（Ｂ）はターゲット認識に用いられる正規化相関のテンプレ−ト画像と対象画像の説明図である。まず、第１マーク抽出処理（Ｓ４０）で計測した第１マーク、例えば第１マーク１ａ、１ｂ、１ｃ、１ｄ４点のタ−ゲットの重心位置の中から、任意のタ−ゲットを選ぶ。正規化相関のテンプレ−ト画像は、選ばれたターゲットの重心位置（画像座標）を中心とする、Ｍ×Ｍ画素の画像とする。また、第２マーク概略位置測定（Ｓ５０）で算出したタ−ゲットの概略位置（画像座標）を中心として、Ｎ×Ｎ画素の画像を対象画像とする。
【００６８】
次に、対象画像に対して、式（６）に示す正規化相関によるテンプレ−トマッチングを施し、相関値が最大となる位置を求める。この相関値が最大値となる位置にて重ね合わせが達成され、最大値となる位置にてタ−ゲットが認識されたとみなす。ここでのテンプレ−ト画像の中心座標を等倍画像上の画像座標に換算し、検出点とする。
Ａ＝｛Ｍ^２×Σ（Ｘｉ×Ｔｉ）−ΣＸｉ×ΣＴｉ｝／
［｛Ｍ^２×ΣＸｉ^２−（ΣＸｉ)^２｝×｛Ｍ^２×ΣＴｉ^２−（ΣＴｉ)^２｝］
（６）
ここで、Ａは相関値、Ｍはテンプレ−ト画像の正方サイズ、Ｘｉは対象画像、Ｔｉはテンプレ−ト画像とする。また、画像の正方サイズＮ、Ｍは可変であるが、処理時間の高速化をはかるため、Ｎ、Ｍはタ−ゲットが十分格納できるのを前提にできるだけ小さくするのがよい。
【００６９】
図８に戻り、第２マークの精密位置測定を行うために、第２マークのサブピクセルエッジ検出を行う（Ｓ６４）。第２マークのサブピクセルエッジ検出を行う対象画像は、Ｓ６２でタ−ゲットと認識された検出点を中心としてＮ×Ｎ画素の画像とする。対象画像に存在する濃淡波形に、式（７）に示すガウス関数の二次微分であるラプラシアン・ガウシアン・フィルタ（ＬＯＧフィルタ）を施し、演算結果の曲線の２箇所のゼロ交差点、つまりエッジをサブピクセルで検出する。ここで、サブピクセルとは一画素よりも細かい精度で位置検出を行うことを言う。
∇^２・Ｇ（ｘ）＝｛（ｘ^２−２σ^２）／２πσ^６｝・ｅｘｐ（−ｘ^２／２σ^２） （７）
ここで、σはガウス関数のパラメ−タ−である。
【００７０】
次に、ターゲットの重心位置を検出し（Ｓ６６）、戻しとする（Ｓ６８）。ここでは、式（７）を用いて求めたｘ、ｙ方向のエッジ位置より、その交点をタ−ゲットの重心位置とする。なお、第２マークの精密位置測定は、Ｓ６２〜Ｓ６６に開示した処理に限定されるものではなく、他の重心位置検出法、例えばモーメント法やテンプレートマッチング法をさらに改良して利用するなど、どのような求め方をしても良い。
【００７１】
図７に戻り、全タ−ゲット重心位置の確認をし、一見明白な過誤のないことを確認する（Ｓ７０）。即ち、ターゲット認識されたターゲットの位置検出が適切であるか確認する。操作者による確認の便宜のために、検出されたターゲットの位置を表示部９に表示する。過誤がない場合には、Ｓ８０に行く。過誤があれば、不適切なターゲットの位置を修正する（Ｓ７５）。例えば、Ｓ６２で演算された相関値が低いターゲットや、重心検出位置が概略検出位置とあまりにかけ離れてしまったターゲットは、表示部９上にそのターゲット表示を赤くするなど、操作者に明確にわかるように表示する。すると、過誤のあったターゲットに関しては、操作者によるマニュアルにて計測しなおす（マウスで重心位置を指定する）。なお、ここで過誤のあったターゲット位置を無理に修正しなくとも、あとのキャリブレーションパラメータを求めるためのＳ９０の処理過程によっても、異常点として検出されるので、取り除くことが可能である。
【００７２】
そして、Ｓ３０〜Ｓ７５の処理を、レンズ収差の測定に必要な画像分繰り返す（Ｓ８０）。例えば、撮影された画像が５枚であれば、全ての５枚について繰り返しても良く、またレンズ収差の測定に必要な画像分に到達していれば、撮影された画像の全部を繰り返して処理しなくてもよい。
【００７３】
レンズ収差の測定に必要な画像分の計測処理を終了したら、次に演算処理部７のカメラの内部パラメータ演算処理を用いて、レンズ収差のキャリブレーション要素を求める処理に移る（Ｓ９０）。ここでは、キャリブレーション要素の演算対象として、チャート１上の第２マークについて、概略マーク位置測定部５と精密マーク位置測定部の処理により対応づけがなされ重心位置が求められている全ての第２マークについて行う。
【００７４】
（IV）：カメラの内部パラメータ演算処理（セルフキャリブレーション付きバンドル調整法）
演算処理部７のカメラの内部パラメータ演算処理としては、例えば写真測量分野で使用されている「セルフキャリブレーション付きバンドル調整法」を用いる。ここで、「バンドル調整」とは、被写体、レンズ、ＣＣＤ面を結ぶ光束（バンドル）は同一直線上になければならないという共線条件に基づき、各画像の光束１本毎に観測方程式をたて、最小２乗法によりカメラの位置と傾き（外部標定要素）と第２マークの座標位置を同時調整する方法である。「セルフキャリブレーション付き」とはさらに、キャリブレーション要素、即ちカメラの内部定位（レンズ収差、主点、焦点距離）を求めることができる方法である。セルフキャリブレーション付きバンドル調整法（以下単に「バンドル調整法」という）の共線条件基本式は、次の（式８）と（式９）である。
【００７５】
【数２】

【数３】

【００７６】
この（式８）と（式９）は、第１マーク抽出処理で説明した単写真標定の共線条件式（５）を基本式とするものである。即ちバンドル調整法は、（式８）と（式９）を用いて、複数画像から最小二乗近似して、各種解を算出する手法であり、各撮影位置のカメラの外部標定要素を同時に求めることが可能となる。即ち、カメラのキャリブレーション要素を求めることが可能となる。
【００７７】
次に、内部定位の補正モデル（レンズ収差）として、放射方向レンズ歪を有する場合の一例を次の（式１０）に示す。
【数４】

補正モデルはこれに限らず、使用レンズにあてはまるものを選択すればよい。これら計算は、基準点を地上座標と画像座標で６点以上あれば、逐次近似解法によって算出される。なお、演算処理部７では、逐次近似解法の閾値によって、閾値以上となった場合の誤差の大きいチャート１上の第２マークを省くことによって、正確なキャリブレーション要素を求めることが可能となる。そこで、ターゲット位置重心位置確認（Ｓ７０）において、誤差の大きい第２マークとして検出されなかった場合でも、Ｓ９０にて過誤のある第２マークを検出して、除去することが可能である。
【００７８】
図７に戻り、演算処理部７によるキャリブレーション要素を求める演算処理結果を判断し（Ｓ１００）、演算処理が収束しなかったり、或いは得られたキャリブレーション要素が適正と思われないものであった場合、Ｓ１１０にて対処する。Ｓ１１０では、過誤のある第２マークを含む画像を選択する。Ｓ９０におけるキャリブレーション終了時点で、演算処理部７によりどの画像のどの第２マークに過誤があるか判明しているので、その各画像における該当ターゲット検出点を表示して、確認する。
【００７９】
そして、操作者はマニュアル操作にて過誤のある第２マークを修正する（Ｓ１２０）。即ち、過誤のある第２マークの重心位置座標がずれて表示されているので、過誤のある第２マークとして表示されているマークを、適性として表示されている重心位置に移動させることで、修正が行われる。そして、過誤のある第２マークの位置修正が完了したか判断し（Ｓ１３０）、完了していればＳ９０のキャリブレーション要素演算に戻り、キャリブレーション要素を演算しなおす。他方、他に修正箇所があれば、Ｓ１１０に戻って、過誤のある第２マークの位置修正操作を繰り返す。
【００８０】
キャリブレーション要素を求める演算処理結果が適性であれば、結果を表示部９に表示する（Ｓ１４０）。図１０は、キャリブレーション要素の演算処理結果の一例を示す説明図である。例えば、表示部９への表示には、キャリブレーション要素である焦点距離、主点位置、歪曲パラメータを表示する。レンズ収差を示すディストーションについては、補正前曲線１０２、補正後曲線１０４、理想に補正された場合１０６、についてグラフィック表示するとわかりやすい。
【００８１】
さらに、キャリブレーションした結果に基づいて、ディストーション補正した画像を、画像処理部８にて作成して表示部９に表示することもできる。こうすれば、ディストーションの大きいカメラにて撮影した画像も、ディストーション補正されて表示する画像表示装置を提供することが可能となる。
【００８２】
次に、本発明のキャリブレーション結果診断装置について説明する。キャリブレーション結果診断装置は、キャリブレーション装置により計測されたレンズに対するキャリブレーション要素を利用して、当該レンズで撮影した立体視可能な画像データにより三次元計測する場合の精度を確認する場合に好適である。即ち、キャリブレーション結果診断装置は、厳密にキャリブレーションを行い、精度よく三次元計測を行いたい場合や、平面チャートを用いて計測したレンズに対するキャリブレーション要素に過誤がないかさらに検証したい場合に利用する。
【００８３】
図１１は本発明のキャリブレーション結果診断装置の一実施の形態を説明する構成ブロック図である。図において、立体基準チャート２０は、ターゲット２０ａ〜２０ｈの位置が三次元的に正確に測定されているもので、ターゲットの数、高さ、並びに平面的な座標は三次元計測に適するように適宜に定められている。カメラ２２は、立体基準チャート２０をステレオ撮影するもので、レンズ収差の補償対象となるカメラ２と同種のカメラを用いて、且つカメラ２と同じ焦点距離で撮影するのが良い。基準ステレオ画像データ２４は、立体基準チャート２０をカメラ２２にてステレオ視できるように撮影した画像データで、典型的には左撮影位置２２Ｌと右撮影位置２２Ｒの一組の撮影位置によって撮影されている。
【００８４】
ターゲット位置測定部２６では、基準ステレオ画像データ２４に撮影されているターゲット２０ａ〜２０ｈを抽出して、このターゲット２０ａ〜２０ｈの画像座標系での位置を測定する。表示部２８は、ＣＲＴや液晶等の表示装置で、例えば基準ステレオ画像データ２４やターゲット位置測定部２６で測定された立体基準チャート２０のターゲット位置を表示する。演算処理部３０は、ターゲット位置測定部２６で測定されたターゲット２０ａ〜２０ｈの画像座標系での位置情報から、相互標定や絶対標定に必要なパラメータを演算するもので、レンズ収差補償パラメータ記憶部１１に記憶されたキャリブレーション要素が用いられる。キャリブレーション要素は、キャリブレーション装置により計測されたレンズに対するものである。演算処理部３０での標定用演算には、航空写真測量等で使用されるステレオ画像による表面形状の凸凹測定に必要な演算手法が用いられる。
【００８５】
立体基準チャートターゲット記憶部３２には、立体基準チャート２０の全ターゲット２０ａ〜２０ｈの位置が記憶されている。キャリブレーション判定部としての収差補償パラメータ適性度判定部３４は、演算処理部３０で演算された立体基準チャート２０のターゲット位置と、立体基準チャートターゲット記憶部３２に記憶されたターゲット２０ａ〜２０ｈの位置とを比較して、レンズ収差補償パラメータ記憶部１１に記憶されたキャリブレーション要素によるレンズ収差の補償が適切であるか判定する。
【００８６】
ステレオ画像生成手段３６は、演算処理部３０にて相互標定された基準ステレオ画像データ２４を読込んで、基準ステレオ画像データ２４のステレオ画像を生成するもので、例えば画像演算処理を高速に行うプロセッサが使用される。ここでステレオ画像とは、カメラ２２で撮影された右撮影方向と左撮影方向の一組のステレオ撮影された画像を偏位修正して、立体視できるように調整したものをいう。立体画像表示部３８は、ステレオ画像生成手段３６により形成された立体画像を立体表示する画像装置である。なお、ターゲット位置測定部２６、演算処理部３０、収差補償パラメータ適性度判定部３４並びにステレオ画像生成手段３６は、デジタル図化機あるいはパソコンの中に構成することもできる。
【００８７】
図１２は本発明のキャリブレーション結果診断装置を用いたキャリブレーション結果診断を説明するフローチャートである。まず、キャリブレーション結果診断装置に基準ステレオ画像データ２４を読込ませる（Ｓ２００）。このとき、立体基準チャートターゲット記憶部３２に対して、基準ステレオ画像データ２４に表示されたターゲット２０ａ〜２０ｈの正確な位置を入力しておくとよい。
【００８８】
次に、ターゲット位置測定部２６によって、基準ステレオ画像データ２４を表示部２８に表示する（Ｓ２１０）。この際に、ターゲット位置測定部２６は基準ステレオ画像データ２４にターゲットとして認識された画素を操作者が識別しやすいように画像処理してもよい。
【００８９】
次に、ターゲット位置測定部２６によって、基準ステレオ画像データ２４の左右（ステレオ）画像の各画像における対応するターゲット座標をそれぞれ計測する（Ｓ２２０）。典型的には、操作者が表示部２８に表示された画像上で、マウスを用いてターゲットを指示すると、ターゲット位置測定部２６によってターゲットの画像座標値が読み取られる。なお、ターゲット座標の画像座標値は、予め基準ステレオ画像データ２４の段階で、各画像毎に読込んで、ターゲット座標値としてキャリブレーション結果診断装置に入力してもよい。
【００９０】
次に、演算処理部３０によって、読込まれたターゲットの画像座標値により相互標定の演算処理を行う（Ｓ２３０）。この際に、読込まれたターゲットの画像座標値に対して演算処理部３０によって、レンズ収差補償パラメータ記憶部１１に記憶されたキャリブレーション要素によるレンズ収差の補償がなされる。また、立体基準チャートターゲット記憶部３２に記憶されたターゲット２０ａ〜２０ｈの正確な位置を用いると、演算処理部３０によって、絶対標定が可能となる。相互標定であればモデル座標値が、絶対標定であれば三次元座標値での比較が可能となる。なお、キャリブレーション装置により計測されたレンズに対するキャリブレーション要素が劣悪であれば、立体視自体が行えない状態であるため、標定ができないという事実によっても確認可能となる。
【００９１】
（Ｖ）：相互標定
ここで、相互標定の詳細について説明する。標定計算は航空写真測量等で用いられているもので、相互標定と絶対標定の２段階計算により、左右それぞれの撮像装置の位置等が求められる。相互標定では、以下の共面条件式によりステレオ撮影パラメータを求める。
【００９２】
図１３はモデル座標系ＸＹＺと左右のカメラ座標系ｘｙｚを用いて標定計算を説明する図である。モデル座標系ＸＹＺの原点Ｏを左側の投影中心にとり、右側の投影中心を結ぶ線をＸ軸にとるようにする。縮尺は、基線長ｌを単位長さにとる。このとき求めるパラメータは、左側のカメラのＺ軸の回転角κ１、Ｙ軸の回転角φ１、右側のカメラのＺ軸の回転角κ２、Ｙ軸の回転角φ２、Ｘ軸の回転角ω２の５つの回転角となる。この場合左側のカメラのＸ軸の回転角ω１は０なので、考慮する必要ない。
【００９３】
【数５】

このような条件にすると、（式１１）の共面条件式は（式１２）のようになり、この式を解けば各パラメータが求める。
【数６】

ここで、モデル座標系ＸＹＺとカメラ座標系ｘｙｚの間には、次に示すような座標変換の関係式が成り立つ。
【数７】

【００９４】
（式１１）〜（式１３）の式を用いて、次の手順により、未知パラメータを求める。
Ｖ−１…初期近似値は通常０とする。
Ｖ−２…共面条件式（式１２）を近似値のまわりにテーラー展開し、線形化したときの微分係数の値を（式１３）中の２つの式により求め、観測方程式をたてる。
Ｖ−３…最小二乗法をあてはめ、近似値に対する補正量を求める。
Ｖ−４…近似値を補正する。
Ｖ−５…補正された近似値を用いて、Ｖ−２〜Ｖ−５までの操作を収束するまで繰り返す。
【００９５】
以上の相互標定計算において、レンズ収差補償パラメータ記憶部１１に記憶されたキャリブレーション要素、即ちチャート１により求められたキャリブレーション要素を利用して計算する。具体的には、主点位置及びレンズ収差値Δｘ、Δｙはキャリブレーション要素により補正して計算する。また、画面距離（焦点距離）ｃもそのキャリブレーション値を利用して計算する。
【００９６】
さらに、絶対標定を行う場合には、立体基準チャートターゲット記憶部３２に記憶されたターゲット２０ａ〜２０ｈの位置を用いる。すると、縮尺が分るので、ターゲットの実座標上の三次元値を用いての比較が可能となる。
【００９７】
図１２に戻り、収差補償パラメータ適性度判定部３４により、レンズ収差補償パラメータ記憶部１１に記憶されたキャリブレーション要素によるレンズ収差の補償が適切であるか判定する（Ｓ２４０）。即ち、収差補償パラメータ適性度判定部３４は、演算処理部３０で演算された立体基準チャート２０のターゲット位置と、立体基準チャートターゲット記憶部３２に記憶されたターゲット２０ａ〜２０ｈの位置とを比較して、許容される誤差の範囲内であれば「適性」とする。許容される誤差の範囲は、計測条件から求めることができるが、好ましくはレンズ収差を補償した撮影データを利用する者の許容誤差範囲によって判定する。なお、チャート１によるキャリブレーション要素が誤差を含んでいれば、上述した相互標定演算が収束しない場合がありうる。その際は、収差補償パラメータ適性度判定部３４により、「不適性」として判定する。
【００９８】
ここでキャリブレーション要素の判定を終了してもよいが、以上の相互標定データと立体基準チャートターゲット記憶部３２に記憶されたターゲット２０ａ〜２０ｈの位置を利用して、ステレオ画像を作成し、確認することも可能となる。ステレオ画像を作成する場合は、さらに以下の処理を行う。
【００９９】
ステレオ画像生成手段３６により、基準ステレオ画像データ２４のステレオ画像を作成する（Ｓ２５０）。相互標定による外部標定要素演算によって、ステレオ画像の位置と傾きがわかるので、さらにキャリブレーション要素を加味して、ステレオ画像生成手段３６により最終的なステレオ画像を作成する。
【０１００】
そして、表示部２８に作成したステレオ画像を表示させる（Ｓ２６０）。あるいは画像データを出力させる。表示部２８に立体画像モニタがあれば、ここにステレオ画像を表示して視覚的に確認することができる。この場合、チャート１によるキャリブレーション要素が悪ければ、立体視のできない（例えば縦視差の除去できていない）画像が表示されるので確認できる。あるいは、基準座標位置と計算した測定座標位置を表示することによっても、立体的な位置の違いを表示部２８上で確認可能となる。仮に、ステレオモニタが無ければ、ステレオ画像データを出力して、他の表示装置にて表示確認も可能となる。
【０１０１】
表示されたステレオ画像をみて、気に入らない精度であれば、チャート１とカメラ２を利用したキャリブレーション作業全体をやり直して、レンズ収差のキャリブレーション要素を求める。
【０１０２】
図１４は本発明のキャリブレーション結果診断装置の第２の実施の形態を説明する構成ブロック図である。なお、図１４において前記図１１と同一作用をするものには同一符号を付して、説明を省略する。図において、基準演算処理部３１は、基準ステレオ画像データと共に用意された基準キャリブレーション要素を用いて、基準ステレオ画像データの標定演算を行い、基準点のモデル座標値又は三次元座標値を求める。収差補償パラメータ適性度判定部３４は、演算処理部３０で演算された基準点のモデル座標値又は三次元座標値と、基準演算処理部３１で演算された基準点のモデル座標値又は三次元座標値とを比較して、キャリブレーション要素の適否判断を行う。
【０１０３】
このように構成された装置においては、カメラ２２のレンズに対する基準キャリブレーション要素が予め用意されている。そして、収差補償パラメータ適性度判定部３４に用いる立体基準チャート２０における基準点の座標値が、基準演算処理手段３１で演算された基準点のモデル座標値又は三次元座標値として与えられる。そこで、例えばレンズ製造の分野の品質管理のように、多量にレンズを取扱う場合の個別レンズのキャリブレーション要素のバラツキを管理するのに適している。
【０１０４】
なお、本発明のキャリブレーション結果診断装置に用いられる基準ステレオ画像データ２４として、カメラ２と同じタイプのカメラによって、カメラ２と同じ焦点距離で予め撮影した２枚の基準ステレオ画像データを用いる場合を説明したが、本発明はこれに限定されるものではなく、例えば立体基準チャート２０の基準ステレオ画像座標データでもよい。また、立体基準チャートターゲット記憶部３２に立体基準チャート２０の全ターゲットの位置を記憶させて、レンズ収差のキャリブレーション要素によりレンズ収差を補償して観測したターゲットの位置と比較する場合を説明したが、本発明はこれに限定されるものではなく、例えば立体視できる程度にレンズ収差が補償されていればよい用途では、立体基準チャート２０の全ターゲットの位置情報を用いなくても良い。
【０１０５】
また、基準ステレオ画像データ、基準ステレオ画像座標データ、基準キャリブレーション要素等の基準データは、立体基準チャート２０を所持しない需要者に対しては、使用するカメラにあわせてフレキシブルディスク、ＣＤ−ＲＯＭ、Ｗｅｂサイトからのインターネットによるダウンロード等で供給するとよい。
【０１０６】
【発明の効果】
以上説明したように、本発明のキャリブレーション装置によれば、キャリブレーション用チャートを、キャリブレーションを行うカメラで、少なくとも２方向から撮影したキャリブレーション用画像から第１マークを抽出する抽出部と、前記抽出部にて抽出された第１マークを用いた射影変換により、前記キャリブレーション用画像における前記第２マークの概略位置をもとめる概略マーク位置測定部と、前記キャリブレーション用画像に対して、前記第２マークの概略位置近傍で前記第２マーク位置の撮影されている位置を求める精密マーク位置測定部と、前記キャリブレーション用チャートにおける第２マークの位置と、この第２マークに対応する前記キャリブレーション用画像における第２マークの位置から、前記レンズのキャリブレーション要素を算出する演算処理部と備える構成としているので、レンズ収差を補正するのに必要なキャリブレーション要素が簡便に測定できる。そこで、特に立体視のように僅かなレンズ収差でも問題となる用途において、レンズ収差の大きな安価なレンズでステレオ撮影した画像であっても、キャリブレーション要素を用いてレンズ収差の影響の少ない画像として作成することができ、容易に立体視できる。
【０１０７】
また、キャリブレーション用チャートは、少なくとも３箇所以上設けられた第１マークと、該第１マークに対して外観上識別可能に設けられた第２マークを有するという簡易な形態であるため、立体的に基準点が配置された立体基準チャートを用いてレンズ収差を測定する場合に比較して、キャリブレーションが簡便に行える。
【図面の簡単な説明】
【図１】 本発明の第１の実施の形態を説明する全体構成ブロック図である。
【図２】 キャリブレーション用チャートの一例を示す平面図である。
【図３】 第１マークの一例を示す説明図である。
【図４】 第２マークの一例を示す説明図である。
【図５】 レンズ収差を計測する場合のカメラ配置を説明する図で、（Ａ）はカメラ配置を示した立体図であり、（Ｂ）は一般的に標準レンズ及び望遠レンズを使用したときのカメラ間隔を示した図である。
【図６】 標準レンズや広角レンズのカメラによるチャートの撮影手順の説明図で、（Ａ１）〜（Ａ４）はカメラの画像、（Ｂ１）〜（Ｂ４）はカメラ画像に対応するチャートとカメラの位置関係を示している。
【図７】 キャリブレーション装置を用いたキャリブレーション方法を説明するフローチャートである。
【図８】 第２マークの精密位置測定の処理手順を説明する詳細フロー図である。
【図９】 （Ａ）は中心投影における画像座標系と対象座標系の説明図、（Ｂ）はターゲット認識に用いられる正規化相関のテンプレ−ト画像と対象画像の説明図である。
【図１０】 キャリブレーション要素の演算処理結果の一例を示す説明図である。
【図１１】 本発明のキャリブレーション結果診断装置の一実施の形態を説明する構成ブロック図である。
【図１２】 本発明のキャリブレーション結果診断装置を用いたキャリブレーション結果診断を説明するフローチャートである。
【図１３】 モデル座標系ＸＹＺと左右のカメラ座標系ｘｙｚを用いて標定計算を説明する図である。
【図１４】 本発明のキャリブレーション結果診断装置の第２の実施の形態を説明する構成ブロック図である。
【符号の説明】
１ チャート（キャリブレーション用チャート）
２ カメラ
３ 画像データ記憶部
４ 抽出部
５ 概略マーク位置測定部
６ 精密マーク位置測定部
７ 演算処理部
８ 画像処理部
９ 表示部
２０ 立体基準チャート
２２ カメラ
２６ ターゲット位置測定手段
３０ 演算処理手段
３２ 立体基準チャートターゲット記憶部
３４ 収差補償パラメータ適性度判定部（キャリブレーション判定部）[0001]
[Industrial application fields]
  The present invention relates to a calibration apparatus and method used for correcting lens aberration of an image photographed by a camera, a digital camera, or the like, or used for performing measurement using an image, and more particularly to an internal parameter of the camera {for example, a principal point of a lens. The present invention relates to a calibration apparatus and method that can easily measure a position, a screen distance (focal distance), a distortion parameter, and the like}.
  The present invention also relates to a planar calibration chart suitable for use in a calibration apparatus and calibration method for easily measuring internal parameters of a camera.The
[0002]
[Prior art]
Conventionally, in the field of photogrammetry and photo measurement, it is important to obtain an image with less aberration. Therefore, in the field of photogrammetry and photo measurement, a high-precision lens with little aberration is used as a lens for a photographing camera. Furthermore, in the field of photogrammetry, the internal parameters of the camera (main point position, screen distance, distortion parameters) can be analyzed analytically by measuring a large number of precisely arranged 3D points from multiple directions. Seeking. In the case of a measurement camera used in the field of photo measurement, the internal parameters of the camera are obtained by precisely measuring the manufactured camera.
[0003]
[Problems to be solved by the invention]
However, the method for obtaining camera internal parameters by photographing and measuring three-dimensional measurement points measured precisely as in the photogrammetry field has the following problems.
(1) Measurement points (targets) are arranged in three dimensions. Therefore, when shooting from a plurality of directions, a blind spot is generated, and a target that cannot be seen between images is generated.
{Circle around (2)} In the captured image, the positional relationship between the three-dimensional measurement points may be reversed, and measurement of the measurement points and association between the images are difficult and cannot be automated.
(3) Therefore, measurement points are measured and correspondence between images is performed by a highly skilled worker. Then, in order to obtain the internal parameters of the camera, high cost and long work are required.
In addition, the method of measuring the manufactured measurement camera precisely requires not only a dedicated tool, but also requires a skilled and specialized engine, making it expensive as a dedicated measurement camera. There is a problem.
[0004]
In recent years, therefore, a calibration method has been proposed in which lens aberration is measured by measuring two-dimensional measurement points printed on a sheet as an alternative to three-dimensional measurement points. However, the method for obtaining the lens aberration by measuring the measurement points printed on the sheet is because the measurement points are two-dimensionally arranged, and therefore the parameters that can be measured are limited to the lens aberration. There is a problem that the focal length cannot be measured. The focal length of the lens is a parameter necessary when performing three-dimensional measurement using a set of images that can be stereoscopically viewed. There is also a calculation formula that calculates the focal length of the lens using a two-dimensional measurement point, but the accuracy is unstable and it is difficult to measure the focal length of the lens stably and accurately. There was a problem.
[0005]
Furthermore, when taking a two-dimensional measurement point printed on a sheet and measuring the lens aberration of a lens that takes a set of images that can be viewed stereoscopically, an error occurs in the calibration of various measured parameters. There may be. However, it has been difficult to find a calibration error using a sheet on which two-dimensional measurement points are printed.
[0006]
  SUMMARY OF THE INVENTION A first object of the present invention is to solve the above-described problems. A calibration apparatus and a calibration method capable of easily measuring internal parameters necessary for correcting lens aberrations in general-purpose cameras and digital cameras. Is to provide.
  A second object of the present invention is to provide a flat calibration chart suitable for use in a calibration apparatus or calibration method for easily measuring internal parameters of a camera.The
[0007]
  The calibration apparatus according to the present invention achieves the first object. As shown in FIGS. 1 and 2, at least three or more first marks are provided, and externally identified with respect to the first marks. An extraction unit 4 that extracts a first mark from a calibration image photographed from at least two directions by a camera 2 that performs calibration on a calibration chart 1 having a second mark that can be provided, and an extraction unit 4 The approximate mark position measuring unit 5 that obtains the approximate position of the second mark in the calibration image by projective transformation using the first mark extracted in step 2, and the second image with respect to the calibration image. A precision mark position measuring unit 6 for obtaining a photographed position of the second mark position in the vicinity of the approximate position of the mark, and for calibration An arithmetic processing unit 7 that calculates a calibration element of the camera 2 from the position of the second mark in the chart 1 and the position of the second mark in the calibration image corresponding to the second mark. The marks are distributed and arranged in at least three quadrants among the quadrants when the calibration chart 1 is equally divided into four sections,On the calibration chartSecond markTotal number ofIs the first markMore than the total number ofHas been placed.
[0008]
The apparatus configured as described above has at least three or more first marks provided on the camera 2 that performs calibration, and a second mark that can be visually distinguished from the first marks. A calibration image obtained by photographing the calibration chart 1 from at least two directions is prepared. The extraction unit 4 extracts the first mark from the calibration image. The approximate mark position measuring unit 5 obtains the approximate position of the second mark in the calibration image by projective transformation using the first mark. The precision mark position measurement unit 6 obtains a position where the second mark position is photographed in the vicinity of the approximate position of the second mark. This is because the influence of the lens aberration is different for each second mark position, so that there is a difference between the position estimated by the approximate mark position measuring unit 5 and the actual position on the calibration image. The arithmetic processing unit 7 calculates a lens calibration element from the position of the second mark in the calibration chart 1 and the position of the second mark in the calibration image corresponding to the second mark. By using this calibration element, calibration including lens aberration compensation for the camera 2 can be performed.
[0009]
Preferably, when the approximate mark position measuring unit 2 of the present invention is configured to obtain the approximate position of the second mark by projective transformation, the approximate position of the second mark can be easily obtained by projective transformation. The lens aberration of the camera 2 appears as a difference between the position roughly estimated by the mark position measuring unit 5 and the position on the actual calibration image.
[0010]
Preferably, the precision mark position measurement unit 6 of the present invention may be configured to determine the second mark position using at least one of template matching and barycentric position detection. When the second mark has an irregular shape or is unclearly displayed on the calibration image, the second mark position can be obtained stably and accurately by using the center of gravity position of the second mark. .
[0011]
Preferably, the arithmetic processing unit 7 of the present invention calculates the second mark from the position of the second mark in the calibration chart 1 and the position of the second mark in the calibration image corresponding to the second mark. It may be configured to calculate distortion. Here, the distortion is a distortion caused by a deviation of the imaging position, Seidel's five aberrations appearing on the lens, and the like.
[0012]
Preferably, the arithmetic processing unit 7 of the present invention selects an appropriate second mark from among the second marks whose positions are calculated by the precision mark position measuring unit when calculating the calibration element of the lens. It may be configured to select. For the selection of the second mark, for example, a bundle adjustment method is preferably used. According to the bundle adjustment method, it is possible to obtain an external orientation element of the camera at each photographing position and to simultaneously classify the second mark with an appropriate position and an inappropriate position.
[0013]
Preferably, the calibration element of the present invention may be configured to include at least one of a focal length of the lens, a principal point position, and a distortion parameter of the lens. The reason why the second mark contributes to distortion is the focal length of the lens, the principal point position, or the distortion parameter of the lens.
[0014]
Preferably, the calibration apparatus according to the present invention further corrects an image captured by a camera that performs calibration to an image compensated for the lens aberration based on the calibration element obtained by the arithmetic processing unit 7. It is good to be comprised so that it may have. When the image processing unit 8 is provided, it is possible to remove the influence of lens aberration from an image photographed by a camera, and is particularly suitable for use in a field where a slight distortion of the image position appears as a large altitude error like a stereoscopic photograph. .
[0015]
Preferably, the calibration device of the present invention may be configured to further include a display unit 9 for displaying a calibration element or distortion in the second mark. Providing the display unit 9 is convenient for the operator because the operator can intuitively determine whether the calculation result is good or bad.
[0016]
  The calibration method according to the present invention achieves the first object. As shown in FIG. 7, at least three or more first marks are provided, and the first marks are provided so as to be visually distinguishable. A first step (S40) for extracting a first mark from a calibration image taken from at least two directions with a camera for performing calibration on the calibration chart 1 having the second mark, and the extracted A second step (S50) for obtaining an approximate position of the second mark in the calibration image by projective transformation from the first mark, and a second step in the vicinity of the approximate position of the second mark with respect to the calibration image. Third step (S69) for accurately obtaining the mark position, and the position of the second mark in the calibration chart 1 And a fourth step (S90) of calculating a calibration element of the camera from the position of the second mark in the calibration image corresponding to the second mark, the first mark being a calibration Are distributed in at least three or more quadrants among the quadrants obtained by equally dividing the distribution chart 1 into four sections,On the calibration chartSecond markTotal number ofIs the first markMore than the total number ofHas been placed.
[0017]
   Preferably, in the calibration method of the present invention, for example, as shown in FIG.Of camera 2The lenses are various lenses (wide-angle lens, standard lens, telephoto lens), and the calibration chart 1 is provided with the first marks that are separated from each other by a predetermined distance. The shooting direction at the time of shooting is preferably selected as an angle at which the focal length of the lens can be accurately calculated.
[0018]
Preferably, in the calibration method of the present invention, for example, as shown in FIGS. 5A and 6, the shooting directions of the various lenses are the first shooting with one of the first marks (I) as the front. A camera position in front of the first mark and another first mark (II, III, IV, V) that is separated from the first mark by the predetermined distance d, and the shooting direction of the lens is the first direction. If the mark (I) includes the second photographing direction in which the optical axis of the lens faces, the camera 2 can be accurately given a desired angle, and the change in the height (depth) direction can be ensured. Thus, the focal length can be accurately calculated.
[0019]
Preferably, in the calibration method of the present invention, for example, as shown in FIG. 5B, the lens is a standard lens or a telephoto lens, and the calibration chart 1 includes the first lenses separated from each other by a predetermined distance. 1 mark is provided, and the lens and the calibration chart 1 are used as positions for photographing the calibration chart 1 by the lens, with reference to the position of the front (I) as one of the first marks. A position (II, III, IV, V) facing the other first mark having an interval (H / 3) of about 1/3 of the shooting distance (H) is selected. Then, the front (I) as one of the first marks is set as the first shooting direction, and the first marks (1e) are set at the shooting positions (II, III, IV, V) of the other first marks, respectively. It is preferable to shoot as the second to fifth shooting directions.
[0020]
  The calibration chart 1 of the present invention achieves the second object. As shown in FIG. 2, the first mark provided in at least three places and the appearance can be distinguished from the first mark. A calibration chart having a second mark provided on the calibration chart, wherein the first mark is distributed in at least three or more quadrants among the quadrants when the calibration chart 1 is equally divided into four sections. Arranged,On the calibration chartSecond markTotal number ofIs the first markMore than the total number ofPlaced and used for camera calibration. The camera calibration includes, for example, the calibration apparatus according to any one of claims 1 to 8 and the calibration method according to any one of claims 9 to 12. Preferably, when the first mark and the second mark are formed on a substantially plane, the calibration chart can be easily handled. Preferably, when the first mark and the second mark are configured to be displayed in an image displayed on a flat display (for example, a liquid crystal display), the flat display is extremely small in elasticity as compared with the sheet. The 1 mark and the 2nd mark are accurately displayed on the flat display.
[0027]
DETAILED DESCRIPTION OF THE INVENTION
The present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing the overall configuration of a first embodiment of the present invention. In the figure, a chart 1 as a calibration chart has a first mark and a second mark printed on a sheet on a plane. The first mark is used to measure the approximate mark position and to correlate the images of the calibration photo set, and to determine the shooting angle at which the camera 2 shots the chart 1. The first mark is provided in at least three places on the chart 1, and is preferably provided in each quadrant when the chart 1 is equally divided into four sections.
[0028]
The second mark designates the position of the image data of the chart 1 photographed by the camera 2 and is also called a target, and is preferably arranged uniformly on the chart 1 with an equal density. The second marks are preferably provided at 30 or more locations on the chart 1, and more preferably about 100 to 200 locations. However, if a large number of second marks are provided, the second mark itself becomes small and difficult to see, and the lens aberration measurement calculation time also increases, so there is an actual upper limit. is there.
[0029]
The chart 1 may use an image display of a flat sheet on which a first mark and a second mark are displayed on a device having a flat display screen such as a notebook computer instead of the flat sheet. A screen using glass, such as a liquid crystal display screen, is suitable as a display device for the chart 1 because the expansion and contraction of the plane due to humidity and temperature is very small compared to paper and plastic. In addition, when using a portable personal computer display screen as a liquid crystal display screen, in addition to having good display accuracy, it has excellent portability. Endure. Details of the chart 1 will be described later.
[0030]
The camera 2 is a camera to be calibrated, and typically has a large lens aberration compared to a photogrammetry or a camera for photo measurement such as a general-purpose optical camera or digital camera. is there. The camera 2 may be a wide-angle lens, a standard lens, or a telephoto lens. The image data storage unit 3 is a storage device that stores the image data of the chart 1 photographed by the camera 2, and an electromagnetic storage medium such as a magnetic disk or a CD-ROM is used. The image data stored in the image data storage unit 3 is preferably stereo image data obtained so that the chart 1 can be viewed in stereo with the camera 2. Typically, the image data is stored in the left shooting position 2L and the right shooting position 2R. Photographed by a set of photographing positions. Preferably, the image data storage unit 3 stores an image of a calibration photo set in such a manner that the shooting angle at which the camera 2 has shot the chart 1 can be determined.
[0031]
The calibration apparatus includes an extraction unit 4, an approximate mark position measurement unit 5, a precise mark position measurement unit 6, an arithmetic processing unit 7, an image processing unit 8, a mark coordinate storage unit 10, and a lens aberration compensation parameter storage unit 11. An image data storage unit 3 and a display unit 9 are provided as external devices. For the calibration device, for example, a computer equipped with Intel Pentium (registered trademark) or Celeron (registered trademark) as a CPU may be used.
[0032]
The extraction unit 4 extracts a first mark from the image data stored in the image data storage unit 3 and performs a first mark extraction process for obtaining an image coordinate value of the first mark. The first mark extraction process is performed as a pre-process for calculating and associating the approximate position of the second mark by the approximate mark position measuring unit 5. The image coordinate value of the first mark is stored in the mark coordinate storage unit 10. In addition, when the 1st mark contains the same design as a 2nd mark, it is good to set it as the image coordinate value of a 1st mark by the 2nd mark position in a 1st mark. Details of the first mark extraction processing by the extraction unit 4 will be described later.
[0033]
The approximate mark position measurement unit 5 performs projective transformation from the image coordinate value of the first mark extracted by the extraction unit 4 to obtain an external orientation element, and uses a single photograph orientation theorem and a collinear conditional expression. The approximate position of the second mark is calculated, and the images in the proofreading photo group are associated with each other. Details of the approximate position calculation processing of the second mark by the approximate mark position measurement unit 5 will be described later.
[0034]
The precision mark position measurement unit 6 recognizes the second mark with respect to the image of the calibration photo set, and precisely calculates the position of the second mark by the center-of-gravity position detection method or the like. If the position of the second mark calculated by the precision mark position measuring unit 6 is significantly different from the position of the other second mark in the image data of the chart 1, the arithmetic processing unit 7 It has a function of excluding the position of the generated second mark. The arithmetic processing unit 7 extracts a second mark suitable for calibration from the second marks calculated by the precision mark position measuring unit 6 and simultaneously adjusts the external orientation element and the target point coordinates. Calculate the camera's internal parameters. The calculated camera internal parameters may be stored in the lens aberration compensation parameter storage unit 11. The internal parameters of the camera include a principal point position, a screen distance, and a distortion parameter. Although only the distortion parameters are obtained here, the spherical aberration, coma, astigmatism, and warp of the image plane constituting the Seidel's five aberrations may be obtained. The internal parameters obtained by the arithmetic processing unit 7 are displayed graphically on the display unit 9. The details of the precision mark position measurement unit 6 and the camera internal parameter calculation process of the calculation processing unit 7 will be described later.
[0035]
The image processing unit 8 rearranges data images of images (particularly images other than the chart 1) taken by the camera 2 using the internal parameters obtained by the arithmetic processing unit 7. Then, the image photographed by the camera 2 is displayed on the display unit 9 as an image with extremely little distortion from which most of the lens aberration is removed. The display unit 9 is an image display device such as a CRT or a liquid crystal display. The mark coordinate storage unit 10 stores the image coordinate value of the first mark, and also stores the management number of the second mark and the image coordinate value thereof. The lens aberration compensation parameter storage unit 11 stores the internal parameters of the camera 2 calculated by the calculation processing unit 7.
[0036]
Next, Chart 1 as a calibration chart will be described. Here, a case where a predetermined mark is printed on a paper or plastic sheet as the chart 1 will be described as an example, but an image in which the predetermined mark is arranged may be displayed on a flat screen.
FIG. 2 is a plan view showing an example of a calibration chart. The chart 1 has a planar sheet shape, and a first mark that is easily visible and a second mark that includes a large number of points are printed on the front side. Here, a total of five first marks are arranged on the chart 1, and are external diamonds, and a symbol common to the second marks is drawn at the center. The first marks 1a, 1b, 1c, and 1d are provided in each quadrant when the chart 1 is divided into four quadrants. The first mark 1a is the upper left quadrant, the first mark 1b is the upper right quadrant, and the first mark 1c. Is in the lower left quadrant, and the first mark 1d is in the lower right quadrant. The first mark 1e is provided at the origin position common to each quadrant. For example, the first marks 1a, 1b, 1c, and 1d are provided at positions equidistant from the first mark 1e. Assuming that the chart 1 is rectangular, the vertical interval between the first marks 1a and 1b and the first mark 1e is h, and the vertical interval between the first marks 1c and 1d and the first mark 1e is l. At this time, the distance d between the first marks 1a, 1b, 1c, and 1d and the first mark 1e satisfies the following relationship.
d = (h²+ L²)^1/2                (1)
[0037]
The first mark and the second mark are printed with desired dimensions in advance or the dimensions are measured. The numerical values of the printing positions of the first mark and the second mark are read into the mark coordinate storage unit 10 of the calibration apparatus, and are used by the approximate mark position measurement unit 5 for the correspondence with the approximate position measurement. Note that the chart 1 may be stored as image data in a storage device of a computer and printed and used at a calibration location. If the positions of the first mark and the second mark are stored in advance in the calibration device and printed on the sheet at the stored coordinates, the measurement work becomes unnecessary, so the work is simple. Become a thing. Or it is good also as a structure which measures the chart 1 precisely, measures the coordinate position of a 1st mark and a 2nd mark, stores the coordinate value in the mark coordinate memory | storage part 10, and uses it.
[0038]
The first mark is used not only for measuring and associating the approximate mark position but also as a target for determining the shooting direction. Furthermore, the central part of the outer shape rhombus of the first mark is used as a template for precise measurement by the precision mark position measuring unit 6 by making the design common to the second mark.
[0039]
FIG. 3 is an explanatory diagram showing an example of the first mark, where (A) shows a rhombus, (B) shows four arrows, and (C) shows a black rectangle. In FIGS. 3A and 3B, the first mark is arranged with rhombus or four arrows so as to surround the same symbol as the second mark, and consideration is given so that the operator can easily see. By making the design easy to see in this way, the first mark can be easily extracted, and even if one shooting angle is selected from a wide shooting angle as the shooting direction of the camera 2, the image was taken. The first mark is not missed from the image. In FIG. 3C, the first mark is a black rectangle, and the central pattern has a color that is reversed from that of the second mark, but detection is easy even in this way. Also, when the measurement is performed by the precision mark position measurement unit 5, the second mark template can be obtained by reversing the gradation of the first mark with respect to the pattern of FIG. .
[0040]
FIG. 4 is an explanatory diagram showing an example of the second mark. (A) is a black circle “●”, (B) is a plus “+”, (C) is a double circle “◎”, (D) is an English character “ (X) and (E) indicate a star mark “★”, (F) indicates a black square “■”, (G) a black triangle “▲”, and (H) indicates a black diamond “◆”. Since many second marks are arranged uniformly on the chart 1, various symbols may be adopted as long as precise position measurement is easy.
[0041]
Next, a procedure for photographing the chart 1 with the camera 2 to be calibrated will be described. FIGS. 5A and 5B are diagrams for explaining a camera arrangement when measuring lens aberration. FIG. 5A is a three-dimensional view showing the camera arrangement, and FIG. 5B is a camera when a standard lens and a telephoto lens are generally used. It is the figure which showed the space | interval. Calibration can be performed if there are two or more images obtained by imaging the chart 1 from different imaging angles. Preferably, when a flat chart printed on a sheet is used as the chart 1, the measurement values of the respective calibration elements, particularly the focal length, are stable and reliable by photographing from three or more photographing angle directions. Will be expensive.
[0042]
When measuring the lens aberration of various lenses, from five directions as shown in FIG. 5A, that is, from the front (I), upper left (II), upper right (III), lower left (IV), lower right (V). Take a picture. The photographing incident angle between the optical axis of the camera 2 and the plane chart is preferably in the range of 10 degrees to 30 degrees when the depth accuracy at the actual photographing site is set to about 1 cm, and is in focus with respect to the focal depth of the lens. In consideration of the limited distance between the two, the range of 12 degrees to 20 degrees is more preferable. Typically, 15 degrees may be adopted as the shooting incident angle between the optical axis of the camera 2 and the plane chart.
[0043]
Hereinafter, with reference to FIG. 6, the photographing procedure of the chart 1 by the cameras of various lenses will be described. FIG. 6 shows camera images at (A1), (A2), (A3), and (A4), and the positional relationship between the chart 1 and the camera 2 corresponding to the camera images is (B1), (B2), ( B3) and (B4). Numbers (I) to (V) correspond to the camera positions in FIG.
[0044]
(I): Shoot from the front so that the first mark and the second mark of Chart 1 are fully filled (FIGS. 6A1 and 6B1). The distortion correction up to the lens periphery is ensured by filling the captured image with the first mark and the second mark as much as possible. Therefore, the shooting distance H changes according to the focal length of the camera.
(II): Next, when a standard lens and a telephoto lens are generally used, the camera is moved to a position about 1/3 of the shooting distance H around the front camera position, for example, in the upper left quadrant. The camera position is changed so that the first mark 1a becomes the center of photographing (FIGS. 6A2 and 6B2). However, in general, when a wide-angle lens is used and the shooting distance H is within about 1 m, the camera 2 may be moved so that the target first mark is in front. Then, with the camera position of the camera 2 as it is, the direction of the camera 2 is directed so that the first mark 1e at the center is the center (FIGS. 6A3 and 6B3). Next, the camera 2 is moved so as to be close to the chart 1 and is photographed so that the first mark and the second mark are filled with respect to the photographed image of the camera 2 (FIG. 6 (A4), (B4)). ).
[0045]
(III): The camera position is changed so that the first mark 1b in the upper right quadrant is the photographing center. Then, the camera is directed so that the first mark 1e at the center is the center as it is, and the first mark and the second mark are fully photographed. (IV): The camera position is changed so that the first mark 1c in the lower left quadrant becomes the photographing center. Then, the camera is directed so that the first mark 1e at the center is the center as it is, and the first mark and the second mark are fully photographed.
(V): The camera position is changed so that the first mark 1a in the lower right quadrant becomes the photographing center. Then, the camera is directed so that the first mark 1e at the center is the center as it is, and the first mark and the second mark are fully photographed.
By such a procedure, the angle of the camera 2 can be ensured as a necessary difference in photographing angle, so that the focal length can be reliably measured.
[0046]
Next, an advantage of the photographing procedure using the first mark when using a planar chart printed on a sheet or a displayed planar screen as the chart 1 will be described. Usually, when shooting a mark printed on a flat sheet, the camera 2 cannot be given a desired angle, and as a result, the screen distance (focal length) cannot be obtained accurately. In other words, if there is no change in the focal length direction (height, depth direction), there is no constraint on the calculated values of the camera's internal parameters, so even if the camera's internal parameters are calculated, it is extremely unreliable. Become. Therefore, the focal length is obtained by measuring the target arranged in three dimensions, but as described in the section “Problems to be solved by the invention”, it is difficult to measure the target arranged in three dimensions. In addition to being unable to automate, it is not easy to create.
[0047]
On the other hand, the flat chart 1 uses a sheet on which the first mark and the second mark are printed or a planar screen on which the first mark and the second mark are printed. The problem of difficulty in association is eliminated. Next, when shooting is performed using the first mark, the camera 2 can be given a desired angle, the change in the height (depth) direction can be ensured, and the focal length can be accurately calculated. For example, if the angle of the camera 2 can be tilted by 10 degrees or more with respect to the chart 1, the focal length can be reliably obtained in the calibration device of the present invention.
[0048]
Here, the distance H between the camera 2 and the chart 1 is determined from the focal length f of a standard lens or a wide-angle lens. For example, in a standard lens with a focal length of 35 mm, the shooting distance H is about 90 cm. Since the mutual distance d between the first marks provided on the chart 1 is 20 cm, for example, when the shooting direction is tilted from the front (I) to the upper left (II) or the like, about 10 degrees is secured as the shooting angle.
Note that the upper limit of the tilt angle in the shooting direction is determined by the depth of focus. That is, if the tilt angle in the shooting direction is large, the distance between the camera 2 and the first mark is different for each first mark, and the image of the first mark in the image is blurred. Therefore, the upper limit of the tilt angle in the shooting direction is, for example, 30 degrees. The actual shooting procedure is as shown in the above (I) to (V). If the shooting is performed so that the first mark and the second mark are inserted in the full screen of the camera, the above conditions are automatically satisfied. This condition is satisfied.
[0049]
When measuring the lens aberration of a telephoto lens or standard lens, the angle of view of the photographic lens becomes narrow and the angle cannot be obtained. Therefore, when the shooting direction is tilted from the front (I) to the upper left (II), etc., 10 degrees is not secured. That is, when the focal length is long, the shooting distance H between the camera 2 and the chart 1 is 1 m or more, and the mutual distance d between the first marks is only about 20 cm. Therefore, as shown in FIG. 5B, the left camera positions (II) and (IV) and the right camera positions (III) and (V) are determined with the front camera position as the center. At this time, the camera is installed at a position where the distance between the left and right camera positions is about 1/3 of the shooting distance H from the front (I) position, and the above left (II), lower left (IV) and upper right are described above. It is only necessary to shoot at (III), lower right (V). The optical axis of the camera may be aligned with the normal direction of the chart 1 but may be directed in the direction of the chart 1.
[0050]
In the above-described embodiment, the shooting position is shown in the five directions of front (I), upper left (II), upper right (III), lower left (IV), and lower right (V). In the lowest case, there are two left and right directions, or three or more directions. In the case of the left and right directions as well, the chart 1 is shot so as to ensure about 10 degrees as the shooting angle.
[0051]
Next, the overall processing flow in the calibration apparatus of the present invention will be described. FIG. 7 is a flowchart for explaining a calibration method using the calibration apparatus. First, the chart 1 is photographed by the camera 2 to be compensated for lens aberration (S10). The photographing method is based on the photographing procedure described with reference to FIG. 5B in the case of a standard lens or a wide-angle lens, and the photographing procedure described in FIG. 6 in the case of a telephoto lens. The image data of each shooting direction shot by the camera 2 is temporarily stored in the image data storage unit 3. Next, the calibration apparatus reads the image data stored in the image data storage unit 3 and displays it on the display unit 9 (S20). Then, the operator selects an image for target association and measurement from the image displayed on the display unit 9 (S30). Then, the extraction unit 4 performs a first mark extraction process for the selected image (S40).
[0052]
(I): First mark extraction process
In the first mark extraction process, a secondary projective transformation formula between the plane coordinates of the chart 1 set on the measurement target surface and its image coordinates (camera side) is determined. Three or more points are measured on the image data. Here, since the second mark is included in the first mark, the position of the first mark is accurately specified by specifying the position of the included second mark. The first mark extraction process includes the following I-iTo I-ivThe processes up to are repeated for the number of the first mark. For example, in the chart 1 shown in FIG. 2, the measurement is performed for the first marks 1a, 1b, 1c, and 1d at the two left and right points.
[0053]
I-iThe operator aligns the mouse cursor position with the second mark in the first mark to be detected on the entire image displayed on the display unit 150 and clicks to obtain the approximate position of the first mark.
I-ii... I-iA local image including the second mark is cut out from the enlarged image with the image coordinates obtained in step 1 as the center and displayed. At this time, an image including the second mark can be used as a template for the second mark precision position measurement.
I-iii... I-iiFor the enlarged image displayed in1The mouse cursor position is aligned with the center of gravity of the mark and clicked, and this image coordinate is set as the center of gravity of the first mark. In order to perform approximate position matching in the subsequent processing, I-iiiAlignment at may not be exact.
I-ivNext, in order to correspond to the management number of the second mark stored in the mark coordinate storage unit 135, I−iiiThe management number of the second mark corresponding to the position of the center of gravity of the first mark measured in is input. At this time, the management number of the input second mark includes I-iiiThe position of the center of gravity of the first mark measured in is stored as the reference point coordinates.
[0054]
In the first mark extraction process, for example, if the measurement order of the first mark on the chart 1 is determined in advance, an automatic numbering process can be performed on the extraction unit 4 side without inputting the management number of the second mark. It is. Further, in the first mark extraction process, for example, the selection image displayed on the display unit 9 is divided into two so that the operator can easily work, and the entire image as shown in FIG. If the enlarged images as shown in FIGS. 3A and 3B are displayed, the position can be easily measured.
[0055]
Next, as another processing procedure of the first mark extraction process, there is a method of measuring only the entire image as shown in FIG. 2 without using an enlarged image. In this case, I-iAs well as I-ivI-iThe management number of the second mark corresponding to the position of the center of gravity of the first mark measured in is input. In this case, since the enlarged image is not used, I−ii, I-iiiCan be omitted. However, since the entire image is displayed, the first mark is displayed in a small size. Therefore, it may be determined whether or not to use the enlarged image according to the preference of the operator.
[0056]
Next, a case where the first mark extraction process is automatically performed by the extraction unit 4 will be described. First, an outer frame portion that does not include the second mark among the first marks is registered as a template. For this registration, for example, the first first mark in the first mark extraction process described above may be registered as a template image. Then, the remaining first marks can be automatically measured in the template matching process. In addition, the position association in the case of the first mark can be easily performed because the position of the first mark is clear from the image. For example, with the first mark arrangement shown in FIG. 2, it is easy to associate five first marks from the detected coordinates. Note that the template matching process is the same as the target recognition process (S62) in the second mark precision position measurement described later, and thus the description thereof is omitted.
[0057]
  Next, a case where the first mark extraction process is further automatically performed by the extraction unit 4 will be described. A template image of the first mark in the first mark extraction process is registered in the extraction unit 4 in advance. Then, the first mark is individually extracted by the template matching process using the template image of the first mark.iIt is possible to omit all the operations for designating the first mark. That is, if the first mark is a mark that is clearly different from the second mark, automatic processing can be performed even if the extraction unit 4 has a virtual template image.
  However, since it is sufficient to measure at least three points for the first mark, even a manual operation is a simple operation.
[0058]
Returning to FIG. 7, the approximate mark position measuring unit 5 associates with the second mark approximate position measurement (S50). The second mark approximate position measurement and association includes a step (II-1) of obtaining an external orientation element and a step (II-2) of calculating an approximate position of the second mark.
(II-1): Step of obtaining an external orientation element
In the approximate mark position measuring unit 5, the image coordinates of the first mark obtained in S40 and the corresponding reference point coordinates are substituted into the secondary projective transformation equation shown in the equation (2), and the observation equation is calculated. b1 to b8 are obtained.
X = (b1 · x + b2 · y + b3) / (b7 · x + b8 · y + 1)
Y = (b4 * x + b5 * y + b6) / (b7 * x + b8 * y + 1) (2)
Here, X and Y indicate reference point coordinates, and x and y indicate image coordinates. Next, the relationship between the reference point coordinates and the image coordinates will be described. FIG. 9A is an explanatory diagram of an image coordinate system and a target coordinate system in central projection. In the case of central projection, an object coordinate system 52 as a reference point coordinate system on which the chart 1 is placed with respect to the projection center point Oc and an image coordinate system 50 on which the film of the camera 2 or the CCD is placed are shown in FIG. It is in such a positional relationship. The coordinates of an object such as a reference mark in the object coordinate system 52 are (X, Y, Z), and the coordinates of the projection center point Oc are (X0, Y0, Z0). The coordinates in the image coordinate system 50 are (x, y), and the screen distance from the projection center point Oc to the image coordinate system 50 is C. ω, φ, and κ represent inclinations at the time of camera photographing with respect to the three axes X, Y, and Z constituting the target coordinate system 52 of the image coordinate system 50, and are called external orientation elements.
[0059]
Then, using the parameters b1 to b8 in the equation (2), the next external orientation element is obtained from the equation (3).
ω = tan^-1(C ・ b8)
φ = tan^-1(-C ・ b7 ・ cosω)
κ = tan^-1(−b4 / b1) (when φ = 0)
κ = tan^-1(−b2 / b5) (when φ ≠ 0, ω = 0)
κ = tan^-1{-(A1, A3-A2, A4) / (A1, A2-A3, A4)} (when φ ≠ 0, ω ≠ 0)
[0060]
  Z0 = C · cosω · {(A2 ² + A3 ² ) / (A1 ² + A4 ² )} ^1/2 + Zm
  X0 = b3− (tan ω · sin κ / cos φ−tan φ · cos κ) × (Zm−Z0)
  Y0 = b6- (tan [omega] .cos [kappa] / cos [phi] -tan [phi] .sin [kappa]) * (Zm-Z0) (3)
  However, A1 = 1 + tan2φ, A2 = B1 + B2 · tanφ / sinω, A3 = B4 + B5 · tanφ / sinω, and A4 = tanφ / (cosφ · tanω). Zm is the average elevation of the reference points of the first marks 1a, 1b, 1c and 1d4. Here, since the reference points of the first marks 1a, 1b, 1c and 1d4 are on the plane coordinates, it can be assumed to be a plane with a constant altitude. C is a focal length, which corresponds to the aforementioned screen distance.
[0061]
(II-2): Step of calculating the approximate position of the second mark
Next, from the principle of single photo orientation, the camera coordinates (xp,...) In the tilted camera coordinate system represented by the image coordinate system 50 with respect to the ground object (X, Y, Z) represented by the target coordinate system 52. yp, zp) is given by equation (4).
[Expression 1]

Here, as described above, X0, Y0, and Z0 are the ground coordinates of the projection center point Oc as shown in FIG.
[0062]
Here, the tilt (ω, φ, κ) of the camera obtained by Equation (3) is substituted into Equation (4), and the rotation matrix is calculated to obtain the rotation matrix elements a11 to a33.
[0063]
  Next, the elements a11 to a33 of the obtained rotation matrix, the camera position (X0, Y0, Z0) obtained by the equation (3), and the reference point coordinates (X, Y, Z) of the target are collinear conditions. Substituting into the expression {expression (5)}, the image coordinates (x, y) of the target are obtained. Here, the collinear conditional expression is a relational expression that is established when the projection center, the photographic image, and the ground object are on a straight line. As a result, the position of the second mark when there is no lens aberration is calculated, so the approximate image coordinates of the target in the image taken with an actual camera with lens aberration are obtained.
  x = -C. {a11 (X-X0) + a12 (YY 0) + A13 (Z-Z0)} / {a31 (X-X0) + a32 (YY 0) + A33 (Z-Z0)}
  y = −C · {a21 (X−X0) + a22 (YY 0) + A23 (Z-Z0)} / {a31 (X-X0) + a32 (YY 0) + A33 (Z-Z0)} (5)
[0064]
By the way, tan in equation (3)^-1In this calculation, since two solutions are obtained, the camera tilts (ω, φ, κ) each have two solutions and are calculated in all ways. The correct answer is obtained by comparing the residuals of the image coordinates of the first marks 1a, 1b, 1c, and 1d4 measured in the first mark extraction process with the corresponding image coordinates of the four points obtained by the equation (5). Ω, φ, and κ are calculated.
In addition, although it demonstrated using the secondary projection transformation formula here as a projection transformation formula, this invention is not limited to this, Even if other projection transformation formulas, such as a tertiary projection transformation formula, are utilized. good.
[0065]
Further, the approximate mark position measuring unit 5 uses, for example, the management number of the second mark added to the reference point coordinate file stored in the mark coordinate storage unit 10 as the target of each first mark (second mark). ) Is assigned to the second mark.
[0066]
Returning to FIG. 7, the precision position measurement unit 6 performs the precision position measurement of the second mark (S60). Hereinafter, the processing procedure of the precise position measurement of the second mark will be described in detail with reference to FIG. First, the precision mark position measurement unit 6 recognizes the target as the second mark (S62). For this target recognition, for example, template matching using normalized correlation is used. Details of target recognition will be described below.
[0067]
(III) Target recognition
FIG. 9B is an explanatory diagram of a normalized correlation template image and target image used for target recognition. First, an arbitrary target is selected from the center of gravity positions of the first marks measured in the first mark extraction process (S40), for example, the first marks 1a, 1b, 1c, and 1d4. The template image of the normalized correlation is an M × M pixel image centered on the center of gravity (image coordinates) of the selected target. Further, an image of N × N pixels is set as the target image with the approximate position (image coordinates) of the target calculated in the second mark approximate position measurement (S50) as the center.
[0068]
Next, the template matching by the normalized correlation shown in Expression (6) is performed on the target image, and the position where the correlation value is maximized is obtained. It is assumed that superposition is achieved at a position where the correlation value becomes the maximum value, and the target is recognized at the position where the correlation value becomes the maximum value. Here, the center coordinates of the template image are converted into image coordinates on the same-size image and set as detection points.
A = {M²× Σ (Xi × Ti) −ΣXi × ΣTi} /
[{M²× ΣXi²-(ΣXi)²} × {M²× ΣTi²-(ΣTi)²}]
(6)
Here, A is the correlation value, M is the square size of the template image, Xi is the target image, and Ti is the template image. Although the square sizes N and M of the image are variable, it is preferable to make N and M as small as possible on the assumption that the target can be sufficiently stored in order to speed up the processing time.
[0069]
Returning to FIG. 8, in order to perform the precise position measurement of the second mark, sub-pixel edge detection of the second mark is performed (S64). The target image on which the sub-pixel edge detection of the second mark is performed is an N × N pixel image centered on the detection point recognized as the target in S62. Laplacian-Gaussian filter (LOG filter), which is the second derivative of the Gaussian function shown in Equation (7), is applied to the grayscale waveform existing in the target image, and two zero crossing points, that is, edges of the calculation result curve are sub Detect by pixel. Here, the sub-pixel means that position detection is performed with an accuracy finer than one pixel.
∇²G (x) = {(x²-2σ²) / 2πσ⁶} · Exp (−x²/ 2σ²(7)
Here, σ is a parameter of the Gaussian function.
[0070]
Next, the position of the center of gravity of the target is detected (S66) and returned (S68). Here, the intersection point is set as the center of gravity of the target from the edge positions in the x and y directions obtained using Expression (7). Note that the precise position measurement of the second mark is not limited to the processing disclosed in S62 to S66, and any other center-of-gravity position detection method such as a moment method or a template matching method may be used. You may ask for this.
[0071]
Returning to FIG. 7, all target center-of-gravity positions are confirmed, and it is confirmed that there is no apparent error (S70). That is, it is confirmed whether or not the position detection of the target recognized is appropriate. For the convenience of confirmation by the operator, the position of the detected target is displayed on the display unit 9. If there is no error, go to S80. If there is an error, an inappropriate target position is corrected (S75). For example, the target whose correlation value calculated in S62 is low or the target whose center of gravity detection position is too far from the approximate detection position can be clearly seen by the operator, such as making the target display red on the display unit 9. To display. Then, with respect to the target with an error, the operator manually measures it again (specifies the position of the center of gravity with the mouse). Even if the erroneous target position is not forcibly corrected here, it can be removed because it is detected as an abnormal point also in the process of S90 for obtaining the calibration parameter later.
[0072]
And the process of S30-S75 is repeated for the image required for the measurement of a lens aberration (S80). For example, if there are five captured images, all five images may be repeated, and if the images necessary for the measurement of lens aberration have been reached, all the captured images are processed repeatedly. You don't have to.
[0073]
When the measurement process for the image necessary for the measurement of the lens aberration is completed, the process proceeds to a process for obtaining a lens aberration calibration element using the camera internal parameter calculation process of the calculation processing unit 7 (S90). Here, as the calculation target of the calibration element, all the second marks on which the center of gravity position is obtained by associating the second mark on the chart 1 by the processing of the approximate mark position measuring unit 5 and the precise mark position measuring unit. Do this for Mark.
[0074]
(IV): Camera internal parameter calculation processing (bundle adjustment method with self-calibration)
As the camera internal parameter calculation processing of the calculation processing unit 7, for example, “bundle adjustment method with self-calibration” used in the photogrammetry field is used. Here, “bundle adjustment” is based on a collinear condition that a light beam (bundle) connecting an object, a lens, and a CCD surface must be on the same straight line, and an observation equation is established for each light beam of each image. In this method, the position and tilt of the camera (external orientation element) and the coordinate position of the second mark are simultaneously adjusted by the least square method. “With self-calibration” is a method by which a calibration element, that is, a camera's internal localization (lens aberration, principal point, focal length) can be obtained. The collinear condition basic expressions of the bundle adjustment method with self-calibration (hereinafter simply referred to as “bundle adjustment method”) are the following (Expression 8) and (Expression 9).
[0075]
[Expression 2]

[Equation 3]

[0076]
These (Expression 8) and (Expression 9) are based on the collinear conditional expression (5) of single photo orientation described in the first mark extraction process. That is, the bundle adjustment method is a method of calculating various solutions by using least square approximation from a plurality of images using (Equation 8) and (Equation 9), and obtaining the external orientation elements of the camera at each photographing position at the same time. Is possible. That is, it is possible to obtain a camera calibration element.
[0077]
Next, as an internal localization correction model (lens aberration), an example in the case of having radial lens distortion is shown in the following (Equation 10).
[Expression 4]

The correction model is not limited to this, and a correction model that is applicable to the lens used may be selected. These calculations are calculated by successive approximation if there are 6 or more reference points in the ground coordinates and the image coordinates. Note that the arithmetic processing unit 7 can obtain an accurate calibration element by omitting the second mark on the chart 1 having a large error when the threshold value of the successive approximation method exceeds the threshold value. Therefore, even if the target position centroid position confirmation (S70) is not detected as the second mark having a large error, it is possible to detect and remove the erroneous second mark in S90.
[0078]
Returning to FIG. 7, the calculation processing result for obtaining the calibration element by the calculation processing unit 7 is determined (S 100), and the calculation process does not converge or the obtained calibration element is not considered appropriate. If this is the case, it is dealt with in S110. In S110, an image including the erroneous second mark is selected. At the end of calibration in S90, the arithmetic processing unit 7 has already determined which second mark of which image has an error, so the corresponding target detection point in each image is displayed and confirmed.
[0079]
Then, the operator corrects the erroneous second mark by manual operation (S120). That is, the coordinates of the center of gravity of the second mark with an error are displayed with a shift, and the correction is made by moving the mark displayed as the second mark with an error to the center of gravity displayed as aptitude. Is done. Then, it is determined whether the correction of the position of the erroneous second mark has been completed (S130). If completed, the process returns to the calibration element calculation of S90, and the calibration element is calculated again. On the other hand, if there are other correction points, the process returns to S110 and repeats the operation of correcting the position of the erroneous second mark.
[0080]
If the calculation processing result for obtaining the calibration element is appropriate, the result is displayed on the display unit 9 (S140). FIG. 10 is an explanatory diagram illustrating an example of the calculation processing result of the calibration element. For example, the focal length, the principal point position, and the distortion parameter, which are calibration elements, are displayed on the display unit 9. The distortion indicating the lens aberration can be easily understood by graphically displaying the pre-correction curve 102, the post-correction curve 104, and the ideally corrected curve 106.
[0081]
Furthermore, based on the result of calibration, a distortion-corrected image can be created by the image processing unit 8 and displayed on the display unit 9. In this way, it is possible to provide an image display device that displays an image captured by a camera with a large distortion after distortion correction.
[0082]
Next, the calibration result diagnostic apparatus of the present invention will be described. The calibration result diagnostic apparatus is suitable for confirming the accuracy when three-dimensional measurement is performed using stereoscopically observable image data captured by the lens, using a calibration element for the lens measured by the calibration apparatus. is there. In other words, the calibration result diagnostic device is used when strict calibration is required and accurate three-dimensional measurement is performed, or when it is desired to further verify whether there is an error in the calibration elements for the lens measured using the flat chart. To do.
[0083]
FIG. 11 is a configuration block diagram for explaining an embodiment of the calibration result diagnostic apparatus of the present invention. In the figure, the three-dimensional reference chart 20 is one in which the positions of the targets 20a to 20h are accurately measured in three dimensions, and the number, height, and plane coordinates of the targets are appropriately set so as to be suitable for three-dimensional measurement. It is stipulated in. The camera 22 shoots the stereoscopic reference chart 20 in stereo, and it is preferable to shoot using the same type of camera as the camera 2 to be compensated for lens aberration and at the same focal length as the camera 2. The reference stereo image data 24 is image data obtained so that the stereo reference chart 20 can be viewed in stereo with the camera 22, and is typically shot at a set of shooting positions of the left shooting position 22L and the right shooting position 22R. Yes.
[0084]
The target position measuring unit 26 extracts the targets 20a to 20h photographed in the reference stereo image data 24, and measures the positions of the targets 20a to 20h in the image coordinate system. The display unit 28 is a display device such as a CRT or a liquid crystal, and displays the target position of the stereoscopic reference chart 20 measured by the reference stereo image data 24 or the target position measuring unit 26, for example. The arithmetic processing unit 30 calculates parameters necessary for relative orientation and absolute orientation from position information in the image coordinate system of the targets 20a to 20h measured by the target position measuring unit 26, and is a lens aberration compensation parameter storage unit. The calibration element stored in 11 is used. The calibration element is for the lens measured by the calibration device. For the calculation for orientation in the calculation processing unit 30, a calculation method necessary for measurement of surface roughness using a stereo image used in aerial photogrammetry or the like is used.
[0085]
The three-dimensional reference chart target storage unit 32 stores the positions of all the targets 20 a to 20 h of the three-dimensional reference chart 20. The aberration compensation parameter suitability determination unit 34 as a calibration determination unit includes a target position of the stereoscopic reference chart 20 calculated by the calculation processing unit 30 and positions of the targets 20 a to 20 h stored in the stereoscopic reference chart target storage unit 32. Are compared to determine whether or not the lens aberration compensation by the calibration element stored in the lens aberration compensation parameter storage unit 11 is appropriate.
[0086]
The stereo image generation means 36 reads the reference stereo image data 24 relative to each other by the arithmetic processing unit 30 and generates a stereo image of the reference stereo image data 24. For example, a processor that performs image calculation processing at high speed is used. used. Here, the stereo image means an image obtained by correcting the displacement of a pair of stereo images taken by the camera 22 in the right shooting direction and the left shooting direction so that the images can be viewed stereoscopically. The stereoscopic image display unit 38 is an image device that stereoscopically displays the stereoscopic image formed by the stereo image generation unit 36. The target position measurement unit 26, the arithmetic processing unit 30, the aberration compensation parameter suitability determination unit 34, and the stereo image generation unit 36 can be configured in a digital plotter or a personal computer.
[0087]
FIG. 12 is a flowchart for explaining calibration result diagnosis using the calibration result diagnosis apparatus of the present invention. First, the calibration result diagnosis apparatus reads the reference stereo image data 24 (S200). At this time, the accurate positions of the targets 20a to 20h displayed in the reference stereo image data 24 may be input to the three-dimensional reference chart target storage unit 32.
[0088]
Next, the reference stereo image data 24 is displayed on the display unit 28 by the target position measurement unit 26 (S210). At this time, the target position measurement unit 26 may perform image processing so that an operator can easily identify pixels recognized as targets in the reference stereo image data 24.
[0089]
Next, the target position measurement unit 26 measures the corresponding target coordinates in each of the left and right (stereo) images of the reference stereo image data 24 (S220). Typically, when an operator designates a target using a mouse on an image displayed on the display unit 28, the target position measurement unit 26 reads an image coordinate value of the target. Note that the image coordinate value of the target coordinate may be read in advance for each image at the stage of the reference stereo image data 24 and input to the calibration result diagnostic apparatus as the target coordinate value.
[0090]
Next, the calculation processing unit 30 performs a relative orientation calculation process based on the read image coordinate values of the target (S230). At this time, the lens aberration is compensated by the calibration element stored in the lens aberration compensation parameter storage unit 11 by the arithmetic processing unit 30 for the read image coordinate values of the target. Further, when the accurate positions of the targets 20a to 20h stored in the three-dimensional reference chart target storage unit 32 are used, the arithmetic processing unit 30 enables absolute orientation. If the relative orientation is used, the model coordinate value can be compared. If the absolute orientation is used, the comparison can be performed using the three-dimensional coordinate value. Note that if the calibration factor for the lens measured by the calibration device is inferior, stereoscopic vision itself cannot be performed, and therefore it can be confirmed by the fact that the orientation cannot be performed.
[0091]
(V): Mutual orientation
Here, the details of relative orientation will be described. The orientation calculation is used in aerial photogrammetry and the like, and the positions and the like of the left and right imaging devices are obtained by two-step calculation of relative orientation and absolute orientation. In relative orientation, stereo imaging parameters are obtained by the following coplanar conditional expression.
[0092]
FIG. 13 is a diagram for explaining the orientation calculation using the model coordinate system XYZ and the left and right camera coordinate systems xyz. The origin O of the model coordinate system XYZ is taken as the left projection center, and the line connecting the right projection centers is taken as the X axis. The scale is based on the base length l as a unit length. The parameters to be obtained at this time are 5 of the left camera Z-axis rotation angle κ1, the Y-axis rotation angle φ1, the right-hand camera Z-axis rotation angle κ2, the Y-axis rotation angle φ2, and the X-axis rotation angle ω2. One rotation angle. In this case, since the rotation angle ω1 of the X axis of the left camera is 0, there is no need to consider it.
[0093]
[Equation 5]

Under such conditions, the coplanar conditional expression of (Expression 11) becomes (Expression 12), and each parameter is obtained by solving this expression.
[Formula 6]

Here, the following relational expression for coordinate transformation is established between the model coordinate system XYZ and the camera coordinate system xyz.
[Expression 7]

[0094]
Using the formulas (Equation 11) to (Equation 13), an unknown parameter is obtained by the following procedure.
V-1: The initial approximate value is normally 0.
V-2: The coplanar conditional expression (Expression 12) is Taylor-expanded around the approximate value, and the value of the differential coefficient when linearized is obtained from the two expressions in (Expression 13), and the observation equation is established.
V-3: A least square method is applied to obtain a correction amount for the approximate value.
V-4: The approximate value is corrected.
V-5: Using the corrected approximate value, the operations from V-2 to V-5 are repeated until convergence.
[0095]
In the relative orientation calculation described above, the calculation is performed using the calibration elements stored in the lens aberration compensation parameter storage unit 11, that is, the calibration elements obtained from the chart 1. Specifically, the principal point position and the lens aberration values Δx and Δy are calculated with correction by a calibration element. The screen distance (focal distance) c is also calculated using the calibration value.
[0096]
Furthermore, when performing absolute orientation, the positions of the targets 20a to 20h stored in the three-dimensional reference chart target storage unit 32 are used. Then, since the scale is known, the comparison using the three-dimensional value on the actual coordinates of the target becomes possible.
[0097]
Returning to FIG. 12, the aberration compensation parameter suitability determination unit 34 determines whether or not the lens aberration compensation by the calibration element stored in the lens aberration compensation parameter storage unit 11 is appropriate (S240). That is, the aberration compensation parameter suitability determination unit 34 compares the target position of the stereoscopic reference chart 20 calculated by the arithmetic processing unit 30 with the positions of the targets 20a to 20h stored in the stereoscopic reference chart target storage unit 32. Therefore, if it is within the allowable error range, it is considered “appropriate”. The allowable error range can be determined from the measurement conditions, but is preferably determined based on the allowable error range of the person using the photographing data compensated for the lens aberration. If the calibration element according to chart 1 includes an error, the relative orientation calculation described above may not converge. In that case, the aberration compensation parameter suitability determination unit 34 determines “unsuitability”.
[0098]
Here, the determination of the calibration element may be finished, but a stereo image is created and confirmed using the above relative orientation data and the positions of the targets 20a to 20h stored in the stereo reference chart target storage unit 32. It is also possible to do. When creating a stereo image, the following processing is further performed.
[0099]
The stereo image generating means 36 creates a stereo image of the reference stereo image data 24 (S250). Since the position and inclination of the stereo image can be known by the external orientation element calculation by the relative orientation, a final stereo image is created by the stereo image generating means 36 with further consideration of the calibration element.
[0100]
Then, the created stereo image is displayed on the display unit 28 (S260). Alternatively, image data is output. If there is a stereoscopic image monitor on the display unit 28, a stereo image can be displayed here for visual confirmation. In this case, if the calibration element according to the chart 1 is bad, an image that cannot be stereoscopically viewed (for example, the vertical parallax cannot be removed) is displayed, which can be confirmed. Alternatively, the difference in the three-dimensional position can be confirmed on the display unit 28 by displaying the reference coordinate position and the calculated measurement coordinate position. If there is no stereo monitor, it is possible to output stereo image data and confirm the display on another display device.
[0101]
When the displayed stereo image is viewed and the accuracy is not satisfactory, the entire calibration operation using the chart 1 and the camera 2 is performed again to obtain a lens aberration calibration element.
[0102]
FIG. 14 is a block diagram illustrating the configuration of the second embodiment of the calibration result diagnostic apparatus according to the present invention. 14 that have the same functions as those in FIG. 11 are denoted by the same reference numerals, and description thereof is omitted. In the figure, a reference calculation processing unit 31 performs a reference calculation of reference stereo image data using a reference calibration element prepared together with reference stereo image data to obtain a model coordinate value or a three-dimensional coordinate value of a reference point. The aberration compensation parameter suitability determination unit 34 includes the model coordinate value or three-dimensional coordinate value of the reference point calculated by the calculation processing unit 30 and the model coordinate value or three-dimensional coordinate of the reference point calculated by the reference calculation processing unit 31. The value is compared to determine whether the calibration element is appropriate.
[0103]
In the apparatus configured as described above, a reference calibration element for the lens of the camera 22 is prepared in advance. Then, the coordinate value of the reference point in the three-dimensional reference chart 20 used for the aberration compensation parameter suitability determination unit 34 is given as the model coordinate value or three-dimensional coordinate value of the reference point calculated by the reference calculation processing means 31. Therefore, it is suitable for managing variations in calibration elements of individual lenses when handling a large amount of lenses, for example, quality control in the field of lens manufacturing.
[0104]
Note that, as the reference stereo image data 24 used in the calibration result diagnosing apparatus of the present invention, two reference stereo image data captured in advance by the same type of camera as the camera 2 at the same focal length as the camera 2 are used. Although described above, the present invention is not limited to this, and may be reference stereo image coordinate data of the stereo reference chart 20, for example. Further, the case has been described in which the positions of all targets of the stereoscopic reference chart 20 are stored in the stereoscopic reference chart target storage unit 32, and the lens aberration is compensated by the lens aberration calibration element and compared with the observed target position. The present invention is not limited to this. For example, in an application where lens aberration is compensated to such an extent that stereoscopic viewing is possible, the position information of all targets of the stereoscopic reference chart 20 may not be used.
[0105]
In addition, reference data such as reference stereo image data, reference stereo image coordinate data, reference calibration elements, and the like can be obtained from a flexible disk, CD-ROM, It may be supplied by downloading from a website via the Internet.
[0106]
【The invention's effect】
As described above, according to the calibration device of the present invention, the extraction chart that extracts the first mark from the calibration image captured from at least two directions with the camera that performs calibration, the calibration chart, With respect to the calibration image, an approximate mark position measurement unit that obtains an approximate position of the second mark in the calibration image by projective transformation using the first mark extracted by the extraction unit, A precision mark position measuring unit that obtains a photographed position of the second mark position in the vicinity of the approximate position of the second mark, the position of the second mark in the calibration chart, and the calibration corresponding to the second mark From the position of the second mark in the image for calibration, Since a structure that includes an arithmetic processing unit for calculating the Deployment element, calibration elements necessary for correcting the lens aberrations can be conveniently measured. Therefore, in applications where even a small amount of lens aberration is a problem, such as in stereoscopic vision, even if the image is taken in stereo with an inexpensive lens with large lens aberration, the image is less affected by lens aberration using the calibration element. It can be created and can be easily stereoscopically viewed.
[0107]
In addition, the calibration chart has a simple form of having a first mark provided at least at three or more locations and a second mark provided so as to be visually identifiable with respect to the first mark. Compared with the case where the lens aberration is measured using a three-dimensional reference chart in which reference points are arranged in the calibration, the calibration can be performed easily.
[Brief description of the drawings]
FIG. 1 is an overall configuration block diagram for explaining a first embodiment of the present invention;
FIG. 2 is a plan view showing an example of a calibration chart.
FIG. 3 is an explanatory diagram illustrating an example of a first mark.
FIG. 4 is an explanatory diagram showing an example of a second mark.
FIGS. 5A and 5B are diagrams for explaining camera arrangement in the case of measuring lens aberration. FIG. 5A is a three-dimensional view showing the camera arrangement, and FIG. 5B is a diagram when a standard lens and a telephoto lens are generally used. It is the figure which showed the camera space | interval.
FIGS. 6A and 6B are explanatory diagrams of a chart photographing procedure using a standard lens or a wide-angle lens camera, in which (A1) to (A4) are camera images, and (B1) to (B4) are charts corresponding to camera images and camera The positional relationship is shown.
FIG. 7 is a flowchart for explaining a calibration method using a calibration apparatus.
FIG. 8 is a detailed flow diagram illustrating a processing procedure for measuring a precise position of a second mark.
9A is an explanatory diagram of an image coordinate system and a target coordinate system in central projection, and FIG. 9B is an explanatory diagram of a normalized correlation template image and target image used for target recognition.
FIG. 10 is an explanatory diagram illustrating an example of a calculation process result of a calibration element.
FIG. 11 is a configuration block diagram illustrating an embodiment of a calibration result diagnostic apparatus according to the present invention.
FIG. 12 is a flowchart illustrating calibration result diagnosis using the calibration result diagnostic apparatus of the present invention.
FIG. 13 is a diagram for explaining orientation calculation using a model coordinate system XYZ and left and right camera coordinate systems xyz.
FIG. 14 is a block diagram illustrating a second embodiment of a calibration result diagnostic apparatus according to the present invention.
[Explanation of symbols]
1 chart (calibration chart)
2 Camera
3 Image data storage
4 extractor
5 Outline mark position measurement unit
6 Precision mark position measurement unit
7 Arithmetic processing part
8 Image processing section
9 Display
20 3D standard chart
22 Camera
26 Target position measuring means
30 arithmetic processing means
32 solid reference chart target storage unit
34 Aberration compensation parameter suitability determination unit (calibration determination unit)

Claims (14)

A calibration chart having at least three or more first marks and a second mark that can be visually distinguished from the first marks is photographed from at least two directions with a camera that performs calibration. An extraction unit for extracting the first mark from the calibration image; and an outline for determining an approximate position of the second mark in the calibration image by projective transformation using the first mark extracted by the extraction unit. A mark position measurement unit; a precision mark position measurement unit that obtains a position where the second mark position is photographed in the vicinity of the approximate position of the second mark with respect to the calibration image; The position of the second mark and the calibration image corresponding to the second mark From the second mark position that, an arithmetic processing unit for calculating the calibration elements of the camera; equipped with,
The first marks are distributed and arranged in at least three or more quadrants among the quadrants when the calibration chart is equally divided into four sections, and the total number of the second marks on the calibration chart Is arranged more than the total number of the first marks,
Calibration device.  The calibration device according to claim 1, wherein the approximate mark position measurement unit is configured to obtain an approximate position of the second mark by projective transformation.  The calibration apparatus according to claim 1, wherein the precision mark position measurement unit is configured to determine the second mark position using at least one of template matching and barycentric position detection.  The calculation processing unit calculates distortion in the second mark from the position of the second mark in the calibration chart and the position of the second mark in the calibration image corresponding to the second mark. The calibration apparatus according to any one of claims 1 to 3.  The calculation processing unit is configured to select an appropriate second mark from among the second marks whose positions are calculated by the precision mark position measurement unit when calculating the calibration element of the lens. The calibration device according to claim 4.  The calibration device according to claim 1, wherein the calibration element includes at least one of a focal length of the lens, a principal point position, and a distortion parameter of the lens.  The image processing unit according to any one of claims 1 to 6, further comprising an image processing unit that corrects an image captured by the camera that performs the calibration into an image in which the lens aberration is compensated based on a calibration element obtained by the arithmetic processing unit. The calibration device according to any one of the above items.  The calibration device according to claim 1, further comprising a display unit that displays a calibration element or distortion in the second mark. A calibration chart having at least three or more first marks and a second mark that can be visually distinguished from the first marks is photographed from at least two directions with a camera that performs calibration. A first step of extracting a first mark from the calibration image obtained; a second step of obtaining an approximate position of the second mark in the calibration image by projective transformation from the extracted first mark; A third step of accurately obtaining the second mark position in the vicinity of the approximate position of the second mark with respect to the calibration image; the position of the second mark in the calibration chart; and the second mark corresponding to the second mark From the position of the second mark in the calibration image, the camera calibre A fourth step of calculating the Deployment element; has,
The first mark, the disposed dispersed in at least three quadrants of the quadrant when equally dividing the calibration chart in four categories, the second mark on the calibration tea over preparative the total number, are arranged more than the total number of the first mark,
Calibration method.  The camera lens is a standard lens or a wide-angle lens; the calibration chart is provided with the first marks separated from each other by a predetermined distance; and when the calibration chart is photographed by the lens The calibration method according to claim 9, wherein the photographing direction is selected as an angle at which the focal length of the lens can be accurately calculated.  The shooting direction of the lens is: a first shooting direction in which one of the first marks is taken as the front; a camera position is provided in front of another first mark that is separated from the first mark by the predetermined distance; The calibration method according to claim 10, further comprising: a second photographing direction in which an optical axis of the lens is directed to the first mark.  The lens is a telephoto lens; the calibration chart is provided with the first marks separated from each other by a predetermined distance; and the lens is used as a position for photographing the calibration chart by the lens; An interval corresponding to a distance of about 1/3 with respect to the imaging distance from the calibration chart is secured in a direction perpendicular to the optical axis of the lens; one imaging position is a first imaging direction and the other imaging is performed. The calibration method according to claim 9, wherein the position is imaged as the second imaging direction. A calibration chart having at least three or more first marks and a second mark that can be visually distinguished from the first marks;
The first marks are distributed and arranged in at least three or more quadrants among the quadrants when the calibration chart is equally divided into four sections, and the total number of the second marks on the calibration chart Is arranged more than the total number of the first marks,
A calibration chart used for camera calibration.  The calibration chart according to claim 13, wherein the first mark and the second mark are displayed on an image displayed on a flat display.

Images