JP4003465B2 - Specific pattern recognition method, specific pattern recognition program, specific pattern recognition program recording medium, and specific pattern recognition apparatus - Google Patents

Description

このように、モデル変形過程では、局所パターン候補点抽出過程で抽出された局所パターン候補点から、少なくとも２つの相異なる局所パターンと符合する局所パターン候補点の組を少なくとも１組抽出することにより、抽出された組の各局所パターン侯補点からなる変形候補が選定される。
【００７５】
また、選定された変形候補の各局所パターン侯補点の位置関係をもとに、認識用モデルに、位置、スケール、角度の変形を加えて、その認識用モデルの全体パターンを認識対象画像に重ね合わせる際の各変形パラメータが求められる。
【００７６】
例えば、図３（ｂ）〜（ｇ）のように、パターンの形状およびパターンの明度に変動がある場合であっても、前述の局所パターン候補点抽出過程において、局所パターン候補点はパターンのスケール、回転、明度などの各変動に影響されにくい方法で検出されているので、各局所パターン候補点は正しく求められているから変形候補を抽出できる。また、図３（ｅ）および（ｇ）に示す局所的な明度ムラやノイズ重畳の程度がひどく、局所パターンが見えない場合や、図３（ｉ）のように一部の局所パターンが障害物に隠されている場合に、複数（実施形態に示すパターンの場合、図４のｐ１〜ｐ４に示す４種類）の局所パターンのうち、幾つかの局所パターンに符合する局所パターン候補点が見つからなかったとしても、最低限２種類の局所パターンに符合する局所パターン候補点があれば変形候補を抽出できる。また、図３（ｈ）のように、連結パターンに符合する部分が隠されている場合には、局所パターンは問題なく見えているのでモデル変形過程において影響はない。
【００７７】
２．高速化処理
次に、高速化のために上記変形候補の絞込みが行われる。前段の局所パターン侯補点抽出過程では、局所パターンの種類毎にパターンが類似している座標位置とともに方向を抽出しているので、その方向が利用される。
【００７８】
すなわち、変形候補の全体パターンに対する局所パターン侯補点の方向の角度と、認識用モデルの全体パターンに対する局所パターンの方向の角度とを比較し、両者の角度差が所定角よりも大きければ、その変形候補は、ありえない変形であるとして除外される。
【００７９】
例えば、図１６中の変形候補ｄｆ１〜ｄｆ４のうち、変形候補ｄｆ４の全体パターンに対する局所パターン侯補点ｃｐ１２の方向の角度と、図４の認識用モデルの全体パターンに対する局所パターンｐ１の方向の角度とが大きく異なっているので、変形候補ｄｆ４が除外されることになる。また、このような場合の両者の角度差よりも小さい値に上記所定角が設定されることになる。これにより、図１６の変形候補ｄｆ１〜ｄｆ４が作成ないし選定されたとき、変形候補ｄｆ１〜ｄｆ３に絞り込まれることになる。
【００８０】
（４）類似度評価過程
１．照合
類似度評価過程では、認識対象画像とこれに対する変形モデルとを照合し、各変形モデルから最適な変形モデルを選択する処理が実行される。
【００８１】
具体的には、モデル変形過程で求められた変形候補の変形パラメータに基づいて、認識用モデルを変形し、この変形した認識用モデルを認識対象画像に重ね合わせ、変形候補と認識対象画像との類似度を評価する処理が実行される。
【００８２】
このとき、連結パターンが認識用モデルにおいてどのように表現されているか（画像か、数式化されているか）や画像の特性や認識の目的（明度ムラやパターン隠れが頻発するかどうか、あるいは単独バターンの認識を目的とするか、複数の類似パターンの識別を目的とするかなど）によって、種々の類似度評価方法から適切な類似度評価方法が選択されて使用される。以下、各類似度評価方法について説明する。
【００８３】
２．連結パターンが画像データである場合の類似度評価方法
図１７は類似度評価方法の説明図である。認識基準画像を含むデジタル画像において、局所パターン間の部分画像が不明瞭なパターンであったり、複雑なパターンであったりする場合には、局所パターン間の部分画像がそのまま連結パターンとして使用される。このとき、図１７の例に示すように、連結パターンを構成する部分画像を、上記変形候補の変形パラメータ（スケール、回転など）に合わせて変形させることにより、連結パターンを変形することができる。そして、この変形した連結パターン候補を含む変形した認識用モデルの全体パターンと、認識対象画像とを、正規化相関マッチングでマッチングすれば、類似度評価値を得ることができる。
【００８４】
３．連結パターンが数式データである場合の類似度評価方法
認識用モデルを構成する連結パターンが、エッジ形状を数式化したものである場合には、変形モデルと認識対象画像との照合方法は、画像の特性に応じて幾つかの方法を選択することができる。
【００８５】
（第１の照合方法）
図１８は連結パターンが数式データである場合の第１の照合方法の説明図である。第１の照合方法では、正規化相関法による照合方法が使用される。図１６に示す変形候補ｄｆ１〜ｄｆ３について、図１８に示すように認識用モデルの輪郭線内にある領域Ｐｉと認識用モデル輪郭の近傍の領域Ｏｉに含まれる画素に対して以下の数式に基づいて相関係数Ｓ１を計算し、最も相関係数が高いものを最適な照合として選択する。
【００８６】
【数１】

【００８７】
第１の照合方法によれば、多数の画素値を利用して照合を行うことができるため、明度ムラに弱い反面、類似した多数のパターンから最も類似したものを選択する必要がある場合やコントラストが低い場合などに信頼性の高い照合が可能となる。
【００８８】
（第２の照合方法）
図１９は連結パターンが数式データである場合の第２の照合方法の説明図である。図１９（ａ）は図１６のｄｆ１による変形モデルに対する照合方法を示し、図１９（ｃ）は図１６のｄｆ２による変形モデルに対する照合方法を示す。
【００８９】
第２の照合方法では、パターンのエッジ付近の明度変化を用いて類似度を評価する方法が使用される。
【００９０】
図１９に示すｌ１１〜ｌ１ｍ，ｌ２１〜ｌ２ｎ（ｍおよびｎはサンプリング点数）のように、図４の局所パターンｐ１の位置を始点とし、変形モデルの連結パターン候補に沿って、時計回りに連結パターン候補上にサンプリング点を複数配置する。続いて、サンプリング点ごとにサンプリング点を中心として連結パターン候補の法線方向に伸びる長さｌの線分（以下「サンプリングライン」という）を設定する。このとき、サンプリングラインに沿って明度を抽出すると、図１９（ｂ），（ｄ）に示すような結果となる。
【００９１】
これらの図に示す各サンプリングラインの明度変化について、サンプリングラインの中点（変形モデルの連結パターン候補上の位置）で、理想的なステップ状（図２０参照）に変化するエッジパターンとの相関係数Ｓ２を、以下の数式により求めれば、エッジ付近の画素のみを用いた照合ができる。
【００９２】
【数２】

【００９３】
最適な照合である図１６のｄｆ１の変形に対しては、図１９（ｂ）に示すごとく、エッジと理想的なステップ状エッジパターンの形状が各々のサンプリングラインにおいてよく類似しているため、相関係数Ｓ２が高くなる。他方、エッジ位置がずれている図１６のｄｆ２の変形に対しては、図１９（ｄ）に示すごとく、エッジと理想的なステップ状エッジパターンの形状が異なるため相関係数Ｓ２が低くなる。このことを利用して、複数の変形モデルから最適な照合を得ることが可能となる。このように、エッジ付近の明度変化のみを用いて照合を行うことにより、パターン内部に明度変動を起こしやすいパターンが存在する場合に信頼性の高い照合を行うことができる。
【００９４】
（第３の照合方法）
第３の照合方法では、第２の照合方法と同じ手順によりサンプリングラインを設定した後、次式により類似度を計算する方法が使用される。
【００９５】
【数３】

【００９６】
第２の照合方法と第３の照合方法との違いは、前者の方法では各サンプリングラインの明度変化を全て合計して、相関係数を１回だけ算出しているのに対し、後者の方法では相関係数を各サンプリングラインごとに計算し、その平均値を取っている点である。
【００９７】
前者の方法は、第１の照合方法において連結パターン候補の周囲だけを切り出して相関係数を算出する処理に相当するので、パターン内部に明度変動が起きやすい部分が存在する場合に、その部分の影響を排除できる特長があるが、連結パターン部分に明度のムラがある場合には誤認識を起こす可能性がある。
【００９８】
これに対し、後者の方法によれば、各サンプリングラインごとに相関係数を計算することにより、連結パターン部分に大きな明度ムラがある場合にも信頼性の高い照合が可能となる特長がある。
【００９９】
また、第１〜第３の照合方法において、計算した相関係数の値は−１〜１の間の数値を取るが、この相関係数は絶対値を取っても良い。こうすることにより、パターンの明暗が反転している場合であっても照合が可能となる。
【０１００】
（第４の照合方法）
図２１は連結パターンが数式データである場合の第４の照合方法の説明図、図２２はエッジ位置の検出方法の説明図である。
【０１０１】
第４の照合方法では、エッジ位置を用いる方法が使用される。第２および第３の方法と同様、変形モデルの連結パターン候補に直交するサンプリングラインを複数設定し、サンプリングラインごとにエッジ位置を検出する。図２１にはサンプリングラインごとにエッジ位置を抽出した状態が示されている。この各エッジ位置と線上パターンとの距離の平均を求め、平均距離が最小となる変形モデルを最適な照合として採用する。
【０１０２】
第４の照合方法において、エッジを検出する方法としては、微分値の最大値を用いる方法が簡便であるが、図２２に示すように、サンプリング線の任意の位置の右側の明度平均値ａ１と左側の明度平均値ａ２を求め、｜ａ１−ａ２｜が最大となる位置をエッジ位置として用いることにより、エッジ付近での局所的な明度変化の影響を受けることなく正しいエッジ位置の検出が可能となる。
【０１０３】
第４の照合方法には、最適な変形モデルの選択と同時に実際の認識対象画像におけるエッジの抽出も行うことができる特長がある。連結パターンが数式により表現されている場合には、抽出されたエッジ位置をもとに連結パターンの数式に最小自乗近似することにより、局所パターンの形状の影響によるマッチングのズレやパターンの歪み（相似変形でない変形）を補正し、認識精度を向上することができる。
【０１０４】
以上のように、モデル生成過程、局所パターン侯補点抽出過程、モデル変形過程、類似度評価過程の４つの処理過程を経ることにより、回転、サイズ変動はもとより、明度ムラ、ピントずれ、コントラスト低下、ノイズ、パターン隠れなどの変動が生じても認識ミスを発生しない頑健な認識が実現されるのである。
【０１０５】
なお、上記４つの処理過程の各過程における処理方法は、複数の特性の異なる方法から目的によって最適な処理方法やパラメータを選ぶ必要がある。したがって、使用者が各処理方法の特性を理解していないと最適な処理方法が採用できない問題があるので、この点を自動化することにより利便性を向上できる。
【０１０６】
具体的には、例えば連結パターンのデータが画像データであるか、エッジ形状の数式化されたデータであるかに応じて、類似度評価過程における照合方法を変更すればよく、あるいはパターン内部にまた別のパターンが存在するかどうかの情報を元に、類似度評価過程において第１の照合方法を用いるか第２の照合方法を用いるかを決定すればよい。この場合、局所パターンおよび連結パターンの特徴と、パターンの認識に対する最適な処理方法や認識パラメータとの関係を記述したデータベースをあらかじめ作成しておき、モデル生成過程において、検出すべきパターンの解析を行って局所パターンおよび連結パターンの分類を行う際、データベースを参照して後段の処理手順を決定するようにすればよい。
【０１０７】
また、時系列的に変動する要因を調べるために、複数の画像に対して認識を実行し、その結果をもとに最適な処理方法やパラメータを選ぶことにより、さらに利便性を向上することができる。具体的には、複数の画像に対して行った認識結果をもとに、明度ムラが多く発生していれば類似度評価過程における第３の照合方法を採用し、そうでなければ第１の照合方法を採用する方法などが挙げられる。この場合も、局所パターンおよび連結パターンの画像間変動の特徴と、パターンの認識に対する最適な処理方法および認識パラメータとの関係を記述したデータベースをあらかじめ作成しておき、複数の画像を用いた認識結果をもとにデータベースを参照して後段の処理手順を決定するようにすればよい。
【０１０８】
また、このように複数の教示用画像を用いる場合には、上記のように処理方法やパラメータだけでなく、認識用モデルの形状の変動も得ることができる。複数の教示用画像から局所パターンデータおよび連結パターンデータを抽出し、ある基準となる教示用画像（通常１枚目の画像）に合わせて他の教示用画像から得られた局所パターンデータおよび連結パターンデータを正規化し、データの平均および変動量を求めることにより、単一の教示用画像のみから認識用モデルを構築する場合よりも信頼性の高い認識が可能な認識用モデルを構築することができる。
【０１０９】
時系列で入力される連続画像に対する処理の場合には、時系列のある時点における認識方法をそれまでの認識結果を用いて変更することにより、認識速度の向上を図ることができる。
【０１１０】
上記連続画像に対する処理に係わる第１の方法として、先に行ったパターン照合の結果をもとに認識用モデルを変更する方法が考えられる。図２３に示すように、パターンが連続して歪みを起こす場合には、上述した相似変形でない変形を補正する場合と同様に、パターンの歪みを補正してパターン照合し、照合の結果を用いて局所パターンあるいは連結パターンを順次変更することにより、時系列で相似変形以外の変形を起こす対象物の追跡が可能となる。
【０１１１】
連続画像に対する処理に係わる第２の方法として、先に行ったパターン照合の結果をもとに処理方法を変更する方法が挙げられる。図２４に示すようにパターンが連続して回転、スケール変動するような連続画像においては、その時点より前の照合結果をもとに局所パターン侯補点抽出過程において局所パターンの位置、回転、スケール変動のパラメータを推定し、局所パターン侯補点抽出の際の探索範囲や局所パターンの回転範囲を限定することによりマッチング処理を効率化したり、抽出する局所パターン侯補点を絞り込むことにより後段の処理過程を効率化することができる。
【０１１２】
【発明の効果】
請求項１記載の発明は、デジタル画像に含まれる特定パターンを有する認識対象画像の位置および向きを特定することができるものであって、前記認識対象画像に対応する基準となる認識基準画像を含むデジタル画像から得られ一の方向がそれぞれに付与された少なくとも２つの局所パターンと、これら局所パターン間を連結する少なくとも１つの連結パターンとから構成される認識用モデルを利用して、画像処理装置が認識対象画像の特定パターンを認識する特定パターン認識方法であって、前記認識用モデルを構成する局所パターンのうち、少なくとも２つの異なる局所パターンの各々に符合する局所パターン候補の位置である局所パターン候補点を、前記認識対象画像を含むデジタル画像から少なくとも１つ抽出し、その少なくとも１つの局所パターン候補点に、対応する局所パターンに付与された方向を割り当てる局所パターン候補点抽出過程と、この局所パターン候補点抽出過程で抽出された局所パターン候補点から、少なくとも２つの相異なる局所パターンと符合する局所パターン候補点の組を少なくとも１組抽出することにより、抽出された組の各局所パターン侯補点からなる変形候補を選定し、選定された変形候補の各局所パターン侯補点の位置関係をもとに、前記認識用モデルに、位置、スケール、角度の変形を加えて、その認識用モデルの全体パターンを前記認識対象画像に重ね合わせる際の各変形パラメータを求めるモデル変形過程と、このモデル変形過程で求められた変形パラメータに基づいて、前記認識用モデルを変形し、この変形した認識用モデルを前記認識対象画像に重ね合わせ、前記変形候補と前記認識対象画像との類似度を評価する類似度評価過程とを含むので、認識対象画像のパターンに、回転、サイズ変動、ピントずれ、明度ムラ、コントラスト低下、ノイズ、パターン隠れ等の変動が生じている場合であっても、認識対象画像の全体のうち、認識対象画像の位置および向きを特定することができる部分が正常に撮影されていれば、その部分に対応する局所パターンをもとに、認識対象画像の全体パターンを推定して認識対象画像との照合を行うことができるから、照明変動や障害物、位置決め誤差などの環境の変動の影響などを受け難い頑健な認識が可能となる。
【０１１３】
請求項２記載の発明は、請求項１記載の特定パターン認識方法において、前記認識用モデルを生成するモデル生成過程をさらに含み、このモデル生成過程では、前記認識基準画像の輪郭線を求め、この輪郭線における曲率値が所定値よりも大きい領域の部分画像から、前記局所パターンを抽出するので、画像のコントラストが高く輪郭線を容易に得ることができる場合には、位置、方向を特定しやすい局所パターンを簡単に抽出することができる。
【０１１４】
請求項３記載の発明は、請求項１記載の特定パターン認識方法において、前記認識用モデルを生成するモデル生成過程をさらに含み、このモデル生成過程では、前記認識基準画像を含むデジタル画像の部分画像について、画素毎に明度変化方向を求め、少なくとも異なる２方向に明度変化のピークが現れる部分の画像のパターンを前記局所パターンとして抽出するので、請求項２記載の発明では輪郭検出が困難な複雑なパターンの場合に、方向画像のヒストグラムを作ってピークが２つある所を探すなどの方法により局所パターンを抽出することができる。
【０１１５】
請求項４記載の発明は、請求項１記載の特定パターン認識方法において、前記連結パターンは画像パターンであるので、連結パターンが複雑な形状をしている場合や不明瞭なパターンである等の理由でエッジ形状を数式で表現することが困難な場合にも連結パターンの表現が可能となる。
【０１１６】
請求項５記載の発明は、請求項１記載の特定パターン認識方法において、前記連結パターンは、直線、少なくとも１つの中間節点を持つ折れ線および曲線のいずれかであり、その連結パターンデータは数式データであるので、認識用モデルのデータ量を小さくすることができる。また、パターン照合および位置検出方法によって精度の向上や歪んだパターンの照合が可能になる。
【０１１７】
請求項６記載の発明は、請求項１記載の特定パターン認識方法において、前記認識用モデルを生成するモデル生成過程をさらに含み、このモデル生成過程では、前記認識基準画像を含むデジタル画像から、前記局所パターンの中心位置を原点とする極座標を、原点を中心とした角度を示す角度軸と原点からの距離を示す距離軸とを有する直交座標に変換して、直交変換画像を作成し、前記局所パターン候補点抽出過程では、前記認識対象画像を含むデジタル画像から、ある注目点を原点とする極座標を直交座標に変換して、直交変換画像を作成し、この直交変換画像と前記モデル生成過程で作成された直交変換画像とを角度軸方向に平行移動させながら、順次類似度を求め、これらの各類似度から最大類似度を抽出し、この最大類似度が所定の基準よりも大きければ、最大類似度が得られたときの直交変換画像の位置を前記局所パターン候補点の位置とするとともに、その最大類似度を与える移動量をもとに、対応する局所パターンに付与された方向を前記局所パターン候補点に割り当てるので、パターンが回転している場合にも局所パターンを抽出するとともに方向も割り当てることができる。
【０１１８】
請求項７記載の発明は、請求項１記載の特定パターン認識方法において、前記認識用モデルを生成するモデル生成過程をさらに含み、このモデル生成過程では、前記認識基準画像を含むデジタル画像から、前記局所パターンの中心位置を原点とする極座標を、原点を中心とした角度を示す角度軸と原点からの距離を示す距離軸とを有する直交座標に変換して、直交変換画像を作成し、この直交変換画像を角度軸に投影して１次元の波形データを作成し、前記局所パターン侯補点抽出過程では、前記認識対象画像を含むデジタル画像から、ある注目点を原点とする極座標を直交座標に変換して、直交変換画像を作成し、この直交変換画像を角度軸に投影して１次元の波形データを作成し、この１次元の波形データと、前記モデル生成過程で作成された波形データとを角度軸方向に平行移動させながら、順次類似度を求め、これらの各類似度から最大類似度を抽出し、この最大類似度が所定の基準よりも大きければ、最大類似度が得られたときの直交変換画像の位置を前記局所パターン候補点の位置とするとともに、その最大類似度を与える移動量をもとに、対応する局所パターンに付与された方向を前記局所パターン候補点に割り当てるので、パターンが回転している場合に簡易な１次元のパターンマッチングによりパターンの類似度および方向を求めることができる。
【０１１９】
請求項８記載の発明は、請求項６記載の特定パターン認識方法において、前記認識対象画像を含むデジタル画像から抽出した直交変換画像を距離軸方向に拡大または縮小して、前記モデル生成過程で作成された直交変換画像とマッチングするので、パターンの回転とともにスケール変化が生じている場合であっても局所パターンを抽出することができる。
【０１２０】
請求項９記載の発明は、請求項１記載の特定パターン認識方法において、前記局所パターン侯補点抽出過程では、局所的に画素値の変化の大きい部分だけを事前に抽出し、その部分のみに対して局所パターン候補点抽出処理を行うので、局所パターン抽出処理の負荷を軽減することができる。
【０１２１】
請求項１０記載の発明は、請求項１記載の特定パターン認識方法において、前記モデル変形過程では、前記変形候補の局所パターン候補点の方向と、対応する前記局所パターンの方向との角度差が所定角より大きい場合、その変形候補を前記類似度評価過程での類似度の評価から除外するので、類似度評価過程における処理の負荷を軽減することができる。
【０１２２】
請求項１１記載の発明は、請求項４記載の特定パターン認識方法において、前記類似度評価過程では、前記連結パターンのデータが画像データである場合、前記モデル変形過程で求められた変形候補の変形パラメータに基づいて、前記連結パターンを変形し、この変形した連結パターンを含む変形した認識用モデルの全体パターンと前記認識対象画像とのマッチングを行うので、連結パターンとして使用するパターンが複雑であったり不明瞭であったりするために連結パターンとして画像を用いている場合に正しいマッチングを行うことができる。
【０１２３】
請求項１２記載の発明は、請求項５記載の特定パターン認識方法において、前記類似度評価過程では、前記連結パターンのデータが線状のパターンを表す数式を用いて表される場合、前記連結パターンから得られた連結パターン候補上にサンプリング点を複数設定し、これらの各サンプリング点を中心として連結パターン候補の法線方向に任意長のサンプリングラインを設定し、設定された全てのサンプリングライン上の画素値の分布と、連結パターン候補の位置で画素値がステップ状に変化する理想パターンとの相関値を算出して類似度とするので、内部パターンを無視してエッジ近傍における類似度だけを評価することができるため、パターン内部に明度ムラや明度が変化するようなパターンが存在する場合にも正しいマッチングを行うことができる。
【０１２４】
請求項１３記載の発明は、請求項５記載の特定パターン認識方法において、前記類似度評価過程では、前記連結パターンのデータが線状のパターンを表す数式を用いて表される場合、前記連結パターンから得られた連結パターン候補上にサンプリング点を複数設定し、これらの各サンプリング点を中心として連結パターン候補の法線方向に任意長のサンプリングラインを設定し、設定されたサンプリングライン上の画素値の分布と、サンプリング点部分で画素値がステップ状に変化する理想パターンとの類似度を調べる処理をサンプリングラインごとに行い、類似度を全てのサンプリングラインに対して求めて平均値を算出し全体の類似度とするので、大きな明度ムラが存在する場合にも正しいマッチングを行うことができる。
【０１２５】
請求項１４記載の発明は、請求項１１から１３のいずれかに記載の特定パターン認識方法において、前記認識対象画像に明暗のコントラスト反転が生じることが既知である場合、類似度として正規化相関値の絶対値を用いるので、明度のコントラスト反転が生じていても正しいマッチングを行うことができる。
【０１２６】
請求項１５記載の発明は、請求項５記載の特定パターン認識方法において、前記類似度評価過程では、前記連結パターンのデータが線状のパターンを表す数式を用いて表される場合、前記連結パターンから得られた連結パターン候補上にサンプリング点を複数設定し、これらの各サンプリング点を中心として連結パターン候補の法線方向に任意長のサンプリングラインを設定し、設定された全てのサンプリングラインの各々についてエッジ位置を求め、エッジ位置と連結パターンの距離の平均値をもって類似度とするので、類似度判定と同時にエッジ抽出を行うことができ、実際の画像上でパターンの輪郭形状を求めることができる。
【０１２７】
請求項１６記載の発明は、請求項１５記載の特定パターン認識方法において、前記サンプリングラインのある注目位置の左右で画素値の平均値を求め、平均画素値の差の絶対値を求め、注目位置を前記サンプリングラインの範囲内で移動させながら、平均画素値の差の絶対値を順次求め、その絶対値が最大となる位置をエッジ位置として用いるので、局所的な特徴によりエッジ位置が著しく不正な位置に検出される危険を回避できる。
【０１２８】
請求項１７記載の発明は、請求項１５記載の特定パターン認識方法において、前記連結パターンの各々について、エッジ位置をもとに最小自乗法を用いて連結パターンの数式に当てはめるので、局所的にエッジ形状が乱れている場合にも正確なエッジ位置の推定が可能となる。
【０１２９】
請求項１８記載の発明は、請求項１記載の特定パターン認識方法において、前記局所パターンおよび連結パターンの特徴と、これらパターンの認識に対する最適な処理方法および認識パラメータのうち少なくとも一方との関係を記述したデータベースを予め作成し、前記認識用モデルを生成するモデル生成過程として、検出するべきパターンの特徴の解析を行い、この解析結果をもとに前記データベースを参照して、前記局所パターン候補点抽出過程、モデル変形過程および類似度評価過程の各過程における処理方法および認識パラメータのうち少なくとも一方を決定する過程をさらに含むので、利用者側がアルゴリズムや認識パラメータを決定する必要がなく、システムとしての使い勝手が向上する。
【０１３０】
請求項１９記載の発明は、請求項１記載の特定パターン認識方法において、前記局所パターンおよび連結パターンの画像間変動の特徴と、それらパターンの認識に対する最適な処理方法および認識パラメータのうち少なくとも一方との関係を記述したデータベースを予め作成し、複数の教示画像に対して試験的に認識を行い、この認識結果をもとに前記データベースを参照して、前記局所パターン候補点抽出過程、モデル変形過程および類似度評価過程の各過程における処理方法および認識パラメータのうち少なくとも一方を決定するので、時系列変化がある対象物にシステムを使用する場合の使い勝手を向上する。
【０１３１】
請求項２０記載の発明は、請求項１記載の特定パターン認識方法において、前記認識用モデルを生成するモデル生成過程をさらに含み、このモデル生成過程では、同じパターンを撮影して得た前記認識基準画像を含むデジタル画像を複数、教示用画像として用いる場合、前記局所パターンデータおよび連結パターンデータの画像間変動を正規化して統合するので、単一の教示用画像のみから認識用モデルを構築する場合よりも信頼性の高い認識が可能な認識用モデルを構築することができる。
【０１３２】
請求項２１記載の発明は、請求項１記載の特定パターン認識方法において、時系列で連続して入力されるデジタル画像に対してパターン照合および位置検出を行い、パターンの追跡を行う場合に、先に行ったパターン照合の結果をもとに前記認識用モデルを変更するので、時系列で相似変形以外の変形を起こす対象物の追跡が可能となる。
【０１３３】
請求項２２記載の発明は、請求項１記載の特定パターン認識方法において、時系列で連続して入力されるデジタル画像に対してパターン照合および位置検出を行い、パターンの追跡を行う場合に、先に行った位置検出の結果をもとに、前記局所パターン候補点抽出過程、モデル変形過程および類似度評価過程の各過程における処理方法を随時変更するので、パターンの探索範囲の絞り込みなどが可能となり、より効率的なパターン照合および位置検出が可能となる。
【０１３４】
請求項２３記載の発明は、デジタル画像に含まれる特定パターンを有する認識対象画像の位置および向きを特定することができるものであって、前記認識対象画像に対応する基準となる認識基準画像を含むデジタル画像から得られ一の方向がそれぞれに付与された少なくとも２つの局所パターンと、これら局所パターン間を連結する少なくとも１つの連結パターンとから構成される認識用モデルを利用して、画像処理装置が認識対象画像の特定パターンを認識する特定パターン認識方法の処理手順を含む特定パターン認識プログラムであって、前記認識用モデルを構成する局所パターンのうち、少なくとも２つの異なる局所パターンの各々に符合する局所パターン候補の位置である局所パターン候補点を、前記認識対象画像を含むデジタル画像から少なくとも１つ抽出し、その少なくとも１つの局所パターン候補点に、対応する局所パターンに付与された方向を割り当てる局所パターン候補点抽出過程と、この局所パターン候補点抽出過程で抽出された局所パターン候補点から、少なくとも２つの相異なる局所パターンと符合する局所パターン候補点の組を少なくとも１組抽出することにより、抽出された組の各局所パターン侯補点からなる変形候補を選定し、選定された変形候補の各局所パターン侯補点の位置関係をもとに、前記認識用モデルに、位置、スケール、角度の変形を加えて、その認識用モデルの全体パターンを前記認識対象画像に重ね合わせる際の各変形パラメータを求めるモデル変形過程と、このモデル変形過程で求められた変形パラメータに基づいて、前記認識用モデルを変形し、この変形した認識用モデルを前記認識対象画像に重ね合わせ、前記変形候補と前記認識対象画像との類似度を評価する類似度評価過程とを含むので、専用の画像処理装置以外の装置であっても、照明変動や障害物、位置決め誤差などの環境の変動の影響などを受け難い頑健な認識が可能となる。
【０１３５】
請求項２４記載の発明の特定パターン認識プログラム記録媒体は、請求項２３記載の特定パターン認識プログラムを前記コンピュータにより読み取り可能に記録したので、専用の画像処理装置以外の装置であっても、照明変動や障害物、位置決め誤差などの環境の変動の影響などを受け難い頑健な認識が可能となる。
【０１３６】
請求項２５記載の発明は、撮像を行う撮像手段と、この撮像手段により撮像された画像信号から２次元マトリクスの各格子点ごとに明度値が記録されたデジタル画像を得る量子化手段と、この量子化手段で得られたデジタル画像のデータを少なくとも記憶する記憶手段とを備え、デジタル画像に含まれる特定パターンを有する認識対象画像の位置および向きを特定することができるものであって、前記認識対象画像に対応する基準となる認識基準画像を含むデジタル画像から得られ一の方向がそれぞれに付与された少なくとも２つの局所パターンと、これら局所パターン間を連結する少なくとも１つの連結パターンとから構成される認識用モデルを利用して、認識対象画像の特定パターンを認識する特定パターン認識装置であって、前記認識用モデルを構成する局所パターンのうち、少なくとも２つの異なる局所パターンの各々に符合する局所パターン候補の位置である局所パターン候補点を、前記認識対象画像を含むデジタル画像から少なくとも１つ抽出し、その少なくとも１つの局所パターン候補点に、対応する局所パターンに付与された方向を割り当てる局所パターン候補点抽出手段と、この局所パターン候補点抽出手段で抽出された局所パターン候補点から、少なくとも２つの相異なる局所パターンと符合する局所パターン候補点の組を少なくとも１組抽出することにより、抽出された組の各局所パターン侯補点からなる変形候補を選定し、選定された変形候補の各局所パターン侯補点の位置関係をもとに、前記認識用モデルに、位置、スケール、角度の変形を加えて、その認識用モデルの全体パターンを前記認識対象画像に重ね合わせる際の各変形パラメータを求めるモデル変形手段と、このモデル変形手段で求められた変形パラメータに基づいて、前記認識用モデルを変形し、この変形した認識用モデルを前記認識対象画像に重ね合わせ、前記変形候補と前記認識対象画像との類似度を評価する類似度評価手段とを含むので、照明変動や障害物、位置決め誤差などの環境の変動の影響などを受け難い頑健な認識が可能となる。
【図面の簡単な説明】
【図１】特定パターン認識装置の構成図である。
【図２】図１中の処理装置によって実行される特定パターン認識方法の処理の流れを示すフローチャートである。
【図３】図１中のカメラおよび量子化器から得られうる各種デジタル画像の例を示す図である。
【図４】モデル生成過程の処理により生成された認識用モデルの例を示す図である。
【図５】認識基準画像を含むデジタル画像から抽出された輪郭線を示す図である。
【図６】図５の輪郭線の曲率（半径の逆数）変化を表すグラフである。
【図７】認識基準画像およびこれを含むデジタル画像中の複数の部分領域を示す図である。
【図８】図７中の各部分領域内の明度変化方向を表すグラフである。
【図９】連結パターンの他の形状例を示す図である。
【図１０】極座標→直交座標変換処理の説明図である。
【図１１】認識対象画像を含むデジタル画像に対する極座標→直交座標変換処理によって得られうる直交変換画像例を示す図である。
【図１２】直交変換画像が単純な縞状のパターンになる場合のマッチング方法の説明図である。
【図１３】直交変換画像が複雑なパターンになる場合のマッチング方法の説明図である。
【図１４】認識対象画像例を示す図である。
【図１５】局所パターン侯補点抽出過程によって図１４の認識対象画像から抽出された局所パターン侯補点を示す図である。
【図１６】モデル変形過程の説明図である。
【図１７】類似度評価方法の説明図である。
【図１８】連結パターンが数式データである場合の第１の照合方法の説明図である。
【図１９】連結パターンが数式データである場合の第２の照合方法の説明図である。
【図２０】理想的なエッジにおける明度変化を示す図である。
【図２１】連結パターンが数式データである場合の第４の照合方法の説明図である。
【図２２】エッジ位置の検出方法の説明図である。
【図２３】パターンに時系列で連続して歪みを生じる状態のデジタル画像例を示す図である。
【図２４】パターンに時系列で連続して相似変形する状態のデジタル画像例を示す図である。
【符号の説明】
１０ 対象物
１１ カメラ
１２ 量子化器
１３ 記憶装置
１４ 画像表示装置
１５ 処理装置
１５１ モデル生成部
１５２ 局所パターン候補点抽出部
１５３ モデル変形部
１５４ 類似度評価部[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a pattern recognition technique applied to electronic circuit positioning and the like, in which a known pattern is stored in advance in a storage means, and this is compared with an input image, so that the known pattern is similar in the image. This is a technique for detecting geometrical parameters such as position, rotation angle, and scale fluctuation when it appears to be deformed, and in particular, an image extracted from an image due to pattern brightness fluctuation, overlap with other patterns, noise generation, etc. The present invention relates to a specific pattern recognition method, a specific pattern recognition program, a specific pattern recognition program recording medium, and a specific pattern recognition device, which can detect a correct geometric parameter even when a feature amount becomes unstable.
[0002]
[Prior art]
In a pattern recognition technology that detects a known pattern from an input image, when the pattern to be detected is only subjected to translation without scale fluctuation or rotation, the pattern image itself is stored in advance as a template. The normalized correlation value between the template and the image at any position on the image (the pixel value of the image cut out to the same size as the template at an arbitrary position of the recognition target image, and the template pixel value There is widely known a normalized correlation pattern matching method for calculating a correlation coefficient between the two and detecting a position where the normalized correlation value is maximized.
[0003]
Further, in the invention described in Japanese Patent Application Laid-Open No. 60-258689 as an example of a recognition method when there is a possibility that the pattern of the recognition target image is rotated, the perimeter of the pattern is obtained from an image obtained by photographing a plurality of patterns. Calculating feature quantities such as area and area, comparing the feature quantity with the feature quantity of the known pattern to detect the desired pattern, and further identifying the rotation angle and position by detecting the straight line part included in the pattern By doing so, it is possible to recognize a pattern in which rotation occurs.
[0004]
[Problems to be solved by the invention]
However, in the normalized correlation method, the position is not correctly detected even when pattern rotation or scale variation occurs as described above, or when partial brightness variation or partial pattern concealment occurs. was there.
[0005]
In addition, the invention described in Japanese Patent Application Laid-Open No. 60-258689 cannot be applied when the pattern to be recognized and the background pattern cannot be separated into a binary image, such as when noise or brightness unevenness is large. was there.
[0006]
The present invention has been made in view of the above circumstances, and a specific pattern recognition method and a specific pattern recognition program capable of robust recognition that are not easily affected by environmental fluctuations such as illumination fluctuations, obstacles, and positioning errors. Another object is to provide a specific pattern recognition program recording medium and a specific pattern recognition device.
[0007]
[Means for Solving the Problems]
The invention according to claim 1 for solving the above-mentioned problem is capable of specifying the position and orientation of a recognition target image having a specific pattern included in a digital image, and is a reference corresponding to the recognition target image. Use a recognition model consisting of at least two local patterns obtained from a digital image including a recognition reference image to be given to each of which one direction is given and at least one connection pattern connecting the local patterns. A specific pattern recognition method in which the image processing apparatus recognizes a specific pattern of a recognition target image, and local pattern candidates that match each of at least two different local patterns among the local patterns constituting the recognition model At least one local pattern candidate point that is the position of the digital image including the recognition target image A local pattern candidate point extraction process for assigning the direction assigned to the corresponding local pattern to the at least one local pattern candidate point, and at least 2 from the local pattern candidate points extracted in this local pattern candidate point extraction process By extracting at least one set of local pattern candidate points that match two different local patterns, a deformation candidate consisting of each local pattern interpolation point of the extracted set is selected, and each local area of the selected deformation candidate is selected. Based on the positional relationship of pattern compensation points, deformation, position, scale, and angle are added to the recognition model, and each deformation parameter when superposing the entire pattern of the recognition model on the recognition target image is set. Based on the desired model deformation process and the deformation parameters obtained in the model deformation process, the recognition model is deformed, The recognition model deformed superimposed on the recognition target image, characterized in that it comprises a similarity evaluation process for evaluating the similarity between the recognition target image and the modified candidate.
[0008]
The invention according to claim 2 further includes a model generation process for generating the recognition model in the specific pattern recognition method according to claim 1, wherein in the model generation process, an outline of the recognition reference image is obtained, The local pattern is extracted from a partial image in a region where the curvature value in the contour line is larger than a predetermined value.
[0009]
According to a third aspect of the present invention, in the specific pattern recognition method according to the first aspect, the method further includes a model generation process for generating the recognition model, and the model generation process includes a partial image of the digital image including the recognition reference image. For each pixel, a brightness change direction is obtained for each pixel, and a pattern of an image in which a peak of brightness change appears in at least two different directions is extracted as the local pattern.
[0010]
According to a fourth aspect of the present invention, in the specific pattern recognition method according to the first aspect, the connection pattern is an image pattern.
[0011]
According to a fifth aspect of the present invention, in the specific pattern recognition method according to the first aspect, the connection pattern is any one of a straight line, a broken line having at least one intermediate node, and a curve, and the connection pattern data is mathematical data. It is characterized by being.
[0012]
According to a sixth aspect of the present invention, in the specific pattern recognition method according to the first aspect of the present invention, the method further includes a model generation process for generating the recognition model, and in the model generation process, from the digital image including the recognition reference image, A polar coordinate having the center position of the local pattern as an origin is converted into an orthogonal coordinate having an angle axis indicating an angle centered on the origin and a distance axis indicating a distance from the origin to create an orthogonal transform image, and In the pattern candidate point extraction process, from the digital image including the recognition target image, polar coordinates having a certain point of interest as an origin are converted into orthogonal coordinates to generate an orthogonal conversion image. In the orthogonal conversion image and the model generation process, While translating the created orthogonal transformation image in the direction of the angle axis, the similarity is obtained sequentially, the maximum similarity is extracted from each of these similarities, and this maximum similarity is predetermined. If it is larger than the reference, the position of the orthogonal transformation image when the maximum similarity is obtained is set as the position of the local pattern candidate point, and the corresponding local pattern is determined based on the movement amount that gives the maximum similarity. The given direction is assigned to the local pattern candidate point.
[0013]
The invention according to claim 7 further includes a model generation process for generating the recognition model in the specific pattern recognition method according to claim 1, wherein in the model generation process, from the digital image including the recognition reference image, Polar coordinates with the center position of the local pattern as the origin are converted into Cartesian coordinates with an angle axis that indicates the angle centered on the origin and a distance axis that indicates the distance from the origin, and an orthogonal transformation image is created to create this orthogonal One-dimensional waveform data is created by projecting the converted image onto the angle axis, and in the local pattern interpolation point extraction process, polar coordinates having a certain point of interest as the origin are used as orthogonal coordinates from the digital image including the recognition target image. Transform to create an orthogonal transform image, project this orthogonal transform image onto the angle axis to create one-dimensional waveform data, and create this one-dimensional waveform data and the model generation process The parallel waveform data is translated in the direction of the angle axis, and the similarities are sequentially obtained, and the maximum similarity is extracted from each of these similarities. If this maximum similarity is greater than a predetermined reference, the maximum similarity is obtained. The position of the orthogonal transformation image when the position is obtained is set as the position of the local pattern candidate point, and the direction given to the corresponding local pattern is determined based on the amount of movement that gives the maximum similarity. It is characterized by assigning to a point.
[0014]
The invention according to claim 8 is the specific pattern recognition method according to claim 6, wherein the orthogonal transformation image extracted from the digital image including the recognition target image is enlarged or reduced in the distance axis direction and created in the model generation process. It matches with the orthogonal transformation image which was made.
[0015]
According to a ninth aspect of the present invention, in the specific pattern recognition method according to the first aspect, in the local pattern interpolation point extraction process, only a portion having a large change in pixel value is extracted in advance, and only that portion is extracted. On the other hand, a local pattern candidate point extraction process is performed.
[0016]
According to a tenth aspect of the present invention, in the specific pattern recognition method according to the first aspect, in the model deformation process, an angle difference between the direction of the local pattern candidate point of the deformation candidate and the direction of the corresponding local pattern is predetermined. If the angle is larger than the corner, the deformation candidate is excluded from the similarity evaluation in the similarity evaluation process.
[0017]
The invention according to claim 11 is the specific pattern recognition method according to claim 4, wherein, in the similarity evaluation process, when the data of the connection pattern is image data, the deformation of the deformation candidate obtained in the model deformation process. Based on the parameters, the connection pattern is deformed, and the entire pattern of the deformed recognition model including the deformed connection pattern is matched with the recognition target image.
[0018]
According to a twelfth aspect of the present invention, in the specific pattern recognition method according to the fifth aspect, in the similarity evaluation process, when the data of the connection pattern is expressed using a mathematical expression representing a linear pattern, the connection pattern A plurality of sampling points are set on the connection pattern candidates obtained from the above, and sampling lines of arbitrary length are set in the normal direction of the connection pattern candidates around these sampling points, and all of the set sampling lines are set. A correlation value between a distribution of pixel values and an ideal pattern in which pixel values change stepwise at the position of a connected pattern candidate is calculated and used as a similarity.
[0019]
According to a thirteenth aspect of the present invention, in the specific pattern recognition method according to the fifth aspect, in the similarity evaluation process, when the data of the connection pattern is expressed using a mathematical expression representing a linear pattern, the connection pattern Set multiple sampling points on the connected pattern candidate obtained from, set an arbitrary length sampling line in the normal direction of the connected pattern candidate around these sampling points, and set the pixel value on the set sampling line The process of examining the similarity between each distribution and the ideal pattern in which the pixel value changes stepwise at the sampling point is performed for each sampling line, and the average is calculated by calculating the similarity for all sampling lines. It is characterized by the similarity.
[0020]
According to a fourteenth aspect of the present invention, in the specific pattern recognition method according to any one of the eleventh to thirteenth aspects, when it is known that contrast inversion of light and darkness occurs in the recognition target image, the normalized correlation value is used as the similarity. The absolute value of is used.
[0021]
According to a fifteenth aspect of the present invention, in the specific pattern recognition method according to the fifth aspect, in the similarity evaluation process, when the data of the connection pattern is expressed using a mathematical expression representing a linear pattern, the connection pattern A plurality of sampling points are set on the connection pattern candidate obtained from the above, and a sampling line of an arbitrary length is set in the normal direction of the connection pattern candidate around each sampling point, and each of all the set sampling lines is set. The edge position is obtained for, and the average value of the distance between the edge position and the connection pattern is used as the similarity.
[0022]
According to a sixteenth aspect of the present invention, in the specific pattern recognition method according to the fifteenth aspect, an average value of pixel values is obtained on the left and right of an attention position with the sampling line, an absolute value of a difference between the average pixel values is obtained, and an attention position The absolute value of the difference between the average pixel values is sequentially obtained while moving within the sampling line range, and the position where the absolute value is maximum is used as the edge position.
[0023]
According to a seventeenth aspect of the present invention, in the specific pattern recognition method according to the fifteenth aspect, each of the connection patterns is applied to a mathematical expression of a connection pattern using a least square method based on an edge position.
[0024]
The invention according to claim 18 is the specific pattern recognition method according to claim 1, wherein the relationship between the characteristics of the local pattern and the connected pattern and at least one of the optimum processing method and recognition parameters for recognition of these patterns is described. As a model generation process for generating the recognition model, the feature of the pattern to be detected is analyzed, and the local pattern candidate point extraction is performed by referring to the database based on the analysis result. The method further includes a step of determining at least one of a processing method and a recognition parameter in each of the process, the model deformation process, and the similarity evaluation process.
[0025]
According to a nineteenth aspect of the present invention, in the specific pattern recognition method according to the first aspect, at least one of a feature of inter-image variation of the local pattern and the connection pattern, an optimum processing method for the recognition of the patterns, and a recognition parameter, A database describing the relationship between the local pattern candidate points, a plurality of teaching images are experimentally recognized, and the local pattern candidate point extraction process and model deformation process are performed by referring to the database based on the recognition results. In addition, at least one of a processing method and a recognition parameter in each process of the similarity evaluation process is determined.
[0026]
The invention according to claim 20 is the specific pattern recognition method according to claim 1, further comprising a model generation process for generating the recognition model, wherein in the model generation process, the recognition standard obtained by photographing the same pattern. When a plurality of digital images including images are used as teaching images, the inter-image variations of the local pattern data and the connection pattern data are normalized and integrated.
[0027]
According to a twenty-first aspect of the present invention, in the specific pattern recognition method according to the first aspect, when pattern matching and position detection are performed on a digital image continuously input in time series, and the pattern is traced, The recognition model is changed on the basis of the result of pattern matching performed in (1).
[0028]
According to a twenty-second aspect of the present invention, in the specific pattern recognition method according to the first aspect, when pattern matching and position detection are performed on a digital image continuously input in time series, and the pattern is traced, On the basis of the result of the position detection performed in the above, the processing method in each process of the local pattern candidate point extraction process, model deformation process, and similarity evaluation process is changed as needed.
[0029]
The invention according to claim 23 is capable of specifying the position and orientation of a recognition target image having a specific pattern included in a digital image, and includes a recognition reference image serving as a reference corresponding to the recognition target image. An image processing apparatus uses a recognition model composed of at least two local patterns obtained from a digital image, each of which is assigned one direction, and at least one connection pattern connecting the local patterns. A specific pattern recognition program including a processing procedure of a specific pattern recognition method for recognizing a specific pattern of a recognition target image, wherein the local pattern matches each of at least two different local patterns among the local patterns constituting the recognition model A local pattern candidate point that is a position of the pattern candidate is displayed as a digital image including the recognition target image. A local pattern candidate point extraction process in which at least one is extracted from and a direction given to the corresponding local pattern is assigned to the at least one local pattern candidate point, and the local pattern candidate extracted in the local pattern candidate point extraction process By extracting at least one set of local pattern candidate points that match at least two different local patterns from the points, a deformation candidate consisting of each local pattern compensation point of the extracted set is selected and selected When the position, scale, and angle of the recognition model are modified based on the positional relationship between the local pattern compensation points of the deformation candidates, and the entire pattern of the recognition model is superimposed on the recognition target image. Based on the model deformation process for obtaining each deformation parameter of the image and the deformation parameter obtained in the model deformation process, the recognition Deforming the model, the recognition model this deformation superposed on the recognition target image, characterized in that it comprises a similarity evaluation process for evaluating the similarity between the recognition target image and the modified candidate.
[0030]
According to a twenty-fourth aspect of the invention, a specific pattern recognition program recording medium records the specific pattern recognition program according to the twenty-third aspect so as to be readable by the computer.
[0031]
According to a twenty-fifth aspect of the present invention, there is provided an imaging unit that performs imaging, a quantization unit that obtains a digital image in which brightness values are recorded for each lattice point of a two-dimensional matrix from an image signal captured by the imaging unit, Storage means for storing at least digital image data obtained by the quantization means, the position and orientation of a recognition target image having a specific pattern included in the digital image can be specified, and the recognition It is composed of at least two local patterns obtained from a digital image including a recognition reference image serving as a reference corresponding to the target image, each of which is assigned one direction, and at least one connection pattern connecting these local patterns. A specific pattern recognition apparatus for recognizing a specific pattern of a recognition target image using a recognition model. At least one local pattern candidate point that is a position of a local pattern candidate that matches each of at least two different local patterns among local patterns constituting the model is extracted from the digital image including the recognition target image, and at least one of the local pattern candidate points is extracted. A local pattern candidate point extraction unit that assigns a direction assigned to a corresponding local pattern to one local pattern candidate point, and at least two different local regions from the local pattern candidate point extracted by the local pattern candidate point extraction unit By extracting at least one set of local pattern candidate points that match the pattern, a deformation candidate composed of each local pattern interpolation point of the extracted set is selected, and each local pattern interpolation point of the selected deformation candidate is selected Based on the positional relationship, position, scale, and angle deformations are added to the recognition model. Model deformation means for obtaining respective deformation parameters when superposing the entire pattern of the recognition model on the recognition target image, and deforming the recognition model based on the deformation parameters obtained by the model deformation means, The recognition model is superimposed on the recognition target image, and similarity evaluation means for evaluating the similarity between the deformation candidate and the recognition target image is included.
[0032]
Here, according to the method of the present invention, the entire pattern is divided into a local pattern of characteristic parts and a connected pattern of non-characteristic parts, and a part similar to the local pattern is detected from the digital image including the recognition target image. By estimating the deformation of the entire pattern (position, rotation, scale fluctuation) based on the information of the part and verifying whether the deformation of the entire pattern estimated using the information of the entire pattern is correct, As a result, it is difficult to separate the pattern from the background using a simple method such as binarization due to noise and lightness unevenness, as well as pattern rotation and scale fluctuations. Even when a part of the image is concealed and the shape appears to change, correct recognition is possible.
[0033]
DETAILED DESCRIPTION OF THE INVENTION
1 is a configuration diagram of a specific pattern recognition apparatus, FIG. 2 is a flowchart showing a flow of a specific pattern recognition method executed by the processing apparatus in FIG. 1, and FIG. 3 is obtained from the camera and quantizer in FIG. It is a figure which shows the example of the various digital images which can be done, and one Embodiment of this invention is described referring these figures.
[0034]
The specific pattern recognition apparatus according to the present embodiment is applied to, for example, positioning of the object 10. As illustrated in FIG. 1, the camera 11, the quantizer 12, the storage device 13, and the image display apparatus are used. 14 and a processing device 15.
[0035]
The camera 11 is a TV camera that takes an image of the object 10 and obtains an image signal. The quantizer 12 records a brightness value for each lattice point of the two-dimensional matrix from the image signal obtained by the camera 11, and includes the recognition target image I10 of the target object 10 in a frame of a predetermined size. Is what you get. The storage device 13 is composed of a semiconductor storage device or the like, and stores the image data of the digital image I12 obtained by the quantizer 12. The image display device 14 is composed of a CRT or LCD, and displays various types of information. The processing device 15 is a central part of a computer composed of a CPU, a RAM, and the like. For example, the processing device 15 is obtained by measuring the position and orientation necessary for positioning the object 10 through pattern recognition processing and measuring the position and orientation. The result is output and displayed on the screen of the image display device 14, or data is output to an external device (not shown).
[0036]
In the apparatus having such a configuration, the recognition target image in the digital image is not always obtained from the quantizer 12 in a constant state. As shown in the example of FIG. Therefore, various fluctuations are added to the recognition target image. That is, as shown in FIG. 3A, the recognition target image may be obtained in an appropriate state, or as shown in FIGS. 3B to 3I, fluctuations that are inconvenient for the recognition processing are recognized. It may be added to the target image. In FIG. 3B, the recognition target image is rotated by an angle that is counterclockwise with respect to the proper direction due to the positioning error of the target object 10. In FIG. 3C and FIG. 3D, the recognition target image becomes larger than the appropriate image size due to the variation in the distance between the object 10 and the camera 11, and the recognition target image is in focus. Is off. In FIG.3 (e), the nonuniformity has arisen in the brightness by the illumination intensity fluctuation | variation of the illuminating device. In FIG. 3 (f) and FIG. 3 (g), contrast is lowered and noise is generated in the image due to a translucent sheet or the like that protects the object 10, respectively. In FIG. 3H, a part of the recognition target image is hidden by an opaque object on the target object 10. In FIG. 3 (i), a part of the local pattern of the recognition target image is visible, but another part is hidden. Moreover, these fluctuations may occur in combination of two or more.
[0037]
In the present embodiment, even if the various variations described above are added to the recognition target image, a plurality of partial images that can specify the position and orientation of the recognition target image can be normally captured when viewed microscopically. Because there are many, the overall pattern of the recognition target image is estimated through the pattern of these partial images (hereinafter, this specific pattern is referred to as “local pattern”), and the position, orientation and scale change of the entire pattern on the frame, etc. Thus, the processing device 15 is configured so that accurate pattern recognition processing can be performed.
[0038]
That is, the processing device 15 includes various processing functions such as a model generation unit 151, a local pattern candidate point extraction unit 152, a model deformation unit 153, and a similarity evaluation unit 154, as shown in FIG. The position and orientation of the object 10 necessary for positioning the object 10 and the like are measured through a local pattern recognition process for specifying the position and orientation of the object 10.
[0039]
As shown in the example of FIG. 3A, the model generation unit 151 prepares a digital image prepared in advance including a recognition target image (hereinafter referred to as “recognition reference image”) obtained in an appropriate state, as shown in the example of FIG. On the other hand, by analyzing the entire pattern structure (shape) of the recognition reference image included therein, the entire pattern of the recognition reference image is classified into a plurality of local patterns and a connection pattern that connects these local patterns. The local pattern and the connection pattern are generated as a recognition model to be used for the recognition process of the recognition target image to be recognized, and the data of the local pattern and the connection pattern is stored and registered in a semiconductor storage device or a hard disk device. (# 1 in FIG. 2).
[0040]
In this embodiment, the recognition model is automatically generated and registered from the recognition reference image. However, the present invention is not limited to this, and the person recognizes by analyzing the entire pattern structure of the recognition reference image in advance. A model for use may be generated and registered.
[0041]
The local pattern candidate point extraction unit 152 needs at least a local pattern candidate point, which is a coordinate position of a local pattern candidate that matches a local pattern of a recognition model, from a digital image including a recognition target image to estimate deformation in the model deformation process. As many local numbers as possible are extracted, and the direction of the local pattern assigned to each local pattern of the recognition model for each local pattern candidate point is also obtained (# 2 in FIG. 2).
[0042]
The model deforming unit 153 uses the local pattern candidate points extracted by the local pattern candidate point extracting unit 152 to obtain local pattern candidate points that match at least two different local patterns among the local patterns constituting the recognition model. By selecting a pair, the position, direction, and scale change of the entire pattern of the recognition target image in the frame are estimated (# 3 in FIG. 2).
[0043]
The similarity evaluation unit 154 verifies whether the estimation by the model deformation unit 153 is correct by evaluating the similarity between the overall pattern of the recognition target image estimated by the model deformation unit 153 and the recognition target image, The collation between the recognition target image and the recognition model is determined (# 4 in FIG. 2).
[0044]
Hereinafter, each of the model generation process, the local pattern compensation point extraction process, the model deformation process, and the similarity evaluation process will be described in order while further adding drawings.
[0045]
(1) Model generation process
1. Create and register a recognition model
FIG. 4 is a diagram showing an example of a recognition model generated by the model generation process. In FIG. 4, p1 to p4 are local patterns. Among these, the local patterns p1 and p3 indicate patterns at both ends of the recognition reference image, and the local patterns p2 and p4 indicate corner pattern of the recognition reference image. ing. These have asymmetric shapes that can be used to identify the position and orientation of the recognition target image, and it is necessary to estimate the pattern deformation in the model deformation process described later. And at least one connection pattern is determined and registered in advance. Moreover, although the information of the direction shown by the arrow is provided to local pattern p1-p4, each of these directions has shown the direction with a recognition reference image. Further, the image data of the recognition reference image is also stored in a semiconductor storage device, a hard disk device, or the like for processing of a local pattern interpolation point extraction process described later.
[0046]
The local pattern and the connection pattern are automatically generated by any one of the following “2. First model automatic generation method” to “3. Second model automatic generation method”.
[0047]
2. First model automatic generation method
FIG. 5 is a diagram showing contour lines extracted from a digital image including a recognition reference image, and FIG. 6 is a graph showing changes in curvature (reciprocal of radius) of the contour lines in FIG.
[0048]
In the first model automatic generation method, the curvature of the contour line is obtained from the contour line of FIG. 5 as shown in FIG. 6, and this curvature of the contour line is used for automatic generation of the local pattern. As shown in FIG. 6, if the curvature value of the contour line is divided by a certain threshold value, the contour line indicating the entire pattern of the recognition reference image is classified into the local patterns p1 to p4 and the connection patterns l1 to l4. be able to. As described above, the contour line is obtained from the digital image including the recognition reference image, the curvature of the contour line is obtained for each position of the contour line, and each curvature value is divided into two by a predetermined threshold value. Specifically, the entire pattern of the recognition reference image can be classified into local patterns p1 to p4 and connected patterns 11 to 14 and these can be generated as a recognition model.
[0049]
3. Second model automatic generation method
FIG. 7 is a diagram showing a recognition reference image and a plurality of partial regions in a digital image including the recognition reference image, and FIG. 8 is a graph showing the brightness change direction in each partial region in FIG. However, FIGS. 8A to 8D show the brightness change directions of the partial regions R1 to R4 in FIG. 7, respectively. Further, as shown in FIG. 7, these graphs showing the respective brightness change directions are obtained when each direction of interest is defined when the direction of the brightness change is defined as 0 ° direction so that the brightness decreases upward on the screen. The brightness change direction is obtained for all the pixels existing in the image, and the histogram is obtained.
[0050]
The second model automatic generation method is effective when, for example, it is difficult to obtain a contour line, and a distribution in the brightness change direction is used as shown in FIG. In the partial areas R1 and R2 in the digital image including the recognition reference image, as shown in FIGS. 8A and 8B, two or more significant peaks appear in the distribution in the brightness change direction, whereas the partial area R3 , R4, as shown in FIGS. 7C and 7D, only one remarkable peak appears at most.
[0051]
As described above, when the distribution of the brightness change direction in each partial area in the digital image including the recognition reference image is examined, and a peak of significant brightness change appears in two or more different directions in one partial area, The pattern of the partial image can be extracted as a local pattern constituting the recognition model. And the connection pattern which connects between them can be created from the extracted local pattern.
[0052]
4). Variations of connected patterns
FIG. 9 is a diagram showing another shape example of the connection pattern. In the example of FIG. 4, the connection pattern is a straight line. However, as shown in the example of FIG. 9, the shape of the connection pattern may be changed according to the shape of the object 10. FIG. 9A shows an example of a polyline-shaped connection pattern having at least one intermediate node. FIG. 9B shows an example of a curved connection pattern. These connection patterns can be registered as mathematical data, but when the edges are unclear or when complex edge patterns such as the leads of QFP chip parts shown in FIG. In this case, the image data of the partial image cut out from the digital image including the recognition reference image is directly registered as the connection pattern data.
[0053]
(2) Local pattern interpolation point extraction process
1. Insensitive to rotation
From the local pattern interpolation point extraction process, a storage device obtained from the camera 11 and the quantizer 12 using a digital image including the recognition model and the recognition reference image instead of the recognition reference image at the time of inspection or the like. The processing is performed on the digital image I12 including the recognition target image stored in FIG.
[0054]
In this process, a local pattern candidate point that is a coordinate position of a local pattern candidate that is highly likely to coincide with a local pattern constituting the recognition model is extracted from the digital image including the recognition target image, and this local pattern candidate is extracted. A process of assigning information of the direction given to the corresponding local pattern to the point is executed.
[0055]
In this case, as shown in FIGS. 3B to 3H, fluctuations such as rotation or scale change may be added to the recognition target image in the digital image. Processing such as extraction of pattern candidate points is performed. Here, a method of performing matching after performing polar coordinate → rectangular coordinate conversion of the image is used. Hereinafter, the polar coordinate → rectangular coordinate conversion and matching will be described.
[0056]
2. Polar coordinate → Cartesian coordinate conversion and matching
FIG. 10 is an explanatory diagram of polar coordinate → rectangular coordinate conversion processing, and FIG. 11 is a diagram illustrating an example of an orthogonal conversion image that can be obtained by polar coordinate → rectangular coordinate conversion processing for a digital image including a recognition target image.
[0057]
In the polar coordinate → rectangular coordinate conversion processing, in order to obtain the reference coordinate position, the digital image including the recognition reference image is centered on a certain point of interest and the screen is turned clockwise at 0 ° as shown in FIG. The pixel value on the line segment of an arbitrary radius is cut out while changing the angle for one round, and as shown in FIG. 10B, the horizontal axis is the angle axis, and the vertical axis is the distance axis from the point of interest. An orthogonal transformation image is generated by transcribing to an orthogonal coordinate system and converting polar coordinates into orthogonal coordinates. In the figure, since the point of interest is a corner point in a partial image of a white right-angled corner on a black background, the orthogonal transformation image has a bright striped pattern with 1/4 of the angle axis.
[0058]
Similarly, as shown in FIG. 11, a process of generating an orthogonal transform image is executed on a digital image including a recognition target image. In the example of FIG. 11A, the attention point is a pixel of a corner point in a partial image of a white right-angled corner on a black background, so that the orthogonal transformation image has an angular axis as shown in FIG. 1/4 is a bright striped pattern. On the other hand, in the example of FIG. 11C, the attention point is a pixel at a position deviating from the corner point, so that the orthogonal transformation image is 1/4 of the angle axis as shown in FIG. Does not result in a bright striped pattern.
[0059]
In either case, the orthogonal transformation image when the point of interest is a corner point pixel is a striped pattern. Therefore, if both bright striped pattern areas at this time are compared, the recognition target image It is possible to determine whether there is a local pattern candidate that matches the local pattern located at the target point of the recognition reference image at the target point, and how much rotation the local pattern candidate has relative to the local pattern. It is possible to detect whether a change is applied. In this embodiment, the coordinate position of a local pattern candidate including information indicating how much rotation variation is added to the local pattern candidate is obtained as a local pattern candidate point, and the local pattern corresponding to the local pattern candidate point is obtained. The process of assigning the information of the direction given to the is executed.
[0060]
More specifically, the orthogonal transformation image pattern (FIG. 10B) based on the partial image of the digital image including the recognition reference image (the image in the circle having the above-mentioned radius centered on the target point) and the recognition target image are obtained. While the relative position of the orthogonal transform image pattern (FIG. 11 (b)) by the partial image of the digital image is moved in the direction of the angle axis (the angle axis is considered to be continuous at 0 ° and 360 ° positions), the normal The correlation values are calculated sequentially.
[0061]
Subsequently, the maximum normalized correlation value is extracted from the obtained normalized correlation values, and it is determined whether or not the maximum normalized correlation value is larger than a predetermined reference value of coincidence.
[0062]
When the maximum normalized correlation value is larger than the reference value, the point of interest in the recognition target image (FIG. 11A) matches the local pattern located at the point of interest of the recognition reference image (FIG. 10A). It is determined that there is a local pattern candidate to be performed. And a local pattern candidate point is calculated | required from the position used as the largest normalized correlation value. Also, based on the amount of movement that gives the maximum correlation value, information in which the direction assigned to the corresponding local pattern is assigned to the local pattern candidate point is created and stored in the storage device 13.
[0063]
On the other hand, when the maximum normalized correlation value is not larger than the reference value, the local position located at the attention point (FIG. 10A) of the recognition reference image is set at the attention point of the recognition target image (FIG. 11C). It is determined that there is no local pattern candidate that matches the pattern. That is, the reference value is set to a value larger than the maximum normalized correlation value in this case.
[0064]
As described above, both the digital image including the recognition reference image and the digital image including the recognition target image are subjected to the polar coordinate → orthogonal coordinate conversion process to generate both orthogonally converted images, and these orthogonally converted images are used. If the local pattern candidate points are extracted from the digital image including the recognition target image, the local pattern candidate points can be extracted without being influenced by the rotation.
[0065]
3. Other matching methods
FIG. 12 is an explanatory diagram of a matching method when the orthogonal transform image becomes a simple striped pattern, and FIG. 13 is an explanatory diagram of a matching method when the orthogonal transform image becomes a complex pattern.
[0066]
FIG. 12 is an orthogonal transform image similar to FIG. 11, but when the orthogonal transform image becomes a simple striped pattern in this way, an orthogonal transform image and a recognition reference image by a partial image of the digital image including the recognition target image. Are projected onto the angle axis of the orthogonal coordinate system as shown in FIGS. 12B and 12D, respectively (the pixel value in the distance axis direction is calculated for each unit of the angle axis). 1-dimensional normalized correlation matching may be performed after the summation process is performed). That is, as shown in FIG. 12, one-dimensional waveform data is created by the above projection, and the similarity is sequentially obtained while the waveform data based on the recognition reference image and the waveform data based on the recognition target image are translated in the angle axis direction. The maximum similarity is extracted from each of these similarities, and if the maximum similarity is larger than a predetermined reference, the position of the orthogonal transformation image when the maximum similarity is obtained is set as the position of the local pattern candidate point. At the same time, the direction given to the corresponding local pattern may be assigned to the local pattern candidate point based on the movement amount that gives the maximum similarity.
[0067]
By using this method, the processing load can be greatly reduced. This method is particularly effective when the local pattern is simple because the projected one-dimensional data reflects the original local pattern shape well, but even if the local pattern is complicated, the rotation angle can be adjusted. It can be used as preprocessing for rough estimation.
[0068]
In the example of FIG. 13, the pattern of the partial image has a shape that is affected by the scale fluctuation, and the pattern of the orthogonal transformation image is more complicated by the influence of the scale fluctuation. Thus, when the orthogonal transform image is compressed or expanded (expanded in FIGS. 13C and 13D) up and down (distance axis direction) under the influence of the scale variation, it is necessary to correct the scale variation. . Therefore, in this case, if the orthogonal transformation image compressed or expanded in the distance axis direction is enlarged or reduced in the positive direction of the distance axis and then matched, it is possible to correctly extract the local pattern candidates. .
[0069]
4). Faster conversion from polar coordinates to Cartesian coordinates and matching
If each pixel of the digital image is the pixel of interest and the above polar coordinate → rectangular coordinate conversion and matching processing are performed, the processing amount becomes enormous and the processing time becomes long. Therefore, if an edge extraction process or the like is performed in advance to extract only a portion having a large change in pixel value, and the above-described processing is performed only for the pixel in this portion, the processing load can be greatly reduced.
[0070]
(3) Model deformation process
1. Candidate for deformation
FIG. 14 is a diagram showing an example of a recognition target image, FIG. 15 is a diagram showing local pattern interpolation points extracted from the recognition target image of FIG. 14 by a local pattern interpolation point extraction process, and FIG. 16 is an explanatory diagram of a model deformation process. It is.
[0071]
In FIG. 15, cp11 to cp14 are local pattern candidate points that code to the local patterns p1 and p3 constituting the recognition model of FIG. cp21 and cp22 are local pattern candidate points that code for the local pattern p2 in FIG. cp31 to cp36 are local pattern candidate points that code p4 in FIG.
[0072]
In the model deformation process, deformation estimation is performed based on these local pattern interpolation points. The deformation estimation is performed by selecting, from the local pattern and local pattern compensation points, a set of local pattern compensation points that are similar to two or more different local patterns, thereby rotating, moving, and scaling the entire pattern. It means to determine the parameter of variation.
[0073]
Specifically, as shown in the example of FIG. 16, the recognition model shown in FIG. 4 and the local pattern interpolation point shown in FIG. Deformation candidates df1 to df4 are created or selected. However, in addition to the following modification candidates, a large number of modification candidates are conceivable, but some of them are shown here.
[0074]

In this way, in the model transformation process, by extracting at least one set of local pattern candidate points that match at least two different local patterns from the local pattern candidate points extracted in the local pattern candidate point extraction process, A deformation candidate consisting of the extracted local pattern interpolation points of the set is selected.
[0075]
In addition, based on the positional relationship between each local pattern interpolation point of the selected deformation candidate, the position, scale, and angle are deformed to the recognition model, and the entire pattern of the recognition model is converted into the recognition target image. Each deformation parameter at the time of superposition is obtained.
[0076]
For example, as shown in FIGS. 3B to 3G, even if there is a change in the pattern shape and the brightness of the pattern, the local pattern candidate point is the pattern scale in the above-described local pattern candidate point extraction process. Since each local pattern candidate point is obtained correctly, it is possible to extract a deformation candidate. 3E and 3G, the local brightness unevenness and the degree of noise superposition are so severe that the local pattern is not visible, or some local patterns are obstructions as shown in FIG. If the pattern is hidden in the pattern, local pattern candidate points that match some local patterns among a plurality of (in the case of the pattern shown in the embodiment, four types shown in p1 to p4 in FIG. 4) local patterns are not found. Even if there are local pattern candidate points that match at least two types of local patterns, a deformation candidate can be extracted. Further, as shown in FIG. 3 (h), when the portion corresponding to the connected pattern is hidden, the local pattern appears without any problem, so there is no influence in the model deformation process.
[0077]
2. High speed processing
Next, the modification candidates are narrowed down for speeding up. In the local pattern interpolation point extraction process in the previous stage, the direction is extracted together with the coordinate position where the pattern is similar for each type of local pattern, so that direction is used.
[0078]
That is, the angle of the direction of the local pattern compensation point with respect to the overall pattern of the deformation candidate is compared with the angle of the direction of the local pattern with respect to the overall pattern of the recognition model, and if the angle difference between the two is larger than a predetermined angle, A deformation candidate is excluded as a possible deformation.
[0079]
For example, among the deformation candidates df1 to df4 in FIG. 16, the angle of the direction of the local pattern complement point cp12 with respect to the entire pattern of the deformation candidate df4 and the angle of the direction of the local pattern p1 with respect to the entire pattern of the recognition model in FIG. Are greatly different from each other, and the deformation candidate df4 is excluded. In addition, the predetermined angle is set to a value smaller than the angle difference between them. Thereby, when the deformation candidates df1 to df4 in FIG. 16 are created or selected, the deformation candidates are narrowed down to the deformation candidates df1 to df3.
[0080]
(4) Similarity evaluation process
1. Collation
In the similarity evaluation process, a process of collating the recognition target image with a deformation model corresponding to the recognition target image and selecting an optimal deformation model from each deformation model is executed.
[0081]
Specifically, based on the deformation parameters of the deformation candidate obtained in the model deformation process, the recognition model is deformed, the deformed recognition model is superimposed on the recognition target image, and the deformation candidate and the recognition target image are combined. A process for evaluating the similarity is executed.
[0082]
At this time, how the connected pattern is expressed in the recognition model (whether it is an image or a mathematical expression), the characteristics of the image, the purpose of recognition (whether uneven brightness or pattern hiding occurs frequently, or a single pattern) Appropriate similarity evaluation methods are selected and used from various similarity evaluation methods depending on whether the purpose is recognition of a plurality of similar patterns, or identification of a plurality of similar patterns. Hereinafter, each similarity evaluation method will be described.
[0083]
2. Similarity evaluation method when connected pattern is image data
FIG. 17 is an explanatory diagram of the similarity evaluation method. In a digital image including a recognition reference image, when a partial image between local patterns is an unclear pattern or a complex pattern, the partial images between local patterns are used as they are as a connection pattern. At this time, as shown in the example of FIG. 17, the connection pattern can be deformed by deforming the partial images constituting the connection pattern according to the deformation parameters (scale, rotation, etc.) of the deformation candidates. Then, if the entire pattern of the deformed recognition model including the deformed connection pattern candidate is matched with the recognition target image by the normalized correlation matching, a similarity evaluation value can be obtained.
[0084]
3. Similarity evaluation method when connected pattern is formula data
When the connected pattern constituting the recognition model is a mathematical expression of the edge shape, several methods can be selected as the matching method between the deformation model and the recognition target image according to the characteristics of the image. it can.
[0085]
(First verification method)
FIG. 18 is an explanatory diagram of a first matching method when the connection pattern is mathematical formula data. In the first matching method, a matching method based on a normalized correlation method is used. For the deformation candidates df1 to df3 shown in FIG. 16, based on the following formulas for pixels included in the area Pi within the outline of the recognition model and the area Oi in the vicinity of the recognition model outline as shown in FIG. The correlation coefficient S1 is calculated, and the one with the highest correlation coefficient is selected as the optimum collation.
[0086]
[Expression 1]

[0087]
According to the first collation method, since collation can be performed using a large number of pixel values, it is weak against lightness unevenness, but it is necessary to select the most similar one from many similar patterns or contrast. Highly reliable verification is possible, for example, when the value is low.
[0088]
(Second verification method)
FIG. 19 is an explanatory diagram of a second matching method when the connection pattern is mathematical formula data. FIG. 19A shows a collation method for the deformation model by df1 in FIG. 16, and FIG. 19C shows a collation method for the deformation model by df2 in FIG.
[0089]
In the second matching method, a method of evaluating the similarity using a change in brightness near the edge of the pattern is used.
[0090]
Like l11 to l1m and l21 to l2n (m and n are the number of sampling points) shown in FIG. 19, the position of the local pattern p1 in FIG. 4 is set as the starting point, and the connection pattern is clockwise along the connection pattern candidates of the deformation model. A plurality of sampling points are arranged on the candidates. Subsequently, for each sampling point, a line segment having a length l (hereinafter referred to as “sampling line”) extending in the normal direction of the connection pattern candidate around the sampling point is set. At this time, when the lightness is extracted along the sampling line, the results shown in FIGS. 19B and 19D are obtained.
[0091]
Regarding the brightness change of each sampling line shown in these figures, the phase relationship with the edge pattern that changes in an ideal step shape (see FIG. 20) at the midpoint of the sampling line (position on the connection pattern candidate of the deformation model) If the number S2 is obtained by the following mathematical formula, collation using only pixels near the edge can be performed.
[0092]
[Expression 2]

[0093]
For the deformation of df1 in FIG. 16 that is the optimal collation, as shown in FIG. 19B, the shape of the edge and the ideal stepped edge pattern are very similar in each sampling line. The relationship number S2 is increased. On the other hand, for the deformation of df2 in FIG. 16 in which the edge position is shifted, as shown in FIG. 19D, the shape of the edge and the ideal stepped edge pattern is different, so the correlation coefficient S2 is lowered. By utilizing this fact, it is possible to obtain an optimal collation from a plurality of deformation models. Thus, by performing collation using only the brightness change near the edge, it is possible to perform highly reliable collation when there is a pattern that is likely to cause brightness fluctuations within the pattern.
[0094]
(Third verification method)
In the third collation method, a method is used in which the sampling line is set by the same procedure as the second collation method and then the similarity is calculated by the following equation.
[0095]
[Equation 3]

[0096]
The difference between the second verification method and the third verification method is that the former method sums all the brightness changes of each sampling line and calculates the correlation coefficient only once, whereas the latter method. Then, the correlation coefficient is calculated for each sampling line, and the average value is taken.
[0097]
The former method corresponds to the process of cutting out only the periphery of the connected pattern candidate and calculating the correlation coefficient in the first matching method. Therefore, when there is a portion where brightness fluctuation is likely to occur within the pattern, Although there is a feature that the influence can be eliminated, there is a possibility of erroneous recognition if the connection pattern portion has uneven brightness.
[0098]
On the other hand, the latter method has a feature that, by calculating a correlation coefficient for each sampling line, highly reliable collation is possible even when there is a large lightness unevenness in the connected pattern portion.
[0099]
In the first to third matching methods, the calculated correlation coefficient value takes a numerical value between −1 and 1, but the correlation coefficient may take an absolute value. By doing so, it is possible to perform collation even when the lightness and darkness of the pattern is reversed.
[0100]
(Fourth verification method)
FIG. 21 is an explanatory diagram of a fourth matching method when the connection pattern is mathematical data, and FIG. 22 is an explanatory diagram of an edge position detection method.
[0101]
In the fourth matching method, a method using an edge position is used. Similar to the second and third methods, a plurality of sampling lines orthogonal to the connection pattern candidates of the deformation model are set, and the edge position is detected for each sampling line. FIG. 21 shows a state where edge positions are extracted for each sampling line. The average of the distance between each edge position and the on-line pattern is obtained, and a deformation model that minimizes the average distance is adopted as the optimum collation.
[0102]
In the fourth collating method, as a method for detecting an edge, a method using the maximum differential value is simple. However, as shown in FIG. 22, the lightness average value a1 on the right side of an arbitrary position of the sampling line By obtaining the left brightness average value a2 and using the position where | a1-a2 | is the maximum as the edge position, it is possible to detect the correct edge position without being affected by local brightness changes near the edge. Become.
[0103]
The fourth matching method has an advantage that an edge in an actual recognition target image can be extracted simultaneously with selection of an optimal deformation model. If the connected pattern is expressed by a mathematical expression, the least square approximation is performed on the mathematical expression of the connected pattern based on the extracted edge position, thereby causing a matching shift or pattern distortion (similarity) due to the influence of the shape of the local pattern. It is possible to correct the deformation (non-deformation) and improve the recognition accuracy.
[0104]
As described above, through the four processing processes of model generation process, local pattern compensation point extraction process, model deformation process, and similarity evaluation process, not only rotation and size fluctuation, but also lightness unevenness, focus shift, contrast reduction Thus, robust recognition that does not cause recognition errors even when fluctuations such as noise and pattern hiding occur is realized.
[0105]
The processing method in each of the above four processing steps needs to select an optimal processing method and parameters according to the purpose from a plurality of methods having different characteristics. Therefore, since there is a problem that an optimum processing method cannot be adopted unless the user understands the characteristics of each processing method, convenience can be improved by automating this point.
[0106]
Specifically, for example, the collation method in the similarity evaluation process may be changed depending on whether the data of the connection pattern is image data or data obtained by formulating the edge shape. Whether to use the first matching method or the second matching method in the similarity evaluation process may be determined based on information on whether another pattern exists. In this case, a database describing the relationship between the features of local patterns and connected patterns and the optimal processing method and recognition parameters for pattern recognition is created in advance, and the pattern to be detected is analyzed during the model generation process. Thus, when the local pattern and the connected pattern are classified, the subsequent processing procedure may be determined with reference to the database.
[0107]
In addition, in order to investigate the factors that fluctuate in time series, it is possible to improve the convenience by executing recognition on multiple images and selecting the optimal processing method and parameters based on the results. it can. Specifically, based on the recognition results obtained for a plurality of images, the third collation method in the similarity evaluation process is adopted if lightness unevenness is large, otherwise the first Examples include a method that uses a collation method. In this case as well, a database describing the relationship between the inter-image variation characteristics of local patterns and connected patterns, the optimum processing method for pattern recognition and recognition parameters is created in advance, and the recognition results using multiple images The subsequent processing procedure may be determined with reference to the database based on the above.
[0108]
In addition, when a plurality of teaching images are used as described above, not only the processing method and parameters as described above but also the variation of the shape of the recognition model can be obtained. Local pattern data and connected pattern data are extracted from a plurality of teaching images, and local pattern data and connected patterns obtained from other teaching images in accordance with a certain teaching image (usually the first image). It is possible to construct a recognition model that can be recognized more reliably than when a recognition model is constructed from only a single teaching image by normalizing the data and obtaining the average and variation amount of the data. .
[0109]
In the case of processing for continuous images input in time series, the recognition speed can be improved by changing the recognition method at a certain point in time series using the recognition results obtained so far.
[0110]
As a first method related to the processing for the continuous image, a method of changing the recognition model based on the result of the pattern matching performed previously can be considered. As shown in FIG. 23, when the pattern is continuously distorted, as in the case of correcting the deformation that is not the similar deformation described above, the pattern distortion is corrected and the pattern matching is performed. By sequentially changing the local pattern or the connected pattern, it is possible to track an object that causes a deformation other than the similar deformation in time series.
[0111]
As a second method related to processing for continuous images, there is a method of changing the processing method based on the result of pattern matching performed previously. As shown in FIG. 24, in a continuous image in which the pattern continuously rotates and scale changes, the position, rotation, and scale of the local pattern are extracted in the local pattern interpolation point extraction process based on the matching result before that point. Estimate the parameters of variation, limit the search range and local pattern rotation range when extracting local pattern interpolation points, and make matching processing more efficient, or narrow down the local pattern interpolation points to be extracted The process can be streamlined.
[0112]
【The invention's effect】
The invention according to claim 1 is capable of specifying the position and orientation of a recognition target image having a specific pattern included in a digital image, and includes a recognition reference image serving as a reference corresponding to the recognition target image. An image processing apparatus uses a recognition model composed of at least two local patterns obtained from a digital image, each of which is assigned one direction, and at least one connection pattern connecting the local patterns. A specific pattern recognition method for recognizing a specific pattern of a recognition target image, the local pattern candidate being a position of a local pattern candidate that matches each of at least two different local patterns among the local patterns constituting the recognition model At least one point is extracted from the digital image including the recognition target image, and at least one of the points is extracted. A local pattern candidate point extraction process for assigning the direction assigned to the corresponding local pattern to the local pattern candidate points, and at least two different local patterns from the local pattern candidate points extracted in this local pattern candidate point extraction process By extracting at least one set of local pattern candidate points that coincides with each other, selecting a deformation candidate consisting of each local pattern interpolation point of the extracted set, and selecting each local pattern interpolation point of the selected deformation candidate Based on the positional relationship, a model deformation process for obtaining each deformation parameter when adding an overall pattern of the recognition model to the recognition target image by adding deformation of the position, scale, and angle to the recognition model. Based on the deformation parameters obtained in the model deformation process, the recognition model is deformed, and the deformed recognition model is Including a similarity evaluation process for evaluating the similarity between the deformation candidate and the recognition target image, so that the pattern of the recognition target image includes rotation, size variation, focus shift, brightness unevenness, Even if fluctuations such as contrast reduction, noise, pattern hiding, etc. occur, if the part that can identify the position and orientation of the recognition target image in the entire recognition target image is captured normally Based on the local pattern corresponding to that part, the entire pattern of the recognition target image can be estimated and collated with the recognition target image, so that fluctuations in the environment such as illumination fluctuations, obstacles, positioning errors, etc. Robust recognition that is difficult to be affected is possible.
[0113]
The invention according to claim 2 further includes a model generation process for generating the recognition model in the specific pattern recognition method according to claim 1, wherein in the model generation process, an outline of the recognition reference image is obtained, Since the local pattern is extracted from the partial image in the region where the curvature value in the contour line is larger than a predetermined value, the position and direction can be easily specified when the contour line can be easily obtained with high image contrast. A local pattern can be easily extracted.
[0114]
According to a third aspect of the present invention, in the specific pattern recognition method according to the first aspect, the method further includes a model generation process for generating the recognition model, and the model generation process includes a partial image of the digital image including the recognition reference image. Therefore, the brightness change direction is obtained for each pixel, and the pattern of the image where the peak of brightness change appears in at least two different directions is extracted as the local pattern. In the case of a pattern, a local pattern can be extracted by a method of creating a histogram of direction images and searching for a place where there are two peaks.
[0115]
According to a fourth aspect of the present invention, in the specific pattern recognition method according to the first aspect, since the connection pattern is an image pattern, the reason is that the connection pattern has a complicated shape or an unclear pattern. Thus, even when it is difficult to express the edge shape with a mathematical expression, the connection pattern can be expressed.
[0116]
According to a fifth aspect of the present invention, in the specific pattern recognition method according to the first aspect, the connection pattern is any one of a straight line, a broken line having at least one intermediate node, and a curve, and the connection pattern data is mathematical data. Therefore, the data amount of the recognition model can be reduced. Further, the pattern matching and the position detection method can improve accuracy and match a distorted pattern.
[0117]
According to a sixth aspect of the present invention, in the specific pattern recognition method according to the first aspect of the present invention, the method further includes a model generation process for generating the recognition model, and in the model generation process, from the digital image including the recognition reference image, A polar coordinate having the center position of the local pattern as an origin is converted into an orthogonal coordinate having an angle axis indicating an angle centered on the origin and a distance axis indicating a distance from the origin to create an orthogonal transform image, and In the pattern candidate point extraction process, from the digital image including the recognition target image, polar coordinates having a certain point of interest as an origin are converted into orthogonal coordinates to generate an orthogonal conversion image. In the orthogonal conversion image and the model generation process, While translating the created orthogonal transformation image in the direction of the angle axis, the similarity is obtained sequentially, the maximum similarity is extracted from each of these similarities, and this maximum similarity is predetermined. If it is larger than the reference, the position of the orthogonal transformation image when the maximum similarity is obtained is set as the position of the local pattern candidate point, and the corresponding local pattern is determined based on the movement amount that gives the maximum similarity. Since the assigned direction is assigned to the local pattern candidate point, the local pattern can be extracted and the direction can be assigned even when the pattern is rotating.
[0118]
The invention according to claim 7 further includes a model generation process for generating the recognition model in the specific pattern recognition method according to claim 1, wherein in the model generation process, from the digital image including the recognition reference image, Polar coordinates with the center position of the local pattern as the origin are converted into Cartesian coordinates with an angle axis that indicates the angle centered on the origin and a distance axis that indicates the distance from the origin, and an orthogonal transformation image is created to create this orthogonal One-dimensional waveform data is created by projecting the converted image onto the angle axis, and in the local pattern interpolation point extraction process, polar coordinates having a certain point of interest as the origin are used as orthogonal coordinates from the digital image including the recognition target image. Transform to create an orthogonal transform image, project this orthogonal transform image onto the angle axis to create one-dimensional waveform data, and create this one-dimensional waveform data and the model generation process The parallel waveform data is translated in the direction of the angle axis, and the similarities are sequentially obtained, and the maximum similarity is extracted from each of these similarities. If this maximum similarity is greater than a predetermined reference, the maximum similarity is obtained. The position of the orthogonal transformation image when the position is obtained is set as the position of the local pattern candidate point, and the direction given to the corresponding local pattern is determined based on the amount of movement that gives the maximum similarity. Since it is assigned to a point, the similarity and direction of the pattern can be obtained by simple one-dimensional pattern matching when the pattern is rotated.
[0119]
The invention according to claim 8 is the specific pattern recognition method according to claim 6, wherein the orthogonal transformation image extracted from the digital image including the recognition target image is enlarged or reduced in the distance axis direction and created in the model generation process. Since matching is performed with the orthogonal transformation image, the local pattern can be extracted even when the scale changes with the rotation of the pattern.
[0120]
According to a ninth aspect of the present invention, in the specific pattern recognition method according to the first aspect, in the local pattern interpolation point extraction process, only a portion having a large change in pixel value is extracted in advance, and only that portion is extracted. On the other hand, since the local pattern candidate point extraction process is performed, the load of the local pattern extraction process can be reduced.
[0121]
According to a tenth aspect of the present invention, in the specific pattern recognition method according to the first aspect, in the model deformation process, an angle difference between the direction of the local pattern candidate point of the deformation candidate and the direction of the corresponding local pattern is predetermined. If the angle is larger than the angle, the deformation candidate is excluded from the similarity evaluation in the similarity evaluation process, so that the processing load in the similarity evaluation process can be reduced.
[0122]
The invention according to claim 11 is the specific pattern recognition method according to claim 4, wherein, in the similarity evaluation process, when the data of the connection pattern is image data, the deformation of the deformation candidate obtained in the model deformation process. Based on the parameters, the connected pattern is deformed, and the entire pattern of the deformed recognition model including the deformed connected pattern is matched with the recognition target image, so that the pattern used as the connected pattern is complicated. Correct matching can be performed when an image is used as a connection pattern because it is unclear.
[0123]
According to a twelfth aspect of the present invention, in the specific pattern recognition method according to the fifth aspect, in the similarity evaluation process, when the data of the connection pattern is expressed using a mathematical expression representing a linear pattern, the connection pattern A plurality of sampling points are set on the connection pattern candidates obtained from the above, and sampling lines of arbitrary length are set in the normal direction of the connection pattern candidates around these sampling points, and all of the set sampling lines are set. Since the correlation value between the distribution of pixel values and the ideal pattern in which the pixel values change stepwise at the position of the connected pattern candidate is calculated as the similarity, the internal pattern is ignored and only the similarity near the edge is evaluated Therefore, correct matching is performed even when there is a pattern with varying brightness or varying brightness within the pattern. Door can be.
[0124]
According to a thirteenth aspect of the present invention, in the specific pattern recognition method according to the fifth aspect, in the similarity evaluation process, when the data of the connection pattern is expressed using a mathematical expression representing a linear pattern, the connection pattern Set multiple sampling points on the connected pattern candidate obtained from, set an arbitrary length sampling line in the normal direction of the connected pattern candidate around these sampling points, and set the pixel value on the set sampling line The process of examining the similarity between each distribution and the ideal pattern in which the pixel value changes stepwise at the sampling point is performed for each sampling line, and the average is calculated by calculating the similarity for all sampling lines. Therefore, correct matching can be performed even when there is a large lightness unevenness.
[0125]
According to a fourteenth aspect of the present invention, in the specific pattern recognition method according to any one of the eleventh to thirteenth aspects, when it is known that a contrast inversion of light and darkness occurs in the recognition target image, a normalized correlation value as a similarity Since the absolute value of is used, correct matching can be performed even if brightness contrast inversion occurs.
[0126]
According to a fifteenth aspect of the present invention, in the specific pattern recognition method according to the fifth aspect, in the similarity evaluation process, when the data of the connection pattern is expressed using a mathematical expression representing a linear pattern, the connection pattern A plurality of sampling points are set on the connection pattern candidate obtained from the above, and a sampling line of an arbitrary length is set in the normal direction of the connection pattern candidate around each sampling point, and each of all the set sampling lines is set. Since the edge position is obtained and the average value of the distance between the edge position and the connected pattern is used as the similarity, the edge can be extracted simultaneously with the similarity determination, and the contour shape of the pattern can be obtained on the actual image. .
[0127]
According to a sixteenth aspect of the present invention, in the specific pattern recognition method according to the fifteenth aspect, an average value of pixel values is obtained on the left and right of an attention position with the sampling line, an absolute value of a difference between the average pixel values is obtained, Since the absolute value of the difference between the average pixel values is sequentially obtained while moving within the range of the sampling line, and the position where the absolute value is the maximum is used as the edge position, the edge position is extremely incorrect due to local features. The danger of being detected at the position can be avoided.
[0128]
According to a seventeenth aspect of the present invention, in the specific pattern recognition method according to the fifteenth aspect, since each of the connection patterns is applied to the expression of the connection pattern using the least square method based on the edge position, the edge is locally edged. Even when the shape is disturbed, the edge position can be accurately estimated.
[0129]
The invention according to claim 18 is the specific pattern recognition method according to claim 1, wherein the relationship between the characteristics of the local pattern and the connected pattern and at least one of the optimum processing method and recognition parameters for recognition of these patterns is described. As a model generation process for generating the recognition model, the feature of the pattern to be detected is analyzed, and the local pattern candidate point extraction is performed by referring to the database based on the analysis result. It further includes the process of determining at least one of the processing method and the recognition parameter in each of the process, the model deformation process, and the similarity evaluation process, so that it is not necessary for the user to determine the algorithm or the recognition parameter, and the system is easy to use. Will improve.
[0130]
According to a nineteenth aspect of the present invention, in the specific pattern recognition method according to the first aspect, at least one of a feature of inter-image variation of the local pattern and the connection pattern, an optimum processing method for the recognition of the patterns, and a recognition parameter, A database describing the relationship between the local pattern candidate points, a plurality of teaching images are experimentally recognized, and the local pattern candidate point extraction process and model deformation process are performed by referring to the database based on the recognition results. In addition, since at least one of the processing method and the recognition parameter in each process of the similarity evaluation process is determined, usability when the system is used for an object having a time series change is improved.
[0131]
The invention according to claim 20 is the specific pattern recognition method according to claim 1, further comprising a model generation process for generating the recognition model, wherein in the model generation process, the recognition standard obtained by photographing the same pattern. When a plurality of digital images including images are used as teaching images, since the inter-image variations of the local pattern data and the connection pattern data are normalized and integrated, a recognition model is constructed only from a single teaching image It is possible to construct a recognition model capable of more reliable recognition.
[0132]
According to a twenty-first aspect of the present invention, in the specific pattern recognition method according to the first aspect, when pattern matching and position detection are performed on a digital image continuously input in time series, and the pattern is traced, Since the recognition model is changed based on the result of the pattern matching performed in the above step, it is possible to track an object that causes a deformation other than the similar deformation in time series.
[0133]
According to a twenty-second aspect of the present invention, in the specific pattern recognition method according to the first aspect, when pattern matching and position detection are performed on a digital image continuously input in time series, and the pattern is traced, Based on the results of the position detection performed in step 1, the processing method in each of the local pattern candidate point extraction process, model deformation process, and similarity evaluation process is changed as needed, so that it is possible to narrow down the pattern search range. Thus, more efficient pattern matching and position detection are possible.
[0134]
The invention according to claim 23 is capable of specifying the position and orientation of a recognition target image having a specific pattern included in a digital image, and includes a recognition reference image serving as a reference corresponding to the recognition target image. An image processing apparatus uses a recognition model composed of at least two local patterns obtained from a digital image, each of which is assigned one direction, and at least one connection pattern connecting the local patterns. A specific pattern recognition program including a processing procedure of a specific pattern recognition method for recognizing a specific pattern of a recognition target image, wherein the local pattern matches each of at least two different local patterns among the local patterns constituting the recognition model A local pattern candidate point that is a position of the pattern candidate is displayed as a digital image including the recognition target image. A local pattern candidate point extraction process in which at least one is extracted from and a direction given to the corresponding local pattern is assigned to the at least one local pattern candidate point, and the local pattern candidate extracted in the local pattern candidate point extraction process By extracting at least one set of local pattern candidate points that match at least two different local patterns from the points, a deformation candidate consisting of each local pattern compensation point of the extracted set is selected and selected When the position, scale, and angle of the recognition model are modified based on the positional relationship between the local pattern compensation points of the deformation candidates, and the entire pattern of the recognition model is superimposed on the recognition target image. Based on the model deformation process for obtaining each deformation parameter of the image and the deformation parameter obtained in the model deformation process, the recognition Since the model includes a similarity evaluation process in which the model is deformed, the deformed recognition model is superimposed on the recognition target image, and the similarity between the deformation candidate and the recognition target image is evaluated. Even in this device, it is possible to perform robust recognition that is not easily influenced by environmental fluctuations such as illumination fluctuations, obstacles, and positioning errors.
[0135]
Since the specific pattern recognition program recording medium according to the twenty-fourth aspect records the specific pattern recognition program according to the twenty-third aspect so as to be readable by the computer, even if the apparatus is other than a dedicated image processing apparatus, the illumination variation And robust recognition that is not easily affected by environmental fluctuations such as obstacles, positioning errors, etc.
[0136]
According to a twenty-fifth aspect of the present invention, there is provided an imaging unit that performs imaging, a quantization unit that obtains a digital image in which brightness values are recorded for each lattice point of a two-dimensional matrix from an image signal captured by the imaging unit, Storage means for storing at least digital image data obtained by the quantization means, the position and orientation of a recognition target image having a specific pattern included in the digital image can be specified, and the recognition It is composed of at least two local patterns obtained from a digital image including a recognition reference image serving as a reference corresponding to the target image, each of which is assigned one direction, and at least one connection pattern connecting these local patterns. A specific pattern recognition apparatus for recognizing a specific pattern of a recognition target image using a recognition model. At least one local pattern candidate point that is a position of a local pattern candidate that matches each of at least two different local patterns among local patterns constituting the model is extracted from the digital image including the recognition target image, and at least one of the local pattern candidate points is extracted. A local pattern candidate point extraction unit that assigns a direction assigned to a corresponding local pattern to one local pattern candidate point, and at least two different local regions from the local pattern candidate point extracted by the local pattern candidate point extraction unit By extracting at least one set of local pattern candidate points that match the pattern, a deformation candidate composed of each local pattern interpolation point of the extracted set is selected, and each local pattern interpolation point of the selected deformation candidate is selected Based on the positional relationship, position, scale, and angle deformations are added to the recognition model. Model deformation means for obtaining respective deformation parameters when superposing the entire pattern of the recognition model on the recognition target image, and deforming the recognition model based on the deformation parameters obtained by the model deformation means, And a similarity evaluation unit that evaluates the similarity between the deformation candidate and the recognition target image and overlays the recognition model on the recognition target image, so that variations in the environment such as illumination variations, obstacles, and positioning errors are included. This makes it possible to achieve robust recognition that is difficult to be influenced by the
[Brief description of the drawings]
FIG. 1 is a configuration diagram of a specific pattern recognition apparatus.
FIG. 2 is a flowchart showing a flow of processing of a specific pattern recognition method executed by the processing device in FIG. 1;
3 is a diagram showing examples of various digital images that can be obtained from the camera and quantizer in FIG. 1. FIG.
FIG. 4 is a diagram illustrating an example of a recognition model generated by a model generation process.
FIG. 5 is a diagram illustrating a contour line extracted from a digital image including a recognition reference image.
6 is a graph showing a change in curvature (reciprocal of radius) of the contour line in FIG. 5;
FIG. 7 is a diagram illustrating a recognition reference image and a plurality of partial regions in a digital image including the recognition reference image.
FIG. 8 is a graph showing the brightness change direction in each partial region in FIG.
FIG. 9 is a diagram illustrating another shape example of a connection pattern.
FIG. 10 is an explanatory diagram of polar coordinate → rectangular coordinate conversion processing;
FIG. 11 is a diagram illustrating an example of an orthogonal transformation image that can be obtained by a polar coordinate → rectangular coordinate transformation process for a digital image including a recognition target image.
FIG. 12 is an explanatory diagram of a matching method when an orthogonal transformation image has a simple striped pattern.
FIG. 13 is an explanatory diagram of a matching method when an orthogonal transformation image has a complicated pattern.
FIG. 14 is a diagram illustrating an example of a recognition target image.
15 is a diagram showing local pattern interpolation points extracted from the recognition target image of FIG. 14 by a local pattern interpolation point extraction process. FIG.
FIG. 16 is an explanatory diagram of a model deformation process.
FIG. 17 is an explanatory diagram of a similarity evaluation method.
FIG. 18 is an explanatory diagram of a first matching method when a connection pattern is mathematical formula data.
FIG. 19 is an explanatory diagram of a second collation method when the connection pattern is mathematical formula data.
FIG. 20 is a diagram showing a change in brightness at an ideal edge.
FIG. 21 is an explanatory diagram of a fourth collation method when the connection pattern is mathematical formula data;
FIG. 22 is an explanatory diagram of an edge position detection method.
FIG. 23 is a diagram illustrating an example of a digital image in a state in which distortion is continuously generated in a time series in a pattern.
FIG. 24 is a diagram illustrating an example of a digital image in a state in which similar deformation is continuously performed in time series.
[Explanation of symbols]
10 Object
11 Camera
12 Quantizer
13 Storage device
14 Image display device
15 Processing device
151 Model generator
152 Local pattern candidate point extraction unit
153 Model deformation part
154 Similarity Evaluation Department

Claims (25)

One direction obtained from a digital image including a recognition reference image serving as a reference corresponding to the recognition target image, the position and orientation of the recognition target image having a specific pattern included in the digital image can be specified The image processing apparatus recognizes a specific pattern of the recognition target image using a recognition model composed of at least two local patterns assigned to each and at least one connected pattern connecting the local patterns. A specific pattern recognition method for
At least one local pattern candidate point that is a position of a local pattern candidate that matches each of at least two different local patterns among the local patterns constituting the recognition model is extracted from the digital image including the recognition target image. , A local pattern candidate point extraction process for assigning the direction given to the corresponding local pattern to the at least one local pattern candidate point;
By extracting at least one set of local pattern candidate points that match at least two different local patterns from the local pattern candidate points extracted in this local pattern candidate point extraction process, each local pattern of the extracted set Select deformation candidates consisting of interpolation points, and add position, scale, and angle deformations to the recognition model based on the positional relationship between the local pattern interpolation points of the selected deformation candidates, and recognize them. A model deformation process for obtaining each deformation parameter when superimposing the entire pattern of the model for recognition on the recognition target image;
Based on the deformation parameters obtained in the model deformation process, the recognition model is deformed, the deformed recognition model is superimposed on the recognition target image, and the similarity between the deformation candidate and the recognition target image is determined. A specific pattern recognition method comprising: a similarity evaluation process to be evaluated. A model generation process for generating the recognition model is further included. In the model generation process, an outline of the recognition reference image is obtained, and the local image is obtained from a partial image in a region where the curvature value in the outline is larger than a predetermined value. The specific pattern recognition method according to claim 1, wherein a pattern is extracted. A model generation process for generating the recognition model is further included. In the model generation process, the brightness change direction is obtained for each pixel of the partial image of the digital image including the recognition reference image, and the brightness change direction is changed in at least two different directions. 2. The specific pattern recognition method according to claim 1, wherein a pattern of an image of a portion where a peak appears is extracted as the local pattern. The specific pattern recognition method according to claim 1, wherein the connection pattern is an image pattern. 2. The specific pattern recognition method according to claim 1, wherein the connection pattern is one of a straight line, a broken line having at least one intermediate node, and a curve, and the connection pattern data is mathematical formula data. A model generation process for generating the recognition model is further included. In this model generation process, polar coordinates with the center position of the local pattern as the origin are taken from the digital image including the recognition reference image as an angle with the origin as the center. Convert to an orthogonal coordinate having an angle axis to indicate and a distance axis to indicate a distance from the origin, and create an orthogonal transformation image,
In the local pattern candidate point extraction process, from the digital image including the recognition target image, polar coordinates having a certain point of interest as an origin are converted into orthogonal coordinates to generate an orthogonal conversion image, and the orthogonal conversion image and the model generation are generated. While translating the orthogonal transform image created in the process in the direction of the angle axis, sequentially obtain the similarity, extract the maximum similarity from each of these similarities, and if this maximum similarity is greater than a predetermined reference The position of the orthogonal transformation image when the maximum similarity is obtained is set as the position of the local pattern candidate point, and the direction given to the corresponding local pattern is determined based on the movement amount that gives the maximum similarity. 2. The specific pattern recognition method according to claim 1, wherein the specific pattern recognition point is assigned to the local pattern candidate point. A model generation process for generating the recognition model is further included. In this model generation process, polar coordinates with the center position of the local pattern as the origin are taken from the digital image including the recognition reference image as an angle with the origin as the center. Converting to an orthogonal coordinate having an angle axis to indicate and a distance axis indicating a distance from the origin, creating an orthogonal transformation image, projecting the orthogonal transformation image on the angle axis to create one-dimensional waveform data, In the local pattern interpolation point extraction process, from the digital image including the recognition target image, polar coordinates having a certain point of interest as the origin are converted into orthogonal coordinates to create an orthogonal conversion image, and this orthogonal conversion image is used as an angle axis. One-dimensional waveform data is created by projecting, and the one-dimensional waveform data and the waveform data created in the model generation process are translated in the direction of the angle axis, and the similarity is sequentially obtained. The maximum similarity is extracted from each of these similarities, and if the maximum similarity is larger than a predetermined reference, the position of the orthogonal transformation image when the maximum similarity is obtained is the position of the local pattern candidate point. The specific pattern recognition method according to claim 1, wherein a direction given to a corresponding local pattern is assigned to the local pattern candidate point based on a movement amount that gives the maximum similarity. The identification according to claim 6, wherein an orthogonal transformation image extracted from a digital image including the recognition target image is enlarged or reduced in a distance axis direction and matched with the orthogonal transformation image created in the model generation process. Pattern recognition method. 2. The local pattern interpolation point extraction process, wherein only a part having a large change in pixel value is extracted in advance, and a local pattern candidate point extraction process is performed only on that part. Specific pattern recognition method. In the model deformation process, when the angle difference between the direction of the local pattern candidate point of the deformation candidate and the direction of the corresponding local pattern is larger than a predetermined angle, the deformation candidate is determined as the similarity in the similarity evaluation process. The specific pattern recognition method according to claim 1, wherein the specific pattern recognition method is excluded from the evaluation. In the similarity evaluation process, when the connection pattern data is image data, the connection pattern is deformed based on a deformation parameter of a deformation candidate obtained in the model deformation process, and includes the deformed connection pattern. 5. The specific pattern recognition method according to claim 4, wherein matching is performed between the entire pattern of the deformed recognition model and the recognition target image. In the similarity evaluation process, when the data of the connection pattern is expressed using a mathematical expression representing a linear pattern, a plurality of sampling points are set on the connection pattern candidates obtained from the connection pattern, A sampling line of arbitrary length is set in the normal direction of the connected pattern candidate centering on the sampling point, and the pixel value changes in a stepped manner at the position of the connected pattern candidate and the distribution of pixel values on all the set sampling lines 6. The specific pattern recognition method according to claim 5, wherein a correlation value with an ideal pattern to be calculated is used as a similarity. In the similarity evaluation process, when the data of the connection pattern is expressed using a mathematical expression representing a linear pattern, a plurality of sampling points are set on the connection pattern candidates obtained from the connection pattern, An arbitrary length sampling line is set in the normal direction of the connected pattern candidate centering on the sampling point, and the distribution of pixel values on the set sampling line and an ideal pattern in which the pixel value changes stepwise at the sampling point part 6. The specific pattern recognition method according to claim 5, wherein a process for checking the similarity is performed for each sampling line, the similarity is obtained for all sampling lines, an average value is calculated, and the overall similarity is obtained. . The specific pattern recognition method according to claim 11, wherein the absolute value of the normalized correlation value is used as the similarity when it is known that contrast inversion of light and dark will occur in the recognition target image. . In the similarity evaluation process, when the data of the connection pattern is expressed using a mathematical expression representing a linear pattern, a plurality of sampling points are set on the connection pattern candidates obtained from the connection pattern, Set a sampling line of arbitrary length in the normal direction of the connected pattern candidate around the sampling point, obtain the edge position for each of all the set sampling lines, and calculate the similarity with the average value of the distance between the edge position and the connected pattern The specific pattern recognition method according to claim 5, wherein: Find the average value of the pixel values at the left and right of the target position with the sampling line, determine the absolute value of the average pixel value difference, and move the target position within the range of the sampling line, 16. The method for recognizing a specific pattern according to claim 15, wherein values are sequentially obtained, and a position where the absolute value is maximum is used as an edge position. The specific pattern recognition method according to claim 15, wherein each of the connection patterns is applied to an expression of a connection pattern using a least square method based on an edge position. Create in advance a database describing the relationship between the features of the local patterns and connected patterns, and the optimum processing method and recognition parameters for recognition of these patterns,
As a model generation process for generating the recognition model, a pattern feature to be detected is analyzed, and the local pattern candidate point extraction process, model deformation process and similar are performed by referring to the database based on the analysis result. 2. The specific pattern recognition method according to claim 1, further comprising a step of determining at least one of a processing method and a recognition parameter in each step of the degree evaluation process. Create in advance a database that describes the relationship between the characteristics of inter-image variation of the local pattern and connected pattern, and the optimum processing method and recognition parameters for recognition of those patterns,
Recognizing a plurality of teaching images on a trial basis, referring to the database based on the recognition results, and processing methods in the local pattern candidate point extraction process, model deformation process, and similarity evaluation process 2. The method for recognizing a specific pattern according to claim 1, wherein at least one of the recognition parameter and the recognition parameter is determined. The method further includes a model generation process for generating the recognition model. In this model generation process, when a plurality of digital images including the recognition reference image obtained by photographing the same pattern are used as teaching images, the local pattern data 2. The specific pattern recognition method according to claim 1, wherein fluctuations between images of the connection pattern data are normalized and integrated. When performing pattern matching and position detection on a digital image that is continuously input in time series and tracking the pattern, the recognition model is changed based on the result of pattern matching performed earlier. The specific pattern recognition method according to claim 1. In the case of performing pattern matching and position detection on a digital image continuously input in time series, and performing pattern tracking, the local pattern candidate point extraction process based on the result of position detection performed previously 2. The specific pattern recognition method according to claim 1, wherein the processing method in each of the model deformation process and the similarity evaluation process is changed as needed. The direction and direction of a recognition target image having a specific pattern included in the digital image can be specified, and is obtained from a digital image including a recognition reference image serving as a reference corresponding to the recognition target image. The image processing device recognizes a specific pattern of the recognition target image using a recognition model composed of at least two local patterns assigned to each and at least one connected pattern connecting the local patterns. A specific pattern recognition program including a processing procedure of a specific pattern recognition method
At least one local pattern candidate point that is a position of a local pattern candidate that matches each of at least two different local patterns among the local patterns constituting the recognition model is extracted from the digital image including the recognition target image. , A local pattern candidate point extraction process for assigning the direction given to the corresponding local pattern to the at least one local pattern candidate point;
By extracting at least one set of local pattern candidate points that match at least two different local patterns from the local pattern candidate points extracted in this local pattern candidate point extraction process, each local pattern of the extracted set Select deformation candidates consisting of interpolation points, and add position, scale, and angle deformations to the recognition model based on the positional relationship between the local pattern interpolation points of the selected deformation candidates, and recognize them. A model deformation process for obtaining each deformation parameter when superimposing the entire pattern of the model for recognition on the recognition target image;
Based on the deformation parameters obtained in the model deformation process, the recognition model is deformed, the deformed recognition model is superimposed on the recognition target image, and the similarity between the deformation candidate and the recognition target image is determined. A specific pattern recognition program including a similarity evaluation process to be evaluated. 24. A recording medium for recording a specific pattern recognition program, wherein the specific pattern recognition program according to claim 23 is recorded so as to be readable by the computer. Imaging means for performing imaging, quantization means for obtaining a digital image in which a brightness value is recorded for each lattice point of the two-dimensional matrix from an image signal captured by the imaging means, and digital obtained by the quantization means Storage means for storing at least image data, which can specify the position and orientation of a recognition target image having a specific pattern included in the digital image, and serves as a reference corresponding to the recognition target image Utilizing a recognition model composed of at least two local patterns obtained from a digital image including a recognition reference image, each of which is assigned one direction, and at least one connection pattern connecting these local patterns A specific pattern recognition apparatus for recognizing a specific pattern of a recognition target image,
At least one local pattern candidate point that is a position of a local pattern candidate that matches each of at least two different local patterns among the local patterns constituting the recognition model is extracted from the digital image including the recognition target image. , A local pattern candidate point extracting means for assigning the direction given to the corresponding local pattern to the at least one local pattern candidate point;
By extracting at least one set of local pattern candidate points that match at least two different local patterns from the local pattern candidate points extracted by the local pattern candidate point extraction means, each local pattern of the extracted set Select deformation candidates consisting of interpolation points, and add position, scale, and angle deformations to the recognition model based on the positional relationship between the local pattern interpolation points of the selected deformation candidates, and recognize them. Model deformation means for obtaining respective deformation parameters when the entire pattern of the model for use is superimposed on the recognition target image;
Based on the deformation parameter obtained by the model deformation means, the recognition model is deformed, the deformed recognition model is superimposed on the recognition target image, and the similarity between the deformation candidate and the recognition target image is determined. A specific pattern recognition apparatus including similarity evaluation means for evaluation.

Images