Nothing Special   »   [go: up one dir, main page]

JP3711892B2 - 3D surface shape measurement method - Google Patents

3D surface shape measurement method Download PDF

Info

Publication number
JP3711892B2
JP3711892B2 JP2001149056A JP2001149056A JP3711892B2 JP 3711892 B2 JP3711892 B2 JP 3711892B2 JP 2001149056 A JP2001149056 A JP 2001149056A JP 2001149056 A JP2001149056 A JP 2001149056A JP 3711892 B2 JP3711892 B2 JP 3711892B2
Authority
JP
Japan
Prior art keywords
measured
laser
back surface
measurement
reflected
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
JP2001149056A
Other languages
Japanese (ja)
Other versions
JP2002340533A (en
Inventor
幸治 船岡
浩之 笹井
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Mitsubishi Electric Corp
Original Assignee
Mitsubishi Electric Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Mitsubishi Electric Corp filed Critical Mitsubishi Electric Corp
Priority to JP2001149056A priority Critical patent/JP3711892B2/en
Publication of JP2002340533A publication Critical patent/JP2002340533A/en
Application granted granted Critical
Publication of JP3711892B2 publication Critical patent/JP3711892B2/en
Anticipated expiration legal-status Critical
Expired - Fee Related legal-status Critical Current

Links

Images

Landscapes

  • Length Measuring Devices By Optical Means (AREA)
  • Measurement Of Optical Distance (AREA)

Description

【0001】
【発明の属する技術分野】
本発明はレーザ変位計を用いた3次元形状測定方法に関し、特に、裏面が平面の透明体の3次元表面形状を測定する方法に関する。
【0002】
【従来の技術】
レーザ変位計を用いた3次元形状測定は広く一般的に用いられる技術であり、図7は、例えば、特開平7−146113号公報に示されたレーザ変位計の測定原理を示す図である。
レーザ変位計の構造は、図7に示すようにレーザ光を使用した三角測距法によるものである。
このレーザ変位計は、レーザ光を被測定物である物体Aに照射するためのレーザ照射器(レーザダイオードなどを有するもの)21と、物体Aからの反射レーザ光をレンズ22を通して受光するイメージセンサ(例えば、CCDが使用される)23とを有するものである。
【0003】
レーザ照射器21から照射されたレーザ光は、物体Aの表面で拡散反射し、この反射レーザ光がレンズ22を通して、イメージセンサ23に捉えられる。
そして、物体Aの表面がレーザ光の照射方向に対して移動したとき、イメージセンサ23上に受光されている反射レーザ光の受光位置が移動するため、この受光位置の移動距離を測定することにより、物体A表面の変位量、すなわち物体Aの表面形状が測定されることになる。
被測定物の3次元表面形状を捉えるには、被測定物あるいはレーザ変位計をxy方向(即ち、所定の平面内)に必要な範囲にわたって移動させ、各点におけるZ方向(即ち、所定の平面に垂直な方向)のデータを取り込むことになる。
なお、図7において、Sは物体Aとレンズ22までの距離、Bは照射レーザ光の中心位置とレンズ22の中心位置との距離、fはレンズ22とイメージセンサ23との距離(即ち、レンス22の焦点距離)、xはレンズ22の中心位置とイメージセンサ23上の受光位置との距離であり、三角距離法によると、次式の関係がある。
S/B = f/x
【0004】
【発明が解決しようとする課題】
図8は、上述した従来の測定方法において、被測定物が透明体の場合に、被測定物の裏面からの反射光外乱を説明するための図である。
図において、1はレーザ変位計、2は裏面が平面の透明な被測定物、21はレーザ照射器、5はレーザ照射器21から出射されるレーザ光、16は被測定物2の表面(測定面)、17は被測定物2の裏面、60はレーザ照射器21から出射されるレーザ光5が裏面17の被測定物2の内部側の面で拡散反射された反射レーザ光、70はレーザ照射器21から出射されるレーザ光5が被測定物2の表面(即ち、測定面)16で拡散反射された反射レーザ光、8は光強度波形、22はレンズ、23はイメージセンサである。
一般的に、透明体(即ち、被測定物2)の表面には微少な凹凸がある。
そのため、レーザ照射器21から被測定物2の表面(測定面)16に出射されたレーザ光5は、被測定物2の表面16の凹凸で拡散反射し、反射レーザ光70が発生する。
【0005】
しかし、レーザ照射器21から出射されたレーザ光5の大部分は、被測定物2の表面(測定面)16を透過し、裏面17に到達する。
そして、裏面17に到達したレーザ光5の大部分は、そのまま被測定物2の外部に透過してゆくが、一部は裏面17の被測定物側内面で拡散反射する。
裏面17の被測定物側内面で拡散反射した反射レーザ光60は、外乱光としてイメージセンサ23に検出される。
裏面17からの拡散反射光である反射レーザ光60の強度が測定面16での反射レーザ光7よりも強い場合、誤測定する問題がある。
この発明は上記のような問題点を解消するためになされたもので、裏面が平面の透明な被測定物の3次元表面形状を安定して測定できる方法を提供するものである。
【0006】
【課題を解決するための手段】
この発明に係る3次元表面形状測定方法は、レーザ照射器から出射されるレーザ光を裏面が平面の透明な被測定物に照射すると共に、その反射レーザ光をレンズで集光し、イメージセンサで受光し、その受光した反射レーザ光のピーク位置の変位に基づき、被測定物表面の裏面に垂直な方向の変位量を測定するレーザ変位計と、被測定物の裏面に平行な平面内において被測定物とレーザ変位計との相対位置を変化させる移動機構とを用いて、被測定物の表面の形状を測定する3次元表面形状測定方法において、レーザ照射器からのレーザ光が被測定物の裏面に垂直に入射するように被測定物を配置し、移動機構により被測定物とレーザ変位計の相対位置を変化させながら測定点を移動し、各測定点においてレーザ照射器からのレーザ光を被測定物の裏面より入射させ、被測定物表面の被測定物内部側の面で反射する反射レーザ光のピーク位置の変位に基づき、被測定物表面の裏面に垂直な方向の変位量を順次測定することにより、被測定物の表面形状を測定するものである。
【0007】
また、この発明に係る3次元表面形状測定方法は、被測定物を透過し、被測定物以外の物で反射したレーザ光が、イメージセンサの視野内に入らないようにしたものである。
【0008】
また、この発明に係る3次元表面形状測定方法は、測定対象点における被測定物表面の裏面に垂直な方向の変位量データを、測定対象点の近傍測定点における被測定物表面の裏面に垂直な方向の変位量データの中央値に置き換える中央値フィルタ処理を行うものである。
【0009】
【発明の実施の形態】
以下、この発明の一実施の形態を図面に基づいて説明する。なお、各図間において、同一符号は同一あるいは相当のものであることを示す。
実施の形態1.
図1は、実施の形態1による3次元表面形状測定方法を説明するための図である。
図において、1はレーザ変位計、2は被測定物、21はレーザ照射器、5はレーザ照射器21から出射されたレーザ光、16は被測定物2の表面(測定面)、17は被測定物2の裏面、6はレーザ照射器21から出射されたレーザ光5が裏面17で拡散反射された反射レーザ光、7はレーザ光5が被測定物2の表面(測定面)16の被測定物内部側の面で拡散反射された反射レーザ光、8は光強度波形、22はレンズ、23はイメージセンサである。
【0010】
また、25はレーザ変位計1を所定の平面内でxy方向に移動させるための移動機構であるXYステージであり、3次元形状を測定する際に用いる。
なお、図1では、レーザ照射器21、レンズ22、イメージセンサ23などで構成されたレーザ変位計1を移動機構(XYステージ)25側に取り付けて、被測定物2の裏面17に平行な平面内において被測定物に対してレーザ変位計1を移動させる場合の構成を概念的に示しているが、移動機構(XYステージ)25は被測定物2の裏面17に平行な平面内において被測定物2とレーザ変位計1との相対位置を変化させればよく、レーザ変位計1を所定の位置に固定し、被測定物2を移動機構(XYステージ)25に取り付けて移動させる構成としてもよいことは言うまでもない。
【0011】
本発明に適用される被測定物2は、アクリル板などの透明な材料で形成されたものであり、かつ、測定対象面(即ち、測定面)である表面16と対向する裏面17は平面である。
そして、本発明による3次元表面形状測定方法では、従来の測定方法とは異なり、図1に示すように、被測定物2の裏面17側をレーザ変位計1に向けると共に、照射器21から出射されるレーザ光5が被測定物2の裏面17に垂直に入射するように配置して、被測定物2の表面形状を測定することを特徴とする。
レーザ照射器21から出射されたレーザ光5は、大部分が、裏面17と測定対象面である表面16を透過するが、裏面17および表面16の微少な凹凸で一部が拡散反射し、裏面17で拡散反射された反射レーザ光6と表面16の被測定物内部側の面で拡散反射された反射レーザ光7が発生する。
【0012】
しかし、裏面17と表面(測定面)16とでは、屈折効果が異なる。
レーザ照射器21から出射されたレーザ光5は、裏面17では屈折率の低い空気側から屈折率の高い被測定物2(例えば、アクリル材)側に入射するが、表面(測定面)16では、その反対で、入射側(即ち、被測定物内部側)が高屈折率媒質である。
図2は、屈折率が異なる媒質の境界面に入射した光の反射と屈折を説明する図であり、図2(a)は、入射側の媒質の方が屈折率が小さい場合(n1<n2)を、図2(b)はその逆の場合(n1>n2)を示している。
なお、n1は入射側の媒質の屈折率、n2は出射側の媒質の屈折率である。
【0013】
図2(a)の場合、入射角iで入射した光9の大部分が境界面12を透過し、sin(i)/sin(t)=n2/n1を満たす角度t方向への透過光10となる。
しかし、図2(b)の場合、即ち、入射側の媒質の方が屈折率が大きい場合、入射角iが臨界角θ(sinθ=n2/n1)よりも大きいと、光が全く透過せずに全反射する。
例えば、空気とアクリル材の場合、臨界角θは42°である。
【0014】
次に、に微少な凹凸がある境界面12に光が入射した場合について、図3で説明する。
図3(a)は、入射側の方が屈折率が小さい場合で、図3(b)は、入射側の方が屈折率が大きい場合である。
図3(a)の場合、平面部分に入射した光14も、凹凸部分に入射した光15も、大部分が透過するため、反射光が極わずかである。
しかし、図3(b)の場合では、凹凸部に臨界角θ以上で入射した光は全反射し、拡散反射光11となる。
つまり、屈折率が大きい媒質側から光が入射した方が、凹凸による拡散反射光が強くなる。
【0015】
従って、図1のようにレーザ光5を照射すると、被測定物2の裏面17で拡散反射された反射レーザ光6よりも、被測定物表面(測定面)16の被測定物内部側の面で拡散反射された反射レーザ光7の方が強くなる。
その結果、イメージセンサ23では、表面(測定面)16からの反射レーザ光7の方が強く検出されるので、裏面17からの反射レーザ光6による誤測定を大幅に軽減することが可能となり、安定して、測定面の形状を測定することができる。
【0016】
なお、図4は、イメージセンサ23が検出する反射レーザ光7のピーク位置から、測定面である表面16の高さ(即ち、被測定物2の裏面17から表面16までの距離)を算出する算出式を説明するための図である。
図4において、
n:周囲の媒質に対する被測定物の屈折率の比率
x:レンズの中心位置とイメージセンサで検出された拡散光のピーク位置との距離
f:レンズとイメージセンサとの距離
B:照射レーザ光の中心位置とのレンズ中心位置との距離
l:レンズから被測定物裏面までの距離
z:被測定物の裏面から測定面までの距離
t:測定面からの拡散光が裏面を透過した光の出射角度
i:測定面からの拡散光の裏面への入射角度
である。
【0017】
屈折の法則から下記の(1)式が得られ、幾何学的関係から下記の(2)式と(3)式が得られる。
そして、(1)〜(3)式からiとtを消去し、Zについて解くと(4)式が得られる。
(4)式で高さZ(即ち、被測定物の裏面から測定面までの距離)を算出することができる。
【0018】
【数1】

Figure 0003711892
【0019】
上述したように、実施の形態1による3次元表面形状測定方法では、裏面が平面の透明な被測定物の裏面側からレーザ光を照射して表面(測定面)の形状を測定するので、裏面では低屈折率媒質側から高屈折率媒質側にレーザ光が入射し、測定面では、逆に、高屈折媒質側からレーザ光が入射することになる。
その結果、屈折効果の違いにより、裏面よりも測定面での拡散反射率の方が高くなるため、裏面からの反射光で誤測定することがなくなり、安定した表面形状の測定が可能となる。
【0020】
実施の形態2.
図5は、実施の形態2による3次元表面形状測定方法を説明するための図である。
図において、1はレーザ変位計、2は被測定物、21はレーザ照射器、5はレーザ照射器21から出射されたレーザ光、16は被測定物2の表面(測定面)、17は被測定物2の裏面、6はレーザ照射器21から出射されたレーザ光5が裏面17で拡散反射された反射レーザ光、7はレーザ光5が被測定物2の表面(測定面)16の被測定物内部側の面で拡散反射された反射レーザ光、8は光強度波形、22はレンズ、23はイメージセンサ、25はレーザ変位計1を所定の平面内でxy方向に移動させるための移動機構(XYステージ)である。
【0021】
また、18は被測定物2を透過したレーザ光、19は被測定物2を透過したレーザ光18が被測定物以外のもので反射された反射レーザ光、20は被測定物2を固定する固定用部材である。
なお、図5では、レーザ変位計1を移動機構(XYステージ)25側に取り付けて、被測定物2の裏面17に平行な平面内において被測定物に対してレーザ変位計1を移動させる場合の構成を概念的に示しているが、実施の形態1の場合と同様に、移動機構(XYステージ)25は被測定物2の裏面17に平行な平面内において被測定物2とレーザ変位計1との相対位置を変化させればよく、レーザ変位計1を所定の位置に固定し、被測定物2を固定する固定用部材20を移動機構(XYステージ)25に取り付けて移動させる構成としてもよいことは言うまでもない。
【0022】
本実施の形態による3次元表面形状測定方法では、被測定物2を透過したレーザ光18の被測定物2以外のもの(例えば、固定用部材20)で反射された反射レーザ光19が、イメージセンサの視野24の中に入らないように、被測定物2の表面(測定面)16と固定用部材20などとの間に十分な空間を設けたことを特徴とする。
このように被測定物2を配置したことにより、被測定物2を透過したレーザ光18が被測定物以外のもの(例えば、固定用部材20)で拡散反射し、外乱光である反射レーザ光19が発生しても、反射レーザ光19はイメージセンサ23の視野24の外にあり、反射レーザ光19はイメージセンサ23によって検出されることはない。
従って、被測定物2を透過したレーザ光18の反射光外乱の影響をによる誤測定を防止することが可能となり、安定した表面形状の3次元計測が行える。
【0023】
実施の形態3.
図6は、実施の形態3による3次元表面形状測定方法に用いられる中央値フィルタ(メディアンフィルタとも称す)のウィンドの一例を示す図である。
前述した実施の形態1あるいは2による3次元表面形状測定方法では、移動機構(XYステージ)25によって、被測定物2に対してレーザ変位計1を、被測定物2の裏面17に平行な平面内においてxy方向に相対的に移動できる構造になっている。
従って、例えば、xy面に平行な面内においてn×mの碁盤の目状に測定点を設定し、各測定点において、被測定物2の裏面から表面(測定面)までの高さ(切り)を測定することで、3次元表面形状が測定できる。
測定データの中には、被測定物の裏面に付着した異物等の影響で突発的に誤測定したデータが含まれることがある。
【0024】
本実施の形態による3次元表面形状測定方法では、測定データを中央値フィルタ処理することにり、上述のような問題点を解決することを特徴としている。
図6の例では、中央値フィルタのウィンドは3×3であり、対象測定点の値z(i,j)は、その値とその近傍8点の合計9点のデータを並べ替え、中央値を対象点の値と置き換える。
なお、対象測定点の値z(i,j)とは、xy平面(被測定物2の裏面17に平行な面)のx=i、y=jにおける被測定物2の裏面17から表面(測定面)16までの距離である。
このような中央値フィルタ処理を行うことによって、被測定物2の裏面17に付着した異物等の影響で突発的に発生した異常データについても、周囲の正常なデータで置き換えられるため、信頼性の高い3次元形状測定結果が得られる。
【0025】
本実施の形態では、3×3の中央値フィルタを用いたが、さらに異常値除去効果が要求される場合は、5×5のようにより大きなウィンドの中央値フィルタを用いても良い。
また、本実施例では、被測定物がアクリルの場合について説明したが、例えばガラスについても、屈折率が1.5程度と空気よりも大きいため、同様に3次元形状測定が可能である。
以上説明したように、実施の形態3による3次元表面形状測定方法では、実施の形態1あるいは2による3次元表面形状測定方法により測定した結果データを、隣接あるいは近傍の測定データの中央値に置き換える中央値フィルタ処理をすることによって、付着した異物等で突発的に誤測定したデータを除去することが可能となり、より安定した3次元表面形状測定を行うことができる。
【0026】
【発明の効果】
この発明による3次元表面形状測定方法によれば、レーザ照射器から出射されるレーザ光を裏面が平面の透明な被測定物に照射すると共に、その反射レーザ光をレンズで集光し、イメージセンサで受光し、その受光した反射レーザ光のピーク位置の変位に基づき、被測定物表面の裏面に垂直な方向の変位量を測定するレーザ変位計と、被測定物の裏面に平行な平面内において被測定物とレーザ変位計との相対位置を変化させる移動機構とを用いて、被測定物の表面の形状を測定する3次元表面形状測定方法において、レーザ照射器からのレーザ光が被測定物の裏面に垂直に入射するように被測定物を配置し、移動機構により被測定物とレーザ変位計の相対位置を変化させながら測定点を移動し、各測定点においてレーザ照射器からのレーザ光を被測定物の裏面より入射させ、被測定物表面の被測定物内部側の面で反射する反射レーザ光のピーク位置の変位に基づき、被測定物表面の裏面に垂直な方向の変位量を順次測定することにより、被測定物の表面形状を測定するので、透明な被測定物の裏面での拡散反射光よりも表面(即ち、測定面)での拡散反射光の方が強くなり、裏面からの拡散反射光で誤測定することを防止することが可能となり、透明な被測定物の3次元表面形状を安定して測定するができる。
【0027】
また、この発明による3次元表面形状測定方法によれば、被測定物を透過し、前記被測定物以外の物で反射したレーザ光が、イメージセンサの視野内に入らないようにしたので、被測定物を透過したレーザ光18の反射光外乱の影響をによる誤測定を防止することが可能となり、透明な被測定物の3次元表面形状を安定して測定するができる。
【0028】
また、この発明による3次元表面形状測定方法によれば、測定対象点における被測定物表面の裏面に垂直な方向の変位量データを、測定対象点の近傍測定点における被測定物表面の裏面に垂直な方向の変位量データの中央値に置き換える中央値フィルタ処理を行うので、被測定物の裏面に付着した異物等で突発的に誤測定したデータを除去することが可能となり、より安定して、透明な被測定物の3次元表面形状を測定するができる。
【図面の簡単な説明】
【図1】 実施の形態1による3次元表面形状測定方法を説明するための図である。
【図2】 屈折率の異なる媒質の境界面に入射した光の屈折と反射を説明するための図である。
【図3】 微少凹凸を有した面に入射した光の反射を説明する図である。
【図4】 被測定物の裏面から測定面である表面までの距離を算出する算出式を説明するための図である。
【図5】 実施の形態2による3次元表面形状測定方法を説明するための図である。
【図6】 実施の形態2による3次元表面形状測定方法で用いられる中央値フィルタを説明するための図である。
【図7】 レーザ変位計の測定原理を示す図である。
【図8】 透明体の表面形状測定時の裏面からの反射光外乱を説明するための図である。
【符号の説明】
1 レーザ変位計 2 被測定物
5 レーザ照射器から出射されるレーザ光
6 被測定物裏面で反射された反射レーザ光
7 被測定物表面の被測定物内部側の面で反射された反射レーザ光
8 光強度波形 9 入射光
10 透過光 11 反射光
12 境界面 13 凹凸
14 平面部分に入射した光 15 凹凸部分に入射した光
16 表面(測定面) 17 裏面
18 被測定物を透過したレーザ光
19 被測定物以外のもので反射されたの反射レーザ光
20 被測定物の固定用部材 21 レーザ照射器
22 レンズ 23 イメージセンサ
24 イメージセンサの視野 25 移動機構(XYステージ)[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a three-dimensional shape measuring method using a laser displacement meter, and more particularly to a method for measuring a three-dimensional surface shape of a transparent body having a flat back surface.
[0002]
[Prior art]
Three-dimensional shape measurement using a laser displacement meter is a widely used technique, and FIG. 7 is a diagram illustrating a measurement principle of a laser displacement meter disclosed in, for example, Japanese Patent Application Laid-Open No. 7-146113.
The structure of the laser displacement meter is based on a triangulation method using laser light as shown in FIG.
The laser displacement meter includes a laser irradiator (having a laser diode or the like) 21 for irradiating an object A that is an object to be measured, and an image sensor that receives reflected laser light from the object A through a lens 22. (For example, a CCD is used) 23.
[0003]
The laser light emitted from the laser irradiator 21 is diffusely reflected on the surface of the object A, and this reflected laser light is captured by the image sensor 23 through the lens 22.
When the surface of the object A moves with respect to the laser light irradiation direction, the light receiving position of the reflected laser light received on the image sensor 23 moves. By measuring the moving distance of this light receiving position, Then, the displacement amount of the surface of the object A, that is, the surface shape of the object A is measured.
In order to capture the three-dimensional surface shape of the object to be measured, the object to be measured or the laser displacement meter is moved over the necessary range in the xy direction (that is, in a predetermined plane), and the Z direction at each point (that is, the predetermined plane) Data in a direction perpendicular to the direction).
In FIG. 7, S is the distance between the object A and the lens 22, B is the distance between the center position of the irradiation laser beam and the center position of the lens 22, and f is the distance between the lens 22 and the image sensor 23 (ie, the lens). 22 is the distance between the center position of the lens 22 and the light receiving position on the image sensor 23. According to the triangular distance method, there is a relationship of the following expression.
S / B = f / x
[0004]
[Problems to be solved by the invention]
FIG. 8 is a diagram for explaining reflected light disturbance from the back surface of the measurement object when the measurement object is a transparent body in the conventional measurement method described above.
In the figure, 1 is a laser displacement meter, 2 is a transparent object to be measured whose back surface is flat, 21 is a laser irradiator, 5 is a laser beam emitted from the laser irradiator 21, and 16 is a surface of the object to be measured 2 (measurement). Surface), 17 is the back surface of the object 2 to be measured, 60 is a reflected laser light in which the laser light 5 emitted from the laser irradiator 21 is diffusely reflected by the surface of the back surface 17 on the inside of the object 2 to be measured, and 70 is a laser. A laser beam 5 emitted from the irradiator 21 is a reflected laser beam diffusely reflected by the surface (ie, measurement surface) 16 of the object 2 to be measured, 8 is a light intensity waveform, 22 is a lens, and 23 is an image sensor.
Generally, the surface of the transparent body (that is, the object to be measured 2) has minute irregularities.
Therefore, the laser beam 5 emitted from the laser irradiator 21 to the surface (measurement surface) 16 of the object to be measured 2 is diffusely reflected by the unevenness of the surface 16 of the object to be measured 2, and the reflected laser light 70 is generated.
[0005]
However, most of the laser light 5 emitted from the laser irradiator 21 passes through the front surface (measurement surface) 16 of the DUT 2 and reaches the back surface 17.
And most of the laser beam 5 reaching the back surface 17 is transmitted as it is to the outside of the device under test 2, but a part is diffusely reflected on the inside surface of the back surface 17 on the device under test object side.
The reflected laser beam 60 diffusely reflected by the inner surface of the object 17 on the back surface 17 is detected by the image sensor 23 as disturbance light.
When the intensity of the reflected laser beam 60, which is diffusely reflected light from the back surface 17, is stronger than the reflected laser beam 7 on the measurement surface 16, there is a problem of erroneous measurement.
The present invention has been made to solve the above-described problems, and provides a method capable of stably measuring the three-dimensional surface shape of a transparent object having a flat back surface.
[0006]
[Means for Solving the Problems]
The three-dimensional surface shape measurement method according to the present invention irradiates a laser beam emitted from a laser irradiator onto a transparent object to be measured whose back surface is flat, condenses the reflected laser beam with a lens, and uses an image sensor. A laser displacement meter that measures the amount of displacement in the direction perpendicular to the back surface of the object to be measured based on the displacement of the peak position of the reflected laser light received, and a surface to be measured in a plane parallel to the back surface of the object to be measured. In a three-dimensional surface shape measuring method for measuring the shape of the surface of an object to be measured using a moving mechanism that changes the relative position between the object to be measured and the laser displacement meter, the laser light from the laser irradiator is applied to the object to be measured. The object to be measured is placed so that it is perpendicularly incident on the back surface, the measurement point is moved while changing the relative position of the object to be measured and the laser displacement meter by the moving mechanism, and the laser beam from the laser irradiator is emitted at each measurement point. Measured The amount of displacement in the direction perpendicular to the back surface of the object to be measured is sequentially measured based on the displacement of the peak position of the reflected laser light that is incident from the back surface of the object and is reflected by the surface inside the object to be measured on the surface of the object to be measured. Thus, the surface shape of the object to be measured is measured.
[0007]
The three-dimensional surface shape measuring method according to the present invention is such that laser light that has passed through the object to be measured and reflected by an object other than the object to be measured does not enter the field of view of the image sensor.
[0008]
Further, the three-dimensional surface shape measuring method according to the present invention provides the displacement amount data in the direction perpendicular to the back surface of the object to be measured at the measurement target point, and is perpendicular to the back surface of the object surface at the measurement point in the vicinity of the measurement target point. A median filter process is performed to replace the median value of the displacement data in various directions.
[0009]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In addition, between each figure, it shows that the same code | symbol is the same or equivalent.
Embodiment 1 FIG.
FIG. 1 is a diagram for explaining a three-dimensional surface shape measuring method according to the first embodiment.
In the figure, 1 is a laser displacement meter, 2 is an object to be measured, 21 is a laser irradiator, 5 is a laser beam emitted from the laser irradiator 21, 16 is a surface (measurement surface) of the object to be measured 2, and 17 is an object to be measured. The back surface of the measurement object 2, 6 is a reflected laser light in which the laser light 5 emitted from the laser irradiator 21 is diffusely reflected by the back surface 17, and 7 is a surface of the measurement object 2 on the surface (measurement surface) 16 of the measurement object 2. Reflected laser light diffusely reflected on the surface inside the measurement object, 8 is a light intensity waveform, 22 is a lens, and 23 is an image sensor.
[0010]
Reference numeral 25 denotes an XY stage, which is a moving mechanism for moving the laser displacement meter 1 in the xy direction within a predetermined plane, and is used when measuring a three-dimensional shape.
In FIG. 1, a laser displacement meter 1 including a laser irradiator 21, a lens 22, an image sensor 23, and the like is attached to the moving mechanism (XY stage) 25 side, and a plane parallel to the back surface 17 of the object 2 to be measured. 1 schematically shows the configuration in the case where the laser displacement meter 1 is moved with respect to the object to be measured, but the moving mechanism (XY stage) 25 is measured in a plane parallel to the back surface 17 of the object 2 to be measured. The relative position between the object 2 and the laser displacement meter 1 may be changed. The laser displacement meter 1 may be fixed at a predetermined position, and the object to be measured 2 may be attached to the moving mechanism (XY stage) 25 and moved. Needless to say, it is good.
[0011]
An object to be measured 2 applied to the present invention is formed of a transparent material such as an acrylic plate, and the back surface 17 facing the surface 16 that is a measurement target surface (that is, a measurement surface) is a flat surface. is there.
In the three-dimensional surface shape measuring method according to the present invention, unlike the conventional measuring method, the back surface 17 side of the object to be measured 2 is directed to the laser displacement meter 1 and emitted from the irradiator 21 as shown in FIG. It arrange | positions so that the laser beam 5 performed may inject into the back surface 17 of the to-be-measured object 2 perpendicularly | vertically, and the surface shape of the to-be-measured object 2 is measured.
Most of the laser light 5 emitted from the laser irradiator 21 is transmitted through the back surface 17 and the surface 16 that is the measurement target surface, but a part of the back surface 17 and the surface 16 is slightly diffusely reflected and reflected by the back surface 17. The reflected laser beam 6 diffusely reflected by 17 and the reflected laser beam 7 diffusely reflected by the surface 16 inside the object to be measured are generated.
[0012]
However, the refraction effect is different between the back surface 17 and the front surface (measurement surface) 16.
The laser light 5 emitted from the laser irradiator 21 is incident on the object 2 (for example, acrylic material) having a high refractive index from the air side having a low refractive index on the back surface 17, but on the surface (measuring surface) 16. On the contrary, the incident side (that is, the inside of the object to be measured) is a high refractive index medium.
FIG. 2 is a diagram for explaining reflection and refraction of light incident on a boundary surface of media having different refractive indexes. FIG. 2A shows a case where the refractive index of the medium on the incident side is smaller (n1 <n2). 2 (b) shows the opposite case (n1> n2).
Here, n1 is the refractive index of the medium on the incident side, and n2 is the refractive index of the medium on the outgoing side.
[0013]
In the case of FIG. 2A, most of the light 9 incident at the incident angle i passes through the boundary surface 12, and the transmitted light 10 in the angle t direction satisfying sin (i) / sin (t) = n2 / n1. It becomes.
However, in the case of FIG. 2B, that is, when the incident side medium has a higher refractive index, if the incident angle i is larger than the critical angle θ (sin θ = n2 / n1), no light is transmitted. Total reflection.
For example, in the case of air and an acrylic material, the critical angle θ is 42 °.
[0014]
Next, a case where light is incident on the boundary surface 12 having minute unevenness will be described with reference to FIG.
3A shows a case where the refractive index is smaller on the incident side, and FIG. 3B shows a case where the refractive index is larger on the incident side.
In the case of FIG. 3A, most of the light 14 incident on the plane portion and the light 15 incident on the concavo-convex portion are transmitted, so that the reflected light is very small.
However, in the case of FIG. 3B, the light incident on the concavo-convex portion at a critical angle θ or more is totally reflected and becomes diffusely reflected light 11.
That is, the diffuse reflected light due to the unevenness becomes stronger when the light is incident from the medium side having a large refractive index.
[0015]
Therefore, when the laser beam 5 is irradiated as shown in FIG. 1, the surface of the object to be measured (measurement surface) 16 on the inner side of the object to be measured rather than the reflected laser light 6 diffusely reflected by the back surface 17 of the object to be measured 2. The reflected laser light 7 diffusely reflected by is stronger.
As a result, in the image sensor 23, the reflected laser light 7 from the front surface (measurement surface) 16 is detected more strongly, so that erroneous measurement due to the reflected laser light 6 from the back surface 17 can be greatly reduced. The shape of the measurement surface can be measured stably.
[0016]
In FIG. 4, the height of the surface 16 that is the measurement surface (that is, the distance from the back surface 17 of the DUT 2 to the surface 16) is calculated from the peak position of the reflected laser light 7 detected by the image sensor 23. It is a figure for demonstrating a calculation formula.
In FIG.
n: Ratio of refractive index of measured object to surrounding medium x: Distance between center position of lens and peak position of diffused light detected by image sensor f: Distance between lens and image sensor B: Irradiation laser light Distance from the center position to the lens center position l: Distance from the lens to the back surface of the object to be measured z: Distance from the back surface of the object to be measured to the measurement surface t: Emission of light transmitted from the measurement surface through which diffused light has passed through the back surface Angle i is an incident angle of the diffused light from the measurement surface to the back surface.
[0017]
The following equation (1) is obtained from the law of refraction, and the following equations (2) and (3) are obtained from the geometric relationship.
Then, when i and t are eliminated from the equations (1) to (3) and solved for Z, the equation (4) is obtained.
The height Z (that is, the distance from the back surface of the object to be measured to the measurement surface) can be calculated by the equation (4).
[0018]
[Expression 1]
Figure 0003711892
[0019]
As described above, in the three-dimensional surface shape measurement method according to the first embodiment, the shape of the surface (measurement surface) is measured by irradiating laser light from the back surface side of the object to be measured whose back surface is flat. Then, laser light is incident on the high refractive index medium side from the low refractive index medium side, and on the contrary, the laser light is incident on the measurement surface from the high refractive medium side.
As a result, the diffuse reflectance on the measurement surface is higher than that on the back surface due to the difference in the refraction effect, so that no erroneous measurement is performed with the reflected light from the back surface, and a stable surface shape can be measured.
[0020]
Embodiment 2. FIG.
FIG. 5 is a diagram for explaining a three-dimensional surface shape measuring method according to the second embodiment.
In the figure, 1 is a laser displacement meter, 2 is an object to be measured, 21 is a laser irradiator, 5 is a laser beam emitted from the laser irradiator 21, 16 is a surface (measurement surface) of the object to be measured 2, and 17 is an object to be measured. The back surface of the measurement object 2, 6 is a reflected laser light in which the laser light 5 emitted from the laser irradiator 21 is diffusely reflected by the back surface 17, and 7 is a surface of the measurement object 2 on the surface (measurement surface) 16 of the measurement object 2. Reflected laser light diffusely reflected on the surface inside the measurement object, 8 is a light intensity waveform, 22 is a lens, 23 is an image sensor, and 25 is a movement for moving the laser displacement meter 1 in the xy direction within a predetermined plane. It is a mechanism (XY stage).
[0021]
Reference numeral 18 denotes a laser beam transmitted through the object to be measured 2, reference numeral 19 denotes a reflected laser beam reflected from the object other than the object to be measured, and reference numeral 20 denotes a fixed object to be measured 2. It is a fixing member.
In FIG. 5, the laser displacement meter 1 is attached to the moving mechanism (XY stage) 25 side, and the laser displacement meter 1 is moved relative to the object to be measured within a plane parallel to the back surface 17 of the object 2 to be measured. As in the case of the first embodiment, the moving mechanism (XY stage) 25 is configured so that the measured object 2 and the laser displacement meter are in a plane parallel to the back surface 17 of the measured object 2. The laser displacement meter 1 is fixed at a predetermined position, and the fixing member 20 for fixing the object 2 to be measured is attached to the moving mechanism (XY stage) 25 and moved. Needless to say.
[0022]
In the three-dimensional surface shape measuring method according to the present embodiment, the reflected laser beam 19 reflected by something other than the measured object 2 (for example, the fixing member 20) of the laser beam 18 transmitted through the measured object 2 is an image. A sufficient space is provided between the surface (measurement surface) 16 of the object to be measured 2 and the fixing member 20 so as not to enter the field of view 24 of the sensor.
By arranging the device under test 2 in this way, the laser light 18 that has passed through the device under test 2 is diffusely reflected by something other than the device under test (for example, the fixing member 20), and reflected laser light that is disturbance light. Even if 19 occurs, the reflected laser light 19 is outside the field of view 24 of the image sensor 23, and the reflected laser light 19 is not detected by the image sensor 23.
Accordingly, it is possible to prevent erroneous measurement due to the influence of the reflected light disturbance of the laser beam 18 that has passed through the DUT 2, and stable three-dimensional measurement of the surface shape can be performed.
[0023]
Embodiment 3 FIG.
FIG. 6 is a diagram illustrating an example of a window of a median filter (also referred to as a median filter) used in the three-dimensional surface shape measurement method according to the third embodiment.
In the three-dimensional surface shape measuring method according to the first or second embodiment described above, the moving mechanism (XY stage) 25 is used to place the laser displacement meter 1 with respect to the object 2 to be measured and a plane parallel to the back surface 17 of the object 2 to be measured. It has a structure that can move relatively in the xy direction.
Therefore, for example, measurement points are set in an n × m grid pattern in a plane parallel to the xy plane, and the height (cutting) from the back surface to the surface (measurement surface) of the DUT 2 is measured at each measurement point. ) Can be measured to measure a three-dimensional surface shape.
The measurement data may include data that is suddenly measured incorrectly due to the influence of foreign matter or the like attached to the back surface of the object to be measured.
[0024]
The three-dimensional surface shape measurement method according to the present embodiment is characterized in that the above-described problems are solved by subjecting measurement data to median filtering.
In the example of FIG. 6, the window of the median filter is 3 × 3, and the value z (i, j) of the target measurement point is obtained by rearranging the total 9 points of data and its neighboring 8 points. Replace with the value of the target point.
Note that the value z (i, j) of the target measurement point is the surface (from the back surface 17 of the DUT 2 at x = i, y = j on the xy plane (the plane parallel to the back surface 17 of the DUT 2). It is the distance to (measurement surface) 16.
By performing such median filter processing, abnormal data suddenly generated due to the influence of foreign matter or the like adhering to the back surface 17 of the object to be measured 2 can be replaced with surrounding normal data. A high three-dimensional shape measurement result can be obtained.
[0025]
In the present embodiment, a 3 × 3 median filter is used. However, when an extraordinary value removal effect is required, a median filter having a larger window such as 5 × 5 may be used.
In the present embodiment, the case where the object to be measured is acrylic has been described. However, since the refractive index of glass, for example, is about 1.5, which is larger than air, three-dimensional shape measurement is possible in the same manner.
As described above, in the three-dimensional surface shape measurement method according to the third embodiment, the result data measured by the three-dimensional surface shape measurement method according to the first or second embodiment is replaced with the median value of adjacent or nearby measurement data. By performing the median filter processing, it is possible to remove data that has been suddenly measured erroneously due to adhered foreign matter or the like, and more stable three-dimensional surface shape measurement can be performed.
[0026]
【The invention's effect】
According to the three-dimensional surface shape measuring method of the present invention, the laser beam emitted from the laser irradiator is irradiated onto the transparent object to be measured whose back surface is flat, and the reflected laser beam is condensed by the lens. A laser displacement meter that measures the amount of displacement in a direction perpendicular to the back surface of the object to be measured based on the displacement of the peak position of the reflected laser beam received, and in a plane parallel to the back surface of the object to be measured In a three-dimensional surface shape measurement method for measuring the shape of the surface of a measurement object using a moving mechanism that changes the relative position between the measurement object and a laser displacement meter, the laser light from the laser irradiator is measured by the measurement object. The object to be measured is placed so that it is perpendicularly incident on the back surface of the laser, the measurement point is moved while changing the relative position of the object to be measured and the laser displacement meter by the moving mechanism, and the laser beam from the laser irradiator is measured at each measurement point. Based on the displacement of the peak position of the reflected laser beam that is incident from the back surface of the object to be measured and reflected by the surface inside the object to be measured, the amount of displacement in the direction perpendicular to the back surface of the object to be measured is sequentially Since the surface shape of the object to be measured is measured by measuring, the diffuse reflected light on the surface (that is, the measurement surface) is stronger than the diffuse reflected light on the back surface of the transparent object to be measured. It is possible to prevent erroneous measurement with the diffuse reflected light, and the three-dimensional surface shape of the transparent object to be measured can be stably measured.
[0027]
Further, according to the three-dimensional surface shape measuring method of the present invention, the laser beam that has passed through the object to be measured and reflected by an object other than the object to be measured is prevented from entering the field of view of the image sensor. It is possible to prevent erroneous measurement due to the influence of the reflected light disturbance of the laser beam 18 that has passed through the measurement object, and the three-dimensional surface shape of the transparent measurement object can be stably measured.
[0028]
Further, according to the three-dimensional surface shape measuring method according to the present invention, the displacement amount data in the direction perpendicular to the back surface of the measurement object surface at the measurement target point is transferred to the back surface of the measurement object surface at the measurement point near the measurement target point. Since the median filtering process is performed to replace the median value of the displacement data in the vertical direction, it is possible to remove data that has been accidentally measured erroneously due to foreign matter adhering to the back surface of the object to be measured. The three-dimensional surface shape of a transparent object to be measured can be measured.
[Brief description of the drawings]
FIG. 1 is a diagram for explaining a three-dimensional surface shape measuring method according to a first embodiment.
FIG. 2 is a diagram for explaining refraction and reflection of light incident on a boundary surface of media having different refractive indexes.
FIG. 3 is a diagram for explaining the reflection of light incident on a surface having minute irregularities.
FIG. 4 is a diagram for explaining a calculation formula for calculating a distance from the back surface of the object to be measured to the surface that is a measurement surface;
FIG. 5 is a diagram for explaining a three-dimensional surface shape measuring method according to a second embodiment.
FIG. 6 is a diagram for explaining a median filter used in the three-dimensional surface shape measurement method according to the second embodiment.
FIG. 7 is a diagram showing the measurement principle of a laser displacement meter.
FIG. 8 is a diagram for explaining disturbance of reflected light from the back surface when measuring the surface shape of a transparent body.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 Laser displacement meter 2 Object to be measured 5 Laser beam emitted from laser irradiator 6 Reflected laser beam reflected on the back surface of the object to be measured 7 Reflected laser beam reflected on the surface inside the object to be measured on the surface of the object to be measured 8 Light intensity waveform 9 Incident light 10 Transmitted light 11 Reflected light 12 Boundary surface 13 Concavity and convexity 14 Light incident on the flat portion 15 Light incident on the concave and convex portion 16 Surface (measurement surface) 17 Back surface 18 Laser light 19 transmitted through the object to be measured Reflected laser light 20 reflected by something other than the object to be measured 20 Member for fixing the object to be measured 21 Laser irradiator 22 Lens 23 Image sensor 24 Field of view of image sensor 25 Moving mechanism (XY stage)

Claims (3)

レーザ照射器から出射されるレーザ光を裏面が平面の透明な被測定物に照射すると共に、その反射レーザ光をレンズで集光し、イメージセンサで受光し、その受光した反射レーザ光のピーク位置の変位に基づき、被測定物表面の裏面に垂直な方向の変位量を測定するレーザ変位計と、前記被測定物の裏面に平行な平面内において前記被測定物と前記レーザ変位計との相対位置を変化させる移動機構とを用いて、前記被測定物の表面の形状を測定する3次元表面形状測定方法において、
前記レーザ照射器からのレーザ光が前記被測定物の裏面に垂直に入射するように前記被測定物を配置し、
前記移動機構により前記被測定物と前記レーザ変位計の相対位置を変化させながら測定点を移動し、各測定点において前記レーザ照射器からのレーザ光を前記被測定物の裏面より入射させ、前記被測定物表面の被測定物内部側の面で反射する反射レーザ光のピーク位置の変位に基づき、被測定物表面の裏面に垂直な方向の変位量を順次測定することにより、被測定物の表面形状を測定することを特徴とする3次元表面形状測定方法。
The laser beam emitted from the laser irradiator irradiates a transparent object to be measured with a flat back surface, and the reflected laser beam is collected by a lens, received by an image sensor, and the peak position of the received reflected laser beam. A laser displacement meter that measures the amount of displacement in a direction perpendicular to the back surface of the object to be measured based on the displacement of the object, and a relative relationship between the object to be measured and the laser displacement meter in a plane parallel to the back surface of the object to be measured. In a three-dimensional surface shape measuring method for measuring the shape of the surface of the object to be measured using a moving mechanism that changes the position,
Arranging the device under test so that the laser beam from the laser irradiator is perpendicularly incident on the back surface of the device under test,
The measurement point is moved while changing the relative position of the object to be measured and the laser displacement meter by the moving mechanism, and laser light from the laser irradiator is incident from the back surface of the object to be measured at each measurement point, Based on the displacement of the peak position of the reflected laser beam reflected on the surface of the object to be measured on the surface inside the object to be measured, by sequentially measuring the amount of displacement in the direction perpendicular to the back surface of the object to be measured, A three-dimensional surface shape measuring method, characterized by measuring a surface shape.
被測定物を透過し、前記被測定物以外の物で反射したレーザ光が、イメージセンサの視野内に入らないようにしたことを特徴とする請求項1に記載の3次元表面形状測定方法。2. The three-dimensional surface shape measuring method according to claim 1, wherein laser light that has passed through the object to be measured and reflected by an object other than the object to be measured does not enter the field of view of the image sensor. 測定対象点における被測定物表面の裏面に垂直な方向の変位量データを、前記測定対象点の近傍測定点における被測定物表面の裏面に垂直な方向の変位量データの中央値に置き換える中央値フィルタ処理を行うことを特徴とする請求項1または2に記載の3次元表面形状測定方法。Median value for replacing displacement amount data in the direction perpendicular to the back surface of the object surface at the measurement target point with the median value of displacement amount data in the direction perpendicular to the back surface of the object surface at the measurement point near the measurement object point The three-dimensional surface shape measuring method according to claim 1 or 2, wherein filtering is performed.
JP2001149056A 2001-05-18 2001-05-18 3D surface shape measurement method Expired - Fee Related JP3711892B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2001149056A JP3711892B2 (en) 2001-05-18 2001-05-18 3D surface shape measurement method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
JP2001149056A JP3711892B2 (en) 2001-05-18 2001-05-18 3D surface shape measurement method

Publications (2)

Publication Number Publication Date
JP2002340533A JP2002340533A (en) 2002-11-27
JP3711892B2 true JP3711892B2 (en) 2005-11-02

Family

ID=18994275

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2001149056A Expired - Fee Related JP3711892B2 (en) 2001-05-18 2001-05-18 3D surface shape measurement method

Country Status (1)

Country Link
JP (1) JP3711892B2 (en)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2004351427A (en) * 2003-05-27 2004-12-16 Tamagawa Seiki Co Ltd Method for automatically correcting stop at top dead center of press machine
JP5666870B2 (en) * 2009-11-09 2015-02-12 シャープ株式会社 Optical distance measuring device, electronic apparatus, and optical distance measuring device calibration method
JP5335709B2 (en) * 2010-01-28 2013-11-06 株式会社日立ハイテクノロジーズ Method and apparatus for measuring track error of concrete track
CN102322815B (en) * 2011-06-12 2013-08-14 浙江省计量科学研究院 High-precision and high-volume measurement device and method based on three-dimensional laser scanning

Also Published As

Publication number Publication date
JP2002340533A (en) 2002-11-27

Similar Documents

Publication Publication Date Title
JP5595693B2 (en) System and method for determining shape of plate glass
CN101641566B (en) Measuring instrument and method for determining geometric properties of profiled elements
EP2426457A2 (en) Inspection device for defect inspection
JP2010513925A (en) A method for automatically and quantitatively analyzing the distortion of molded glass for vehicles using reflected optical images.
JP2009139248A (en) Defect detecting optical system and surface defect inspecting device for mounting defect detecting image processing
US9797833B2 (en) Method for determining the refractive power of a transparent object, and corresponding device
JP4568800B2 (en) Droplet state measuring apparatus and camera calibration method in the apparatus
JP2012021781A (en) Method and device for evaluating surface shape
CN111288902B (en) Double-field-of-view optical coherence tomography imaging system and material thickness detection method
TW201732263A (en) Method and system for optical three-dimensional topography measurement
US20150177160A1 (en) Non-Imaging Coherent Line Scanner Systems and Methods for Optical Inspection
US10508950B2 (en) Transparent measuring probe for beam scanning
KR20210087541A (en) Device and method for optical measurement of the inner contour of a spectacle frame
KR101875467B1 (en) 3-dimensional shape measurment apparatus and method thereof
JP2012127675A (en) Method and apparatus for evaluating front-surface shape
JP3711892B2 (en) 3D surface shape measurement method
JP6485616B2 (en) Appearance measurement system, image processing method, and program
JP4864734B2 (en) Optical displacement sensor and displacement measuring apparatus using the same
TWI553291B (en) System for measuring transparent object by fringe projection
JP4683270B2 (en) Lens meter
JP2001066127A (en) Optical surface inspecting mechanism and its device
JP2013029462A (en) Flaw measuring method
KR101833055B1 (en) 3-dimensional measuring system
JP7332417B2 (en) Measuring device and measuring method
JP4007473B2 (en) Wavefront shape measurement method

Legal Events

Date Code Title Description
RD01 Notification of change of attorney

Free format text: JAPANESE INTERMEDIATE CODE: A7421

Effective date: 20040708

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20050627

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20050726

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20050808

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20080826

Year of fee payment: 3

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20090826

Year of fee payment: 4

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20090826

Year of fee payment: 4

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20100826

Year of fee payment: 5

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20110826

Year of fee payment: 6

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20110826

Year of fee payment: 6

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20120826

Year of fee payment: 7

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20120826

Year of fee payment: 7

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20130826

Year of fee payment: 8

LAPS Cancellation because of no payment of annual fees