CN104618650A - Movement detection method and device through combination between front image acquisition device and side image acquisition device - Google Patents
Movement detection method and device through combination between front image acquisition device and side image acquisition device Download PDFInfo
- Publication number
- CN104618650A CN104618650A CN201510050451.0A CN201510050451A CN104618650A CN 104618650 A CN104618650 A CN 104618650A CN 201510050451 A CN201510050451 A CN 201510050451A CN 104618650 A CN104618650 A CN 104618650A
- Authority
- CN
- China
- Prior art keywords
- target
- collecting device
- monitored
- motion
- side image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Image Analysis (AREA)
- Closed-Circuit Television Systems (AREA)
Abstract
The invention provides a movement detection method through combination between a front image acquisition device and a side image acquisition device. The movement detection method comprises the following steps that images in a target area are acquired by the front image acquisition device and the side image acquisition device at the same time; the front image acquisition device acquires the front image, and detects the first movement direction of a target, to be monitored, in the front image; the side image acquisition device acquires the side image, and detects the second movement direction of the target, to be monitored, in the side image; when the first movement direction and the second movement direction are the vertical movement direction, the front image acquisition device conducts close-up operation on the target to be monitored. The invention further provides a movement detection device through combination between the front image acquisition device and the side image acquisition device. The movement detection device comprises the front image acquisition device, the side image acquisition device and a control device, wherein the control device is connected with the front image acquisition device and connected with the side image acquisition device.
Description
Technical field
The invention belongs to technical field of image processing, particularly relate to a kind of front, side image collecting device associating method for testing motion and equipment.
Background technology
In Online class, need to carry out the shooting of camera lens feature to move both vertically (such as the taking the floor) of the personnel of listening to the teacher.Require that system accurately can detect the information that moves both vertically of object like this.Usually the acquisition of these information depends on algorithm and carries out moving object detection and analysis to the image collected, and provide accurately positional information to control the feature shooting of camera lens and to follow the tracks of.
In order to effectively gather image, two side cameras of installation front camera or symmetry are usually adopted to gather image.When an employing front camera collection image, front camera cannot be distinguished moving both vertically of moving object and move along the depth of camera optical axis direction.When two side cameras by symmetry gather image, the associated working by the camera of two sides calculates the position of moving object, but this method needs to carry out coordinate system transformation, and computational process is complicated.
Summary of the invention
The invention provides a kind of front, side image collecting device associating method for testing motion and equipment, to solve the problem.
The invention provides the method for testing motion of a kind of front, the associating of side image collecting device.Said method comprises the following steps: direct picture collecting device and side image collecting device gather the image of target area simultaneously; Described direct picture collecting device gathers direct picture, and detects first direction of motion of target to be monitored in described direct picture; Described side image collecting device gathers side image, and detects second direction of motion of target to be monitored in described side image; When described first direction of motion and described second direction of motion be move both vertically direction time, described direct picture collecting device carries out feature shooting to described target to be monitored.
The present invention also provides the motion detection device of a kind of front, the associating of side image collecting device, and the said equipment comprises: direct picture collecting device, side image collecting device, control appliance; Wherein, described control appliance is connected with described side image collecting device between two with described direct picture collecting device; Described direct picture collecting device, for gathering the direct picture of target area, and detects first direction of motion of target to be monitored in described direct picture, will detect that first direction of motion of target to be monitored sends to described control appliance; Described direct picture collecting device is also for carrying out feature shooting to described target to be monitored; Described side image collecting device, for gathering the side image of target area, and detects second direction of motion of target to be monitored in described side image, will detect that second direction of motion of target to be monitored sends to described control appliance; Described control appliance, for receiving first direction of motion of the target to be monitored that described direct picture collecting device sends, and receive second direction of motion of the target to be monitored that described side image collecting device sends, when described first direction of motion and described second direction of motion be move both vertically direction time, described control appliance controls described direct picture collecting device and carries out feature shooting to described target to be monitored.
Compared to prior art, according to front provided by the invention, the method for testing motion of side image collecting device associating and equipment, the present invention adopts direct picture collecting device and side image collecting device Collect jointly and detection, the detection that monitoring objective carries out the direction of motion is treated by direct picture collecting device, verified the direction of motion of target to be monitored by side image collecting device simultaneously, when the direction of motion of the target to be monitored that direct picture collecting device and side image collecting device detect is vertical direction, treat monitoring objective again and carry out feature shooting.Such testing process avoids and moves both vertically because single direct picture collecting device detects to distinguish and to move the flase drop brought along the depth of camera optical axis direction, improve the accuracy rate detected, simultaneously, method of the present invention does not need complex process side coordinate system being mapped to front coordinate system in implementation process, improves the efficiency of detection.
Accompanying drawing explanation
Accompanying drawing described herein is used to provide a further understanding of the present invention, and form a application's part, schematic description and description of the present invention, for explaining the present invention, does not form inappropriate limitation of the present invention.In the accompanying drawings:
Figure 1 shows that the flow chart of method for testing motion of front that preferred embodiment according to the present invention provides, the associating of side image collecting device;
Figure 2 shows that the structure chart of motion detection device of front that preferred embodiment according to the present invention provides, the associating of side image collecting device;
Figure 3 shows that the installation site schematic diagram of the direct picture collecting device that preferred embodiment according to the present invention provides and side image collecting device.
Embodiment
Hereinafter also describe the present invention in detail with reference to accompanying drawing in conjunction with the embodiments.It should be noted that, when not conflicting, the embodiment in the application and the feature in embodiment can combine mutually.
Be illustrated in figure 1 the flow chart of method for testing motion of front that preferred embodiment according to the present invention provides, the associating of side image collecting device.As shown in Figure 1, preferred embodiment of the present invention provide front, side image collecting device associating method for testing motion comprise step 101-104.
Step 101: direct picture collecting device and side image collecting device gather the image of target area simultaneously;
Described side image collecting device is one or more.
The setting height(from bottom) of described direct picture collecting device is the height higher than human body when standing.
The setting height(from bottom) of described side image collecting device is the height higher than human body during sitting, and is less than the height of human body when standing.
Step 102: described direct picture collecting device gathers direct picture, and detects first direction of motion of target to be monitored in described direct picture;
When comprising multiple moving target in described direct picture, by Feature Points Matching algorithm and/or coordinate position matching algorithm, the moving target of the moving target in described direct picture with described side image is mated, using match and the moving target all with the direction that moves both vertically as target described to be monitored to be tracked.
Determine that in image, the method for moving target comprises: frame differential method, background subtraction method and optical flow method.
Step 103: described side image collecting device gathers side image, and detects second direction of motion of target to be monitored in described side image;
Step 104: when described first direction of motion and described second direction of motion be move both vertically direction time, described direct picture collecting device carries out feature shooting to described target to be monitored.
The direction of motion of described target to be monitored be move both vertically direction and be more than or equal to preset value at the described move distance moved both vertically on direction time, described direct picture collecting device carries out feature shooting to described target to be monitored.
Described feature shooting is to target following to be monitored described in image, and adjusting focal length bust shot.
Early warning height is set, if when the height of described target to be monitored is greater than early warning height, described direct picture collecting device carries out feature shooting to described target to be monitored.
Citing is below described in detail:
In Online class, as shown in Figure 3, install front camera and side camera, 1 front camera is installed on the dead ahead in classroom, 1 side camera is installed on the side in classroom.Wherein, the setting height(from bottom) of front camera is the standing height higher than the highest student of height in classroom, and front camera is the image that depression angle gathers student; The setting height(from bottom) H of side camera is the sitting height higher than student the highest during sitting in classroom, is less than the standing height of the student that height is the shortest when standing simultaneously.By camera collection classroom, front middle school student's regional display, obtain the image A of 680 × 480 pixel sizes, and the direction of motion of image A middle school student M (target to be monitored) detected.While front camera carries out the collection of student's regional display, side camera also gathers the picture in student region simultaneously, obtains the image B of 680 × 480 pixel sizes, and the student M in detected image B.
Then, the testing result of front camera is verified in conjunction with side camera.For front camera, the motion that stand up motion and the motion of front and back depth of student M are all thought in vertical direction.Suppose that student M actual motion is front and back depth motions, now the testing result of front camera moves both vertically.The testing result of side camera is front and back depth motions.Because both result of determination are inconsistent, therefore, judge that this student M is not in vertical motion, front camera is without the need to carrying out feature shooting to student M.And when the actual motion of student M be move both vertically, the testing result of front camera moves both vertically, and the testing result of side camera also moves both vertically, and both testing results are identical, then judge that student M is in vertical motion, front camera carries out feature shooting to student M.The testing result of therefore using side camera can help front camera to tell the situation of depth motion before and after moving target, can not judge by accident, cause tracking error.
In addition, determining that student M is when being in vertical motion, also needing to judge whether the amplitude that moves both vertically of student M is greater than preset value, if be greater than preset value, then carry out feature shooting to this student M, front camera adjusting focal length amplifies the image of student M.Like this, the little trick of student M can be prevented and cause the tracking action of camera, thus reducing rubbish camera lens.
In the above embodiments, in the image that camera collects, only have a moving target, if the image gathered has multiple moving target, following method will be adopted process in the present embodiment.
Suppose front camera collection to image A in comprise two moving targets, moving target 1 and moving target 2 respectively, simultaneously, side camera collection to image B in also comprise two moving targets, need to distinguish the moving target of two in image B, in two moving targets of resolution image B, which is the moving target 1 in image A, which is the moving target 2 in image A, for Feature Points Matching algorithm, the moving target 1 how determined in image B and moving target 2 are described in detail below, to front camera collection to image A in moving target 1 and and moving target 2 carry out characteristic pattern collection, the characteristic pattern obtaining moving target 1 is cross, the characteristic pattern of moving target 2 is triangle, respectively using cruciform pattern and triangle pattern as characteristic body template to side camera collection to image B mate, find out pixel identical with cruciform pattern or triangle pattern pixel in image B, if obtaining in image B with same pixel number in cruciform pattern is 350, and judge this same pixel number be greater than preset value (this preset value be pre-set for judging the critical value whether two images mate, such as 320), then determine moving target 1 in image B, in like manner, determine moving target 2 in image B.Respectively the direction of motion of the moving target 1 in image A and image B and moving target 2 is judged further.Judge when passing through, when obtaining the moving target 1 in image A and the moving target 1 in image B all for moving both vertically, meanwhile, when the moving target 2 when the moving target 2 in image A is for moving both vertically and in image B is for horizontal movement, camera is pursuit movement target 1 only, carries out feature shooting.When the moving target 1 in image A and the moving target 1 in image B are all for moving both vertically, simultaneously, when the moving target 2 in image A and the moving target 2 in image B are all for moving both vertically, camera pursuit movement target 1 and moving target 2, carry out feature shooting.
In another preferred embodiment, side camera collection to image B in arrange early warning height, the position of early warning height is the sitting height higher than student the highest during sitting in classroom, be less than the standing height of the student that height is the shortest when standing simultaneously, early warning height is exceeded when there being student in image B, front camera follows the tracks of the student exceeding early warning height, carries out feature shooting.
Figure 2 shows that the motion detection device structure chart of front that preferred embodiment according to the present invention provides, the associating of side image collecting device.As shown in Figure 2, the motion detection device of what preferred embodiment of the present invention provided be applied to front, the associating of side image collecting device, the said equipment comprises: direct picture collecting device 201, side image collecting device 202, control appliance 203; Wherein, described control appliance 203 is connected with described side image collecting device 202 between two with described direct picture collecting device 201; Described direct picture collecting device 201, for gathering the direct picture of target area, and detects first direction of motion of target to be monitored in described direct picture, will detect that first direction of motion of target to be monitored sends to described control appliance 203; Described direct picture collecting device 201 is also for carrying out feature shooting to described target to be monitored; Described side image collecting device 202, for gathering the side image of target area, and detects second direction of motion of target to be monitored in described side image, will detect that second direction of motion of target to be monitored sends to described control appliance 203; Described control appliance 203, for receiving first direction of motion of the target to be monitored that described direct picture collecting device 201 sends, and receive second direction of motion of the target to be monitored that described side image collecting device 202 sends, when described first direction of motion and described second direction of motion be move both vertically direction time, described control appliance 203 controls described direct picture collecting device 201 and carries out feature shooting to described target to be monitored.
In addition, about the specific operation process of the said equipment with described in said method, therefore repeat no more in this.
In sum, the method for testing motion of front of the present invention, the associating of side image collecting device and equipment, the present invention adopts direct picture collecting device and side image collecting device Collect jointly and detection, the detection that monitoring objective carries out the direction of motion is treated by direct picture collecting device, verified the direction of motion of target to be monitored by side image collecting device simultaneously, when the direction of motion of the target to be monitored that direct picture collecting device and side image collecting device detect is vertical direction, then treats monitoring objective and carry out feature shooting.Such testing process avoids and moves both vertically because single direct picture collecting device detects to distinguish and to move the flase drop brought along the depth of image capture device optical axis direction, improve the accuracy rate detected, simultaneously, method of the present invention does not need complex process side coordinate system being mapped to front coordinate system in implementation process, improves the efficiency of detection.
The foregoing is only the preferred embodiments of the present invention, be not limited to the present invention, for a person skilled in the art, the present invention can have various modifications and variations.Within the spirit and principles in the present invention all, any amendment done, equivalent replacement, improvement etc., all should be included within protection scope of the present invention.
Claims (10)
1. a method for testing motion for front, the associating of side image collecting device, is characterized in that, comprise the following steps:
Direct picture collecting device and side image collecting device gather the image of target area simultaneously;
Described direct picture collecting device gathers direct picture, and detects first direction of motion of target to be monitored in described direct picture;
Described side image collecting device gathers side image, and detects second direction of motion of target to be monitored in described side image;
When described first direction of motion and described second direction of motion be move both vertically direction time, described direct picture collecting device carries out feature shooting to described target to be monitored.
2. method according to claim 1, it is characterized in that, the direction of motion of described target to be monitored be move both vertically direction and be more than or equal to preset value at the described move distance moved both vertically on direction time, described direct picture collecting device carries out feature shooting to described target to be monitored.
3. method according to claim 1, it is characterized in that, when comprising multiple moving target in described direct picture, by Feature Points Matching algorithm and/or coordinate position matching algorithm, the moving target of the moving target in described direct picture with described side image is mated, using match and the moving target all with the direction that moves both vertically as target described to be monitored to be tracked.
4. method according to claim 3, is characterized in that, determines that in image, the method for moving target comprises: frame differential method, background subtraction method and optical flow method.
5. method according to claim 1, is characterized in that, described side image collecting device is one or more.
6. method according to claim 1, is characterized in that, described feature shooting is to target following to be monitored described in image, and adjusting focal length bust shot.
7. method according to claim 1, is characterized in that, the setting height(from bottom) of described direct picture collecting device is the height higher than human body when standing.
8. method according to claim 1, is characterized in that, the setting height(from bottom) of described side image collecting device is the height higher than human body during sitting, and is less than the height of human body when standing.
9. method according to claim 1, is characterized in that, arranges early warning height, if when the height of described target to be monitored is greater than early warning height, described direct picture collecting device carries out feature shooting to described target to be monitored.
10. a motion detection device for front, the associating of side image collecting device, is characterized in that, comprising: direct picture collecting device, side image collecting device, control appliance; Wherein, described control appliance is connected with described side image collecting device between two with described direct picture collecting device;
Described direct picture collecting device, for gathering the direct picture of target area, and detects first direction of motion of target to be monitored in described direct picture, will detect that first direction of motion of target to be monitored sends to described control appliance; Described direct picture collecting device is also for carrying out feature shooting to described target to be monitored;
Described side image collecting device, for gathering the side image of target area, and detects second direction of motion of target to be monitored in described side image, will detect that second direction of motion of target to be monitored sends to described control appliance;
Described control appliance, for receiving first direction of motion of the target to be monitored that described direct picture collecting device sends, and receive second direction of motion of the target to be monitored that described side image collecting device sends, when described first direction of motion and described second direction of motion be move both vertically direction time, described control appliance controls described direct picture collecting device and carries out feature shooting to described target to be monitored.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510050451.0A CN104618650A (en) | 2015-01-30 | 2015-01-30 | Movement detection method and device through combination between front image acquisition device and side image acquisition device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510050451.0A CN104618650A (en) | 2015-01-30 | 2015-01-30 | Movement detection method and device through combination between front image acquisition device and side image acquisition device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN104618650A true CN104618650A (en) | 2015-05-13 |
Family
ID=53152914
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510050451.0A Pending CN104618650A (en) | 2015-01-30 | 2015-01-30 | Movement detection method and device through combination between front image acquisition device and side image acquisition device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104618650A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107328361A (en) * | 2017-07-19 | 2017-11-07 | 苏州经贸职业技术学院 | A kind of base station terrace parameter measurement system and method |
| CN110119692A (en) * | 2019-04-19 | 2019-08-13 | 华南理工大学 | A kind of plane portrait detection method based on dual camera |
| CN111265835A (en) * | 2020-03-03 | 2020-06-12 | 淮安信息职业技术学院 | Throwing exercise training equipment and control method thereof |
| CN114071133A (en) * | 2022-01-14 | 2022-02-18 | 苏州高视半导体技术有限公司 | Semiconductor chip detection image imaging method |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101068342A (en) * | 2007-06-05 | 2007-11-07 | 西安理工大学 | Video frequency motion target close-up trace monitoring method based on double-camera head linkage structure |
| JP2009077092A (en) * | 2007-09-20 | 2009-04-09 | Hitachi Ltd | Multi-camera system |
| CN102063607A (en) * | 2009-11-16 | 2011-05-18 | 日电（中国）有限公司 | Method and system for acquiring human face image |
| CN102096930A (en) * | 2011-01-30 | 2011-06-15 | 吴柯维 | Student standing and sitting detection method for intelligent recorded broadcasting system for teaching |
| CN102096812A (en) * | 2011-01-30 | 2011-06-15 | 吴柯维 | Teacher blackboard writing action detection method for intelligent teaching recording and playing system |
| CN102622766A (en) * | 2012-03-01 | 2012-08-01 | 西安电子科技大学 | Multi-objective optimization multi-lens human motion tracking method |
| CN103327250A (en) * | 2013-06-24 | 2013-09-25 | 深圳锐取信息技术股份有限公司 | Method for controlling camera lens based on pattern recognition |
| CN103686092A (en) * | 2013-12-30 | 2014-03-26 | 深圳锐取信息技术股份有限公司 | Dual system detection method and system for rising of students |
-
2015
- 2015-01-30 CN CN201510050451.0A patent/CN104618650A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101068342A (en) * | 2007-06-05 | 2007-11-07 | 西安理工大学 | Video frequency motion target close-up trace monitoring method based on double-camera head linkage structure |
| JP2009077092A (en) * | 2007-09-20 | 2009-04-09 | Hitachi Ltd | Multi-camera system |
| CN102063607A (en) * | 2009-11-16 | 2011-05-18 | 日电（中国）有限公司 | Method and system for acquiring human face image |
| CN102096930A (en) * | 2011-01-30 | 2011-06-15 | 吴柯维 | Student standing and sitting detection method for intelligent recorded broadcasting system for teaching |
| CN102096812A (en) * | 2011-01-30 | 2011-06-15 | 吴柯维 | Teacher blackboard writing action detection method for intelligent teaching recording and playing system |
| CN102622766A (en) * | 2012-03-01 | 2012-08-01 | 西安电子科技大学 | Multi-objective optimization multi-lens human motion tracking method |
| CN103327250A (en) * | 2013-06-24 | 2013-09-25 | 深圳锐取信息技术股份有限公司 | Method for controlling camera lens based on pattern recognition |
| CN103686092A (en) * | 2013-12-30 | 2014-03-26 | 深圳锐取信息技术股份有限公司 | Dual system detection method and system for rising of students |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107328361A (en) * | 2017-07-19 | 2017-11-07 | 苏州经贸职业技术学院 | A kind of base station terrace parameter measurement system and method |
| CN110119692A (en) * | 2019-04-19 | 2019-08-13 | 华南理工大学 | A kind of plane portrait detection method based on dual camera |
| CN111265835A (en) * | 2020-03-03 | 2020-06-12 | 淮安信息职业技术学院 | Throwing exercise training equipment and control method thereof |
| CN114071133A (en) * | 2022-01-14 | 2022-02-18 | 苏州高视半导体技术有限公司 | Semiconductor chip detection image imaging method |
| CN114071133B (en) * | 2022-01-14 | 2022-05-24 | 苏州高视半导体技术有限公司 | Semiconductor chip detection image imaging method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10452931B2 (en) | Processing method for distinguishing a three dimensional object from a two dimensional object using a vehicular system | |
| CN105631390B (en) | Method and system for spatial finger positioning | |
| WO2014155979A1 (en) | Tracking processing device and tracking processing system provided with same, and tracking processing method | |
| CN103716594A (en) | Panorama splicing linkage method and device based on moving target detecting | |
| CN104966062A (en) | Video monitoring method and device | |
| CN105447853A (en) | Flight device, flight control system and flight control method | |
| WO2015024407A1 (en) | Power robot based binocular vision navigation system and method based on | |
| CN104954747B (en) | Video monitoring method and device | |
| CN104102069B (en) | A kind of focusing method of imaging system and device, imaging system | |
| CN103716595A (en) | Linkage control method and device for panoramic mosaic camera and dome camera | |
| KR20110012626A (en) | Location and distance mesuring appratus its method usnig camera | |
| CN101902616A (en) | Quick stereo positioning method for video monitoring | |
| CN104618650A (en) | Movement detection method and device through combination between front image acquisition device and side image acquisition device | |
| WO2017175470A1 (en) | Speckle measurement device and speckle measurement method | |
| CN102354364B (en) | Three-dimensional barrier detecting method of monitoring system with single video camera | |
| CN109308718A (en) | A kind of space personnel positioning apparatus and method based on more depth cameras | |
| CN104751146A (en) | Indoor human body detection method based on 3D (three-dimensional) point cloud image | |
| CN104952104A (en) | Three-dimensional human body gesture estimating method and device thereof | |
| KR20130114899A (en) | Image sensing method using dual camera and apparatus thereof | |
| CN106524995B (en) | Detect the localization method of target object space length in real time based on visible images | |
| JP2009210331A (en) | Camera calibration apparatus and camera calibration method | |
| KR101469099B1 (en) | Auto-Camera Calibration Method Based on Human Object Tracking | |
| KR101620580B1 (en) | Method and system for dectecting run | |
| CN103323216A (en) | Device and method for detecting parallax error of collimator | |
| CN201839377U (en) | Whole scene infrared separation automatic tracking device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C53 | Correction of patent of invention or patent application | ||
| CB02 | Change of applicant information |
Address after: Suzhou City, Jiangsu province 215121 Suzhou Industrial Park Road No. 8 only Weiting Technology Venture Building 7 floor Applicant after: Kuo Di education and science company limited Address before: Suzhou City, Jiangsu province 215121 Suzhou Industrial Park Road No. 8 only Weiting Technology Venture Building 7 floor Applicant before: Suzhou Kuodi Network Technology Co., Ltd. |
|
| COR | Change of bibliographic data |
Free format text: CORRECT: APPLICANT; FROM: SUZHOU KUODI NETWORK TECHNOLOGY CO., LTD. TO: CODYY EDUCATION TECHNOLOGY CO., LTD. |
|
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20150513 |
|
| WD01 | Invention patent application deemed withdrawn after publication |