WO2022078437A1

WO2022078437A1 - Three-dimensional processing apparatus and method between moving objects

Info

Publication number: WO2022078437A1
Application number: PCT/CN2021/123774
Authority: WO
Inventors: 左忠斌; 左达宇
Original assignee: 左忠斌
Priority date: 2020-10-15
Filing date: 2021-10-14
Publication date: 2022-04-21

Abstract

Embodiments of the present invention provide a device and method for realizing three-dimensional (3D) processing between moving objects. The device comprises a target object 3D information collection device and a first processor; the target object 3D information collection device is used for continuously scanning objects to be processed when said objects move, and at a plurality of moments, separately obtaining a plurality of images that can be synthesized into a 3D model; and the first processor is used for performing calculation on the 3D models of at least two objects to be processed at the same moment. It is proposed for the first time that the 3D model construction of the objects is achieved by means of a visual 3D collection device having a simple structure, and thus, mutual matching between the objects can be achieved, the collection speed is high, and the precision is high.

Description

A three-dimensional processing device and method between moving objects

technical field

The invention relates to the technical field of topography measurement, in particular to the technical field of 3D topography measurement.

Background technique

At present, object-to-object matching is required in many scenarios, such as the use of helicopters for maritime rescue, which requires the helicopter and the surface boat to align with each other; or two trucks that need to relay cargoes. That is to say, for two objects that are in different motions at the same time, they are in motion with each other, so that when aligning, docking, inserting and other actions are to be performed, clear images of both are required for further operations. In the prior art, two-dimensional images are used, which brings about large errors. Of course, in the prior art, a combination of a two-dimensional image and a rangefinder is also used, but the problems of complicated equipment, high cost, and difficult operation are caused. Therefore, there is an urgent need for a device and method for 3D acquisition through a simple structure to achieve fast and accurate matching between objects.

In addition, when collecting 3D information, the commonly used methods include the use of machine vision, structured light, laser ranging, and lidar. Structured light, laser ranging, and lidar all require an active light source to be emitted to the target, which will affect the target in some cases, and the cost of the light source is high. Moreover, the structure of the light source is relatively precise and easy to be damaged. The machine vision method is to collect pictures of objects from different angles, and match and stitch these pictures to form a 3D model, which is low-cost and easy to use. When collecting pictures from different angles, multiple cameras can be set at different angles of the object to be tested, or pictures can be collected from different angles by rotating a single or multiple cameras. But no matter which of the two methods, the acquisition position of the camera needs to be set around the target (referred to as the surround type), but this method requires a large space to set the acquisition position for the image acquisition device. The applicable scenarios are extremely limited.

In the prior art, it has also been proposed to use an empirical formula including rotation angle, target size, and object distance to define the camera position, so as to take into account the synthesis speed and effect. In practice, however, this is found to be feasible in wrap-around 3D acquisition, where the target size can be measured in advance. However, in an open space, it is difficult to measure the target in advance. For example, it is necessary to collect and obtain 3D information of streets, traffic intersections, buildings, tunnels, traffic flows, etc. (not limited to this). This makes this approach ineffective. Even small fixed targets, such as furniture, human body parts, etc., can be measured in advance, but this method is still limited: the size of the target is difficult to accurately determine, especially in some applications. Frequent replacements bring a lot of extra work per measurement and require specialized equipment to accurately measure irregular targets. The measurement error leads to the camera position setting error, which will affect the acquisition and synthesis speed and effect; the accuracy and speed need to be further improved.

Although the prior art also has methods for optimizing the wrap-around capturing device, when the capturing direction of the camera of the 3D capturing and synthesizing device deviates from the direction of its rotation axis, there is no better optimization method in the prior art.

Therefore, there is an urgent need for a technology that can collect spatial 3D information accurately, efficiently, and is applicable to a wide range of scenarios, and match and optimize it with the 3D information of the object, so as to guide the matching action between the object and the space.

SUMMARY OF THE INVENTION

In view of the above problems, it is proposed that the present invention provides an apparatus and method for realizing 3D processing between moving objects that overcomes the above problems or at least partially solves the above problems.

Embodiments of the present invention provide a device and method for realizing 3D processing between moving objects, including a device for collecting 3D information of a target object and a first processor;

The target object 3D information acquisition device is used to continuously scan the object to be processed when the object to be processed moves, and obtain multiple images that can synthesize the three-dimensional model at multiple times;

The first processor is configured to perform a matching operation or a mismatch operation on the three-dimensional models of the at least two objects to be processed at the same time.

In an optional embodiment, the first processor is further configured to synthesize the three-dimensional model of the object to be processed.

In an optional embodiment, it further includes a second processor, located in the target object 3D information acquisition device, for synthesizing a three-dimensional model of the object to be processed opposite to it;

In an optional embodiment, when the apparatus for collecting the 3D information of the target object collects the above-mentioned image of the object to be processed at any time, the image is stamped with a time stamp.

In an optional embodiment, the spatial 3D information acquisition device includes an image acquisition device and a rotation device;

The image acquisition device is connected with the rotating device, and the rotating device drives it to rotate;

The angle α between the optical axes of the image acquisition device at two adjacent acquisition positions satisfies the following conditions:

Among them, R is the distance from the rotation center to the surface of the target object, T is the sum of the object distance and the image distance during acquisition, d is the length or width of the photosensitive element of the image acquisition device, F is the lens focal length of the image acquisition device, and u is the experience coefficient.

In an optional embodiment, u<0.498, for better synthesis effect, preferably u<0.411, especially preferably u<0.359, in some applications, u<0.281, or u<0.169, or u<0.041, or u<0.028.

In an optional embodiment, the optical acquisition ports of the image acquisition device are all facing away from the direction of the rotation axis.

In an optional embodiment, the device for collecting 3D information of the target object is arranged on the object to be processed.

In an optional embodiment, the device for collecting 3D information of the target object is provided separately from the object to be processed.

In an optional embodiment, after the processing is completed, the processor outputs the processing result to the display device, the printing device, and/or the action execution device.

Inventions and technical effects

1. For the first time, it is proposed to construct a 3D model of an object through a visual 3D acquisition device with a simple structure, so that the mutual matching or mismatch between objects can be realized, and the acquisition speed is fast and the precision is high.

2. For the first time, it is proposed to optimize the camera acquisition position by measuring the distance between the rotation center and the target, and the distance between the image sensing element and the target, so as to take into account the speed and effect of 3D construction.

3. Through the self-rotating acquisition device, it is possible to quickly scan the object without being limited by the occasion, so that the matching or mismatching between objects is easier to achieve.

4. By flexibly setting the location of the acquisition device (located outside the target or on the target), it can be applied to different occasions.

5. Ensure the accuracy and speed of 3D synthesis by means of time stamps.

Description of drawings

Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are for the purpose of illustrating preferred embodiments only and are not to be considered limiting of the invention. Also, the same components are denoted by the same reference numerals throughout the drawings. In the attached image:

FIG. 1 shows a schematic structural diagram of a 3D information collection device provided by an embodiment of the present invention;

FIG. 2 shows another schematic structural diagram of a 3D information collection device provided by an embodiment of the present invention;

FIG. 3 shows a schematic structural diagram of a 3D information collection device including a telescopic device provided by an embodiment of the present invention.

FIG. 4 shows another schematic structural diagram of a 3D information collection device including a telescopic device provided by an embodiment of the present invention.

FIG. 5 shows a schematic structural diagram of a 3D information collection device including a pitch device provided by an embodiment of the present invention.

FIG. 6 shows another schematic structural diagram of a 3D information collection device including a pitching device provided by an embodiment of the present invention.

FIG. 7 shows a schematic structural diagram of a 3D information collection device including a telescopic and tilt device provided by an embodiment of the present invention.

FIG. 8 shows another schematic structural diagram of a 3D information collection device including a telescopic and tilt device provided by an embodiment of the present invention.

FIG. 9 shows a schematic diagram of a 3D information acquisition device located outside a target object for acquisition and matching provided by an embodiment of the present invention.

FIG. 10 shows another schematic diagram of the 3D information collection apparatus provided by an embodiment of the present invention for collection and matching outside the target object.

FIG. 11 shows a schematic diagram of a 3D information acquisition device located on a target object for acquisition and matching provided by an embodiment of the present invention.

FIG. 12 shows a schematic diagram of a mismatch between 3D information collection provided by an embodiment of the present invention that is located outside the target object.

FIG. 13 shows a schematic diagram of a 3D information collection apparatus provided by an embodiment of the present invention in which a collection apparatus is located on a target object and collects mismatches.

The corresponding relationship between the reference numerals in the accompanying drawings and the components is as follows:

1 image acquisition device;

2 rotating device;

3 carrying device;

4 telescopic device;

5 pitching device;

6 3D information collection device.

Detailed ways

Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure will be more thoroughly understood, and will fully convey the scope of the present disclosure to those skilled in the art.

3D information acquisition device structure

In order to solve the above technical problems, the present invention provides a 3D information acquisition device, as shown in FIG.

The image acquisition device 1 is connected with the rotating shaft of the rotating device 2 , and the rotating device 2 drives it to rotate. The acquisition direction of the image acquisition device 1 is the direction away from the rotation center. That is, the acquisition direction is directed outward relative to the center of rotation. The optical axis of the image acquisition device 1 may be parallel to the rotation plane, or may form a certain angle with the rotation plane, for example, within the range of -90°-90° based on the rotation plane. Usually, the rotation axis or its extension line (ie, the rotation center line) passes through the image acquisition device, that is, the image acquisition device still rotates in an autorotation manner. This is essentially different from the acquisition method (surround type) in which the traditional image acquisition device rotates around a certain object, that is, it is completely different from the surround type in which the image acquisition device rotates around the target object. The optical collection ports (eg lenses) of the image collection device are all facing away from the direction of the rotation axis, that is to say, the collection area of the image collection device has no intersection with the rotation center line. Optionally, the optical axis of the image acquisition device has an included angle with the horizontal plane, so this method is also quite different from the general autorotation method, especially the target object whose surface is not perpendicular to the horizontal plane can be acquired.

Of course, the rotating shaft of the rotating device can also be connected to the image capturing device through a deceleration device, for example, through a gear set or the like. When the image capturing device rotates 360° on the horizontal plane, it captures an image corresponding to the target at a specific position (the specific shooting position will be described in detail later). This shooting can be performed in synchronization with the rotation action, or after the shooting position stops rotating, and then continues to rotate after shooting, and so on. The above-mentioned rotating device may be a motor, a motor, a stepping motor, a servo motor, a micro motor, or the like. The rotating device (for example, various types of motors) can rotate at a specified speed under the control of the controller, and can rotate at a specified angle, so as to realize the optimization of the collection position. The specific collection position will be described in detail below. Of course, the rotating device in the existing equipment can also be used, and the image capturing device can be installed thereon.

The carrying device 3 is used to carry the weight of the entire equipment, and the rotating device 2 is connected with the carrying device 3 . The carrying device can be a tripod as shown in Figure 1, a base with a support device as shown in Figure 2, and so on. Typically, the rotating device is located in the center part of the carrier to ensure balance. However, in some special occasions, it can also be located at any position of the carrying device. Furthermore, the carrying device is not necessary. The swivel device can be installed directly in the application, eg on the roof of a vehicle.

In another embodiment, as shown in FIGS. 3 and 4 , an image capturing device 1 , a rotating device 2 , a carrying device 3 , and a telescopic device 4 are included.

The image acquisition device 1 is connected with the rotating shaft of the rotating device 2, and the rotating device 2 drives it to rotate. Of course, the rotating shaft of the rotating device can also be connected to the image capturing device through a deceleration device, for example, through a gear set or the like. When the image capturing device rotates 360° on the horizontal plane, it captures an image corresponding to the target at a specific position (the specific shooting position will be described in detail later). This shooting can be performed in synchronization with the rotation action, or after the shooting position stops rotating, and then continues to rotate after shooting, and so on. The above-mentioned rotating device may be a motor, a motor, a stepping motor, a servo motor, a micro motor, or the like. The rotating device (for example, various types of motors) can rotate at a specified speed under the control of the controller, and can rotate at a specified angle, so as to realize the optimization of the collection position. The specific collection position will be described in detail below. Of course, the rotating device in the existing equipment can also be used, and the image capturing device can be installed thereon.

One end of the telescopic device 4 is connected to the rotating device 2, and the other end is connected to the bearing device 3, and is used to expand and contract in a direction perpendicular to the optical axis of the image capture device, so that the image capture device can be positioned at different positions. At each position, it is rotated and scanned by the rotating device, so that a 3D model of the target at that position can be constructed. After scanning a certain position, the telescopic device moves again, so that the image acquisition device moves to another position, repeating the above scanning, and so on, to realize the construction of the internal 3D model of the slender target. It can also be used to scan at different height levels when the surrounding target is high, so as to construct a 3D model of the entire target.

The telescopic device can be various telescopic structures such as telescopic sleeves and telescopic slide rails. Its telescoping can be adjusted manually or under the control of the control unit. The telescopic device may also include a telescopic motor for driving the telescopic unit (eg, a telescopic sleeve) to extend or shorten. After telescopic in place, the length of the telescopic device can be locked by the locking unit to provide stable support for the rotating device. The locking unit may be a mechanical locking unit, such as a locking pin, etc., or an electric locking unit, for example, under the control of the control unit, to lock the telescopic device.

The carrying device 3 is used to carry the weight of the entire device. The carrying device may be a tripod, a base with a supporting device, or the like. Typically, the rotating device is located in the center part of the carrier to ensure balance. However, in some special occasions, it can also be located at any position of the carrying device. Furthermore, the carrying device is not necessary. The rotating device can be installed directly in the application equipment, for example, it can be installed on the top of the walking robot.

Through the design of the telescopic device, the image acquisition device can collect information at different heights, so that for buildings with high indoor ceilings, comprehensive and accurate acquisition can be achieved.

In another embodiment, as shown in FIGS. 5 and 6 , the 3D information acquisition device includes an image acquisition device 1 , a rotation device 2 , a carrying device 3 , and a pitch device 5 . The image acquisition device 1 is arranged on the tilt device 5, so that the image acquisition device 1 can tilt and rotate along the vertical plane. The pitching device can be rollers, gears, bearings, ball joints, etc. The optical axis of the image acquisition device 1 is usually parallel to the pitch direction, but may also be at a certain angle in some special cases. The pitching device can be adjusted manually, or it can be pitched and rotated under the drive of the motor, so as to realize the precise pitch angle adjustment according to the program control. The tilting device further includes a locking mechanism for locking the tilting device after the tilting angle is adjusted in place and the optical axis of the image capturing device is at a predetermined angle with the horizontal plane, thereby preventing it from rotating in the vertical direction again.

The pitching device 5 is connected with the rotating shaft of the rotating device 2 , and is driven by the rotating device 2 to rotate. Of course, the rotating shaft of the rotating device can also be connected to the pitching device through a reduction gear, for example, through a gear set or the like.

Due to the adjustment of the tilting device, the optical axis of the image acquisition device usually forms a certain angle with the horizontal plane. This allows scanning of targets whose surfaces are not perpendicular to the horizontal. That is, according to the approximate angle between the surface of the target object and the horizontal plane, the tilting device is adjusted so that the optical axis of the image acquisition device is perpendicular to the surface of the target object as much as possible, so as to improve the acquisition accuracy of the details of the target object. Of course, it can also be parallel to the horizontal plane in special cases.

Through the design of the pitch angle, when the field of view of the image acquisition device cannot cover all spaces in the room, especially the upper space, the image acquisition device can be properly tilted upward, thereby making the acquisition range larger.

In another embodiment, as shown in FIGS. 7 and 8 , the telescopic device 4 and the pitching device 5 may be included at the same time. That is, the image capturing device 1 is installed on the pitching device 5 , the pitching device 5 is connected to the rotating device 2 , the rotating device 2 is installed on one end of the telescopic device 4 , and the other end of the telescopic device 4 is installed on the carrying device 3 . In this way, when the indoor space is large and high (such as a church), the image acquisition device can be positioned at different heights in turn through the telescopic rod, and then scanned and acquired in sequence, or the pitch angle can be adjusted to make the image acquisition device Collect more upper space information. Of course, both can be used at the same time depending on the situation.

For all the above embodiments, the acquisition direction of the image acquisition device is the direction away from the rotation center. That is, the acquisition direction is directed outward relative to the center of rotation. The optical axis of the image acquisition device may be parallel to the rotation plane, or may form a certain angle with the rotation plane, for example, within the range of -90°-90° based on the rotation plane. Usually, the rotation axis or its extension line (ie, the rotation center line) passes through the image acquisition device, that is, the image acquisition device still rotates in an autorotation manner. This is essentially different from the acquisition method (surround type) in which the traditional image acquisition device rotates around a certain object, that is, it is completely different from the surround type in which the image acquisition device rotates around the target object. The optical collection ports (eg lenses) of the image collection device are all facing away from the direction of the rotation axis, that is to say, the collection area of the image collection device has no intersection with the rotation center line. Optionally, the optical axis of the image acquisition device has an included angle with the horizontal plane, so this method is also quite different from the general autorotation method, especially the target object whose surface is not perpendicular to the horizontal plane can be acquired.

The above device may further include a ranging device, the ranging device is fixedly connected with the image acquisition device, and the pointing direction of the ranging device is the same as the direction of the optical axis of the image acquisition device. Of course, the distance measuring device can also be fixedly connected to the rotating device, as long as it can rotate synchronously with the image capturing device. Preferably, an installation platform may be provided, the image acquisition device and the distance measuring device are both located on the platform, the platform is installed on the rotating shaft of the rotating device, and is driven and rotated by the rotating device. The distance measuring device can use a variety of methods such as a laser distance meter, an ultrasonic distance meter, an electromagnetic wave distance meter, etc., or a traditional mechanical measuring tool distance measuring device. Of course, in some applications, the 3D acquisition device is located at a specific location, and its distance from the target has been calibrated, and no additional measurement is required.

It can also include a light source, and the light source can be arranged on the periphery of the image acquisition device, on the rotating device and on the installation platform. Of course, the light source can also be set independently, for example, an independent light source is used to illuminate the target. Even when lighting conditions are good, no light source is used. The light source can be an LED light source or an intelligent light source, that is, the parameters of the light source are automatically adjusted according to the conditions of the target object and the ambient light. Usually, the light sources are distributed around the lens of the image capture device, for example, the light sources are ring-shaped LED lights around the lens. Because in some applications it is necessary to control the intensity of the light source. In particular, a diffuser device, such as a diffuser housing, can be arranged on the light path of the light source. Or directly use the LED surface light source, not only the light is softer, but also the light is more uniform. More preferably, an OLED light source can be used, which has a smaller volume, softer light, and has flexible properties, which can be attached to a curved surface.

In order to facilitate the measurement of the actual size of the target, multiple marking points can be set at the position of the target. And the coordinates of these marker points are known. By collecting marker points and combining their coordinates, the absolute size of the 3D composite model is obtained. These marking points can be pre-set points or laser light spots. The method for determining the coordinates of these points may include: ①Using laser ranging: using a calibration device to emit laser light toward the target to form a plurality of calibration point spots, and obtain the calibration point coordinates through the known positional relationship of the laser ranging unit in the calibration device. Use the calibration device to emit laser light toward the target, so that the light beam emitted by the laser ranging unit in the calibration device falls on the target to form a light spot. Since the laser beams emitted by the laser ranging units are parallel to each other, and the positional relationship between the units is known. Then the two-dimensional coordinates on the emission plane of the multiple light spots formed on the target can be obtained. By measuring the laser beam emitted by the laser ranging unit, the distance between each laser ranging unit and the corresponding light spot can be obtained, that is, depth information equivalent to multiple light spots formed on the target can be obtained. That is, the depth coordinates perpendicular to the emission plane can be obtained. Thereby, the three-dimensional coordinates of each spot can be obtained. ②Using the combination of distance measurement and angle measurement: respectively measure the distance of multiple markers and the angle between each other, so as to calculate the respective coordinates. ③ Use other coordinate measurement tools: such as RTK, global coordinate positioning system, star-sensing positioning system, position and pose sensors, etc.

3D information collection and matching process

The so-called matching of two objects means that the space occupied by the two objects has an intersection, or a stable relationship is formed between the spaces occupied by the two objects. For example, the refueling gun is inserted into the refueling port, the marine helicopter is aimed at the sea to be rescued, and the drone is landed on the top of the moving vehicle.

1. The acquisition device is located outside the target object

As shown in Figure 9-10:

(1) Place the 3D acquisition device 6 at a certain position in space to ensure that the object to be matched (hereinafter referred to as the target) is within its scanning field of view.

(2) At time T0, the acquisition device 6 acquires images of multiple objects, and performs 3D synthesis respectively, so as to obtain 3D models of multiple objects in a relatively short period of time. The specific method is as follows:

The rotating device drives the image acquisition device to rotate at a certain speed, and the image acquisition device performs image acquisition at a set position during the rotation process. At this time, the rotation may not be stopped, that is, the image acquisition and the rotation are performed synchronously; or the rotation may be stopped at the position to be acquired, image acquisition is performed, and the rotation continues to the next position to be acquired after the acquisition is completed. The rotating device can be driven by a pre-programmed control unit program. It can also communicate with the upper computer through the communication interface, and control the rotation through the upper computer. In particular, it can also be wired or wirelessly connected to the mobile terminal, and the rotation of the rotating device can be controlled by the mobile terminal (eg, a mobile phone). That is, the rotation parameters of the rotating device can be set through the remote platform, cloud platform, server, host computer, and mobile terminal to control the start and stop of its rotation.

The image acquisition device collects multiple images of the target object, and sends the images to the remote platform, cloud platform, server, host computer and/or mobile terminal through the communication device, and uses the 3D model synthesis method to synthesize the 3D model of the target object. Synthesize the 3D model of the object to be matched. It can be understood that the speed of the entire acquisition and synthesis is very fast.

(3) At time T1, the acquisition device acquires images of multiple objects, and performs 3D synthesis respectively, thereby obtaining 3D models of multiple objects again. It can be understood that the time interval between T0 and T1 may be shorter or longer as required. The shortest interval is: the acquisition device keeps rotating, so that the acquisition is uninterrupted.

(4) By analogy, images of multiple objects are collected at different times, and 3D synthesis is performed respectively, so as to continuously obtain 3D models of multiple objects. In this process, the 3D model information of the target is continuously used to match different targets. For example, when the two trucks are handing over the goods, the 3D models of the two vehicles are continuously obtained, so as to continuously adjust their actions according to the latest vehicle 3D model and its position and attitude information to ensure the alignment of the two cargo boxes, so as to smoothly Complete the handover.

In the above collection process, for the space that needs to be collected at multiple heights, the following process can be used.

The length of the telescopic device is controlled so that the image acquisition device is located at a predetermined position, the rotating device drives the image acquisition device to rotate at a certain speed, and the image acquisition device performs image acquisition at the set position during the rotation. At this time, the rotation may not be stopped, that is, the image acquisition and the rotation are performed synchronously; or the rotation may be stopped at the position to be acquired, image acquisition is performed, and the rotation continues to the next position to be acquired after the acquisition is completed. The rotating device can be driven by a pre-programmed control unit program. It can also communicate with the upper computer through the communication interface, and control the rotation through the upper computer. In particular, it can also be wired or wirelessly connected to the mobile terminal, and the rotation of the rotating device can be controlled by the mobile terminal (eg, a mobile phone). That is, the rotation parameters of the rotating device can be set through the remote platform, cloud platform, server, host computer, and mobile terminal to control the start and stop of its rotation.

The length of the telescopic device is controlled so that the image acquisition device is located at another predetermined position, and the above-mentioned action of the rotating device is repeated, so that the image acquisition device can acquire the image of the target object surrounding the position, and so on, and the acquisition is performed at multiple height positions to obtain images, thereby Build the corresponding 3D model.

The image acquisition device collects multiple images of the target, and sends the images to the remote platform, cloud platform, server, host computer and/or mobile terminal through the communication device, and uses the 3D model synthesis method to perform 3D synthesis of the target.

Of course, the tilting device can also be controlled so that the image acquisition device is tilted to a certain angle, and then rotated and acquired.

In particular, the distance measuring device can be used to measure the corresponding distance parameters in the relevant formula conditions, that is, the distance from the rotation center to the target, and the distance from the sensing element to the target, before or at the same time as the acquisition. The collection position is calculated according to the corresponding conditional formula, and the user is prompted to set the rotation parameters, or the rotation parameters are automatically set.

When measuring the distance before acquisition, the rotating device can drive the distance measuring device to rotate, so as to measure the above two distances at different positions. The two distances measured at multiple measurement points are averaged respectively, and are brought into the formula as the unified distance value collected this time. The average value may be obtained by a summation average method, a weighted average method, or another average value method, or a method of discarding abnormal values and averaging again.

When the distance measurement is performed during the acquisition process, while the rotating device is rotated to the first position for image acquisition, the above-mentioned two distance values are measured, and they are brought into the conditional formula to calculate the interval angle, and the next step is determined according to the angle. Collection location.

2. The acquisition device is located on the target object

In many occasions, there is no additional space or position to place the acquisition device. For this reason, the present invention proposes a solution as shown in Figure 11 (which is also one of the inventions):

(1) The above-mentioned 3D information acquisition device 6 is set on the object to be matched (referred to as a target object), so that the matched object is within its scanning field of view. Usually, at least one 3D information acquisition device is provided on each target. However, in some cases, if the 3D model or information of one of the targets is known, or can be checked in real time, it is only necessary to set the acquisition device on it.

(2) At time T0, the acquisition device collects the image corresponding to the target, and performs 3D synthesis to obtain a 3D model of the target. If two objects to be matched with each other are provided with acquisition devices, each acquisition device acquires an image of the object opposite to it. If a collection device is provided on one of the two objects to be matched with each other, it collects the image of the object opposite to it, and the 3D model of the object collected by it is known and can be given by querying the system. The specific collection and synthesis methods are the same as those described above, and are not repeated here.

(3) At time T1, the acquisition device acquires an image corresponding to the target object, and performs 3D synthesis to obtain a 3D model of the target object. If two objects to be matched with each other are provided with acquisition devices, each acquisition device acquires an image of the object opposite to it. If a collection device is provided on one of the two objects to be matched with each other, it collects the image of the object opposite to it, and the 3D model of the object collected by it is known and can be given by querying the system. The specific collection and synthesis methods are the same as those described above, and are not repeated here. It can be understood that the time interval between T0 and T1 may be shorter or longer as required. The shortest interval is: the acquisition device keeps rotating, so that the acquisition is uninterrupted.

(4) By analogy, the images of the target objects are collected at different times, and 3D synthesis is performed respectively, so as to continuously obtain 3D models of multiple targets. In this process, the 3D model information of the target is continuously used to match different targets. For example, when the two trucks are handing over the goods, the 3D models of the two vehicles are continuously obtained, so as to continuously adjust their actions according to the latest vehicle 3D model and its position and attitude information to ensure the alignment of the two cargo boxes, so as to smoothly Complete the handover.

The difference here is that the matching process can be completed in the acquisition devices of the two objects or in one of the devices respectively, and can also be completed in a server or a cloud platform. That is to say, any acquisition device can independently obtain the 3D model of the opposite target (synthesize it internally, or send it to the device after being synthesized by a third-party server or cloud platform), and can receive or query its own target. The 3D model of the object can be adjusted according to the 3D model of the two objects and their position and attitude information inside the device, so as to achieve the matching of the two objects. In another case, each device can independently synthesize the 3D models of the objects it faces, and send them to the third-party server and cloud platform, so as to realize the integration of the two objects in the third-party server and cloud platform. match. In another case, each device sends the collected pictures to a third-party server and cloud platform, so as to realize the synthesis of 3D models of two objects in the third-party server and cloud platform, and complete the matching at the same time.

In the above, in each acquisition, the time stamp of the acquisition time can be stamped on the image, so as to determine which images were acquired at the same time, and only the images with the same (or very similar) time stamps are used for the 3D model synthesis of the target at this time. .

It can be understood that the above-mentioned matching may be between two objects, or may be a match between one object and two objects, or may be a match between multiple objects and multiple objects.

3D information collection does not match the process

When performing a mismatch operation between objects, it is actually to avoid matching between objects. To truly achieve accurate and timely mismatch, it is also necessary to first model the objects in 3D, so as to avoid objects colliding during the movement process (occupying spaces with intersection). The so-called mismatch means that the space occupied by two objects has no intersection. For example, vehicles stop when pedestrians pass by; vehicles bypass other vehicles to overtake; drones avoid obstacles, etc.

1. The acquisition device is located outside the target object

In some cases, the acquisition device cannot be set on the object. For example, in the intersection monitoring scene, not all vehicles can be equipped with 3D acquisition equipment, which will bring a large cost. At this point, the scheme of Figure 12 can be used:

(1) Place the 3D information acquisition device 6 at a certain position in space to ensure that the object to be mismatched (hereinafter referred to as the target) is within its scanning field of view.

(2) At time T0, the acquisition device collects images of multiple objects, and performs 3D synthesis respectively, thereby obtaining 3D models of multiple objects. The specific method is as follows:

The rotating device drives the image acquisition device to rotate at a certain speed, and the image acquisition device performs image acquisition at a set position during the rotation process. At this time, the rotation may not be stopped, that is, the image acquisition and the rotation are performed synchronously; or the rotation may be stopped at the position to be acquired, image acquisition is performed, and the rotation continues to the next position to be acquired after the acquisition is completed. The rotating device can be driven by a pre-programmed control unit program. It can also communicate with the host computer through the communication interface, and control the rotation through the host computer. In particular, it can also be wired or wirelessly connected with a mobile terminal, and the rotation of the rotating device can be controlled by the mobile terminal (eg, a mobile phone). That is, the rotation parameters of the rotating device can be set through the remote platform, cloud platform, server, host computer, and mobile terminal to control the start and stop of its rotation.

The image acquisition device collects multiple images of the target object, and sends the images to the remote platform, cloud platform, server, host computer and/or mobile terminal through the communication device, and uses the 3D model synthesis method to synthesize the 3D model of the target object. Synthesize 3D models of objects to be mismatched. It can be understood that the speed of the entire acquisition and synthesis is very fast.

(4) By analogy, images of multiple objects are collected at different times, and 3D synthesis is performed respectively, so as to continuously obtain 3D models of multiple objects. In this process, the 3D model information of the target is continuously used to perform mismatch operations on different targets. For example, during the driving process, an autonomous vehicle continuously obtains the 3D model of the vehicle and its surrounding vehicles, as well as the 3D models of obstacles such as trees, so as to continuously adjust its actions according to the latest vehicle 3D model and its position and attitude information. , to ensure that the vehicle does not collide with other vehicles and obstacles (mismatch).

It should be noted that the avoided objects include moving objects and non-moving objects, and the moving objects may be in a moving state or a non-moving state.

The length of the telescopic device is controlled so that the image acquisition device is located at a predetermined position, the rotating device drives the image acquisition device to rotate at a certain speed, and the image acquisition device performs image acquisition at the set position during the rotation process. At this time, the rotation may not be stopped, that is, the image acquisition and the rotation are performed synchronously; or the rotation may be stopped at the position to be acquired, image acquisition is performed, and the rotation continues to the next position to be acquired after the acquisition is completed. The rotating device can be driven by a pre-programmed control unit program. It can also communicate with the host computer through the communication interface, and control the rotation through the host computer. In particular, it can also be wired or wirelessly connected with a mobile terminal, and the rotation of the rotating device can be controlled by the mobile terminal (eg, a mobile phone). That is, the rotation parameters of the rotating device can be set through the remote platform, cloud platform, server, host computer, and mobile terminal to control the start and stop of its rotation.

In particular, the distance measuring device can be used to measure the corresponding distance parameters in the relevant formula conditions, that is, the distance from the rotation center to the target, and the distance from the sensing element to the target, before or at the same time as the acquisition. Calculate the collection position according to the corresponding conditional formula, and prompt the user to set the rotation parameters, or automatically set the rotation parameters.

2. The acquisition device is located on the target object

In many occasions, there is no additional space or position to place the acquisition device. For this reason, the present invention proposes a solution as shown in Figure 13 (which is also one of the inventions):

(1) The above-mentioned 3D information acquisition device 6 is set on the object to be mismatched (referred to as the target), so that the mismatched object is within the scanning field of view. Usually, at least one 3D information acquisition device is provided on each target. However, in some cases, if the 3D model or information of one of the targets is known, or can be checked in real time, it is only necessary to set the acquisition device on it.

(2) At time T0, the acquisition device collects the image corresponding to the target, and performs 3D synthesis to obtain a 3D model of the target. If two objects to be mismatched with each other are provided with acquisition devices, each acquisition device acquires an image of the object opposite to it. If a collection device is provided on one of the two objects that are to be mismatched with each other, it collects the image of the object opposite to it, and the 3D model of the object collected by it is known and can be given by the query system. The specific collection and synthesis methods are the same as those described above, and are not repeated here.

(3) At time T1, the acquisition device acquires an image corresponding to the target object, and performs 3D synthesis to obtain a 3D model of the target object. If two objects to be mismatched with each other are provided with acquisition devices, each acquisition device acquires an image of the object opposite to it. If a collection device is provided on one of the two objects that are to be mismatched with each other, it collects the image of the object opposite to it, and the 3D model of the object collected by it is known and can be given by the query system. The specific collection and synthesis methods are the same as those described above, and are not repeated here. It can be understood that the time interval between T0 and T1 may be shorter or longer as required. The shortest interval is: the acquisition device keeps rotating, so that the acquisition is uninterrupted.

(4) By analogy, the images of the target objects are collected at different times, and 3D synthesis is performed respectively, so as to continuously obtain 3D models of multiple targets. In this process, the 3D model information of the target is continuously used to match different targets. For example, during the driving process, an autonomous vehicle continuously obtains the 3D model of the vehicle and its surrounding vehicles, as well as the 3D models of obstacles such as trees, so as to continuously adjust its actions according to the latest vehicle 3D model and its position and attitude information. , to ensure that the vehicle does not collide with other vehicles and obstacles (mismatch).

The difference from the above is that the process of mismatching (avoiding matching) can be completed in the acquisition devices of the two objects or one of the devices respectively, and can also be completed in a server or a cloud platform. That is to say, any acquisition device can independently obtain the 3D model of the opposite target (synthesize it internally, or send it to the device after being synthesized by a third-party server or cloud platform), and can receive or query its own target. The 3D model of the object can be adjusted according to the 3D model of the two objects and their position and attitude information inside the device, so as to realize the mismatch between the two objects. In another case, each device can independently synthesize the 3D models of the objects it faces, and send them to the third-party server and cloud platform, so as to realize the integration of the two objects in the third-party server and cloud platform. Mismatch. In another case, each device sends the collected pictures to a third-party server and cloud platform, so as to realize the synthesis of 3D models of two objects in the third-party server and cloud platform, and complete the mismatch at the same time.

It can be understood that the above matching may be between two objects, or a mismatch between one object and two objects, or a mismatch between multiple objects and multiple objects.

Optimization of camera position

In order to ensure that the device can take into account the effect and efficiency of 3D synthesis, in addition to the conventional method of optimizing the synthesis algorithm, the method of optimizing the camera acquisition position can also be adopted. Especially when the acquisition direction of the camera of the 3D acquisition and synthesis device deviates from the direction of its rotation axis, the prior art for such a device does not mention how to better optimize the camera position. Even if some optimization methods exist, they are obtained under different empirical conditions under different experiments. In particular, some existing position optimization methods need to obtain the size of the target object, which is feasible in surround 3D acquisition and can be measured in advance. However, it is difficult to measure in advance in an open space. Therefore, it is necessary to propose a method for optimizing the camera position when the acquisition direction of the camera of the 3D acquisition and synthesis device and the direction of its rotation axis deviate from each other. This is exactly the problem to be solved by the present invention and the technical contribution made.

To this end, the present invention conducts a large number of experiments, and summarizes the following empirical conditions that the interval of camera acquisition is preferably satisfied during acquisition.

When performing 3D acquisition, the included angle α of the optical axis of the image acquisition device at two adjacent positions satisfies the following conditions:

in,

R is the distance from the center of rotation to the surface of the target,

T is the sum of the object distance and the image distance during acquisition, that is, the distance between the photosensitive unit of the image acquisition device and the target object.

d is the length or width of the photosensitive element (CCD) of the image acquisition device. When the above two positions are along the length direction of the photosensitive element, d is the length of the rectangle; when the above two positions are along the width direction of the photosensitive element, d is the rectangle. width.

F is the focal length of the lens of the image acquisition device.

u is the empirical coefficient.

Usually, a distance measuring device, such as a laser distance meter, is configured on the acquisition device. Adjust its optical axis to be parallel to the optical axis of the image acquisition device, then it can measure the distance from the acquisition device to the surface of the target object. Using the measured distance, according to the known positional relationship between the distance measuring device and the various components of the acquisition device, you can Get R and T.

When the image acquisition device is in any one of the two positions, the distance from the photosensitive element to the surface of the target object along the optical axis is taken as T. In addition to this method, multiple averaging methods or other methods can also be used. The principle is that the value of T should not deviate from the distance between the image and the object during acquisition.

In the same way, when the image acquisition device is in any one of the two positions, the distance from the center of rotation to the surface of the target object along the optical axis is taken as R. In addition to this method, multiple averaging methods or other methods can also be used, the principle of which is that the value of R should not deviate from the radius of rotation at the time of acquisition.

Usually, the size of the object is used as a method for estimating the position of the camera in the prior art. Because the size of the object will change with the change of the measured object. For example, after collecting 3D information of a large object, when collecting small objects, it is necessary to re-measure the size and re-calculate. The above-mentioned inconvenient measurements and multiple re-measurements will bring about measurement errors, resulting in incorrect camera position estimation. In this scheme, based on a large number of experimental data, the empirical conditions that the camera position needs to meet are given, and there is no need to directly measure the size of the object. In the empirical conditions, d and F are fixed parameters of the camera. When purchasing a camera and lens, the manufacturer will give the corresponding parameters without measurement. However, R and T are only a straight line distance, which can be easily measured by traditional measurement methods, such as straightedge and laser rangefinder. At the same time, in the apparatus of the present invention, the acquisition direction of the image acquisition device (eg, camera) is away from the direction of its rotation axis, that is, the orientation of the lens is substantially opposite to the rotation center. In this case, it is easier to control the angle α of the optical axis between the two positions of the image acquisition device, and it is only necessary to control the rotation angle of the rotary drive motor. Therefore, it is more reasonable to use α to define the optimal position. Therefore, the empirical formula of the present invention makes the preparation process convenient and quick, and also improves the arrangement accuracy of the camera position, so that the camera can be set in an optimized position, thereby taking into account the 3D synthesis accuracy and speed at the same time.

According to a large number of experiments, in order to ensure the speed and effect of synthesis, u should be less than 0.498. For better synthesis effect, u<0.411 is preferred, especially u<0.359. In some applications, u<0.281, or u<0.169, or u<0.041, or u<0.028.

Using the device of the present invention, experiments are carried out, and some experimental data are as follows, in mm. (The following data are only limited examples)

The above data are only obtained from experiments to verify the conditions of the formula, and do not limit the invention. Even the absence of these data does not affect the objectivity of the formula. Those skilled in the art can adjust the parameters of the equipment and the details of the steps to conduct experiments, and other data obtained are also in line with the conditions of the formula.

3D model synthesis method

The multiple images acquired by the image acquisition device are sent to the processing unit, and the following algorithm is used to construct a 3D model. The processing unit may be located in the acquisition device, or may be located remotely, such as a cloud platform, a server, a host computer, and the like.

The specific algorithm mainly includes the following steps:

Step 1: Perform image enhancement processing on all input photos. The following filters are used to enhance the contrast of the original photo and suppress noise at the same time.

In the formula: g(x, y) is the gray value of the original image at (x, y), f(x, y) is the gray value of the original image after enhancement by Wallis filter, and m _g is the local gray value of the original image. sg is the local grayscale standard deviation of the original image, _mf is the local _grayscale target value of the transformed image, and _sf is the localized grayscale standard deviation target value of the transformed image. c∈(0,1) is the expansion constant of the image variance, and b∈(0,1) is the image luminance coefficient constant.

The filter can greatly enhance the image texture patterns of different scales in the image, so it can improve the number and accuracy of feature points when extracting image point features, and improve the reliability and accuracy of matching results in photo feature matching.

Step 2: Extract feature points from all the input photos, and perform feature point matching to obtain sparse feature points. The SURF operator is used to extract and match the feature points of the photo. The SURF feature matching method mainly includes three processes, feature point detection, feature point description and feature point matching. This method uses Hessian matrix to detect feature points, uses Box Filters to replace second-order Gaussian filtering, uses integral image to accelerate convolution to improve calculation speed, and reduces the dimension of local image feature descriptors, to speed up matching. The main steps include ① constructing the Hessian matrix to generate all interest points for feature extraction. The purpose of constructing the Hessian matrix is to generate image stable edge points (mutation points); ② constructing the scale space feature point positioning, which will be processed by the Hessian matrix Each pixel point is compared with 26 points in the two-dimensional image space and scale space neighborhood, and the key points are initially located. (3) The main direction of the feature point is determined by using the harr wavelet feature in the circular neighborhood of the statistical feature point. That is, in the circular neighborhood of the feature points, the sum of the horizontal and vertical harr wavelet features of all points in the 60-degree sector is counted, and then the sector is rotated at intervals of 0.2 radians, and the harr wavelet eigenvalues in the region are counted again. The direction of the sector with the largest value is used as the main direction of the feature point; (4) a 64-dimensional feature point description vector is generated, and a 4*4 rectangular area block is taken around the feature point, but the direction of the obtained rectangular area is along the main direction of the feature point. direction. Each sub-region counts the haar wavelet features of 25 pixels in the horizontal and vertical directions, where the horizontal and vertical directions are relative to the main direction. The haar wavelet features are 4 directions after the horizontal value, after the vertical value, after the absolute value of the horizontal direction and the sum of the absolute value of the vertical direction. These 4 values are used as the feature vector of each sub-block area, so there are 4* 4*4=64-dimensional vector as the descriptor of the Surf feature; ⑤ Matching of feature points, the matching degree is determined by calculating the Euclidean distance between the two feature points. The shorter the Euclidean distance, the better the matching degree of the two feature points. .

Step 3: Input the coordinates of the matched feature points, and use the beam method to adjust the position and attitude data of the sparse target object 3D point cloud and the camera to obtain the sparse target object model 3D point cloud and position model coordinates. ; Take sparse feature points as the initial value, perform dense matching of multi-view photos, and obtain dense point cloud data. There are four main steps in this process: stereo pair selection, depth map calculation, depth map optimization, and depth map fusion. For each image in the input dataset, we select a reference image to form a stereo pair for computing the depth map. So we can get a rough depth map for all images, these depth maps may contain noise and errors, and we use its neighborhood depth map to perform a consistency check to optimize the depth map for each image. Finally, depth map fusion is performed to obtain a 3D point cloud of the entire scene.

Step 4: Use dense point cloud to reconstruct the target surface. Including several processes of defining octrees, setting function spaces, creating vector fields, solving Poisson equations, and extracting isosurfaces. The integral relationship between the sampling point and the indicator function is obtained from the gradient relationship, the vector field of the point cloud is obtained according to the integral relationship, and the approximation of the gradient field of the indicator function is calculated to form the Poisson equation. The approximate solution is obtained by matrix iteration according to the Poisson equation, the isosurface is extracted by the moving cube algorithm, and the model of the measured object is reconstructed from the measured point cloud.

Step 5: Fully automatic texture mapping of the target model. After the surface model is constructed, texture mapping is performed. The main process includes: ① texture data acquisition through image reconstruction of the target surface triangle mesh; ② visibility analysis of the reconstructed model triangle. Use the calibration information of the image to calculate the visible image set of each triangular face and the optimal reference image; 3. The triangular face is clustered to generate texture patches. According to the visible image set of the triangular surface, the optimal reference image and the neighborhood topology relationship of the triangular surface, the triangular surface is clustered into several reference image texture patches; ④The texture patches are automatically sorted to generate texture images. Sort the generated texture patches according to their size relationship, generate a texture image with the smallest enclosing area, and obtain the texture mapping coordinates of each triangular surface.

It should be noted that the above algorithm is the algorithm used in the present invention, the algorithm cooperates with the image acquisition conditions, and the time and quality of synthesis are taken into account when using this algorithm. However, it can be understood that conventional 3D synthesis algorithms in the prior art can also be used in conjunction with the solution of the present invention.

Applications

In order to realize the delivery of goods between the two trucks without stopping, the 3D information collection device of the present invention can be arranged on the roofs of the two trucks. Scan and collect the images of the opposite vehicles separately, and transmit them to the server. The server synthesizes the 3D models of the two vehicles at different times according to the multiple images, and at the same time, continuously adjusts the operation of the two trucks according to their 3D models and position and attitude information. Track and attitude, and finally realize the matching and alignment of the cargo box.

The three-dimensional model of the target object is imported into the processor, and the optimal alignment is carried out according to the existing algorithm, that is, the alignment of the two is realized according to the three-dimensional shape of the object. All of the above operations can be implemented in a remote platform, cloud platform, server, host computer and/or mobile terminal.

The above is the 3D model obtained by each scanning device and imported into the processor for matching, or the pictures of each scanning device can be directly imported into the processor, and the synthesis of the 3D model of the space, the synthesis of the 3D model of the object, and their match between. This can simplify the structure and cost of acquisition hardware. For example, it is not necessary to provide a processor capable of processing large data in the scanning device, and only simple control is required. The collected images can be transmitted to the cloud platform (equivalent to a processor) for centralized processing through 4G, 5G or other communication networks. This is also one of the inventive points of the present invention.

In order to realize the driving of the autonomous vehicle on the road, the 3D information collection device of the present invention can be arranged on the roof of the vehicle. Scan and collect the images of the opposite vehicle and fixed obstacles respectively, and transmit them to the server. The server synthesizes the 3D models of the vehicle, its surrounding vehicles and obstacles at different times according to multiple images, and at the same time according to their 3D models and The position and attitude information continuously adjusts the running trajectory and attitude of the vehicle, and finally realizes the obstacle avoidance of the autonomous vehicle.

The above is that each scanning device obtains the 3D model separately and then imports it into the processor for mismatching. It is also possible to directly import the pictures of each scanning device into the processor, and the synthesis of the spatial 3D model, the synthesis of the 3D model of the object, and mismatch between them. This can simplify the structure and cost of acquisition hardware. For example, it is not necessary to provide a processor capable of processing large data in the scanning device, and only simple control is required. The collected images can be transmitted to the cloud platform (equivalent to a processor) for centralized processing through 4G, 5G or other communication networks. This is also one of the inventive points of the present invention.

The above-mentioned target object, target object, and object all represent objects for which three-dimensional information is pre-acquired. It can be a solid object, or it can be composed of multiple objects. The three-dimensional information of the target includes a three-dimensional image, a three-dimensional point cloud, a three-dimensional grid, a local three-dimensional feature, a three-dimensional size, and all parameters with the three-dimensional feature of the target. The so-called three-dimensional in the present invention refers to having three directional information, XYZ, especially depth information, which is essentially different from having only two-dimensional plane information. It is also fundamentally different from some definitions that are called three-dimensional, panoramic, holographic, and three-dimensional, but actually only include two-dimensional information, especially not depth information.

The acquisition area mentioned in the present invention refers to the range that can be photographed by an image acquisition device (eg, a camera). The image acquisition device in the present invention can be CCD, CMOS, camera, video camera, industrial camera, monitor, camera, mobile phone, tablet, notebook, mobile terminal, wearable device, smart glasses, smart watch, smart bracelet and Image acquisition capabilities for all devices.

In the description provided herein, numerous specific details are set forth. It will be understood, however, that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.

Similarly, it is to be understood that in the above description of exemplary embodiments of the invention, various features of the invention are sometimes grouped together into a single embodiment, figure, or its description. This disclosure, however, should not be construed as reflecting an intention that the invention as claimed requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the Detailed Description are hereby expressly incorporated into this Detailed Description, with each claim standing on its own as a separate embodiment of this invention.

Those skilled in the art will understand that the modules in the device in the embodiment can be adaptively changed and arranged in one or more devices different from the embodiment. The modules or units or components in the embodiments may be combined into one module or unit or component, and further they may be divided into multiple sub-modules or sub-units or sub-assemblies. All features disclosed in this specification (including accompanying claims, abstract and drawings) and any method so disclosed may be employed in any combination, unless at least some of such features and/or procedures or elements are mutually exclusive. All processes or units of equipment are combined. Each feature disclosed in this specification (including accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.

Furthermore, it will be understood by those skilled in the art that although some of the embodiments herein include certain features, but not others, included in other embodiments, that combinations of features of the different embodiments are intended to be within the scope of the present invention And form different embodiments. For example, in the claims, any of the claimed embodiments may be used in any combination.

Various component embodiments of the present invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art should understand that, in practice, a microprocessor or a digital signal processor (DSP) may be used to implement some or all functions of some or all of the components in the apparatus based on the present invention according to the embodiments of the present invention. The present invention can also be implemented as apparatus or apparatus programs (eg, computer programs and computer program products) for performing part or all of the methods described herein. Such a program implementing the present invention may be stored on a computer-readable medium, or may be in the form of one or more signals. Such signals may be downloaded from Internet sites, or provided on carrier signals, or in any other form.

It should be noted that the above-described embodiments illustrate rather than limit the invention, and that alternative embodiments may be devised by those skilled in the art without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by one and the same item of hardware. The use of the words first, second, and third, etc. do not denote any order. These words can be interpreted as names.

By now, those skilled in the art will recognize that, although various exemplary embodiments of the present invention have been illustrated and described in detail herein, the present invention may still be implemented in accordance with the present disclosure without departing from the spirit and scope of the present invention. The content directly determines or derives many other variations or modifications consistent with the principles of the invention. Accordingly, the scope of the present invention should be understood and deemed to cover all such other variations or modifications.

Claims

A device for realizing 3D processing between moving objects, characterized in that it comprises a device for collecting 3D information of a target object and a first processor;

The target object 3D information acquisition device is used to continuously scan the object to be processed when the object to be processed moves, and obtain multiple images that can synthesize the three-dimensional model at multiple times;

a first processor, configured to perform a matching operation or a mismatch operation on the three-dimensional models of at least two objects to be processed at the same time;

The 3D information acquisition device includes an image acquisition device and a rotation device;

The image acquisition device is connected with the rotating device, and the rotating device drives it to rotate;

The angle α between the optical axes of the image acquisition device at two adjacent acquisition positions satisfies the following conditions:

Among them, R is the distance from the rotation center to the surface of the target object, T is the sum of the object distance and the image distance during acquisition, d is the length or width of the photosensitive element of the image acquisition device, F is the lens focal length of the image acquisition device, and u is the experience coefficient.
The apparatus of claim 1, wherein the first processor is further configured to synthesize a three-dimensional model of the object to be processed.
The device according to claim 1, further comprising a second processor, located in the target object 3D information acquisition device, for synthesizing a three-dimensional model of the object to be processed opposite to it.
The device according to claim 1, wherein when the device for collecting the 3D information of the target object collects the image of the object to be processed at any time, the image is stamped with a time stamp.
The device of claim 1, wherein: u<0.498, or u<0.411, or u<0.359, or u<0.281, or u<0.169, or u<0.041, or u<0.028.
The device according to claim 1, wherein the optical acquisition ports of the image acquisition device are all facing away from the direction of the rotation axis.
The device of claim 1, wherein the device for collecting 3D information of the target object is arranged on the object to be processed.
The device according to claim 1, wherein the 3D information acquisition device of the target object is arranged separately from the object to be processed.
The device according to claim 1, wherein after the processing is completed, the first processor outputs the processing result to the display device, the printing device, and/or the action execution device.
A method for realizing 3D processing between moving objects, characterized in that it comprises a target object 3D information acquisition device and a first processor;

The target object 3D information acquisition device is used to continuously scan the object to be processed when the object to be processed moves, and obtain multiple images that can synthesize the three-dimensional model at multiple times;

a first processor, configured to perform a matching operation or a mismatch operation on the three-dimensional models of the at least two objects to be processed at the same moment;

The 3D information acquisition device includes an image acquisition device and a rotation device;

The image acquisition device is connected with the rotating device, and the rotating device drives it to rotate;

The angle α between the optical axes of the image acquisition device at two adjacent acquisition positions satisfies the following conditions:

Among them, R is the distance from the rotation center to the surface of the target object, T is the sum of the object distance and the image distance during acquisition, d is the length or width of the photosensitive element of the image acquisition device, F is the lens focal length of the image acquisition device, and u is the experience coefficient.
The method of claim 10, wherein the first processor is further configured to synthesize a three-dimensional model of the object to be processed.
The method of claim 10, further comprising: a second processor, located in the target object 3D information acquisition device, for synthesizing a three-dimensional model of the object to be processed opposite to it.
The method according to claim 10, characterized in that, when the device for collecting the 3D information of the target object collects the above-mentioned image of the object to be processed at any time, the image is stamped with a time stamp.
The method of claim 10, wherein: u<0.498, or u<0.411, or u<0.359, or u<0.281, or u<0.169, or u<0.041, or u<0.028.
The method according to claim 10, wherein the optical acquisition ports of the image acquisition device are all facing away from the direction of the rotation axis.
The method of claim 10, wherein the 3D information acquisition device of the target object is arranged on the object to be processed.
The method according to claim 10, wherein the 3D information acquisition device of the target object is arranged separately from the object to be processed.
The method of claim 10, wherein after the processing is completed, the first processor outputs the processing result to the display device, the printing device, and/or the action execution device.