CN114427835A - A method and device for detecting the center of a support hole under low light in a mine - Google Patents

Abstract

The invention relates to a method and a device for detecting the center of a lower supporting hole of a mine under low illumination. The device comprises: acquiring a plurality of laser pictures, and marking pixel points where line laser blocks in the laser pictures are located and pixel points where picture backgrounds are located by adopting an HSV color model to obtain marked pictures. And extracting the outline of the marked picture to obtain the number of the line laser blocks and the length and the width of the line laser blocks. And determining the pixel type of the pixel where the line laser block is located according to the number, the length and the width of the line laser blocks, and calibrating the pixel type to the line laser block. And integrating all calibrated line laser blocks to form an integrated image. And obtaining the three-dimensional coordinates of the pixel points in the integrated image according to the width of the line laser block and the internal reference of the imaging device. And extracting the contour of the integrated image to obtain the contours of the anchor net and the anchor net belt, and obtaining the three-dimensional coordinates of the center of the anchor net hole and the center of the anchor net hole according to the three-dimensional coordinates of pixel points in the contour. The invention can realize the automatic determination of the position of the anchor rod.

Description

A method and device for detecting the center of a support hole under low light in a mine

technical field

The invention relates to the technical field of mine support and three-dimensional vision sensors, in particular to a method and device for detecting the center of a support hole under low light in a mine.

Background technique

In coal mining operations, the rock mass and coal mine are usually crushed by a roadheader to form a roadway space for easy operation. Before mining the roadway, safety measures must be taken before the coal mine can be fully exploited. With the deepening of the mining depth In order to ensure that the roadway will not collapse and drop gravel during the mining process, the rock wall of the roadway needs to be supported.

Considering the geological and economic conditions, the support is mainly divided into: bolt support, anchor mesh support and anchor mesh belt support according to different geological conditions. In order to prevent the anchor net from being damaged in the process of driving the anchor rod, it is necessary to find a suitable position for the anchor rod to pass through the anchor net and the anchor net belt smoothly. At present, the position of the anchor rod is manually determined. Although manual work can generally determine the appropriate position for bolting, the manual position determination brings the following problems: 1) The position that may be found due to human factors is not an ideal position; 2) There are certain safety hazards in manual identification; 3) It is not in line with the development of fully automatic anchor drilling machines.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a method and device for detecting the center of a support hole under low light in a mine, which can automatically realize the determination of the position of the bolt.

For achieving the above object, the present invention provides the following scheme:

A method for detecting the center of a support hole under low light in a mine, comprising:

Acquire a plurality of laser pictures according to time series, and the laser pictures are pictures formed by the line laser generating device in the process of scanning the support system; the support system includes a support bracket and an anchor net, and the anchor net is arranged at the on the support bracket;

For any laser image, the HSV color model is used to mark the pixel points where the line laser blocks in the laser image are located and the pixel points where the image background is located to obtain the marked image;

Perform contour extraction on the marked picture to obtain the number of the line laser blocks and the length and width of each of the line laser blocks;

According to the number of the line laser blocks and the length and width of each line laser block, determine the pixel type of the pixel where each line laser block is located, and calibrate the pixel type to each of the line laser blocks; the pixel type includes background pixels, Anchor web pixels and anchor web band pixels;

Integrate all calibrated line laser blocks to form an integrated image;

Based on the width of each line laser block, the laser calibration relationship, the internal reference of the imaging device and the scanning angle of the line laser generating device, the three-dimensional coordinates of each pixel in the integrated image in the world coordinate system are obtained. The laser calibration relationship is the relationship between the calibrated distance between the support system and the laser generating device and the width of the line laser block;

Perform contour extraction on the integrated image to obtain the anchor net contour and the anchor net belt contour, and according to the three-dimensional coordinates of each pixel in the anchor net contour under the world coordinate system and the The three-dimensional coordinates in the coordinate system are obtained to obtain the three-dimensional coordinates of the center of gravity of the anchor mesh in the world coordinate system and the three-dimensional coordinates of the center of the anchor mesh with holes in the world coordinate system.

Optionally, performing contour extraction on the marked picture to obtain the number of the line laser blocks and the length and width of each of the line laser blocks, specifically including:

Perform contour extraction on the marked picture to obtain the number of the line laser blocks and the contour coordinates of each of the line laser blocks;

The length and width of each of the line laser blocks are calculated according to the contour coordinates of each of the line laser blocks.

Optionally, performing contour extraction on the integrated image to obtain an anchor mesh outline and an anchor mesh belt outline, and according to the three-dimensional coordinates of each pixel in the anchor mesh outline under the world coordinate system and the anchor mesh strip outline. The three-dimensional coordinates of each pixel in the world coordinate system can obtain the three-dimensional coordinates of the anchor mesh center of gravity in the world coordinate system and the three-dimensional coordinates of the anchor mesh belt hole center in the world coordinate system, including:

Perform image segmentation on the integrated image to obtain a first image and a second image; the first image includes anchor mesh pixels and background pixels; the second image includes anchor mesh belt pixels and background pixels;

Perform contour extraction on the first image and the second image respectively to obtain the anchor net contour and the anchor net belt contour, and according to the three-dimensional coordinates of each pixel in the anchor net contour under the world coordinate system and the anchor net belt The three-dimensional coordinates of each pixel in the outline in the world coordinate system are obtained to obtain the three-dimensional coordinates of the anchor mesh center of gravity in the world coordinate system and the three-dimensional coordinates of the anchor mesh belt hole center in the world coordinate system.

Optionally, determining the pixel type of the pixel where each line laser block is located according to the number of the line laser blocks and the length and width of each line laser block specifically includes:

Determine the aspect ratio of each line laser block according to the length and width of each line laser block;

If the number of the line laser blocks in one of the laser pictures is greater than the first set threshold and the aspect ratio of the line laser blocks is greater than the second set threshold, the pixel type of the line laser block is background pixels;

If the number of the line laser blocks in one of the laser pictures is greater than the first set threshold and the aspect ratio of the line laser blocks is less than the third set threshold, the pixel type of the line laser block is anchor web pixel;

If the line laser block is a straight line and appears in the first set number of pictures, the pixel type of the line laser block is anchor mesh pixels;

The line laser block is a straight line and appears in the diagram of the second set number and the number of the line laser block is less than the fourth set threshold and the aspect ratio of the line laser block is greater than the fifth set threshold. The pixel type of the line laser block is anchor mesh belt;

The line laser block is a straight line and appears in the graph of the second set number and the number of line laser blocks is less than the fourth set threshold and the aspect ratio of the line laser block is less than the sixth set threshold. If the threshold is set, the pixel type of the line laser block is background.

Optionally, the world coordinate system of each pixel in the integrated image is obtained based on the width of each of the line laser blocks, the laser calibration relationship, the internal reference of the imaging device, and the scanning angle of the line laser generator. The three-dimensional coordinates below, specifically include:

Obtain the distance from the line laser generator to the support system according to the width of the line laser block and the laser calibration relationship;

Determine the coordinates of the line laser block in the Z direction in the world coordinate system according to the distance from the line laser generating device to the support system and the scanning angle of the line laser generating device;

Based on the internal reference of the imaging device and the coordinates of the line laser block in the Z direction in the world coordinate system, the three-dimensional coordinates of each pixel in the integrated image in the world coordinate system are obtained.

A device for detecting the center of a support hole under low light in a mine, comprising:

Support system, scanning imaging subsystem and data processing subsystem;

The support system includes a support bracket, an anchor mesh belt and an anchor mesh, the anchor mesh is arranged on the support bracket, and the anchor mesh strip is used to fix the anchor mesh on the support bracket ;

The scanning imaging subsystem includes: a line laser generating device and an imaging device, the anchor mesh and the anchor mesh belt are arranged on the outgoing optical path of the line laser generating device, and the imaging device is arranged on the anchor mesh and the anchor mesh. On the reflection light path of the anchor mesh belt to the line laser generating device, the imaging device is used for taking a plurality of laser pictures;

The data processing subsystem is connected to the imaging device, and the data processing subsystem is used to obtain the laser pictures taken by the imaging device, and to detect the center of the support hole under the low light of the mine as described above. Laser images are processed.

Optionally, the data processing subsystem includes: an image processing module, a three-dimensional reconstruction module and a contour extraction module connected in sequence; the image processing module is connected to the imaging device;

The image processing module is used to obtain the laser pictures taken by the imaging device, determine the pixel type of the pixel where each of the line laser blocks is located according to the number of the line laser blocks and the length and width of the line laser blocks The pixel type is calibrated to each of the line laser blocks and all the calibrated line laser blocks are integrated to form an integrated image; the pixel types include background pixels, anchor mesh pixels and anchor mesh belt pixels;

The three-dimensional reconstruction module is used to obtain the world coordinates of each pixel in the integrated image based on the width of each of the line laser blocks, the laser calibration relationship, the internal reference of the imaging device, and the scanning angle of the line laser generator. The three-dimensional coordinates under the system; the laser calibration relationship is the relationship between the distance between the calibrated support system and the laser generating device and the width of the line laser block;

The contour extraction module is used to perform contour extraction on the integrated image to obtain the anchor net contour and the anchor net belt contour, and according to the three-dimensional coordinates of each pixel in the anchor net contour under the world coordinate system and the anchor net belt. The three-dimensional coordinates of each pixel in the outline in the world coordinate system are obtained to obtain the three-dimensional coordinates of the anchor mesh center of gravity in the world coordinate system and the three-dimensional coordinates of the anchor mesh belt hole center in the world coordinate system.

Optionally, the data processing subsystem further includes: a camera calibration module, where the camera calibration module is used for calibrating the internal parameters of the imaging device and calibrating the laser calibration relationship.

Optionally, the three-dimensional reconstruction module specifically includes: a distance determination sub-module, a Z-coordinate determination sub-module, and a three-dimensional coordinate determination sub-module;

The distance determination submodule is used to obtain the distance from the line laser generator to the support system according to the width of the line laser block and the laser calibration relationship;

The Z coordinate determination submodule is used to determine the coordinates of the line laser block in the Z direction in the world coordinate system according to the distance from the line laser generator to the support system and the scanning angle of the line laser generator;

The three-dimensional coordinate determination sub-module is configured to obtain the three-dimensional coordinates of each pixel in the integrated image in the world coordinate system based on the internal reference of the imaging device and the coordinates of the line laser block in the Z direction in the world coordinate system.

Optionally, the image processing module includes: an image HSV segmentation submodule, a contour extraction submodule, a type judgment submodule, and a type image generation submodule;

The HSV segmentation sub-module is used to mark the pixel points where the line laser blocks in the laser picture are located and the pixel points where the picture background is located by using the HSV color model to obtain the marked picture;

The contour extraction sub-module is configured to perform contour extraction on the marked picture to obtain the number of the line laser blocks and the length and width of each of the line laser blocks;

The type judging submodule is used to determine the pixel type of the pixel where each line laser block is located according to the number of the line laser blocks and the length and width of each line laser block, and demarcate the pixel type to each of the line laser blocks ;

The image generation sub-module of the type is used to integrate all the calibrated line laser blocks to form an integrated image.

According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects: the present invention obtains a plurality of laser pictures according to time series, and the laser pictures are pictures formed by the line laser generating device irradiated on the support system; The support system includes a support bracket and an anchor net, and the anchor net is arranged on the support bracket; for any laser picture, the pixel points and pictures where the line laser blocks in the laser picture are located are determined by the HSV color model. Mark the pixels where the background is located to obtain the marked picture; perform contour extraction on the marked picture to obtain the number of the line laser blocks and the length and width of each line laser block; according to the number of the line laser blocks and the length and width of each line laser block to determine the pixel type of the pixel where each line laser block is located and calibrate the pixel type to each line laser block; the pixel types include background pixels, anchor mesh pixels and anchor mesh belt pixels Integrate all the calibrated line laser blocks to form an integrated image; obtain the integrated image based on the width of each of the line laser blocks, the laser calibration relationship, the internal reference of the imaging device and the scanning angle of the line laser generator The three-dimensional coordinates of each pixel in the image under the world coordinate system; the contour extraction is performed on the integrated image to obtain the anchor net contour and the anchor net belt contour, and the three-dimensional coordinates of each pixel in the anchor net contour under the world coordinate system are obtained. The coordinates and the three-dimensional coordinates of each pixel in the outline of the anchor mesh belt under the world coordinate system are obtained to obtain the three-dimensional coordinates of the center of gravity of the anchor mesh under the world coordinate system and the three-dimensional coordinates of the center of the anchor mesh belt hole under the world coordinate system, which can realize Automatic determination of anchor position.

Description of drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings required in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some of the present invention. In the embodiments, for those of ordinary skill in the art, other drawings can also be obtained according to these drawings without creative labor.

1 is a flowchart of a method for detecting the center of a support hole provided by an embodiment of the present invention;

2 is a schematic structural diagram of a support hole center detection device provided by an embodiment of the present invention;

3 is a schematic structural diagram of a scanning imaging system provided by an embodiment of the present invention;

4 is a schematic diagram of a type of a pixel provided by an embodiment of the present invention;

5 is a schematic diagram of scanning provided by an embodiment of the present invention;

6 is a schematic diagram of an anchor net outline extraction type provided by an embodiment of the present invention;

FIG. 7 is a schematic diagram of an anchor mesh belt outline extraction type provided by an embodiment of the present invention.

Symbol Description:

1-Support system, 2-Scan imaging system, 11-Support bracket, 12-Anchor mesh, 13-Anchor mesh belt, 14-Hook, 21-Sensor bracket, 22-Line laser generator, 23-Imaging device, 211- tripod, 212- sensor base, 221- servo bracket, 222- line laser generator, 223- servo.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

In order to make the above objects, features and advantages of the present invention more clearly understood, the present invention will be described in further detail below with reference to the accompanying drawings and specific embodiments.

The embodiments of the present invention provide a method and device for detecting the center of a support hole under low light in a mine, which solves the problems of poor accuracy and low safety in manually determining the center of the support hole, and promotes the development of an automatic bolting machine. As shown in Figure 1, the detection method for the center of the support hole under low light in the mine includes:

Step 101: Acquire a plurality of laser pictures according to time series, and the laser pictures are pictures formed by the imaging device during the scanning of the support system by the line laser generator; the support system includes a support bracket and an anchor net, so The anchor net is arranged on the support bracket.

Step 102 : For any laser picture, use the HSV color model to mark the pixel points where the line laser blocks are located and the pixel points where the picture background is located in the laser picture to obtain a marked picture.

Step 103: Perform contour extraction on the marked picture to obtain the number of the line laser blocks and the length and width of each of the line laser blocks.

Step 104: Determine the pixel type of the pixel where each line laser block is located according to the number of the line laser blocks and the length and width of each line laser block, and calibrate the pixel type to each of the line laser blocks; the pixel type includes Background Pixels, Anchor Pixels, and Anchor Band Pixels.

Step 105: Integrate all the calibrated line laser blocks to form an integrated image.

Step 106: Based on the width of each line laser block, the laser calibration relationship, the internal reference of the imaging device, and the scanning angle of the line laser generating device, obtain the three-dimensional image of each pixel in the integrated image in the world coordinate system. Coordinates, and the laser calibration relationship is the relationship between the calibrated distance between the support system and the laser generating device and the width of the line laser block.

Step 107: Perform contour extraction on the integrated image to obtain the anchor net contour and the anchor net belt contour, and according to the three-dimensional coordinates of each pixel point in the anchor net contour in the world coordinate system and each pixel in the anchor net belt contour. The three-dimensional coordinates of the point in the world coordinate system are obtained to obtain the three-dimensional coordinates of the center of gravity of the anchor mesh in the world coordinate system and the three-dimensional coordinates of the center of the anchor mesh with holes in the world coordinate system. The anchor mesh hole and the anchor mesh belt hole are collectively referred to as the support hole. According to the three-dimensional coordinates of the anchor mesh hole gravity center and the three-dimensional coordinate of the anchor mesh belt hole center are the center coordinates of the support hole, and the position of the anchor rod is determined according to the center coordinate of the support hole.

In practical applications, integrating all the calibrated line laser blocks to form an integrated image specifically includes:

The coordinates of the laser block in the single image are the coordinates of the integrated laser block in the integrated image, which is similar to the layer in PhotoShop. Where is it in the layer, and it is still in the same position after integration.

In practical applications, before using the HSV color model to mark the pixels where the line laser blocks are located in the laser picture and the pixels where the picture background is located, the marked picture further includes:

A median filter with a size of 3×3 is used for the laser image to remove noise points.

In practical applications, the number of the line laser blocks and the length and width of each line laser block are obtained by performing contour extraction on the marked picture, specifically including:

Perform contour extraction on the marked picture to obtain the number of the line laser blocks and the contour coordinates of each of the line laser blocks.

The length and width of each of the line laser blocks are calculated according to the contour coordinates of each of the line laser blocks.

In practical applications, the number of line laser blocks and the contour coordinates of each line laser block are obtained by performing contour extraction on the marked picture, specifically including:

The marked image is processed by using the function cv::findContour in OpenCV to obtain the number of line laser blocks and the contour coordinates of each line laser block.

In practical applications, calculating the length and width of each line laser block according to the contour coordinates of each line laser block specifically includes:

The length and width of each line laser block are obtained by using the function cv::boundingRect function in OpenCV and the outline coordinates of each line laser block.

In practical application, the outline of the integrated image is extracted to obtain the outline of the anchor network and the outline of the anchor network, and the three-dimensional coordinates of each pixel in the outline of the anchor network in the world coordinate system and the outline of the anchor network are obtained. The three-dimensional coordinates of each pixel in the outline in the world coordinate system are obtained to obtain the three-dimensional coordinates of the anchor mesh center of gravity in the world coordinate system and the three-dimensional coordinates of the anchor mesh belt hole center in the world coordinate system, including:

Perform image segmentation on the integrated image to obtain a first image as shown in Figure 6 and a second image as shown in Figure 7; the first image includes anchor mesh pixels and background pixels; the second image includes anchor mesh belts pixels and background pixels.

Perform contour extraction on the first image and the second image respectively to obtain the anchor net contour and the anchor net belt contour, and according to the three-dimensional coordinates of each pixel in the anchor net contour under the world coordinate system and the anchor net belt The three-dimensional coordinates of each pixel in the outline in the world coordinate system are obtained to obtain the three-dimensional coordinates of the anchor mesh center of gravity in the world coordinate system and the three-dimensional coordinates of the anchor mesh belt hole center in the world coordinate system.

In practical applications, determining the pixel type of the pixel where each line laser block is located according to the number of the line laser blocks and the length and width of each line laser block specifically includes:

The aspect ratio of each line laser block is determined according to the length and width of each line laser block.

If the number of the line laser blocks in one of the laser pictures is greater than the first set threshold and the aspect ratio of the line laser blocks is greater than the second set threshold, the pixel type of the line laser block is background pixels.

If the number of the line laser blocks in one of the laser pictures is greater than the first set threshold and the aspect ratio of the line laser blocks is less than the third set threshold, the pixel type of the line laser block is anchor web pixel.

If the line laser block is a straight line and appears in the first set number of graphs, the pixel type of the line laser block is anchor mesh pixels.

The line laser block is a straight line and appears in the diagram of the second set number and the number of the line laser block is less than the fourth set threshold and the aspect ratio of the line laser block is greater than the fifth set threshold. The pixel type of the line laser block is anchor mesh belt.

The line laser block is a straight line and appears in the graph of the second set number and the number of line laser blocks is less than the fourth set threshold and the aspect ratio of the line laser block is less than the sixth set threshold. If the threshold is set, the pixel type of the line laser block is background.

In practical applications, the world coordinates of each pixel in the integrated image are obtained based on the width of each of the line laser blocks, the laser calibration relationship, the internal reference of the imaging device, and the scanning angle of the line laser generator. The three-dimensional coordinates under the system, including:

The distance from the line laser generator to the support system is obtained according to the width of the line laser block and the laser calibration relationship.

The coordinates of the line laser block in the Z direction in the world coordinate system are determined according to the distance from the line laser generating device to the supporting system and the scanning angle of the line laser generating device.

Based on the internal reference of the imaging device and the coordinates of the line laser block in the Z direction in the world coordinate system, the three-dimensional coordinates of each pixel in the integrated image in the world coordinate system are obtained.

As shown in FIG. 5 , in practical application, the coordinates of the line laser block in the Z direction in the world coordinate system are determined according to the distance from the line laser generator to the support system and the scanning angle of the line laser generator, which specifically includes: :

Calculate the depth of the line laser block according to the formula Z=scosθ, that is, the coordinate of the line laser block in the Z direction in the world coordinate system, where s represents the distance from the line laser generator to the support system, according to the width of the line laser block and the laser The calibration relationship is obtained, θ represents the rotation angle of the steering gear, that is, the scanning angle of the line laser generator. After obtaining the depth information Z, the values X and Y of the other two dimensions can be calculated through the camera internal parameters.

In practical applications, the three-dimensional coordinates of each pixel in the integrated image in the world coordinate system are obtained based on the internal reference of the imaging device and the coordinates of the line laser block in the Z direction in the world coordinate system, specifically including:

得到整合后图像中各像素点在世界坐标系下的三维坐标(X,Y,Z)。According to the formula

The three-dimensional coordinates (X, Y, Z) of each pixel in the integrated image in the world coordinate system are obtained.

表示成像装置的内参构成的矩阵的逆矩阵，f_x表示x方向焦距，f_y表示y方向焦距，c_x表示x方向主点坐标，c_y表示y方向主点坐标，

表示线激光块在图像坐标系下的坐标构成的矩阵即像素坐标，u表示水平方向，v表示竖直方向。in,

Represents the inverse matrix of the matrix formed by the internal parameters of the imaging device, f _x represents the focal length in the x direction, f _y represents the focal length in the y direction, c _x represents the coordinates of the principal point in the x direction, and c _y represents the coordinates of the principal point in the y direction,

The matrix representing the coordinates of the line laser block in the image coordinate system is the pixel coordinates, u represents the horizontal direction, and v represents the vertical direction.

As shown in FIG. 2 and FIG. 3 , an embodiment of the present invention further provides a device for detecting the center of a support hole under low light in a mine using the above method, including:

Support system 1, scanning imaging subsystem and data processing subsystem.

The support system 1 includes a support bracket 11, an anchor mesh belt 13 and an anchor mesh 12, the anchor mesh 12 is arranged on the support bracket 11, and the anchor mesh belt 13 is used to attach the anchor mesh 12 Fixed on the support bracket 11; hooks 14 are distributed on both sides of the specific support bracket 11 for placing the anchor mesh 12 and the anchor mesh belt 13; the anchor mesh 12 is placed on the support bracket 11; the anchor mesh belt 13 is placed On the anchor net 12 and fixedly connected with the support bracket 11, it is used to connect the anchor net 12 into a piece.

The scanning imaging subsystem includes: a line laser generating device 22 and an imaging device 23 , the anchor mesh 12 and the anchor mesh belt 13 are arranged on the outgoing optical path of the linear laser generating device 22 , and the imaging device 23 is arranged On the reflected light path of the anchor net 12 and the anchor net belt 13 to the line laser generating device 22 , the imaging device 23 is used for taking a plurality of laser pictures.

The data processing subsystem is connected to the imaging device 23, and the data processing subsystem is used to obtain the laser pictures taken by the imaging device 23, and to detect the center of the support hole under the low light in the mine as described above. The laser pictures are processed.

As an optional implementation manner, the data processing subsystem includes: an image processing module, a three-dimensional reconstruction module, and a contour extraction module connected in sequence; the image processing module is connected to the imaging device 23 ; the image processing module It is used to obtain the laser pictures taken by the imaging device 23, determine the pixel type of the pixel where each of the line laser blocks is located according to the number of the line laser blocks and the length and width of the line laser blocks, and calibrate the pixel type to each of the line laser blocks and integrate all the calibrated line laser blocks to form an integrated image; the pixel types include background pixels, anchor mesh pixels and anchor mesh belt pixels; the three-dimensional reconstruction module is used for The width of the line laser block, the laser calibration relationship, the internal reference of the imaging device, and the scanning angle of the line laser generator can obtain the three-dimensional coordinates of each pixel in the integrated image in the world coordinate system, and the laser calibration relationship In order to calibrate the relationship between the distance between the support system and the laser generating device and the width of the line laser block, the contour extraction module is used to perform contour extraction on the integrated image to obtain the anchor net contour and The outline of the anchor mesh is obtained according to the three-dimensional coordinates of each pixel in the outline of the anchor mesh under the world coordinate system and the three-dimensional coordinates of each pixel in the outline of the anchor mesh under the world coordinate system to obtain the center of gravity of the anchor mesh in the world coordinate. The three-dimensional coordinates in the system and the three-dimensional coordinates of the center of the anchor mesh belt hole in the world coordinate system.

As an optional implementation manner, the scanning imaging subsystem further includes: a sensor bracket 21, where the sensor bracket 21 is used to place the line laser generating device 22 and the imaging device 23; the line laser generating device 22 is fixed on the sensor bracket 21 , used for vertical scanning of the support system 1; the imaging device 23 is placed above the sensor support 21 for recording the trajectory of the line laser generating device 22 to obtain multiple laser pictures during the scanning process.

The sensor bracket 21 is divided into a tripod 211 and a sensor base 212. The sensor base 212 is placed above the tripod 211, the tripod 211 is fixed in the world coordinate system, and the tripod 211 is used to adjust the height of the sensor bracket 21. The sensor base 212 has a cylindrical shape as a whole, and the laser generating device 22 is placed inside, and the imaging device 23 is placed above.

The imaging device 23 is a camera.

As an optional embodiment, the line laser generating device 22 includes: a rotating device and a line laser generator 222; the line laser generator 222 is arranged on the rotating device, and the rotating device includes a steering gear 223 and a steering gear bracket 221, The steering gear 223 is fixedly connected to the steering gear bracket 221 for generating pitching motion; the line laser generator 222 is fixedly connected to the steering gear 223 for vertical scanning.

The rotating device is divided into a steering gear bracket 221 and a steering gear 223. The steering gear bracket 221 is fixed on the sensor base 212, the steering gear 223 is fixed on the steering gear bracket 221, and the line laser generator 222 is fixed on the steering gear. 223 on the reels.

As an optional implementation manner, the data processing subsystem further includes: a camera calibration module, where the camera calibration module is used for calibrating the internal parameters of the imaging device 23 and calibrating the laser calibration relationship.

As an optional implementation manner, the camera calibration module includes:

The internal parameter calibration sub-module is used to realize the internal parameter calibration of the imaging device by using the Zhang Zhengyou checkerboard calibration method.

The relationship calibration sub-module is used to hit the laser on the checkerboard at different distances, measure the width of the laser block on the laser image to calibrate the laser calibration relationship, that is, mark the laser line on the checkerboard, and use image processing to extract the width of the laser line. The relationship between the width of the laser line and the distance can be obtained by changing the different distances of the checkerboard and conducting multiple sets of experiments. The process of determining the laser calibration relationship is: line laser calibration formula: s=μw, s is the distance from the scanned position to the line laser generator, μ is the ratio of distance and width, w is the width of the line laser block, the specific calibration method In order to collect multiple groups of s and w with different distances, the parameter μ is then calculated by the least square method.

As an optional implementation manner, the three-dimensional reconstruction module specifically includes: a distance determination sub-module, a Z-coordinate determination sub-module, and a three-dimensional coordinate determination sub-module.

The distance determination sub-module is used to obtain the distance from the line laser generator 22 to the support system 1 according to the width of the line laser block and the laser calibration relationship.

The Z coordinate determination submodule is used to determine the coordinate of the line laser block in the Z direction in the world coordinate system according to the distance from the line laser generator 22 to the support system 1 and the scanning angle of the line laser generator.

The three-dimensional coordinate determination sub-module is configured to obtain the three-dimensional coordinates of each pixel in the integrated image based on the internal reference of the imaging device 23 and the coordinates of the line laser block in the Z direction in the world coordinate system.

As an optional implementation manner, the image processing module includes: an image HSV segmentation submodule, a contour extraction submodule, a type judgment submodule, and a type image generation submodule.

The HSV segmentation sub-module is configured to use the HSV color model to mark the pixel points where the line laser blocks are located in the laser picture and the pixel points where the picture background is located to obtain a marked picture.

The contour extraction sub-module is configured to perform contour extraction on the marked picture to obtain the number of the line laser blocks and the length and width of each of the line laser blocks.

The type judging submodule is used to determine the pixel type of the pixel where each line laser block is located according to the number of the line laser blocks and the length and width of each line laser block, and demarcate the pixel type to each of the line laser blocks .

The image generation sub-module of the type is used to integrate all the calibrated line laser blocks to form an integrated image.

As an optional implementation manner, the image processing module further includes: an image filtering sub-module, configured to use median filtering with a size of 3×3 on the laser image to remove noise points.

The embodiment of the present invention also provides a device for detecting the center of a supporting hole in a mine under low light. Different from the above-mentioned device for detecting the center of a supporting hole in a low-light mine, the device for detecting the center of a supporting hole in a low-light mine in this embodiment is As shown in FIG. 2, the support system 1 is divided into a support bracket 11, an anchor net 12, an anchor net belt 13, and a hook 14. The hook 14 is installed on the support bracket 11, and there are a total of 6 hooks 14, which are evenly distributed. On both sides of the support bracket 11, and the opposite hooks 14 on both sides are on the same horizontal line, the anchor net 12 is hung on the hook 14, there are three anchor nets 12, and the anchor net belt 13 presses the anchor net 12 and is stuck on the hook 14. In the middle, there are 2 anchor mesh belts 13 in total.

The embodiment of the present invention provides the workflow of the above-mentioned mine support hole center detection system under low light:

Hit the line laser on the support system 1. Since the support system 1 (anchor mesh 12, anchor mesh belt 13, background) has different depth information, line segments with different numbers and lengths will be formed, as shown in Figure 4 and shown in Figure 5.

The number of line laser blocks and the way of judging the length and width:

First, the image is denoised by median filtering, and the pixel coordinate where the line laser block is located is marked as 255 and the background is marked as 0 by the HSV judgment method. Call the cv::findContour function to extract the boundary of the image. After the extraction is completed, it will return the number of pixel blocks (line laser blocks) and the outline coordinates of each line laser block. Call the cv::boundingRect function according to the outline coordinates of each line laser block. Calculate the length and width of each pixel block, and use the length and width to calculate the aspect ratio of each pixel block.

Pixel type determination:

According to the number and length of the line laser blocks and the number of frames collected in the time series, it can be inferred where the line laser hits the support system. The pixel judgment method of the laser block on the image is shown in Table 1 and Figure 4:

Table 1 Pixel type determination table

Image integration:

Integrate the pixels of the line laser blocks on a series of images into a single image consisting of background pixel 0, anchor mesh pixel 1, and anchor mesh pixel 2.

Depth calculation method of line laser block:

According to the calibrated laser calibration relationship and the width of the line laser block, the distance from the support to the laser generator is calculated. Since the center of the laser line generator and the camera are on the same horizontal plane, the angle θ and Z=scosθ can be calculated by the rotation angle of the steering gear. Depth Z of line laser block pixels.

Pixel 3D coordinate calculation method:

Using the calibrated camera internal parameters, through the formula

Calculate the three-dimensional coordinates of each pixel in the world coordinate system.

At this time, the obtained information package support type (background, anchor net and anchor net belt, generated by time series) image and the three-dimensional coordinates of each pixel in the world coordinate system.

The anchor net + background and the anchor net belt + background in the generated support type picture are generated into two pictures, one contains the anchor net information, and the other contains the anchor net belt information.

Use the cv::findContour function to extract the boundary of the anchor network for the picture containing the anchor network information, calculate the area of the outline through cv::contourArea, and determine whether the outline is an anchor network outline according to the size, and then the boundary corresponds to the world coordinate system. The three-dimensional coordinates contained in the pixel coordinates below can obtain its center of gravity, and the coordinates of the center of gravity can be equal to the center coordinates.

Use the cv::findContour function to extract the boundary of the anchor mesh for the picture containing the information of the anchor mesh, calculate the area of the contour through cv::contourArea, and judge whether the outline is the anchor mesh outline according to the size. The corresponding three-dimensional coordinates contained in the pixel coordinates in the world coordinate system are used to obtain its center of gravity, and the coordinates of the center of gravity can be equal to the three-dimensional center coordinates of the anchor mesh with holes.

The present invention has the following technical effects:

1. Solve the problems of poor accuracy and low safety in manually determining the center position of the support hole.

2. Promote the development of fully automatic drilling and bolting machine, and realize the automatic determination of the position of the bolt.

The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same and similar parts between the various embodiments can be referred to each other.

In this paper, specific examples are used to illustrate the principles and implementations of the present invention. The descriptions of the above embodiments are only used to help understand the methods and core ideas of the present invention; meanwhile, for those skilled in the art, according to the present invention There will be changes in the specific implementation and application scope. In conclusion, the contents of this specification should not be construed as limiting the present invention.

Claims (10)

1. A method for detecting the center of a protective hole under low illumination of a mine is characterized by comprising the following steps:

acquiring a plurality of laser pictures according to a time sequence, wherein the laser pictures are pictures shot by an imaging device of a line laser generating device in the process of scanning a support system; the supporting system comprises a supporting bracket and an anchor net, and the anchor net is arranged on the supporting bracket;

for any laser picture, marking a pixel point where a line laser block in the laser picture is located and a pixel point where a picture background is located by adopting an HSV color model to obtain a marked picture;

extracting the outline of the marked picture to obtain the number of the line laser blocks and the length and the width of each line laser block;

determining the pixel type of the pixel of each line laser block according to the number of the line laser blocks and the length and width of each line laser block, and calibrating the pixel type to each line laser block; the pixel types comprise background pixels, anchor mesh pixels and anchor mesh band pixels;

integrating all calibrated line laser blocks to form an integrated image;

obtaining three-dimensional coordinates of each pixel point in the integrated image under a world coordinate system based on the width of each line laser block, a laser calibration relation, the internal parameters of the imaging device and the scanning angle of the line laser generating device, wherein the laser calibration relation is the relation between the distance from the calibrated support system to the laser generating device and the width of the line laser block;

and extracting the contour of the integrated image to obtain an anchor net contour and an anchor net belt contour, and obtaining a three-dimensional coordinate of the anchor net hole gravity center in the world coordinate system and a three-dimensional coordinate of the anchor net hole center in the world coordinate system according to the three-dimensional coordinate of each pixel point in the anchor net contour in the world coordinate system and the three-dimensional coordinate of each pixel point in the anchor net belt contour in the world coordinate system.

2. The method for detecting the center of the pilot hole under low illumination of the mine as claimed in claim 1, wherein the extracting the contour of the marked picture to obtain the number of the line laser blocks and the length and width of each line laser block specifically comprises:

extracting the contour of the marked picture to obtain the number of the line laser blocks and contour coordinates of each line laser block;

and calculating the length and the width of each line laser block according to the contour coordinates of each line laser block.

3. The method for detecting the center of the protective hole under the low illumination of the mine according to claim 1, wherein the extracting of the contour of the integrated image is performed to obtain an anchor net contour and an anchor net belt contour, and a three-dimensional coordinate of the gravity center of the anchor net hole under a world coordinate system and a three-dimensional coordinate of the center of the anchor net hole under the world coordinate system are obtained according to a three-dimensional coordinate of each pixel point under the world coordinate system in the anchor net contour and a three-dimensional coordinate of each pixel point under the world coordinate system in the anchor net belt contour, and specifically comprises:

carrying out image segmentation on the integrated image to obtain a first image and a second image; the first image comprises anchor mesh pixels and background pixels; the second image comprises anchor swath pixels and background pixels;

and respectively extracting the outlines of the first image and the second image to obtain an anchor net outline and an anchor net belt outline, and obtaining a three-dimensional coordinate of the anchor net hole gravity center under a world coordinate system and a three-dimensional coordinate of the anchor net hole center under the world coordinate system according to the three-dimensional coordinate of each pixel point in the anchor net outline under the world coordinate system and the three-dimensional coordinate of each pixel point in the anchor net belt outline under the world coordinate system.

4. The method for detecting the center of the pilot hole under low illumination of the mine as claimed in claim 1, wherein the determining the pixel type of the pixel of each line laser block according to the number of the line laser blocks and the length and width of each line laser block specifically comprises:

determining the length-width ratio of each line laser block according to the length and the width of each line laser block;

the number of the line laser blocks in one laser picture is larger than a first set threshold value, the length-width ratio of the line laser blocks is larger than a second set threshold value, and then the pixel type of the line laser blocks is a background pixel;

the number of the line laser blocks in one laser picture is larger than the first set threshold value, and the length-width ratio of the line laser blocks is smaller than a third set threshold value, so that the pixel type of the line laser blocks is an anchor network pixel;

the line laser blocks are straight lines, and if the line laser blocks appear in a first set number of graphs, the pixel types of the line laser blocks are anchor net pixels;

the line laser blocks are a straight line and appear in a second set number of graphs, the number of the line laser blocks is smaller than a fourth set threshold, the length-width ratio of the line laser blocks is larger than a fifth set threshold, and then the pixel type of the line laser blocks is an anchor mesh belt;

the line laser blocks are straight lines and appear in the second set number of graphs, the number of the line laser blocks is smaller than the fourth set threshold, the length-width ratio of the line laser blocks is smaller than the sixth set threshold, and then the pixel type of the line laser blocks is background.

5. The method for detecting the center of the pilot hole under the low illumination of the mine according to claim 1, wherein the obtaining of the three-dimensional coordinates of each pixel point in the integrated image under the world coordinate system based on the width of each line laser block, the laser calibration relation, the internal parameters of the imaging device and the scanning angle of the line laser generating device specifically comprises:

obtaining the distance from the line laser generating device to the support system according to the width of the line laser block and the laser calibration relation;

determining the coordinate of the line laser block in the Z direction under a world coordinate system according to the distance from the line laser generating device to the supporting system and the scanning angle of the line laser generating device;

and obtaining the three-dimensional coordinates of each pixel point in the integrated image in the world coordinate system based on the internal reference of the imaging device and the coordinates of the line laser block in the Z direction in the world coordinate system.

6. The utility model provides a mine low light is support hole center detection device down which characterized in that includes:

the system comprises a support system, a scanning imaging subsystem and a data processing subsystem;

the supporting system comprises a supporting bracket, an anchor net belt and an anchor net, wherein the anchor net is arranged on the supporting bracket, and the anchor net belt is used for fixing the anchor net on the supporting bracket;

the scanning imaging subsystem includes: the line laser imaging device comprises a line laser generating device and an imaging device, wherein the anchor net and the anchor net belt are arranged on an emergent light path of the line laser generating device, the imaging device is arranged on a reflection light path of the anchor net and the anchor net belt to the line laser generating device, and the imaging device is used for shooting a plurality of laser pictures;

the data processing subsystem is connected with the imaging device and used for acquiring a laser picture shot by the imaging device and processing the laser picture according to the method for detecting the center of the protective hole under the low illumination of the mine as claimed in any one of the claims 1 to 5.

7. The mine low-light lower supporting hole center detection device as claimed in claim 6, wherein the data processing subsystem comprises: the system comprises an image processing module, a three-dimensional reconstruction module and a contour extraction module which are sequentially connected; the image processing module is connected with the imaging device;

the image processing module is used for acquiring the laser picture shot by the imaging device, determining the pixel type of the pixel of each line laser block according to the number of the line laser blocks and the length and width of the line laser blocks, calibrating the pixel type to each line laser block, and integrating all calibrated line laser blocks to form an integrated image; the pixel types comprise background pixels, anchor mesh pixels and anchor mesh band pixels;

the three-dimensional reconstruction module is used for obtaining three-dimensional coordinates of all pixel points in the integrated image under a world coordinate system based on the width of each line laser block, the laser calibration relation, the internal parameters of the imaging device and the scanning angle of the line laser generation device; the laser calibration relation is the relation between the distance between the calibrated support system and the laser generating device and the width of the line laser block;

the contour extraction module is used for extracting the contour of the integrated image to obtain an anchor net contour and an anchor net belt contour and obtaining a three-dimensional coordinate of the anchor net hole gravity center in the world coordinate system and a three-dimensional coordinate of the anchor net hole center in the world coordinate system according to the three-dimensional coordinate of each pixel point in the anchor net contour in the world coordinate system and the three-dimensional coordinate of each pixel point in the anchor net belt contour in the world coordinate system.

8. The mine low light down hole guard center detection device of claim 7, wherein the data processing subsystem further comprises: and the camera calibration module is used for calibrating the internal reference of the imaging device and calibrating the laser calibration relation.

9. The mine low-light lower guard hole center detection device according to claim 7, wherein the three-dimensional reconstruction module specifically comprises: a distance determination submodule, a Z coordinate determination submodule and a three-dimensional coordinate determination submodule;

the distance determining submodule is used for obtaining the distance from the line laser generating device to the supporting system according to the width of the line laser block and the laser calibration relation;

the Z coordinate determination submodule is used for determining the coordinate of the line laser block in the Z direction under a world coordinate system according to the distance from the line laser generating device to the support system and the scanning angle of the line laser generating device;

and the three-dimensional coordinate determination submodule is used for obtaining the three-dimensional coordinates of all pixel points in the integrated image in the world coordinate system based on the internal reference of the imaging device and the coordinates of the line laser block in the Z direction in the world coordinate system.

10. The mine low-light lower guard hole center detection device as claimed in claim 7, wherein the image processing module comprises: the image HSV segmentation submodule, the contour extraction submodule, the type judgment submodule and the type image generation submodule are connected;

the HSV segmentation submodule is used for marking pixel points where line laser blocks in the laser picture are located and pixel points where the picture background is located by adopting an HSV color model to obtain marked pictures;

the contour extraction submodule is used for extracting the contour of the marked picture to obtain the number of the line laser blocks and the length and the width of each line laser block;

the type judgment sub-module is used for determining the pixel type of the pixel where each line laser block is located according to the number of the line laser blocks and the length and the width of each line laser block and calibrating the pixel type to each line laser block;

and the type image generation submodule is used for integrating all the calibrated line laser blocks to form an integrated image.

Images