JP7567559B2 - Installation position display system and work vehicle - Google Patents

Description

The present invention relates to an installation position display system and a work vehicle. In particular, the present invention relates to an installation position display system for a work vehicle having a boom that can be raised and lowered via a swivel base attached to a running body, and a work vehicle equipped with the installation position display system.

Traditionally, construction sites for civil engineering, building, and other construction work are leveled by construction work such as excavation and filling in according to the state of the natural terrain. At the leveled construction site, the ground is excavated and the excavated areas are backfilled in order to construct structures above and below ground. In addition, at the construction site, ground improvement work is carried out according to the state of the ground and the use of the plot. However, at the construction site, it is difficult to grasp the location of the underground structures and the scope of the work carried out on the ground such as filling in, backfilling, and ground improvement work after each work is completed. Therefore, a technology is known that uses augmented reality technology to visually grasp the state of the underground structures and ground at the construction site. For example, see Patent Document 1.

The display control device described in Patent Document 1 calculates a virtual state model showing the state of the construction site as seen from the imaging direction of the camera based on imaging data of the construction site captured by a camera, information on the position and posture of the user holding the camera, and information on a virtual model of the construction site. The display device combines and displays the imaging data and the calculated virtual state model. The user can check the positions of underground structures at the construction site, the extent of backfilling, and the extent of ground improvement work in the virtual model superimposed on the imaging data of the construction site.

At the construction site, various work vehicles such as mobile cranes and backhoes move around the site and perform various tasks at their destinations. For example, the mobile cranes move to a specified location within the construction site while loaded with a heavy load, or transport heavy loads to a desired location while supported by outriggers. When moving around the construction site, the mobile cranes can only move on ground that has the bearing capacity to support the vehicle weight. When transporting heavy loads, the mobile cranes can only be installed on ground that has the bearing capacity to support the vehicle weight and the moment generated during transportation. Therefore, it is necessary to know whether the ground along the movement route or at the installation location has sufficient bearing capacity.

Meanwhile, the bearing capacity of the ground at the construction site varies depending on the content and circumstances of the work to develop the natural terrain, backfilling associated with construction work, ground improvement work, etc. The display control device of Patent Document 1 can confirm the location of underground structures at the construction site, the extent of backfilling, and the extent of ground improvement work using VR technology. However, the display control device cannot determine whether the ground has the bearing capacity to allow the mobile crane to be moved or installed.

JP 2015-11368 A

The object of the present invention is to provide a work vehicle and an installation position display system that can provide information for determining the movement path or installation position of a work vehicle based on the work vehicle, the type of work to be performed by the work vehicle, and the bearing capacity of the ground.

The inventors have studied an installation position display system and a work vehicle that can provide information for determining the movement path or installation position of a work vehicle based on the work vehicle, the content of the work performed by the work vehicle, and the bearing capacity of the ground. As a result of extensive study, the inventors have come up with the following configuration.

The installation position display system according to one embodiment of the present invention is an installation position display system that displays the ground surface that can be worked on by a work vehicle having a boom that can be raised and lowered via a swivel base attached to a running body.

The installation position display system includes an imaging device that captures a real image, an imaging position detection device that detects the imaging position and imaging direction of the imaging device, an information acquisition device that acquires work vehicle information related to the work vehicle, work information related to the work of the work vehicle, and ground information related to the state of the ground, a display device, and an image processing device. The image processing device detects an image portion of the ground surface included in the real image. The image processing device calculates at least one of a movable range in which the work vehicle can move on the ground surface and a workable range in which the work vehicle can work, based on the work vehicle information, the work information, and the ground information. The image processing device generates a virtual image showing at least one of the movable range and the workable range in accordance with the shape of the image portion of the ground surface. The image processing device also generates a composite image by superimposing the virtual image on the image portion of the ground surface in the real image.

As described above, the image processing device generates a virtual image showing the movable range or the workable range of the work vehicle according to the shape of the image portion of the ground surface. Furthermore, the image processing device generates a composite image in which the virtual image is superimposed on the image portion of the ground surface in the real image. In other words, the installation position display system calculates the range of ground that can support the work vehicle while moving or the work vehicle while working based on the type of the work vehicle and the content of the work. In this way, the installation position display system can provide information for determining the movement path or installation position of the work vehicle based on the work vehicle, the content of the work by the work vehicle, and the bearing capacity of the ground.

In another aspect, the image processing device in the installation position display system of the present invention generates the composite image from the acquired ground information by further superimposing the value of the allowable bearing capacity of the ground on the image portion of the ground surface in the real image.

As described above, the image processing device further displays the value of the ground's allowable bearing capacity. Therefore, the operator can select a more optimal movement route and installation position based on the value of the ground's allowable bearing capacity. This allows the installation position display system to provide information for determining the movement route or installation position of the work vehicle based on the work vehicle, the content of the work performed by the work vehicle, and the bearing capacity of the ground.

According to another aspect, the information acquisition device in the installation position display system of the present invention acquires the current position of the work vehicle contained in the work vehicle information, and the image processing device displays a warning on the display device when the work vehicle is moving and not located within the calculated movable range, or when the work vehicle is working and not located within the workable range.

As described above, the image processing device notifies the operator that the ground's bearing capacity is insufficient for the type of work vehicle and the type of work to be performed. This allows the installation position display system to provide information for determining the work vehicle's travel path or installation position based on the work vehicle, the type of work to be performed by the work vehicle, and the ground's bearing capacity.

According to another aspect, the image processing device in the installation position display system of the present invention calculates the moving support force, which is the support force of the ground required to support the load when the work vehicle is traveling, or the working support force, which is the support force of the ground required to support the load when the work vehicle is working, from the attitude information of the work vehicle acquired from the work vehicle and the weight of the cargo being transported, and displays a warning on the display device when the moving support force or the working support force exceeds a reference value based on the allowable support force of the ground.

As described above, the image processing device monitors the work load that occurs while the work vehicle is traveling or working. For example, when work is performed under conditions different from those based on the acquired work information, and the work support force, which is the support force required by the ground to support the work load that occurs, exceeds a reference value based on the allowable support force of the ground, the image processing device notifies the display device of a warning to the work vehicle performing the work that the support force of the ground is insufficient. This allows the installation position display system to provide information for determining the movement path or installation position of the work vehicle based on the work vehicle, the content of the work performed by the work vehicle, and the support force of the ground.

From another perspective, the installation position display system of the present invention is a mobile terminal in which the imaging device, the imaging position detection device, the information acquisition device, the image processing device, and the display device are integrated.

As described above, the installation position display system includes each device in a mobile terminal, so that the positional relationship between the position of the work vehicle and the movable range or workable range can be visually confirmed from a location other than the driver's seat of the work vehicle. This allows the installation position display system to provide information for determining the movement path or installation position of the work vehicle based on the work vehicle, the content of the work performed by the work vehicle, and the bearing capacity of the ground.

The work vehicle according to one embodiment of the present invention is provided with a boom that can be raised and lowered via a swivel base on the running body, and is equipped with the above-mentioned installation position display system.

As described above, the work vehicle equipped with the installation position display system generates a virtual image showing the workable range of the work vehicle by the installation position display system, which is adapted to the shape of the image portion of the ground surface. Furthermore, the work vehicle generates a composite image by superimposing the virtual image on the image portion of the ground surface in the real image by the image processing device. In other words, the work vehicle calculates the range of the ground that can support the work load, which is the reaction force during movement or work, based on the content of the work. As a result, the work vehicle equipped with the installation position display system can provide information for determining the movement path or installation position of the work vehicle based on the work vehicle, the content of the work by the work vehicle, and the supporting capacity of the ground.

According to one embodiment of the present invention, the installation position display system and a work vehicle equipped with the installation position display system can provide information for determining the movement path or installation position of the work vehicle based on the work vehicle and the content of the work performed by the work vehicle.

FIG. 1 is a block diagram showing the overall configuration of an installation position display system according to a first embodiment of the present invention. FIG. 2 is a process diagram showing a process for generating a composite image of the installation position display system according to the first embodiment of the present invention. FIG. 3 is a diagram showing a state in which the ground surface is detected from a real image by the installation position display system according to the first embodiment of the present invention. 4A and 4B show the generated virtual image, where (A) is a diagram showing a planar image of the generated virtual image, and (B) is a diagram showing the virtual image that can be superimposed on a real image. FIG. 5 is a diagram showing a composite image generated by the installation position display system according to the first embodiment of the present invention. FIG. 6 is a diagram showing a usage state of a mobile terminal including the installation position display system according to the first embodiment of the present invention. FIG. 7 is a diagram showing the display of the allowable supporting force value in the installation position display system according to the first embodiment of the present invention. FIG. 8 is a diagram showing a display regarding a warning by the installation position display system according to the first embodiment of the present invention. FIG. 9 is a side view showing the overall configuration of a mobile crane according to a second embodiment of the present invention. FIG. 10 is a block diagram showing a control configuration for a mobile crane according to a second embodiment of the present invention.

Each embodiment will be described below with reference to the drawings. In each drawing, the same parts are given the same reference numerals, and the description of the same parts will not be repeated. Note that the dimensions of the components in each drawing do not faithfully represent the actual dimensions of the components or the dimensional ratios of each component.

[Embodiment 1]
<Overall composition>
An installation position display system 1 according to a first embodiment of the present invention will be described below with reference to Fig. 1. Fig. 1 is a block diagram showing the overall configuration of the installation position display system 1 according to the first embodiment of the present invention.

As shown in FIG. 1, the installation position display system 1 is a system that displays a position to which a mobile crane MC, a work vehicle equipped with a boom that can be rotated and raised, can be moved or installed. The installation position display system 1 includes a photographing device 2, a photographing position detection device 3, an information acquisition device 4, a display device 5, and an image processing device 6.

The imaging device 2 is a device that photographs the construction site where the mobile crane MC moves or works. The imaging device 2 is capable of photographing a predetermined range in order to photograph the surroundings of the mobile crane MC. The imaging device 2 is capable of acquiring a control signal from the image processing device 6. The imaging device 2 is also capable of transmitting a real image RI, which is a photographed image, to the image processing device 6.

The imaging device 2 is, for example, composed of a 360-degree camera with a 360-degree viewing angle. The imaging device 2 may be a camera with a rotation mechanism that can scan a 360-degree range. The imaging device 2 may also be linked to the control device of the mobile crane MC in order to capture images of the extension direction of the boom as seen from the operator's seat of the mobile crane MC. The imaging device 2 may also be provided in a remote control terminal that remotely controls the mobile crane MC, or in a wearable terminal worn by the operator.

The photographing position detection device 3 is a device that detects the absolute coordinates of the photographing device 2. The photographing position detection device 3 is provided integrally with the photographing device 2 or in the vicinity of the photographing device 2. The photographing position detection device 3 is, for example, composed of multiple GNSS receivers provided in the photographing device 2. The GNSS receiver is a receiver that constitutes a global navigation satellite system. The GNSS receiver receives ranging radio waves from satellites and calculates latitude, longitude, and altitude in a global coordinate system. The photographing position detection device 3 can acquire a control signal from the image processing device 6. The photographing position detection device 3 can transmit the latitude, longitude, and altitude in the global coordinate system of the multiple GNSS receivers provided in the photographing device 2 to the image processing device 6 as the absolute coordinates of the photographing device 2 (hereinafter simply referred to as "position coordinates P").

In this embodiment, the photographing position detection device 3 is composed of multiple GNSS receivers installed in the photographing device 2, but is not limited to this. The photographing position detection device 3 may be a combination of a GNSS receiver and a gyro sensor or a direction sensor. The photographing position detection device 3 may also be multiple beacons. The photographing position detection device 3 may calculate the position coordinates P and orientation of the photographing device 2 by photographing a marker installed at a predetermined position at the construction site whose absolute coordinates are known.

The information acquisition device 4 is a device that acquires various information. The information acquisition device 4 has a wired or wireless communication device 4a and an input device 4b. That is, the information acquisition device 4 can be connected to, for example, an external network N and acquire information from an external server S. The information acquisition device 4 can also be connected to a control device of the mobile crane MC and acquire information from the control device. The information acquisition device 4 can also acquire information input by the input device 4b. The information acquisition device 4 acquires work vehicle information IV including information such as the vehicle weight and posture information of the mobile crane MC, work information IW including information on the transport work such as the weight, shape, lifting position, and hanging position of the load W transported by the mobile crane MC, and ground information IG including the area filled with earth, the area backfilled, the area of the underground structure, the area improved by the ground, and the allowable bearing force B1, which is the external force that the ground in each state can support, from an external server S or a control device of the mobile crane MC. Additionally, the information acquisition device 4 acquires work vehicle information IV, work information IW, and ground information IG through input from the operator via the input device 4b.

The display device 5 is a device that displays a composite image CI of a real image RI and a virtual image VI. The display device 5 also has an operating tool 5a that operates the image capture device 2, the image capture position detection device 3, the information acquisition device 4, and the image processing device 6. The display device 5 is configured, for example, with a touch panel that can input position information on the display screen by touching the display screen. The display device 5 can input operation signals for the image capture device 2 and the image processing device 6 by touching the screen using the touch panel as the operating tool 5a. The operating tool 5a may be provided separately from the display device 5. The display device 5 may be provided in a remote control terminal that remotely controls the mobile crane MC, or in a wearable terminal worn by the operator.

The image processing device 6 is a device that generates a composite image CI by superimposing a real image RI and a virtual image VI. The image processing device 6 may be configured with a CPU, ROM, RAM, HDD, etc. connected via a bus. Alternatively, the image processing device 6 may be configured with a one-chip LSI, etc. The image processing device 6 can control the image capture device 2, the image capture position detection device 3, and the information acquisition device 4 based on an operation signal from an operation tool 5a provided on the display device 5. The image processing device 6 stores various programs and data for controlling the image capture device 2, the image capture position detection device 3, and the information acquisition device 4. The image processing device 6 stores specification data such as the angle of view and focal length of the camera that is the image capture device 2.

The image processing device 6 is connected to the image capturing device 2 by wire or wirelessly. The image processing device 6 can transmit a control signal to the image capturing device 2. In other words, the image processing device 6 can control the image capturing device 2. The image processing device 6 can also acquire a real image RI captured by the image capturing device 2.

The image processing device 6 is connected to the photographing position detection device 3 by wire or wirelessly. The image processing device 6 can transmit a control signal to the photographing position detection device 3. In other words, the image processing device 6 is configured to be able to control the photographing position detection device 3. The image processing device 6 can also acquire position information detected by the photographing position detection device 3. The image processing device 6 can calculate the position coordinate Pc, which is the coordinate of the photographing position of the photographing device 2, and the photographing direction D (see FIG. 2), from multiple position coordinates P acquired from the photographing position detection device 3. The image processing device 6 may calculate the position coordinate Pc and the photographing direction D by photographing a marker installed at a predetermined position whose absolute coordinates are known at the construction site. The image processing device 6 calculates the position coordinate Pc and the photographing direction D of the photographing device 2 using the acquired image of the marker, an IMU (acceleration, angular velocity, geomagnetism, etc.) sensor, and image processing (SLAM: Simultaneous Localization and Mapping, etc.) technology.

The image processing device 6 is connected to the information acquisition device 4 by wire or wirelessly. The image processing device 6 can transmit a control signal to the information acquisition device 4. In other words, the image processing device 6 can control the information acquisition device 4. The image processing device 6 can also acquire the work vehicle information IV, work information IW, and ground information IG acquired by the information acquisition device 4.

The image processing device 6 can generate a virtual image VI showing at least one of the movable range MA, which is the range of the ground surface GS within which the mobile crane MC can move, and the workable range WA, which is the range of the ground surface GS within which the mobile crane MC can work, based on the calculated position coordinates Pc, which is the shooting position of the imaging device 2, and the shooting direction D (see Figure 2), and the acquired work vehicle information IV, work information IW, and ground information IG.

The image processing device 6 is connected to the display device 5 by wire or wirelessly. The image processing device 6 can transmit a control signal and a composite image CI of a real image RI acquired from the imaging device 2 and a generated virtual image VI to the display device 5. The image processing device 6 can also acquire an operation signal input from the operating tool 5a of the display device 5.

<Generation process of composite image CI>
Next, the process of generating a composite image CI will be described with reference to Figs. 1 to 5. Fig. 2 is a process diagram showing the process of generating a composite image CI of the installation position display system 1 according to the first embodiment of the present invention. Fig. 3 is a diagram showing a state in which the ground surface GS is detected from the real image RI by the installation position display system 1 according to the first embodiment of the present invention. Fig. 4(A) is a diagram showing a planar image of a virtual image VI generated by the installation position display system 1 according to the first embodiment of the present invention. Fig. 4(B) is a diagram showing a virtual image VI that can be superimposed on the real image RI generated by the installation position display system 1 according to the first embodiment of the present invention. Fig. 5 is a diagram showing a composite image CI generated by the installation position display system 1 according to the first embodiment of the present invention.

As shown in FIG. 2, the installation position display system 1 can link each device to perform each of the steps of information acquisition process S1, ground surface detection process S2, virtual image generation process S3, and composite image generation process S4. By performing each step, the installation position display system 1 can generate a composite image CI in which a virtual image VI showing at least one of the movable range MA and the workable range WA of the mobile crane MC is superimposed on a real image RI of the construction site (see FIG. 1).

As shown in Figures 1 and 2, the information acquisition process S1 is a process for acquiring various information. In the information acquisition process S1, when the operating tool 5a of the display device 5 is operated, the installation position display system 1 captures a real image RI of the construction site using the imaging device 2. The installation position display system 1 acquires the real image RI captured by the imaging device 2 using the image processing device 6. The installation position display system 1 also acquires multiple position coordinates P calculated by the imaging position detection device 3 using the image processing device 6. The installation position display system 1 also acquires work vehicle information IV, work information IW, and ground information IG from an external server S or the control device of the mobile crane MC using the information acquisition device 4.

2 and 3, the ground surface detection process S2 is a process of detecting the range of the ground surface GS in the real image RI. In the ground surface detection process S2, the installation position display system 1 detects the range of the ground surface GS (shaded portion), which is the image portion of the ground surface in the real image RI, by the image processing device 6. In other words, the image processing device 6 detects the range of the ground surface GS in the real image RI that is not covered by objects C that are obstacles to the mobile crane MC, such as buildings, materials, and vehicles, and does not include a range of a predetermined distance from the obstacles. The image processing device 6 detects the range of the ground surface GS, for example, by unsupervised learning of machine learning. The image processing device 6 calculates the feature amounts of each part in the real image RI. The image processing device 6 divides the calculated feature amounts into groups by a method such as clustering. Furthermore, the image processing device 6 detects the groups of the ground surface GS based on the database acquired by machine learning. In this way, the installation position display system 1 detects the ground surface GS from the real image RI by the image processing device 6, where the mobile crane MC can move without interfering with obstacles. The image processing device 6 may detect the ground surface GS using SLAM technology, such as Lidar (Laser Imaging Detection and Ranging) or ToF (Time-of-Flight) sensors.

2 and 4(A), the virtual image generation process S3 is a process for generating at least one of the movable range MA and the workable range WA of the mobile crane MC, which are virtual image VI. In the virtual image generation process S3, the installation position display system 1 calculates, using the image processing device 6, at least one of the moving support force B2, which is the support force of the ground required to support the mobile crane MC during movement, and the work support force B3, which is the support force of the ground required to support the mobile crane MC during a specified operation, based on the work vehicle information IV and the work information IW. Next, based on the acquired ground information IG including the allowable bearing capacity B1 of the ground, the ground surface GS detected from the real image RI by the image processing device 6, and the calculated moving bearing capacity B2 and working bearing capacity B3 of the mobile crane MC, the installation position display system 1 calculates at least one of the movable range MA (hatched area), which is the range of the ground surface GS having the moving bearing capacity B2 capable of supporting the mobile crane MC while moving, and the workable range WA (dark hatched area), which is the range of the ground surface GS having the working bearing capacity B3 capable of supporting the mobile crane MC while working. The movable range MA is included in the workable range WA.

Furthermore, as shown in FIG. 2 and FIG. 4(B), in the virtual image generation process S3, the installation position display system 1 calculates the position coordinates Pc and the shooting direction D of the imaging device 2 from the position coordinates P of the multiple GNSS receivers using the image processing device 6. Based on the specification data of the imaging device 2 and the calculated position coordinates Pc and shooting direction D of the imaging device 2, the installation position display system 1 generates a virtual image VI in which at least one of the calculated movable range MA and workable range WA is converted into a shape that matches the shape of the ground surface GS as seen from the imaging device 2 in the shooting direction D.

As shown in FIG. 5, the composite image generation process S4 is a process of synthesizing the real image RI and the virtual image VI (see FIG. 2). In the composite image generation process S4, the installation position display system 1 generates a composite image CI by superimposing the virtual image VI on the real image RI using specific coordinates in the real image RI as a reference by the image processing device 6. The composite image CI is an image in which at least one of the movable range MA and the workable range WA on the real image RI is displayed in a distinguishable manner. In the composite image CI, the movable range MA (hatched part) and the workable range WA (dark hatched part) are superimposed as the virtual image VI on the real image RI, which includes the construction site features C, the ground surface GS (shaded part), etc. The installation position display system 1 displays the composite image CI on the display device 5 (see FIG. 1).

The installation position display system 1 thus configured calculates at least one of the movable range MA, which is the range of the ground having a movable support force B2 capable of supporting the mobile crane MC while moving at the current construction site, and the workable range WA, which is the range of the ground having a work support force B3 capable of supporting the mobile crane MC while working, based on the work vehicle information IV, such as the vehicle weight and boom length of the mobile crane MC, the weight of the load W to be transported and the transport position, and the ground information IG, which is related to the support force of the ground. By referring to the composite image CI, the operator of the mobile crane MC recognizes on the real image RI the virtual image VI, which includes the movable range MA or workable range WA of the mobile crane MC according to the progress of the construction at the construction site. This allows the installation position display system 1 to provide information for determining the movement path or installation position of the mobile crane MC based on the work vehicle information IV, work information IW, and ground information IG.

The installation location display system 1 may be composed of a mobile terminal 7 in which the photographing device 2, the photographing location detection device 3, the information acquisition device 4, the display device 5, and the image processing device 6 are integrated. Figure 6 is a diagram showing the use state of the mobile terminal 7 including the installation location display system 1 according to the present invention.

As shown in FIG. 6, the installation position display system 1 is composed of, for example, a mobile terminal 7 equipped with a touch panel. The mobile terminal 7 is composed of a camera which is an image capture device 2, a GNSS receiver and a gyro sensor which are an image capture position detection device 3, a communication device 4a such as a wireless LAN which is an information acquisition device 4, a touch panel which is a display device 5, an operating tool 5a, and an image processing device 6 (see FIG. 2). The installation position display system 1 generates a synthetic image CI based on a real image RI captured from any position desired by the worker.

In the installation position display system 1 configured in this manner, each device is included in a portable mobile terminal 7, so that the positional relationship between the position of the mobile crane MC and the movable range MA or the workable range WA can be confirmed in the composite image CI from any location other than the operator's seat of the mobile crane MC. In this way, the installation position display system 1 can display the range in which the mobile crane MC can move and work at a construction site based on the operator's desired image.

[Modification of the first embodiment]
Next, a modified example of the installation position display system 1 according to the first embodiment will be described with reference to Fig. 7 and Fig. 8. Fig. 7 is a diagram showing a display of the value of the allowable supporting force B1 in the installation position display system 1 according to the first embodiment of the present embodiment. Fig. 8 is a diagram showing a display regarding an alarm by the installation position display system 1 according to the first embodiment of the present embodiment.

As shown in FIG. 7, the installation position display system 1 may display the allowable bearing capacity B1 of a specified location on the screen of the display device 5 as a numerical value. In the composite image generation step S4 (see FIG. 2), the installation position display system 1 generates a composite image CI by superimposing the virtual image VI on the real image RI using specific coordinates on the real image RI as a reference by the image processing device 6. Furthermore, the installation position display system 1 displays the acquired allowable bearing capacity B1 of the ground superimposed on the composite image CI. In other words, the installation position display system 1 displays a distribution map of the allowable bearing capacity B1 and the numerical value of the allowable bearing capacity B1 superimposed on the composite screen. The installation position display system 1 may also be configured to display the moving bearing capacity B2 or working bearing capacity B3 required for the allowable bearing capacity B1.

The installation position display system 1 configured in this manner can display in greater detail at least one of the movable range MA, which is the range of the ground having a movable support force B2 capable of supporting the mobile crane MC while it is moving, and the workable range WA, which is the range of the ground having a working support force B3 capable of supporting the mobile crane MC while it is working, at a construction site. This allows the installation position display system 1 to provide the operator with information for selecting the optimal movement route and installation position based on the value of the ground's allowable support force B1.

As shown in FIG. 8, the installation position display system 1 may display information about a warning when the mobile crane MC is not located in a ground range suitable for work. In the information acquisition step S1, the installation position display system 1 acquires the position coordinate Pm, which is the coordinate of the current position of the mobile crane MC, by the information acquisition device 4. The installation position display system 1 displays the acquired position coordinate Pm of the mobile crane MC by superimposing it on the composite image CI. Furthermore, the installation position display system 1 acquires the state of the mobile crane MC from the control device of the mobile crane MC. When the position coordinate Pm of the mobile crane MC during movement is not included in the movable range MA, or when the position coordinate Pm of the mobile crane MC during work is not included in the workable range WA, the installation position display system 1 notifies the operator by a warning display Wd using a figure, color, text, etc., which is a warning display on the composite image CI of the display device 5 (see FIG. 1).

The installation position display system 1 configured in this manner notifies the operator that the allowable bearing force B1 of the ground is insufficient for the moving bearing force B2 required to support the load generated when the mobile crane MC is traveling, or for the working bearing force B3 required to support the load generated when the mobile crane MC is working. This allows the installation position display system 1 to actively provide information for determining whether the moving path or installation position of the mobile crane MC is appropriate based on the work vehicle information IV, work information IW, and ground information IG.

[Second embodiment]
Next, a crane 11, which is a work vehicle (rough terrain crane) equipped with an installation position display system 1 according to a second embodiment of the present invention, will be described with reference to Fig. 9. Fig. 9 is a side view showing the overall configuration of the crane 11 according to the second embodiment of the present invention. Fig. 10 is a block diagram showing the control configuration of the crane 11 according to the second embodiment of the present invention. Note that in this embodiment, the crane 11 (rough terrain crane) will be described as the work vehicle, but an all-terrain crane, truck crane, loaded truck crane, aerial work vehicle, etc. may also be used. In other words, the present invention is applicable to a work vehicle in which a boom that can be raised and lowered via a swivel base is provided on a traveling body.

As shown in FIG. 9, the crane 11 is a mobile crane that can be moved to any location. The crane 11 has a vehicle 12, a crane device 16 that is a working device, and an installation position display system 1.

The vehicle 12 is a vehicle that transports the crane device 16. The vehicle 12 has multiple wheels 13 and runs using an engine 14 as a power source. The vehicle 12 is provided with outriggers 15. The outriggers 15 have hydraulically extendable extension beams on both sides of the width of the vehicle 12 and hydraulic jack cylinders that can be extended in a direction perpendicular to the ground.

The crane device 16 is a work device that hoists cargo W with a wire rope. The crane device 16 includes a swivel base 17, a boom 19, a jib 19a, a main hook block 20, a sub-hook block 21, a hydraulic cylinder 22 for raising and lowering the load, a main winch 23, a main wire rope 24, a sub-winch 25, a sub-wire rope 26, and a cabin 27.

The swivel table 17 is a drive device that allows the crane device 16 to rotate. The swivel table 17 is mounted on the frame of the vehicle 12 via an annular bearing. The swivel table 17 is configured to be freely rotatable around the center of the annular bearing. The swivel table 17 is provided with a hydraulic swivel motor 18, which is an actuator. The swivel hydraulic motor 18 is an actuator that is rotated by a swivel valve 31 (see FIG. 10), which is an electromagnetic proportional switching valve. The swivel table 17 is provided with a swivel sensor 36 (see FIG. 10), which is a swivel angle detection unit that detects the swivel angle θz (see FIG. 10) and swivel speed of the swivel table 17. The GNSS receiver 28 is provided on the swivel table 17.

The swivel camera 17b (see FIG. 10), which is the imaging device 2, is provided in front of the swivel camera 17 and behind the swivel camera 17. The swivel camera 17b can capture images in any direction using a camera operating tool (not shown).

The boom 19 is a movable support that supports the main wire rope 24 and the sub wire rope 26 so that the load W can be lifted. The boom 19 is composed of multiple boom members. The base end of the base boom member of the boom 19 is swingably mounted at approximately the center of the swivel base 17. The boom 19 is configured to be freely extendable in the axial direction by moving each boom member with an actuator, a hydraulic cylinder for extension and retraction (not shown). The hydraulic cylinder for extension and retraction is operated by an extension valve 32 (see FIG. 10), which is an electromagnetic proportional switching valve.

The boom 19 is provided with a jib 19a and a boom camera 19b (see FIG. 10). The boom 19 is also provided with an extension sensor 37 that detects the extension length lb of the boom 19, and a hoisting sensor 38 (see FIG. 10) that detects the hoisting angle θx (see FIG. 4).

The main hook block 20 and the sub-hook block 21 are hoisting devices for suspending cargo W. The main hook block 20 is provided with a number of hook sheaves around which the main wire rope 24 is wound, and a main hook 20a for suspending cargo W. The sub-hook block 21 is provided with a sub-hook 21a for suspending cargo W.

The hoisting hydraulic cylinder 22 is an actuator that raises and lowers the boom 19 and maintains the position of the boom 19. The end of the cylinder section of the hoisting hydraulic cylinder 22 is swingably connected to the swivel base 17, and the end of the rod section is swingably connected to the base boom member of the boom 19. The hoisting hydraulic cylinder 22 is extended and retracted by the hoisting valve 33 (see Figure 10), which is an electromagnetic proportional switching valve.

The main winch 23 and the sub winch 25 are winding devices that wind up and unwind the main wire rope 24 and the sub wire rope 26. The main winch 23 is rotated by a main hydraulic motor (not shown) that serves as an actuator for the main drum around which the main wire rope 24 is wound. The sub winch 25 is rotated by a sub hydraulic motor (not shown) that serves as an actuator for the sub drum around which the sub wire rope 26 is wound.

The main hydraulic motor is rotated by the main valve 34 (see FIG. 10), which is an electromagnetic proportional switching valve. The sub hydraulic motor is rotated by the sub valve 35 (see FIG. 10), which is an electromagnetic proportional switching valve. The main winch 23 and the sub winch 25 are provided with winding sensors 39 (see FIG. 10) that detect the payout amounts lf (see FIG. 10) of the main wire rope 24 and the sub wire rope 26, respectively.

The cabin 27 is a driver's seat covered by a housing. The cabin 27 is mounted on the swivel base 17. The cabin 27 is provided with a driving operation tool 29 for operating the vehicle 12, a crane device operation tool 30 for operating the crane device 16, a display device 5, etc. (see FIG. 10).

As shown in FIG. 10, the control device 40 controls the actuators of the crane device 16 via each operating valve. The control device 40 may be configured with a CPU, ROM, RAM, HDD, etc. connected via a bus, or may be configured with a one-chip LSI, etc. The control device 40 stores various programs and data for controlling the operation of each actuator, switching valve, sensor, etc.

The control device 40 is connected to the swivel base camera 17b, the boom camera 19b, the travel operation tool 29, and the crane device operation tools 30, which include a swivel operation tool, a hoisting operation tool, a telescopic operation tool, a main drum operation tool, and a sub-drum operation tool. The control device 40 can acquire images from the swivel base camera 17b, images from the boom camera 19b, and the operation amounts of the travel operation tool 29 and the crane device operation tool 30.

The control device 40 is connected to the swing valve 31, the extension valve 32, the elevation valve 33, the main valve 34, and the sub valve 35. The control device 40 can transmit an actuation signal to each valve to operate each actuator to the target actuation amount.

The control device 40 is connected to the rotation sensor 36, the extension sensor 37, the elevation sensor 38, and the winding sensor 39. The control device 40 can acquire the rotation angle θz, extension length Lb, elevation angle θx, and payout amount lf of the main wire rope 24 or sub-wire rope 26 (hereinafter simply referred to as "wire rope") of the rotating table 17.

The control device 40 is connected to the GNSS receiver 28 of the rotating base 17. The control device 40 can calculate the position coordinate P of the center of rotation of the rotating base 17 from the signal of the GNSS receiver 28 of the rotating base 17.

The control device 40 generates a control signal based on the amount of operation of the travel operating device 29 and the crane device operating device 30.

The crane 11 configured in this way can move the crane device 16 to any position by driving the vehicle 12. The crane 11 can also increase the lifting height and working radius of the crane device 16 by adjusting the boom 19 to any hoisting angle θx and extension length Lb using each operating tool. The crane 11 can also transport cargo W by operating the swivel base 17 and boom 19.

The installation position display system 1 is composed of a rotating base camera 17b which is an image capture device 2, a GNSS receiver 28 on the rotating base 17 which is an image capture position detection device 3, an information acquisition device 4, an image processing device 6, and a display device 5.

The photographing position detection device 3 acquires the position coordinate P of the rotation center of the swivel base 17. The photographing position detection device 3 is the GNSS receiver 28 of the swivel base 17. The photographing position detection device 3 can transmit the position coordinate P of the GNSS receiver 28 of the swivel base 17 to the image processing device 6 via the information acquisition device 4.

The information acquisition device 4 is connected to the control device 40 of the crane 11. The information acquisition device 4 can acquire the real image RI acquired by the rotating base camera 17b and the work vehicle information IV via the control device 40. The information acquisition device 4 can also acquire the work vehicle information IV, the work information IW, and the ground information IG from the control device 40 or an external server S. The information acquisition device 4 can also acquire the position coordinate P of the rotation center of the rotating base 17 as the position coordinate Pm of the crane 11 from the control device 40.

The display device 5 is composed of a touch panel. The display device 5 is provided inside the cabin 27 of the crane 11. The display device 5 is positioned and oriented so that it can be seen by the operator in the cockpit.

The image processing device 6 can acquire the real image RI, work vehicle information IV, work information IW, and ground information IG captured by the swivel base camera 17b via the information acquisition device 4. The image processing device 6 can also acquire the position coordinate Pm of the crane 11, the rotation angle θz of the swivel base 17, and the rotation radius from the center of rotation to the swivel base camera 17b via the information acquisition device 4. The image processing device 6 can calculate the position coordinate Pc and shooting direction D (see FIG. 2) of the swivel base camera 17b from the acquired position coordinate P of the swivel base 17, the rotation angle θz, and the rotation radius detected by the rotation sensor 36. The image processing device 6 may be configured integrally with the control device 40 of the crane 11.

The installation position display system 1 for the crane 11 can perform each of the above-mentioned information acquisition process S1, ground surface detection process S2, virtual image generation process S3, and composite image generation process S4 by linking the rotating base camera 17b, GNSS receiver 28, rotation sensor 36, information acquisition device 4, image processing device 6, and display device 5 (see FIG. 2). By performing each process, the installation position display system 1 can generate a composite image CI in which a virtual image VI showing at least one of the movable range MA and the workable range WA of the crane 11 is superimposed on a real image RI of the construction site.

With the crane 11 configured in this way, the installation position display system 1 generates a virtual image VI showing the workable range WA of the crane 11 in accordance with the shape of the ground surface GS in the real image RI. Furthermore, the image processing device 6 generates a composite image CI in which the virtual image VI is superimposed on the ground surface GS in the real image RI. In other words, the crane 11 calculates the range of the ground that can support the work load applied to the ground from the wheels or outriggers 15 during movement or work. This allows the crane 11 to present the operator with the range in which the crane 11 can move and work at the construction site.

[Modification of the second embodiment]
The crane 11 may also be configured to include a mobile terminal 7 including the installation position display system 1 described above. When the mobile terminal 7 is located inside the cabin 27, it is connected to the control device 40 of the crane 11. The mobile terminal 7 acquires a real image RI captured by the rotating base camera 17b as the imaging device 2, and acquires a signal from the GNSS receiver 28 of the rotating base 17 as the imaging position detection device 3. When the mobile terminal 7 is located outside the cabin 27, it acquires a real image RI captured by the built-in camera as the imaging device 2, and acquires a signal from the GNSS receiver of the mobile terminal 7 as the imaging position detection device 3. The mobile terminal 7 can switch between the imaging device 2 and the imaging position detection device 3 to be used depending on the positional relationship with the crane 11. The mobile terminal 7 may also be capable of remotely operating the crane 11.

The crane 11 may also warn the operator by the installation position display system 1 when the work support force B3 required to support the work load from the crane 11 transmitted to the ground during work exceeds a reference value based on the allowable support force B1. The installation position display system 1 calculates the work support force B3 required by the ground to support the work load from the posture information including the rotation angle θz of the rotating base 17 of the crane 11, the extension length Lb and the hoisting angle θx of the boom 19, and the payout amount lf of the wire rope, which are acquired from the control device 40 of the crane 11 by the image processing device 6, and the weight of the luggage W being transported. For example, when a luggage W heavier than the weight of the luggage W included in the work information IW is transported, the installation position display system 1 displays a warning on the display device 5 that the support force of the ground is insufficient when the work support force B3 exceeds a reference value based on the allowable support force B1 of the ground. This allows the crane 11 installation position display system 1 to indicate whether the crane 11 is within a workable range at a construction site based on the work load actually transferred from the crane 11 to the ground.

The above-mentioned embodiment is merely a representative example, and various modifications can be made without departing from the gist of the embodiment. Of course, the present invention can be implemented in various other forms, and the scope of the present invention is indicated by the claims, and further includes the equivalent meanings set forth in the claims, and all modifications within the scope of the claims.

REFERENCE SIGNS LIST 1 Installation position display system 2 Photography device 3 Photography position detection device 4 Information acquisition device 5 Display device 6 Image processing device 7 Portable terminal IV Work vehicle information IW Work information IG Ground information RI Real image VI Virtual image CI Synthetic image GS Ground surface MA Movable range WA Workable range MC Mobile crane

Claims (6)

An installation position display system that displays a ground surface on which a work vehicle having a boom that can be raised or lowered via a swivel base on a traveling body can work,
An imaging device for capturing a real image;
a photographing position detection device for detecting a photographing position and a photographing direction of the photographing device;
an information acquisition device that acquires work vehicle information related to the work vehicle, work information related to the work of the work vehicle, and ground information related to the state of the ground;
an image processing device that detects an image portion of the ground surface included in the real image, calculates at least one of a movable range within which the work vehicle can move on the ground surface and a workable range within which the work vehicle can work, based on the work vehicle information, the work information, and the ground information, generates a virtual image showing at least one of the movable range and the workable range in accordance with a shape of the image portion of the ground surface, and generates a composite image by overlaying the virtual image on the image portion of the ground surface in the real image;
and a display device that displays the composite image. The image processing device includes:
The installation position display system according to claim 1 , wherein the composite image is generated by further superimposing a value of the allowable bearing capacity of the ground on the image portion of the ground surface in the real image from the acquired ground information. The information acquisition device includes:
Acquire a current position of the work vehicle included in the work vehicle information;
The image processing device includes:
The installation location display system according to claim 1 or 2, wherein when the work vehicle is moving and not located within the calculated movable range, or when the work vehicle is working and not located within the workable range, a warning is displayed on the display device. The image processing device includes:
An installation position display system as described in any one of claims 1 to 3, which calculates a moving support force, which is the support force of the ground required to support the load when the work vehicle is traveling, or a working support force, which is the support force of the ground required to support the load when the work vehicle is working, from work vehicle posture information obtained from the work vehicle and the weight of the cargo being transported, and displays a warning on the display device when the moving support force or the working support force exceeds a standard value based on the allowable support force of the ground. The installation location display system according to any one of claims 1 to 4, wherein the photographing device, the photographing location detection device, the information acquisition device, the image processing device, and the display device are integrated into a portable terminal. A work vehicle having a boom that can be raised and lowered via a swivel base attached to a traveling body, the work vehicle comprising the installation position display system according to any one of claims 1 to 5.

Images