JP6969807B2 - Work support system - Google Patents

Description

The present invention relates to a work support system that supports workers' on-site work using an MR (Mixed Reality) device.

Conventionally, a system that automatically arranges equipment at a work site using a VR (virtual reality) device has been known. In this system, a worker wears an HMD (head-mounted display), sets a flag in the shooting range of the camera, and measures the distance such as the frontage and depth of the land based on this. Then, the material data drawn in the 3D image is fused with the virtual space displayed on the HMD, and it is configured to automatically provide the optimum equipment layout based on the patterned and systemized layout data. There is.

In addition, a technique of displaying a virtual reality generated by a computer on an HMD and using it for sales promotion of furniture or education of construction workers has also been proposed. For example, Patent Document 1 describes a "virtual reality system" that changes the layout of furniture in a virtual reality space displayed on an HMD and proposes interior remodeling to an HMD wearer. Patent Document 2 describes an "education support system" in which a learning model is placed in a virtual reality space displayed by an HMD and the learner learns the reinforcement work of reinforcing bars with the same feeling as an actual construction site. There is.

Japanese Unexamined Patent Publication No. 2018-13562 Japanese Unexamined Patent Publication No. 2018-128510

However, according to the system using the conventional VR device, the HMD wearer experiences the work in a simulated reality space, which has the following problems.
(1) Since work such as measurement, inspection, and maintenance cannot be performed unless a person goes to the site, the work cannot be performed until the worker arrives at the site, and the number of sites that one worker can handle. However, the efficiency of on-site work was generally inefficient.
(2) Construction machinery and civil engineering machinery need to acquire many skills for operation and also need to gain experience through practical training, so in order to train inexperienced workers to the point where they can work on-site alone. It took a lot of time.
(3) It is difficult to pass on the technology by desk learning using documents and videos, and the know-how of veterans is lost due to the aging of skilled engineers, and there is an urgent need to deal with the shortage of human resources.

The present invention has been made in view of the above problems, and an object of the present invention is to provide a work support system that enhances the efficiency of on-site work and contributes to human resource development.

In order to solve the above problems, the work support system of the present invention (1) is provided with a MR device carried by the operator (6) (5 1). The MR device, a light transmissive display and (5 11) and the camera (5 14) is provided. The display has a real space transparent unit (31) and a virtual space display unit (32) in a complex manner. Further, the work support system includes a database (12) and a support server (13). The database stores a plurality of work support information that supports on-site work by workers. The support server is connected to the MR device and the database via the network, reads the work support information from the database, and transfers the work support information to the MR device. Then, the MR device captures the scene of the work site in the real space transparent part of the display, transmits the image of the work site taken by the camera to the support server, and sends the work support information received from the support server to the virtual space of the display. It is configured to be displayed on the display unit.

In this onset bright, MR device (51) includes a display (511) formed in the shield-shaped, and a handle (512) to carry the display in the hands of workers.
In the reference form , the MR device (11) is carried by a worker who inspects and maintains the grinder. The MR device watermarks the scene of the work site where the inspection and maintenance of the grinder is performed on the transparent part of the real space of the display (111).
In a preferred embodiment of the present invention, the support server reads the identification code (25) attached to the equipment or equipment at the work site from the image of the camera received from the MR device, and obtains the work support information from the database based on the identification code. read out.

According to the work support system having the above configuration, the worker can easily perform the on-site work based on the work support information received from the support server while visually recognizing the on-site equipment or equipment through the display of the MR device. Therefore, the efficiency of on-site work is improved, and it becomes possible to train inexperienced workers in a short period of time.

It is a perspective view of the site where the inspection and maintenance work of the excavator is performed. It is a block diagram of the work support system which shows one Embodiment of this invention. It is a perspective view which shows the appearance of an MR device. It is a front view which shows the control panel of an excavator. It is a schematic diagram which illustrates the display screen of an MR device. It is a flowchart which shows the overall flow of the field work. It is a flowchart which shows the flow of preparation, inspection, maintenance process. It is a flowchart which shows the work support program. It is a flowchart which shows the procedure which evaluates the technical readiness level of a worker. It is (a) perspective view and (b) side view which shows the modification of the MR device.

(One embodiment)
Hereinafter, an embodiment of the present invention will be described with reference to the drawings. As shown in FIG. 1, the work support system 1 of this embodiment is applied to a work site 3 for inspecting and servicing an excavator 2. At the work site 3, a work table 4 on which the excavator 2 is placed is installed. A drill bit 5 for drilling the ground is attached to the excavator 2, and equipment or equipment (not shown) such as a motor for rotating the drill bit 5 and a control panel is built in the excavator 2. FIG. 1 shows a worker 6 wearing an MR device 11 and working toward a control panel of an excavator 2.

As shown in FIG. 2, the work support system 1 includes an MR device 11 carried by a worker 6, a database 12 that stores work support information related to on-site work, and a support server 13 that provides work support information to the MR device 11. It is composed of an instructor mobile terminal 14 carried by an instructor in a remote location. The MR device 11, the database 12, the support server 13, and the instructor mobile terminal 14 are connected to each other via a network (not shown) such as the Internet so that bidirectional communication is possible.

As shown in FIG. 3, the MR device 11 is an HMD attached to the head of the worker 6 and includes a flexible band member 110 attached to and detached from the head. A light-transmitting display 111 capable of seeing through the work site 3 and a camera 112 for photographing the scene of the work site 3 are attached to the band member 110. The display 111 is rotatably supported up and down with respect to the band member 110, and is arranged at a used position that covers the eyes of the worker and a non-used position that opens.

Further, as shown in FIG. 2, the MR device 11 includes a line-of-sight sensor 113 for detecting the line of sight of the worker 6, a microphone 114 and a speaker 115 used for conversation with an instructor or another worker 6, and inspection or maintenance results. A memory 116 for storing work information such as, a communication unit 117, a control unit 118, and the like are provided. The communication unit 117 manages communication between the MR device 11, the support server 13, and the instructor mobile terminal 14. The control unit 118 controls the entire MR device 11 such as automatically controlling the focus of the camera 112 according to the output of the line-of-sight sensor 113.

The database 12 contains a site information storage unit 121 that stores site information such as the geographical position, geology, and weather conditions of the work site 3, and a work history storage unit 122 that stores work history such as work contents and progress status for each work day. , Worker information storage unit 123 that stores worker information such as the identification number of worker 6 and technical proficiency, parts list storage unit 124 that records the parts list used for parts management, and form information such as estimates or contract documents. A form information storage unit 125 to be stored and a support information storage unit 126 for storing work support information for supporting on-site work by the worker 6 are provided.

The support server 13 is provided with a monitor 130, a communication unit 131 connected to the network, a communication data storage unit 132, and a control unit 133. The communication data storage unit 132 stores communication data between the server 13 and the MR device 11, and communication data between the server 13 and the instructor mobile terminal 14. The control unit 133 is provided with a video analysis unit 134, a support information reading unit 135, a support information updating unit 136, and an image generation unit 137.

The image analysis unit 134 analyzes the image taken by the camera 112 of the MR device 11 and identifies the identification code of the device or equipment included in the image. For example, as shown in FIG. 4, the control panel 20 of the excavator 2 is provided with a tag 25 having a unique identification code attached to a device or equipment such as a switch 21, a relay 22, an instrument 23, and a wiring cable 24. Has been done. The support information reading unit 135 of the control unit 133 reads work support information necessary for on-site work from the support information storage unit 126 of the database 12 based on the identification code of the tag 25.

The support information updating unit 136 updates the work support information of the support information storage unit 126 based on the work report transmitted from the MR device 11 when the on-site work is completed. The image generation unit 137 generates a 3D image for displaying the work support information read by the support information reading unit 135 on the display 111 of the MR device 11. FIG. 5 illustrates the screen 30 of the display 111 that is visible to the eyes of the worker 6. This screen 30 is configured by combining the real space transparent unit 31 and the virtual space display unit 32 in a complex manner, and the scene of the work site 3 (relay 22 is taken out from the control panel 2) in the real space transparent unit 31. Is shown), and work support information (part list 33 is shown) is displayed on the virtual space display unit 32.

Then, the image generation unit 137 executes a process of adjusting the color, size, shape, position, etc. of the 3D image displayed on the virtual space display unit 32 so that the work support information is displayed in an easy-to-see manner by the worker 6. do. As a result, the MR device 11 transparently copies the scene of the work site 3 to the real space transparent unit 31 of the display 111, transmits the image of the work site 3 taken by the camera 112 to the support server 13, and receives it from the support server 13. The work support information is displayed on the virtual space display unit 32 of the display 111.

Next, the operation of the work support system 1 configured as described above will be described. The flowchart shown in FIG. 6 shows the overall flow of field work using the MR device 11. At the start of the work, the worker 6 attaches the MR device 11 to his / her head (S51) and sees through the work site 3 (see FIG. 1) through the display 111. The instructor 7 activates the mobile terminal 14 or the support server 13 in a remote location away from the work site 3, and confirms the image of the work site 3 taken by the camera 112 of the MR device 11 on the monitor 130 (S52).

Next, the worker 6 carries out work such as inspection and maintenance of the excavator 2 (see FIG. 1) (S53), and during this work period, the instructor 7 performs on-site work according to the support program 60 shown in FIG. (S54). Subsequently, the worker 6 saves the work results such as inspection and maintenance in the memory 116 at any time (S55). Then, both the worker 6 and the instructor 7 confirm the completion of the work (S56), and if there is a defect, the process is repeated again. After that, both 6 and 7 create a work report and send it to the support server 13 (S57), and the work this time is completed.

FIG. 7 shows a work support program 60 executed in cooperation with the MR device 11 and the support server 13. Here, first, the MR device 11 transmits the scene of the work site 3 to the real space transmission unit 31 of the display 111 (S61). Next, the camera 112 follows the line of sight of the worker 6 and photographs the equipment at the work site 3 (S62). Next, the communication unit 117 transfers the image of the camera 112 from the MR device 11 to the support server 13 (S63). Subsequently, the support server 13 analyzes the image of the camera 112 by the image analysis unit 134, and the control unit 133 reads the identification code of the tag 25 attached to the device or equipment based on the analysis result (S64).

Next, the support information reading unit 135 of the server 13 searches the database 12 based on the read identification code, and the manual, specifications, parts list, etc. related to the device or equipment corresponding to the identification code are searched from the support information storage unit 126. Read out one or more work support information to help the worker in the current field work (S65). Subsequently, the image generation unit 137 processes the work support information so that it is easy to see and generates a 3D image (S66), and the communication unit 131 transmits the image data from the server 13 to the MR device 11 (S67).

Then, the MR device 11 displays a 3D image on the virtual space display unit 32 of the display 111 (S68), and the worker 6 performs on-site work with reference to the work support information displayed in the 3D image (S69). Therefore, the worker 6 can receive the support information necessary for the work from the server 13 while visually recognizing the situation of the work site 3 by using the MR device 11, and can efficiently perform the work on the site hands-free. Further, in addition to the support information from the server 13, the worker 6 can directly receive guidance or advice from the instructor 7 using the microphone 114 and the speaker 115, so that the inexperienced worker 6 should be trained in a short period of time. Is possible.

FIG. 8 shows a procedure for performing on-site work using work history information. For example, in the field work of the excavator 1, the work history information is used in the preparation process (S70), the inspection process (S80), and the maintenance process (S90). In the preparatory process, a model of equipment arrangement is displayed based on the previous work history information, the worker 6 and the instructor 7 confirm the current situation (S71), confirm the work procedure (S72), set the work goal, and set the work goal. Record (S73). In the inspection process, the inspection items of this time are confirmed by referring to the past work history information (S81), the inspections are executed in the order of items (S82), the inspection results are recorded (S83), and the supervisor's inspection is performed as necessary. Ask for witness (S84). In the maintenance process, maintenance items are confirmed (S91), equipment is repaired or parts are replaced (S92), and maintenance results are recorded (S93). Then, the recorded information of each process is transmitted to the server 13, and the history information of the database 12 is updated.

FIG. 9 shows a procedure for updating worker information according to the technical readiness level of the worker 6. Here, first, the support server 13 reads the worker information from the worker information storage unit 123 of the database 12 (S101). The worker information storage unit 123 includes the current technical readiness level of the worker 6. Next, the actual working time required by the worker 6 is measured (S102). Subsequently, the actual working time is compared with the standard working time predetermined for each process (S103). Then, the technical readiness level in the worker information is updated based on the comparison result (S104). In this way, it is possible to formulate a future work plan according to the technical readiness level of the worker 6.

(Other embodiments)
In the above embodiment, the HMD type device 11 mounted on the head of the worker is exemplified as the MR device, but the shield type device 51 as shown in FIG. 10A can also be used. The MR device 51 includes a light-transmitting shield-shaped display 511, a shield-shaped display 511, a portable handle 512, a line-of-sight sensor 513 that detects the line of sight of the worker 6, and a camera that captures a scene of the work site. 514 is provided. According to the MR device 51, as shown in FIG. 10B, a plurality of workers 6 can share one display 511 and experience on-site work.

Further, in the above embodiment, the inspection / maintenance work of the excavator 2 is exemplified as the use of the work support system 1, but the system of the present invention can be applied to a ground excavation work site, a civil engineering work machine, a construction crane, or the like. It is also possible to apply it to the operation or operation of. In addition, the present invention is not limited to the above embodiment, and the shape and configuration of each part can be appropriately changed and implemented without departing from the spirit of the invention.

1 ... work support system, 3 ... work site, 6 ... workers, 7 ... instructors,
11 ... MR device, 111 ... Display, 112 ... Camera,
113 ... line-of-sight sensor, 114 ... microphone, 115 ... speaker,
12 ... database, 13 ... support server, 14 ... instructor mobile terminal,
31 ... Real space transparent unit, 32 ... Virtual space display unit.

Claims (7)

An MR device (514) carrying a light-transmitting display (511) and a camera (514), which is carried by a worker (6) and has a real space transparent unit (31) and a virtual space display unit (32) in a complex manner. 51) and
A database (12) that stores a plurality of work support information that supports on-site work by the worker, and
Includes a support server (13) that is connected to the MR device and the database via a network, reads the work support information from the database, and transfers it to the MR device.
The MR device includes the display formed in a shield shape and a handle (512) for carrying the display by the worker's hand.
The MR device transmits the scene of the work site to the real space transmission unit of the display, transfers the image of the work site taken by the camera to the support server, and receives the work support from the support server. A work support system configured to display information on the virtual space display unit of the display. The support server reads the identification code (25) attached to the equipment or equipment at the work site from the image of the camera received from the MR device, and reads the work support information from the database based on the identification code. The work support system according to claim 1. The work support system according to claim 1 or 2 , wherein the MR device transmits a work report to the support server when the work is completed. The work support system according to any one of claims 1 to 3 , wherein the MR device transmits recorded information to the support server after parts replacement. The database stores worker information including the technical readiness level of the worker, and the support server has a standard work time determined for each work process and the actual work time of the worker required for the work process. The work support system according to any one of claims 1 to 4 , which updates the technical readiness level of the worker based on the comparison result with the above. The work support system according to any one of claims 1 to 5 , wherein the MR device includes a control unit (118) that automatically controls the focus of the camera according to the movement of the line of sight of the worker. The work support system according to any one of claims 1 to 6 , wherein the MR device includes a microphone (114) and a speaker (115) in which the worker talks with an instructor at a remote location away from the work site.

Images