JP5025567B2 - Operation content reproduction system and operation content reproduction program - Google Patents

Description

  The present invention relates to an operation content reproduction technique for acquiring an operation image obtained by capturing an operation screen on a terminal and operation log information representing an operation on the terminal and reproducing the operation image.

  Conventionally, various studies have been made to monitor terminals connected via a network. For example, in a system in which a remote control server and a client terminal are connected via a communication network, when the operator remotely operates the terminal, the client terminal transmits a monitor screen displayed on the monitor to the remote control server. Then, the received monitor screen is displayed on the monitor, and the captured image recording means of the remote control server records the monitor screen of the client terminal as a captured image (operation image) in the operation event log at a predetermined interval. There is another type in which a log reproduction processing unit of a remote control server reproduces a captured image recorded in an operation event log (see Patent Document 1). In this system, it is possible to monitor the operation content of the terminal by the user by reproducing the operation image stored in the data storage unit on the monitoring monitor.

  Furthermore, in order to visually confirm the user operation on the image, the time position and spatial coordinate data in the video when the user clicks on a specific subject in the video is saved as a click operation event log, and the video is being played back In addition, a technique for superimposing (synthesizing) and displaying the click operation event log is known (see Patent Document 2). With this technique, it is possible to visually confirm a click operation such as what scene the user clicked at the time of reproduction.

JP 2006-108947 A (paragraph numbers 0094 to 0099, FIGS. 1 and 17) Japanese Patent Laying-Open No. 2004-110096 (paragraph numbers [0022] to 0030, FIG. 3)

  In the technology for reproducing the operation image of the terminal and confirming the operation content of the terminal, it is necessary to effectively grasp the operation content without increasing the burden on the administrator.

  However, in the technique of Patent Document 1, even if it is desired to monitor by paying attention to a specific dangerous operation, the operation other than that frequently appears, so that the monitoring work is distracted, and the file can be duplicated and portable. It becomes difficult to reliably find an operation that is prohibited in some cases, such as copying to a memory.

  Further, in the technique of Patent Document 2, a user's click operation on an image that was not visible in real time is displayed as an image at the playback time when the playback is clicked during playback, and information for the administrator to grasp the operation content is displayed. Although it can be provided, the user's operation cannot be grasped in advance at the time of reproduction. Therefore, the administrator must always watch the reproduced screen, and the burden on the administrator is large.

  In view of the above situation, an object of the present invention is to provide a technique for effectively grasping an operation on a terminal by reproducing an operation image on the terminal.

  In order to solve the above problems, the operation content reproduction system of the present invention records operation image information including an operation image obtained by capturing the operation screen on the terminal in the operation content reproduction system for reproducing the operation screen on the terminal. An operation image information recording unit for recording, an operation log information recording unit for recording operation log information in the terminal, and detecting a specific preceding operation and recording operation log information as operation log information of the preceding operation. A previous operation log information acquisition unit acquired from the operation unit, and subsequent operation log information which is operation log information of a subsequent operation following the previous operation from the operation log information recording unit based on the previous operation log information and the extraction condition A subsequent operation log information acquisition unit for generating operation transition information based on the preceding operation log information and the subsequent operation log information A line information creation unit, and a reproduction image creation unit that obtains operation image information corresponding to the preceding operation log information from the operation image information recording unit and creates a reproduction image based on the transition information and the acquired operation image information And.

  With this configuration, when a specific operation is detected, a reproduction image is generated from the operation image related to the specific operation and the transition information, and therefore, the administrator performs any operation after the specific operation. Since it is possible to grasp in advance, it is possible to easily grasp the content of the operation on the terminal without always paying attention to the reproduced image.

  In such an operation content reproduction system, operation images are reproduced and displayed from moment to moment, and it is desirable that the transition information can be confirmed in the displayed operation image in order to effectively grasp the operation content. Therefore, in one preferred embodiment of the present invention, the operation log information corresponding to the date / time information of the operation image to be displayed is used as the preceding operation log information.

  The technical features of the operation content reproduction system according to the present invention described above can also be applied to similar operation content reproduction programs. For example, an operation log information recording unit that records operation log information in the terminal, and an operation image information recording unit that records operation image information including an operation image obtained by capturing an operation screen in the terminal, In an operation content reproduction program for an operation content reproduction system for reproducing an operation screen, a preceding operation log information acquisition function for detecting a specific preceding operation and acquiring preceding operation log information which is operation log information of the preceding operation; A subsequent operation log information acquisition function for acquiring subsequent operation log information that is operation log information of a subsequent operation following the preceding operation based on the preceding operation log information and the extraction condition; the preceding operation log information and the subsequent operation log A migration information creation function for creating operation migration information based on information, and an operation image corresponding to the preceding operation log information Gets the broadcast, is realized on the computer and reproduced image generating function of generating a reproduced image based on the transition information and the acquired operation image information. Naturally, such an operation content reproduction program can also obtain the effects described in the operation content reproduction system described above, and can further incorporate the above-described additional technology.

[First Embodiment]
Hereinafter, a first embodiment of the present invention will be described with reference to the drawings. In the image display system of the present invention, as shown in FIG. 1, a plurality of terminals C are connected to a management server S via a communication network (hereinafter referred to as a network) N. The network N is a CSMA / CD (Carrier Sense Multiple Access with Collision Detection) system including a cable for physically transmitting information in a limited facility such as a company or a school and a relay device such as a LAN switch or a hub. Ethernet (trademark) type LAN, but this network N is constructed by an intranet using Internet technology in addition to the Ethernet type LAN, or WAN (Wide Area Network) ).

  The management server S includes a general-purpose computer 1 and includes a display 2 and an input device 3 (keyboard 3a and mouse 3b). The terminal C includes a computer main body 4 to which a display 5 and an input device 6 (a keyboard 6a and a mouse 6b) are connected.

   FIG. 2 shows a functional block diagram of the management server S and terminal C constituting the operation content reproduction system of the present invention. The terminal C generates operation log information representing various operations in the network I / F 10 and terminal C for transmitting / receiving various information to / from the management server S and other terminals C via the network N, and sends them to the management server S. The operation log information generation unit 11 to transmit, the operation image information including the operation image obtained by capturing the operation screen on the terminal C, and the operation image information generation unit 12 to transmit to the management server S and the input device 6 A GUI unit 13 that acquires input and inputs / outputs with the display 5 is provided.

  Normally, the operation log information generation unit 11 and the operation image information generation unit 12 execute the processing by reading the hardware (program, module, etc.) that executes the processing into hardware. It may be configured by a combination of the above, or may be configured only by hardware combining logic and the like.

  In the present embodiment, the operation log information includes operation contents, user identification information, terminal identification information, operation target information, date information, and environment information. The operation target information includes the name of the application that performs the operation, the name of the file to be operated (hereinafter collectively referred to as a file name), the position of the icon to be operated, the position of the mouse operation, and the operation. Position information indicating the window position of the application is included, and the environment information is various settings and environments in the terminal C, for example, whether or not the USB memory is used. The user identification information is information that can uniquely identify the user, and a user name, a unique number assigned exclusively to each user, or the like can be used. The terminal identification information uniquely identifies the terminal C. This is identifiable information such as a terminal name, an IP address, a MAC address, and a unique number assigned exclusively to each terminal.

  On the other hand, the operation image information in the present embodiment includes an operation image obtained by capturing an operation screen of the terminal C, terminal identification information, user identification information, and date / time information. The various information described above is not limited to these, and other information may be used as long as the object of the present invention is achieved.

  The operation log information generation unit 11 generates information representing various operations on the terminal C at a predetermined timing. In the present embodiment, the predetermined timing represents the timing at which the predetermined operation is performed and every predetermined interval, but is not limited to this, and other timings may be used. The generated operation log information is transmitted to the management server S via the network I / F 10 and the network N. The operation log information may be transmitted at the same time as the operation log information is generated, or the operation log information generated at a predetermined interval may be accumulated and then transmitted together.

  The operation image information generation unit 12 captures an operation screen displayed on the display 5 of the terminal C at a predetermined time interval via the GUI unit 13 to acquire an operation image, and also acquires operation image information including the operation image. Generate. Note that the operation image information generation timing is not limited to this, and may be other timings, and may be synchronous / asynchronous with the operation log information generation timing. The generated operation image information is transmitted to the management server S via the network I / F 10 and the network N. The operation image information may be transmitted at the same time as the operation log information is generated, or the operation image information generated at a predetermined interval may be accumulated and then transmitted together.

  The management server S acquires a network I / F 20 for transmitting and receiving various information to and from the terminal C via the network N, an operation log information recording unit 21 for acquiring and storing operation log information transmitted from the terminal C, and a terminal The operation image information recording unit 22 that acquires the operation image information transmitted from C, stores and manages it, the preceding operation log information that detects the preceding operation and acquires the operation log information of the preceding operation from the operation log information recording unit 21 The acquisition unit 23, the subsequent operation log information acquisition unit 24 that acquires the subsequent operation log information related to the subsequent operation following the previous operation from the operation log information recording unit 21 based on the acquired previous operation log information and a predetermined extraction condition, A migration information creation unit 25 that creates operation migration information from the operation log information and the subsequent operation log information, and the operation image information corresponding to the preceding operation log information A playback image creation unit 26 that creates a playback image based on the acquired operation image information and migration information, an image display unit 27 that displays the playback image created by the playback image creation unit 27, and A GUI unit 30 is provided for acquiring input from the input device 3 and controlling input / output with the display 2.

  Usually, the operation log information recording unit 21, the operation image information recording unit 22, the preceding operation information acquisition unit 23, the subsequent operation log information acquisition unit 24, the transition information generation unit 25, the reproduction image generation unit 26, and the image display unit 27 are The processing is executed by loading the processing means (program, module, etc.) into hardware, but these may be configured in combination with hardware, or only hardware combining logic etc. You may comprise.

  The operation log information recording unit 21 and the operation image information recording unit 22 respectively acquire and manage operation log information and operation image information transmitted from the terminal C via the network I / F 20.

  The preceding operation log information acquisition unit 23 acquires operation image information from the operation image information recording unit 22, determines the presence / absence of operation log information corresponding to the acquired operation image information, and operates corresponding to the acquired operation image information When it is determined that the log information exists, it is further determined whether or not the operation log information is related to the preceding operation. If it is determined that the operation log information is the operation log information related to the preceding operation, the operation log information is preceded by Acquired as operation log information. In the present embodiment, the preceding operation is a predetermined operation, and various operations such as file copying and deletion can be set as the predetermined operation. The acquired preceding operation log information is sent to the subsequent operation log information acquisition unit 24 and the migration information creation unit 25.

  The subsequent operation log information acquisition unit 24, based on the previous operation log information acquired by the previous operation log information acquisition unit 23 and a predetermined extraction condition, the operation log information of the subsequent operation following the previous operation (hereinafter, the subsequent operation log). (Referred to as information) from the operation log information recording unit 21. Specifically, an operation log that has the same terminal identification information as the terminal identification information of the preceding operation log information, has operation date information after the date and time information of the preceding operation log information, and satisfies a predetermined extraction condition Information is acquired from the operation log information recording unit 21. The acquired subsequent operation log information is sent to the migration information creation unit 25.

  The migration information creation unit 25 creates migration information based on the preceding operation log information acquired by the preceding operation log information acquisition unit 23 and the subsequent operation log information acquired by the subsequent operation log information acquisition unit 24. The transition information in the present embodiment is information indicating a transition from the state in the preceding operation to the state in the subsequent operation. For example, information indicating the contents of the subsequent operation, information indicating the movement from the position of the preceding operation to the position of the subsequent operation, time until the subsequent operation is performed from the preceding operation, information indicating the risk of the subsequent operation, operation of the subsequent operation Information representing the method can be used. Of course, the migration information can be appropriately changed as long as the object of the present invention is achieved. The created transition information is sent to the reproduction image creation unit 26 together with the preceding operation log information.

  The reproduction image creation unit 26 acquires operation image information corresponding to the preceding operation log information acquired by the preceding operation log information acquisition unit 23 from the preceding operation log information acquisition unit 23, and acquires the acquired operation image information and the transition information generation unit. A reproduction image is created based on the transition information created in step 25. The association between the operation log information and the operation image information is performed based on each terminal identification information and date information. The created reproduced image is sent to the image display unit 27 and displayed on the display 2 of the management server S via the GUI unit 30.

  Next, a specific processing flow of the present embodiment will be described with reference to the flowchart of FIG. The specific operation in this embodiment is an operation for opening a file or a file selection operation. Prior to the processing described below, operation log information and operation image information are transmitted from each terminal C to the management server S. It is assumed that they are recorded in the operation log information recording unit 21 and the operation image information recording unit 22 of the management server S.

  An example of operation log information and operation image information at this time is shown in FIG. The figure shows operation log information and operation image information arranged in chronological order from the top to the bottom. The operation log information has a name such as Operation001, and the operation image information has a name such as Image001. Yes. Note that the operation log information and operation image information shown in this figure are for one terminal C, and display of terminal identification information and the like is omitted. The right column of the operation log information indicates the operation content and environment information included in the operation log information, and the operation image information and the operation log information on the same line correspond to each other. In the present embodiment, when the date and time information of the operation image information and the operation log information match or the difference between them is equal to or less than a predetermined value, the operation log information and the operation image information correspond to each other.

  First, when the administrator operates the input device 3 to instruct the management server S to start playback of the operation content via the GUI unit 30, the following processing is started. At this time, the start date and time and the end date and time for executing reproduction are also instructed.

  In the normal reproduction state, the preceding operation log information acquisition unit 23 sequentially acquires the operation image information from the specified start date and time to the end date and time from the operation image information recording unit 22, and uses the acquired operation image information as an image. The image display unit 27 sequentially sends the operation images included in the operation image information (# 01 → No branch of # 13 → # 11 → # 12).

  Simultaneously with the above reproduction operation, the preceding operation log information acquisition unit 23 determines whether or not operation log information corresponding to the acquired operation image information exists in the operation log information recording unit 21 (# 02). In the example of FIG. 4, when the operation image information of Image000 is acquired by the preceding operation log information acquisition unit 23, it is determined that there is no operation log information corresponding to the operation image information Image000 (No branch of # 02) ).

  In this case, the preceding operation log information acquisition unit 23 is notified to the image display unit 27 that there is no operation log information corresponding to the acquired operation image information, and the image display unit 27 stores valid transition information. The presence or absence is determined (# 13). In this case, since there is no migration information (No branch of # 13), the image display unit 27 displays the operation image included in the operation image information Image000 on the display 2 (see FIG. 5A).

  On the other hand, when the operation image information Image001 is acquired by the preceding operation log information acquisition unit 23 (# 01), it is determined that the corresponding operation log information Operation000 exists (Yes branch of # 02).

  Next, the preceding operation log information acquisition unit 23 determines whether or not the operation log information corresponding to the acquired operation image information is operation log information related to the specific operation (# 03), and the operation log information of the specific operation (Yes branch of # 03), the operation log information is acquired as preceding operation log information (# 04). When Image001 is acquired as the operation image information described above, the preceding operation log information acquisition unit 23 determines whether or not the operation content of the operation log information Operation000 is a specific operation. In this case, since the operation content of the operation log information Operation000 is “USB memory mounted”, it is determined that the operation is not a specific operation (No branch of # 03).

  The preceding operation log information acquisition unit 23 notifies the image display unit 27 that the corresponding operation log information is not the preceding operation log information, and the image display unit 27 determines whether there is valid transition information (# 13). In this case, since there is no migration information (No branch of # 13), the image display unit 27 displays the operation image included in the operation image information Image001 on the display 2.

  On the other hand, when Image002 is acquired by the preceding operation log information acquisition unit 23 (# 01), the corresponding operation log information Operation001 exists (Yes branch of # 02), and the operation content is “file A as application W. The operation log information Operation001 is acquired as the preceding operation log information (# 04), and the subsequent operation log information acquisition unit 24 and the migration information creation unit 25 are selected. Sent to.

  The subsequent operation log information acquisition unit 24 that acquired the previous operation log information acquires the subsequent operation log information based on the previous operation log information and a predetermined extraction condition (# 05). Specifically, the subsequent operation log information acquisition unit 24 has the same terminal identification information as the terminal identification information of the previous operation log information among the operation log information recorded in the operation log information recording unit 21. The operation log information having date information after the date information and satisfying a predetermined condition is acquired as subsequent operation log information. The acquired subsequent operation log information is sent to the migration information creation unit 25.

  When operation log information Operation001 is acquired as preceding operation log information, operation log information Operation002 to Operation010 having the same terminal identification information as operation log information Operation001 and date / time information after operation log information Operation001 is extracted. Is done. Here, if the predetermined extraction condition is “operation related to the same file”, operation log information Operation002, Operation003, and Operation004 with “File A” that is the operation target of the operation log information Operation001 are the subsequent operation log information and Become.

  The subsequent operation log information may be one operation log information or a plurality of operation log information. In the former case, of the operation log information satisfying the above conditions, the operation log information immediately after the preceding operation log information is the subsequent operation log information.

  The migration information creation unit 25 that has acquired the preceding operation log information and the subsequent operation log information from the preceding operation log information acquisition unit 23 and the subsequent operation log information acquisition unit 24 obtains the migration information based on the preceding operation log information and the subsequent operation log information. Create (# 06). The created transition information is sent to the reproduction image creation unit 26 together with the preceding operation log information.

  As described above, the transition information in the present embodiment is information indicating the content of the subsequent operation, information indicating movement from the position of the preceding operation to the position of the subsequent operation, time until the subsequent operation is performed from the preceding operation, etc. This is information indicating a transition from the state in the preceding operation to the state in the subsequent operation. In the case of the above-described example, the migration information includes character information reflecting the operation contents of the subsequent operation log information such as “File A is saved → selected → copied to USB memory”, and icon / mouse position information. Information indicating the trajectory, time until subsequent operation, and the like.

  The reproduction screen creation unit 26 that has acquired the preceding operation log information and the migration information from the migration information creation unit 25 obtains the operation image information corresponding to the preceding operation log information, and the operation image and the migration information included in the obtained operation image information. A reproduction image is created from the above (# 08). At this time, it is determined whether or not the acquired operation image information is after the date and time information of the last subsequent operation log information (# 09). If it is after (Yes branch of # 09), the migration information is discarded (# 10). ). The created reproduced image is displayed on the display 2 by the image display unit 27 via the GUI unit 30 (# 11).

  In the above example, the operation image information corresponding to the preceding operation log information Operation001 is Image002, and for the operation image included in the operation image information Image002, the character string “file A is saved → selected → USB memory as migration information” A reproduced image on which “is copied” is superimposed is created (see FIG. 5B).

  When the operation image information Image003 is acquired by the preceding operation log information acquisition unit 23, there is no corresponding operation log information (No branch of # 02), and there is migration information (# 13). Therefore, the preceding operation log information acquisition unit 23 notifies the reproduction image creation unit 26 to that effect, and the reproduction image creation unit 26 reproduces from the operation image included in the operation image information Image003 and the above-described transition information. Create an image.

  If the operation image information Image004 is acquired by the preceding operation log information acquisition unit 23, the corresponding operation log information Operation002 exists (Yes branch of # 02), but “save” is not a specific operation. (No branch at # 03). In this case as well, since the transition information exists (Yes branch of # 13), the preceding operation log information acquisition unit 23 notifies the reproduction image generation unit 26 to that effect, and the reproduction image generation unit 26 creates a reproduction image from the operation image included in the operation image information Image004 and the above-described transition information.

  By repeating the above process, a reproduced image shown in FIG. 5 is obtained.

  In the above processing, if the predetermined extraction condition is “operation related to the same application”, the operation log information Operation002 and Operation005 having the same operation target application as the operation target application “application W” of the operation log information Operation001 will follow. If it becomes operation log information and the transition information at this time is information indicating the content of the subsequent operation, the reproduced image of FIG. 6 is obtained. At this time, it is also possible to include the position information of the application window in the operation log information and display the transition information and the application window related to a predetermined extraction condition so as not to overlap (see FIG. 6D). .

  It is also possible to set the extraction condition based on the environment information. For example, in the above example, if the predetermined extraction condition is “USB memory mounted”, the operation log information from Operation 002 to Operation 005 is the subsequent operation log. It becomes information and the reproduced image of FIG. 7 is obtained.

[Second Embodiment]
Next, a second embodiment of the operation content reproduction system of the present invention will be described with reference to the drawings. FIG. 8 is a functional block diagram in the present embodiment, and the same reference numerals are given to the same functional units as those in the first embodiment.

  The operation content reproduction system of this embodiment is different from the first embodiment in that an interpolation image creation unit 28 is provided. The interpolated image creating unit 28 normally executes the processing by reading the means (program, module, etc.) for executing the processing into hardware, but these may be configured in combination with hardware. However, it may be configured only by hardware combining logic and the like.

  When there is no operation image information that matches the date / time information of the preceding operation log information, the interpolation image creating unit 28 creates the operation screen information on which the date / time when the preceding operation log information was generated is displayed. Specifically, the operation image information is created based on the operation screen information and the preceding operation log information immediately before the date and time information of the preceding operation log information, or the operation image information immediately before and immediately after the date and time information of the preceding operation log information. A method of creating an operation image from the included operation image by interpolation can be used. In the present embodiment, the former process will be described.

  FIG. 9 is a flowchart showing the processing flow of the present embodiment. First, as in the first embodiment, the administrator starts the reproduction of the operation content and instructs the start date and time and the end date and time to execute the reproduction, so that the following processing is started.

  First, the instructed start date and time is acquired as the reproduction date and time (# 21), and operation log information having the same date and time information as the reproduction date and time is acquired from the operation log information recording unit 21 (# 22). If the operation log information is acquired at this time (Yes branch of # 23), it is determined whether or not the acquired operation log information is the preceding operation log information (# 24).

  If it is determined that the preceding operation log information is the preceding operation log information (Yes branch of # 24), the preceding operation log information is sent to the subsequent operation log information acquisition unit 24 and the migration information creation unit 25.

  The subsequent operation log information acquisition unit 24 that acquired the previous operation log information acquires the subsequent operation log information based on the previous operation log information and a predetermined extraction condition (# 25). The acquired subsequent operation log information is sent to the migration information creation unit 25.

  The migration information creation unit 25 that has acquired the preceding operation log information and the subsequent operation log information creates migration information based on the preceding operation log information and the subsequent operation log information (# 26). The created transition information is sent to the reproduction image creation unit 26 together with the preceding operation log information.

  The reproduction screen creation unit 26 that has acquired the preceding operation log information and the migration information from the migration information creation unit 25 inquires of the operation image information recording unit 22 whether there is operation image information having the same date and time information as the previous operation log information. (# 27) If it exists (Yes branch of # 27), the operation image information is acquired (# 28), and a reproduction image is created from the operation image and the transition information included in the acquired operation image information. (# 29). The created reproduction image is displayed on the display 2 by the image display unit 27 via the GUI unit 30 (# 30), and a predetermined reproduction interval time is added to the current reproduction date and time, and the reproduction date and time is updated ( # 31). At this time, it is determined whether or not the updated reproduction date and time is after the date and time information of the last subsequent operation log information (# 32), and if it is after (Yes branch of # 32), the transition information is discarded (# 33). ).

  Thereafter, the current reproduction date and time and the end date and time are compared (# 34). If the end date and time have not been reached (No branch at # 34), the above-described processing is repeated.

  If there is no operation image information having date and time information that matches the date and time information of the preceding operation log information (No branch at # 27), an interpolation image is created by the interpolation image creation unit 28 (# 35). Specifically, operation screen information having date / time information immediately before the date / time information of the operation log information acquired in # 24 is acquired from the operation image information recording unit 22, and the acquired operation image information and its operation log information are acquired. Based on the above, an interpolated image is created. The created interpolated image is transferred to the reproduction image creating unit 26.

  The reproduction image creation unit 26 that has acquired the interpolation image creates a reproduction image based on the transition information and the interpolation image (# 29), and the created reproduction image is displayed on the display 2 by the image display unit 27 (#). 30).

  On the other hand, when there is no operation log information having the same date and time information as the playback date and time (No branch of # 23), the playback screen creation unit 26 acquires the operation image information of the playback date and time from the operation image information recording unit 22 (# 28) and a reproduced image is created (# 29). The created reproduction image is sent to the image display unit 27 and displayed on the display 2 (# 30).

  The above-described processing is repeated until the reproduction date / time reaches the designated end date / time (Yes branch of # 34).

[Third Embodiment]
Next, a third embodiment of the operation content reproduction system of the present invention will be described. In the present embodiment, the operation of the preceding operation log information acquisition unit 23 is different from that in the first embodiment and the second embodiment described above.

  The preceding operation log information acquisition unit 23 in the present embodiment uses the operation log information corresponding to the operation image information including the operation image displayed by the image display unit 27 as the preceding operation log information.

  The flow of processing at this time will be described based on the flowchart of FIG. The same processing steps as those in the first embodiment are denoted by the same reference numerals.

  In the normal reproduction state, as in the first embodiment, the image display unit 27 acquires the operation image information from the specified start date and time to the end date and time from the operation image information recording unit 22 and stores the operation image information in the operation image information. The operation images included are sequentially displayed (# 01 → No branch of # 13 → # 11 → # 12).

  Simultaneously with the above-described reproduction operation, the preceding operation log information acquisition unit 23 determines whether operation log information corresponding to the operation image information acquired by the image display unit 26 exists in the operation log information recording unit 21. (# 02). If operation log information corresponding to the operation image information exists (Yes branch of # 02), the operation log information is acquired as preceding operation log information (# 04) and sent to the subsequent operation log information acquisition unit 24. .

  The subsequent operation log information acquisition unit 24 that acquired the previous operation log information acquires the subsequent operation log information by comparing the previous operation log information with a predetermined condition. However, in the present embodiment, unlike the first embodiment, the same terminal identification as the terminal identification information of the preceding operation log information is satisfied from the operation log information recorded in the operation log information recording unit 21 by satisfying a predetermined condition. Of the operation log information having information and date / time information after the date / time information of the preceding operation log information, only the earliest operation log information is acquired as subsequent operation log information.

  The subsequent processing is the same as the processing flow in the first embodiment, and the reproduced image shown in FIG. 11 is obtained by repeating the above processing. In this embodiment, as can be seen from the figure, the transition information also changes as the playback image changes.

[Fourth Embodiment]
Next, a fourth embodiment of the operation content reproduction system of the present invention will be described. In the present embodiment, the operation of the subsequent operation log information acquisition unit 24 is different from the first embodiment and the second embodiment described above.

  The subsequent operation log information acquisition unit 24 in the present embodiment acquires subsequent operation log information based on an operation related table (not shown) that records the relationship between operations.

  In the operation relation table, operations having a correlation between the first operation and the second operation are recorded in association with each other. FIG. 12 is an example of the contents of the operation-related table. “Select” and “Copy”, “Select” and “Delete”, “USB memory installation” and “Select” are the first operation and the second, respectively. It is recorded as an operation. At this time, the subsequent operation log information acquisition unit 24 acquires the second operation associated with the first operation using the operation content of the preceding operation log information as the first operation, and the operation log information recording unit 21. From the operation log information having the same terminal identification information as the terminal identification information of the preceding operation log information, the date and time information after the date and time information of the preceding operation log information, and the operation content being the second operation. Acquire operation log information as subsequent operation log information.

  Next, the flow of processing in this embodiment will be described using the flowchart of FIG. It is assumed that the operation log information and the operation image information of FIG. 4 are recorded in the operation log information recording unit 21 and the operation image information recording unit 22 prior to the processing of FIG. Further, it is assumed that the specific operation is the first operation recorded in the operation related table.

  First, one piece of operation image information is acquired from the operation image information recording unit 22 (# 01), and the presence / absence of corresponding operation log information is determined (# 02). In this example, when Image001 is first acquired, it is determined that Operation000 exists as the corresponding operation log information. Here, the operation content of the operation log information Operation000 is “attaching USB memory”, and it is determined that the operation is the specific operation. (Yes branch of # 03). Therefore, the operation log information Operation000 is acquired as preceding operation log information (# 04).

  The preceding operation log information Operation000 is sent to the subsequent operation log information acquisition unit 24, and the subsequent operation log information acquisition unit 24 is associated with “USB memory mounting” that is the operation content of the preceding operation log information Operation000 from the operation related table. “Select” is acquired as the second operation. The subsequent operation log information acquisition unit 24 further acquires the earliest operation log information whose operation content is “selected” among the operation log information after the previous operation log information Operation000, that is, Operation003 as the subsequent operation log information ( # 05).

  At this time, transition information is created from the preceding operation log information Operation000 and the subsequent operation log information Operation003 (# 06), and the reproduced image of FIG. 13A is created (# 08).

  Thereafter, when the operation image information Image006 is acquired by the same processing, Operation003 becomes the preceding operation log information, and the subsequent operation log information at this time becomes Operation004, and the reproduced image of FIG. 13C is created.

  By defining the preceding operation and the subsequent operation as related operations in this way, it is possible to generate a reproduction image reflecting transition information representing a series of related operations, and accurately grasp the content of the operation at the terminal C. It becomes possible.

[Another embodiment]
(1) In the above-described embodiment, the content of the operation is used as the migration information, but various information useful for grasping the content of the operation can be used. For example, the time until the operation is executed can be displayed together with the operation content. By displaying the time until the operation is executed together with the operation content as the transition information on the reproduced image, the administrator can grasp the time until the operation to be noticed, thereby avoiding the oversight of the operation. it can.

  (2) In the above-described embodiment, the content of the operation is displayed on the reproduced image as transition information. However, the transition information displayed on the reproduced image is not limited to character information. For example, if an operation with the mouse is performed, the transition information is displayed on the reproduced image by displaying a diagram of the mouse or connecting the position at which the subsequent operation is performed with a straight line from the position at which the previous operation is performed. be able to. Displaying such a diagram or the like on the reproduced image is preferable because the administrator can be alerted to watch the operation.

  (3) The arrangement of the functional units of the operation content reproduction system of the present invention can be appropriately changed to the terminal C and the management server S. For example, it is possible to construct a stand-alone system in which all functional units are provided in the terminal C. In addition, the operation log information recording unit 21 and the operation image information recording unit 22 can be arranged in the terminal C or a separately installed server. Further, the transition information creation unit 25, the reproduction image creation unit 26, and the image display unit 27 can be arranged in the terminal C. Of course, the arrangement of the functional units is not limited to the above, and can be appropriately changed as long as the object of the present invention is achieved.

  (4) In the above-described embodiment, the subsequent operation log information is acquired and the operation log information and the operation image information are associated with each other based on the terminal identification information, but may be based on the user identification information.

  (5) In the above-described embodiment, the transition information is displayed on the operation image. However, the display position of the transition information is not limited to this, and is displayed in the vicinity of the operation image (see FIG. 14). Other display positions can be used. Thus, it is preferable to display the transition information so that it does not overlap with the operation image because the contents of the operation image can be easily grasped together with the transition information.

  (6) In the above-described embodiment, information indicating the content of the subsequent operation is used as the transfer information. However, as described above, various information can be used as the transfer information. FIG. 15 is an example in which a change in position information from a position related to a preceding operation to a position related to a subsequent operation is used as transition information. In this example, after file A is selected (see FIG. 15A), “Send” and “USB memory” are selected from the menu list (see FIG. 15B), so that file A becomes USB. It shows how it is copied to memory. At this time, when the selection operation of the file A is acquired as the preceding operation, the copy operation to the USB memory is acquired as the subsequent operation, and the operation from the operation position of the preceding operation to the operation position of the subsequent operation is related to the subsequent operation. The window position is represented as a dotted line as transition information. By using such transition information, the administrator can know the position related to the subsequent operation in advance, and can grasp the subsequent operation only by gazing at the position, which is preferable.

  FIG. 16 is an example in which the time from the preceding operation to the subsequent operation is used as the transition information. As described above, it is preferable to use the time from the preceding operation to the subsequent operation as the migration information because the administrator can grasp the time until the operation image related to the subsequent operation is displayed.

  FIG. 17 is an example in which a graphic for alerting the administrator is used as the migration information when the subsequent operation is an operation to be noted. In this example, the file A is selected as the preceding operation, the copy of the file A to the USB memory is acquired as the succeeding operation, and “!” For calling attention is displayed as the migration information. At this time, it is preferable that the display position of the transition information is a position related to the subsequent operation because the administrator can pay attention to the position related to the subsequent operation.

  As described above, various kinds of information can be used as the migration information, and can be appropriately changed as long as the object of the present invention is achieved.

System configuration diagram of operation content reproduction system according to the present invention The functional block diagram in 1st Embodiment of the operation content reproduction | regeneration system by this invention The flowchart showing the flow of the process in 1st Embodiment of the operation content reproduction | regeneration system by this invention. Examples of operation log information and operation image information in the first embodiment of the operation content reproduction system according to the present invention Example of replay image in the first embodiment of the operation content replay system according to the present invention Example of replay image in the first embodiment of the operation content replay system according to the present invention Example of replay image in the first embodiment of the operation content replay system according to the present invention The functional block diagram in 2nd Embodiment of the operation content reproduction | regeneration system by this invention The flowchart showing the flow of the process in 2nd Embodiment of the operation content reproduction | regeneration system by this invention. The flowchart showing the flow of the process in 3rd Embodiment of the operation content reproduction | regeneration system by this invention. Examples of replayed images in the third embodiment of the operation content replaying system according to the present invention Examples of operation related tables in the fourth embodiment of the operation content reproduction system according to the present invention Examples of reproduced images in the fourth embodiment of the operation content reproducing system according to the present invention Example of replay image in another embodiment of operation content replay system according to the present invention Example of replay image in another embodiment of operation content replay system according to the present invention Example of replay image in another embodiment of operation content replay system according to the present invention Example of replay image in another embodiment of operation content replay system according to the present invention

Explanation of symbols

S: Management server C: Terminal 10, 20: Network I / F
11: Operation log information generation unit 12: Operation image information generation unit 13, 30: GUI unit 21: Operation log information recording unit 22: Operation image information recording unit 23: Predecessor operation log information acquisition unit 24: Subsequent operation log information acquisition unit 25: Transition information creation unit 26: Playback image creation unit 27: Image display unit

Claims (3)

In the operation content playback system that plays back the operation screen on the terminal,
An operation image information recording unit that records operation image information including an operation image obtained by capturing an operation screen in the terminal;
An operation log information recording unit for recording operation log information in the terminal;
A predecessor operation log information acquisition unit that detects a specific predecessor operation and acquires predecessor operation log information that is operation log information of the predecessor operation from an operation log information recording unit;
A subsequent operation log information acquisition unit that acquires subsequent operation log information that is operation log information of a subsequent operation following the preceding operation from the operation log information recording unit based on the preceding operation log information and the extraction condition;
A migration information creation unit that creates migration information of an operation based on the preceding operation log information and the subsequent operation log information;
Operation content reproduction comprising operation image information corresponding to the preceding operation log information from the operation image information recording unit, and a reproduction image creation unit that creates a reproduction image based on the transition information and the acquired operation image information system. The operation content reproduction system according to claim 1, wherein operation log information corresponding to date and time information of an operation image to be displayed is used as preceding operation log information. An operation log information recording unit for recording operation log information in the terminal; and an operation image information recording unit for recording operation image information including an operation image obtained by capturing the operation screen in the terminal. In the operation content playback program for the operation content playback system
A predecessor operation log information acquisition function for detecting a specific predecessor operation and acquiring predecessor operation log information that is operation log information of the predecessor operation;
A subsequent operation log information acquisition function for acquiring subsequent operation log information that is operation log information of a subsequent operation following the preceding operation based on the preceding operation log information and the extraction condition;
A migration information creation function for creating migration information of operations based on the preceding operation log information and the subsequent operation log information;
An operation content reproduction program for causing a computer to acquire operation image information corresponding to the preceding operation log information and to realize a reproduction image creation function for creating a reproduction image based on the transition information and the acquired operation image information.

Images