WO2024047696A1 - Information processing system, information processing method, and program - Google Patents

Abstract

[Problem] The present invention addresses the problem of providing an information processing system, etc., with which it is possible to efficiently perform healthcare education. [Solution] According to one aspect of the present invention, provided is an information processing system for assisting in healthcare education. This information processing system comprises a control unit. The control unit is configured to execute each of the following steps. In a moving-image acceptance step, an operation moving image of surgery is accepted from a first user. In a display control step, the operation moving image is visibly displayed as a shared moving image to the first user and a second user different from the first user. The shared moving image is configured to enable an annotation to be written by the first user and/or the second user, and the screen on which the shared moving image is displayed is configured to enable a text comment of the first user and/or the second user associated with the annotation to be added.

Description

Information processing system, information processing method and program

The present invention relates to an information processing system, an information processing method, and a program.

In recent years, with the development of medical technology, surgical techniques have become more complex. Along with this, systems to support surgery have also been developed, and for example, Patent Document 1 discloses a technique aimed at improving communication between surgeons.

Japanese Patent Application Publication No. 2019-162339

The technology disclosed in Patent Document 1 is configured so that annotations can be written to specific parts of a video. However, it has been found that inexperienced users (physicians) may not be able to understand the instructor's ideas clearly.

In view of the above circumstances, the present invention aims to provide an information processing system etc. that can efficiently provide medical education.

According to one aspect of the present invention, an information processing system that supports medical education is provided. This information processing system includes a control section. The control unit is configured to perform the following steps. In the video reception step, a surgical operation video is received from the first user. In the display control step, the operation video is displayed as a shared video so that the first user and a second user different from the first user can view it. Here, the shared video is configured to allow the first user and/or the second user to write an annotation, and on the screen that displays the shared video, the first user and/or The second user is configured to be able to add a text comment.

According to the above aspect, it is possible to provide an information processing system and the like that can efficiently provide medical education.

FIG. 1 is a configuration diagram showing an information processing system 1 according to the present embodiment. 2 is a block diagram showing the hardware configuration of a first user terminal 2. FIG. 3 is a block diagram showing the hardware configuration of a server 3. FIG. 2 is a block diagram showing the hardware configuration of a second user terminal 4. FIG. 2 is a block diagram showing functions realized by a control unit 33 and the like in the server 3. FIG. FIG. 2 is an activity diagram showing the flow of information processing according to the present embodiment. FIG. 2 is a schematic diagram for explaining a shared video. This is an example of a screen displaying search results shown on the first user terminal 2.

Hereinafter, embodiments of the present invention will be described using the drawings. Various features shown in the embodiments described below can be combined with each other.

That is, the information processing system according to this embodiment is as shown below.
An information processing system that supports medical education,
Equipped with a control unit,
The control unit is configured to execute the following steps,
In the video reception step, a surgical operation video is accepted from the first user,
In the display control step, the operation video is displayed so as to be visible to the first user and a second user different from the first user as a shared video;
Here, the shared video is configured to allow the first user and/or the second user to write an annotation, and
The information processing system is configured to be able to add a text comment of the first user and/or the second user that is linked to the annotation on a screen on which the shared video is displayed.

By the way, the program for realizing the software appearing in this embodiment may be provided as a non-transitory computer-readable medium, or may be downloaded from an external server. The program may be provided in a manner that allows the program to be started on an external computer and the function thereof is realized on the client terminal (so-called cloud computing).

Furthermore, in this embodiment, the term "unit" may include, for example, a combination of hardware resources implemented by circuits in a broad sense and software information processing that can be concretely implemented by these hardware resources. . In addition, various types of information are handled in this embodiment, and these information include, for example, the physical value of a signal value representing voltage and current, and the signal value as a binary bit collection consisting of 0 or 1. It is expressed by high and low levels or quantum superposition (so-called quantum bits), and communication and calculations can be performed on circuits in a broad sense.

Further, a circuit in a broad sense is a circuit realized by at least appropriately combining a circuit, a circuit, a processor, a memory, and the like. In other words, Application Specific Integrated Circuit (ASIC), programmable logic device (for example, Simple Programmable Logic Device (SPLD)), complex programmable logic Device (Complex Programmable Logic Device: CPLD) and field This includes a field programmable gate array (FPGA) and the like.

1. Hardware Configuration This section describes the hardware configuration of this embodiment.

1.1 Information processing system 1
FIG. 1 is a configuration diagram showing an information processing system 1 according to the present embodiment. The information processing system 1 is an information processing system that supports medical education. The information processing system 1 includes a first user terminal 2, a server 3, and a second user terminal 4, which are connected through a network 5. These components will be further explained. Note that a system exemplified by the information processing system 1 is composed of one or more devices or components. Therefore, even the server 3 alone is an example of a system.

1.2 First user terminal 2
The first user terminal 2 is typically a terminal owned by a user receiving operational guidance. FIG. 2 is a block diagram showing the hardware configuration of the first user terminal 2. As shown in FIG.
The first user terminal 2 has a communication section 21, a storage section 22, a control section 23, a display section 24, an input section 25, and an audio output section 26, and these components are connected to the first user terminal 2. It is electrically connected within the user terminal 2 via a communication bus 20 . Descriptions of the communication unit 21, storage unit 22, and control unit 23 will be omitted because they are substantially the same as the communication unit 31, storage unit 32, and control unit 33 in the server 3, which will be described later.

For example, the display unit 24 may be included in the casing of the first user terminal 2, or may be attached externally. The display unit 24 displays a screen of a graphical user interface (GUI) that can be operated by the user. This is preferably implemented by using display devices such as a CRT display, a liquid crystal display, an organic EL display, and a plasma display depending on the type of the first user terminal 2, for example. Here, the display unit 24 will be described as being included in the casing of the first user terminal 2.

The input unit 25 may be included in the casing of the first user terminal 2, or may be externally attached. For example, the input unit 25 may be integrated with the display unit 24 and implemented as a touch panel. With a touch panel, the user can input tap operations, swipe operations, and the like. Of course, a switch button, a mouse, a QWERTY keyboard, etc. may be used instead of the touch panel. That is, the input unit 25 receives operation inputs made by the user. The input is transferred as a command signal to the control unit 23 via the communication bus 20, and the control unit 23 can perform predetermined control or calculation as necessary.

For example, the audio output unit 26 may be included in the housing of the first user terminal 2, or may be externally attached. The audio output unit 26 outputs audio that can be recognized by the user. The audio output unit 26 may be a non-directional speaker, a directional speaker, or both. Here, the audio output unit 26 will be described as being included in the casing of the first user terminal 2.

1.3 Server 3
FIG. 3 is a block diagram showing the hardware configuration of the server 3. As shown in FIG. The server 3 includes a communication section 31, a storage section 32, and a control section 33, and these components are electrically connected via a communication bus 30 inside the server 3. Each component will be further explained.

Although the communication unit 31 is preferably a wired communication means such as USB, IEEE1394, Thunderbolt (registered trademark), wired LAN network communication, etc., it is also suitable for wireless LAN network communication, mobile communication such as 3G/LTE/5G, Bluetooth (registered trademark) Communication etc. may be included as necessary. That is, it is more preferable to implement it as a set of these plurality of communication means. That is, the server 3 communicates various information with the first user terminal 2 and the second user terminal 4 via the network 5 via the communication unit 31 .

The storage unit 32 stores various information defined by the above description. This may be used, for example, as a storage device such as a solid state drive (SSD) that stores various programs related to the server 3 executed by the control unit 33, or as a temporary storage device related to program calculations. It can be implemented as a memory such as a random access memory (RAM) that stores information (arguments, arrays, etc.). Alternatively, a combination of these may be used. In particular, the storage unit 32 stores various programs related to the server 3 that are executed by the control unit 33.

The control unit 33 processes and controls the overall operation related to the server 3. The control unit 33 is, for example, a central processing unit (CPU) not shown. The control unit 33 implements various functions related to the server 3 by reading predetermined programs stored in the storage unit 32. That is, information processing by software stored in the storage unit 32 is specifically implemented by the control unit 33, which is an example of hardware, and can be executed as each functional unit included in the control unit 33. Regarding these, 2. Further details are provided in Sec. Note that the control section 33 is not limited to a single control section, and may be implemented so as to have a plurality of control sections 33 for each function. It may also be a combination thereof.

1.4 Second user terminal 4
The second user terminal 4 is typically a terminal owned by a person who instructs the first user (for example, may be referred to as an "instructor"). FIG. 4 is a block diagram showing the hardware configuration of the second user terminal 4. As shown in FIG.
The second user terminal 4 includes a communication section 41, a storage section 42, a control section 43, a display section 44, an input section 45, and an audio output section 46, and these components are connected to the second user terminal 4. It is electrically connected within the user terminal 4 via a communication bus 40 . The details of the communication unit 41, storage unit 42, control unit 43, display unit 44, input unit 45, and audio output unit 46 are as described above in the first user terminal 2. Since they are substantially the same as the display section 24, input section 25, and audio output section 26, their explanation will be omitted.

2. Functional Configuration This section describes the functional configuration of this embodiment. As described above, the information processing by the software stored in the storage unit 32 is specifically implemented by the control unit 33, which is an example of hardware, and can be executed as each functional unit included in the control unit 33.

FIG. 5 is a block diagram showing the functions realized by the control unit 33 and the like in the server 3. Specifically, the server 3, which is an example of the information processing system 1, includes a video reception section 331 and a display control section 332. Note that FIG. 5 shows an embodiment in which the server 3 includes a search request reception section 333, a search execution section 334, a payment execution section 335, and a storage management section 336 as functions provided in the server 3.

The video reception unit 331 is configured to be able to execute the video reception step. In this video reception step, a surgical operation video is received from the first user. This will be explained in more detail later.

The display control unit 332 is configured to be able to execute display control steps. In this display control step, various display information is generated to control display contents that can be viewed by the user. Note that the display information may be visual information itself such as a screen, image, icon, text, etc. that is generated in a form that is visible to the user, or, for example, visual information such as a screen, image, icon, text, etc. on various terminals. It may also be rendering information for display. Note that in the information processing system 1 of the present embodiment, the display control unit 332 displays the operation video as a shared video for the first user and a second user who is different from the first user. let Here, the shared video is configured to allow the first user and/or the second user to write an annotation, and on the screen that displays the shared video, the first user and/or The second user is configured to be able to add a text comment.
Details of this display content will be explained later.

The search request receiving unit 333 is configured to be able to execute a search request receiving step. In this search request receiving step, a search request for a user who shares the operation video is received from the first user. Further, the search execution unit 334 is configured to be able to execute a search execution step. In this search execution step, a search for users who share the operation video is executed based on the above-mentioned search request.
Information processing related to these searches will be explained later.

The payment execution unit 335 is configured to be able to execute the payment execution step. In this payment execution step, the first user pays the second user compensation for writing an annotation and/or adding a text comment by the second user.

The storage management unit 336 is configured to be able to execute storage management steps. In this storage management step, various information related to the information processing system 1 of this embodiment is configured to be stored and managed. Typically, the storage management unit 336 is configured to store received videos, annotations, comments, etc. in a storage area. This storage area is exemplified by the storage unit 32 of the server 3, but may also be the storage unit 22 provided in the first user terminal 2 or the storage unit 42 provided in the second user terminal 4. . That is, according to the function of the storage management unit 336, various information related to education can be configured to be viewable not only by the first user but also by the second user. Note that this storage area does not necessarily have to be within the information processing system 1, and the storage management unit 336 can also manage to store various information in an external storage device or the like.

3. Information Processing Method In this section, each step of the information processing method executed by the information processing system 1 described above will be explained using an activity diagram. Note that, as described above, the information processing system 1 of this embodiment is used to support medical education by receiving surgical operation videos from the first user. There is no particular restriction as long as it is performed in an area.
For example, the operation video of a surgery to which this information processing system 1 is applied may be a video related to surgery, ophthalmology, dentistry, urology, otorhinolaryngology, or obstetrics, and the surgical target may be a human. It may also be an animal other than a human. This embodiment will be described based on a case where the surgical operation video is a surgical operation video.

FIG. 6 is an activity diagram showing the flow of information processing according to this embodiment. Specifically, in the information processing method of this embodiment, at least a video reception step and a display control step are executed. The details of the processing of each activity will be described below.

First, in the information processing method of this embodiment, the video reception unit 331 receives a surgery operation video from the first user (activity A101). This operation video is typically a video of the surgery performed by the first user, and there are no particular restrictions on the file extension or the like.

Acceptance of this video can be achieved, for example, by the first user logging into a predetermined web page and uploading the video file.

Subsequently, the display control unit 332 displays this operation video so that the first user and a second user different from the first user can view it as a shared video (activity A102). Here, the shared video is configured to allow the first user and/or the second user to write an annotation, and on the screen that displays the shared video, the first user and/or The second user is configured to be able to add a text comment.

This shared video will be explained with reference to FIG. FIG. 7 is a schematic diagram for explaining a shared video. A video MV1 (shared video) is displayed on the display screen D in FIG. 7, and in addition to the video MV1, an input form IF1 is provided in which a text comment can be posted.

As described above, the shared video is configured to allow the first user and/or the second user to write annotations. That is, the video MV1 is a recording of the operation performed on the organ OG1, and the video MV1 is configured such that an object OBJ1 can be written as an annotation. Notes can be made regarding operations, etc.
The object OBJ1 that functions as this annotation may be any object that is displayed using a known method. Typically, this object OBJ1 may be provided as a handwritten object using a pen tablet device or the like, or may be based on a stamp function prepared in advance.

In addition, in this embodiment, it is configured such that a text comment of the first user and/or the second user can be attached to this annotation. In other words, assuming that object OBJ1 is provided at the playback time of ``13 minutes and 20 seconds'' in video MV1, the text comment associated with this annotation (the comment ``in this direction'' in Figure 7) is , can be posted (assigned) via the input form IF1. Specifically, a text comment can be added by inputting a predetermined text into the input form IF1 and then pressing the button BT2 labeled "ENTER" by a click operation or the like. Note that when the button BT1 written as "CANCEL" is pressed, the comment written in the input form IF1 and the annotation (object OBJ1) before being saved are deleted.
Note that, as shown in FIG. 7, the poster of this comment may be displayed. Further, as shown in FIG. 7, the annotation and text comment may be displayed in association with the time information of the operation video. When the time of the video and the annotation are linked in this way, the configuration may be such that, for example, it is possible to move to a predetermined time in the video based on the content of the annotation or comment. In other words, referring to FIG. 7, by clicking on the comment section "Pinch further down," you can jump to the "16 minutes 56 seconds" portion of the video.

Additionally, once a comment has been added, another user can add a comment. For example, in Figure 7, a comment made by a preceptor named "XY" is described, but a reply to this comment is sent to another user (a first user or a second user different from XY). ) can be done. Note that the reply to this comment may be a text reply. On the other hand, the comment field on display screen D may be provided with a "like" button, a button for various reactions, a button for displaying a stamp, etc., and the above-mentioned reply can be displayed by pressing this button. It may be something that is done.

In addition, the display screen D in FIG. 7 is provided with an object OBJ2 that can control playback/stop of a moving image, an object OBJ3 that can be repeatedly played, and an object OBJ4 that can adjust the volume. In addition, on display screen D in Figure 7, object OBJ5 can be set to change the color when drawing the annotation (OBJ1), and it can be set to change the line width when drawing the annotation (OBJ1). An object OBJ6 is provided. Furthermore, an object OBJ7 that can display the video MV1 in full screen is provided.
Note that the functions that can be provided on the display screen D are not limited to these, and may be added or deleted as necessary.

In this way, the storage management unit 336 stores the displayed shared videos, annotations, comments, etc. as data in a predetermined storage area (activity A103).

Furthermore, in the information processing method of this embodiment, a search process for a second user who provides guidance to the first user may be performed. For example, such a search process may be performed after the video reception unit 331 of the server 3 receives the operation video as shown in the activity diagram of FIG. 6, but is not necessarily limited to this. This search process may be performed before accepting the operation video.

In this search process, first, the search request receiving unit 333 receives a search request from the first user for a user who shares the operation video (activity A104). Furthermore, the search execution unit 334 executes a search for users who share the operation video based on the search request (activity A105).
In addition to this, this search process is accomplished by specifying the second user who shares the operation video (activity A106).

This search request may be based on various information, for example, a first user searches for a predetermined second user based on field of expertise, age, years of experience, affiliation, location, etc. There may be. Furthermore, the second user is identified by identifying the user who requests to share the video from among the second user candidates extracted as a result of the search.

Note that this search process may be as shown below.
That is, the search execution unit 334 may be configured to identify the part where the operation is being performed from the contents of the operation video and search for a user who has expertise in the part.

As explained in FIG. 7, the video posted by the first user includes images of body parts such as the organ OG1. When the search execution unit 334 of the server 3 receives a search request from the first user, the search execution unit 334 determines the region captured in the video and searches for a user who has expertise in surgery for this region. may be configured.

Here, expertise in a region refers to, for example, having specialized knowledge regarding a predetermined region, and typically refers to having expertise according to the classification of clinical medicine. The degree of expertise may be determined based on the number of submitted papers, the position at the hospital to which the person belongs, etc.

FIG. 8 is an example of a screen displayed on the first user terminal 2 that displays search results. As shown in FIG. 8, the search execution unit 334 specifies that the organ OG1 captured in the operation video is the pancreas, and based on this specific content, searches a second doctor who has expertise in the field of pancreas. It may be configured to be presented as a candidate to the user.
The first user who comes into contact with such a display screen D identifies the second user who will share the video by pressing at least one (or more than one) of the buttons BT11, BT21, and BT31. be able to.
Note that on the display screen D shown in FIG. 8, when buttons BT12, BT22, and BT32 are pressed, reviews from contributors of other operation videos are displayed. In this way, in the information processing method of this embodiment, the evaluation regarding the second user may be displayed so that the first user can view it.

Furthermore, in the information processing method of the present embodiment, compensation may be paid to the second user with whom the video is shared. On the other hand, in the information processing method of this embodiment, the second user with whom the video is shared may not be paid (free of charge).

Here, if the information processing method of this embodiment is one in which a consideration is paid to the second user with whom the video is shared, the information processing system of this embodiment as shown in the activity diagram of FIG. 1, payment for the second user may be executed.

Specifically, the payment execution unit 335 of the server 3 executes a process in which the first user pays the second user compensation for writing an annotation and/or adding a text comment by the second user.
This consideration may be appropriately set between the first user and the second user. As an example, the amount of consideration may depend on the number and quality of annotations and comments. Furthermore, if the second user has a high degree of expertise in the body part that is the subject of the operation video, the compensation may be set higher depending on the degree of expertise.
It should be noted that, together with the search results as shown in FIG. 8 described above, the compensation for requesting each candidate of the second user to share or add a comment may be clearly displayed.

As described above, according to the information processing method of this embodiment, the annotation and the text comment linked to this annotation are visible to the first user, so that the first user receiving the education can easily understand the information processing method. can help. Therefore, medical education can be provided efficiently.

4. Others Regarding the information processing system 1 according to the present embodiment, the following aspects may be adopted.

Although the above embodiment has been described as the configuration of the information processing system 1, a program that causes a computer to execute each step of the information processing system 1 may be provided.

In the above embodiment, the first user searches for a candidate for the second user, but conversely, the second user searches for the first user that he or she wants to instruct. may be configured. In this case, for example, the first user can set a tag associated with the operation, and the second user can search for the first user by searching for this tag.

In the above embodiments, the server 3 performed various storage and control operations, but instead of the server 3, a plurality of external devices may be used. That is, using blockchain technology or the like, information regarding the attendance history may be distributed and stored in a plurality of external devices.

Furthermore, it may be provided in each of the following embodiments.

(1) An information processing system that supports medical education, including a control unit, and the control unit is configured to execute the following steps, and in the video reception step, a video reception step is performed by receiving a video request from a first user. The operation video is received, and in the display control step, the operation video is displayed as a shared video for the first user and a second user different from the first user, and here, the operation video is The shared video is configured to allow the first user and/or the second user to write an annotation, and on the screen that displays the shared video, the first user and/or the annotation associated with the annotation are displayed. An information processing system configured to be able to add/or a text comment of the second user.

(2) In the information processing system according to (1) above, in the search request receiving step, a search request of a user who shares the operation video is received from the first user, and in the search execution step, the search request is An information processing system that searches for the user who shares the operation video based on the information processing system.

(3) In the information processing system according to (2) above, in the search execution step, the part where the operation is being performed is specified from the contents of the operation video, and a user who has expertise in the part is searched for. , Information Processing Systems.

(4) In the information processing system according to any one of (1) to (3) above, in the payment execution step, compensation for writing the annotation and/or adding the text comment by the second user is provided. , an information processing system that executes a process of paying from the first user to the second user.

(5) In the information processing system according to any one of (1) to (4) above, in the display control step, the evaluation regarding the second user is displayed so that the first user can view it. , Information Processing Systems.

(6) In the information processing system according to any one of (1) to (5) above, in the display control step, the annotation and the text comment are displayed in association with time information of the operation video. information processing system.

(7) The information processing system according to any one of (1) to (6) above, wherein the surgical operation video is a surgical operation video.

(8) An information processing method, comprising each step of the information processing system described in any one of (1) to (7) above.

(9) A program that causes a computer to execute each step of the information processing system according to any one of (1) to (7) above.
Of course, this is not the case.

Finally, although various embodiments according to the present invention have been described, these are presented as examples and are not intended to limit the scope of the invention. The new embodiment can be implemented in various other forms, and various omissions, substitutions, and changes can be made without departing from the gist of the invention. The embodiment and its modifications are included within the scope and gist of the invention, and are included within the scope of the invention described in the claims and its equivalents.

1: Information processing system 2: First user terminal 3: Server 4: Second user terminal 5: Network 20: Communication bus 21: Communication section 22: Storage section 23: Control section 24: Display section 25: Input section 26 :Audio output section 30:Communication bus 31:Communication section 32:Storage section 33:Control section 40:Communication bus 41:Communication section 42:Storage section 43:Control section 44:Display section 45:Input section 46:Audio output section 331 : Video reception unit 332 : Display control unit 333 : Search request reception unit 334 : Search execution unit 335 : Payment execution unit 336 : Storage management unit BT1 : Button BT2 : Button BT11 : Button BT12 : Button BT21 : Button BT22 : Button BT31 : Button BT32: Button D: Display screen IF1: Input form MI1: Medical instrument MI2: Medical instrument MV1: Video OBJ1: Object OBJ2: Object OBJ3: Object OBJ4: Object OBJ5: Object OBJ6: Object OBJ7: Object OG1: Organ

Claims (9)

1. An information processing system that supports medical education,
   Equipped with a control unit,
   The control unit is configured to execute the following steps,
   In the video reception step, a surgical operation video is accepted from the first user,
   In the display control step, the operation video is displayed so as to be visible to the first user and a second user different from the first user as a shared video;
   Here, the shared video is configured to allow the first user and/or the second user to write an annotation, and
   The information processing system is configured to be able to add a text comment of the first user and/or the second user that is linked to the annotation on a screen on which the shared video is displayed.
2. The information processing system according to claim 1,
   In the search request receiving step, receiving a search request from the first user for a user who shares the operation video,
   In the search execution step, the information processing system executes a search for the user who shares the operation video based on the search request.
3. The information processing system according to claim 2,
   In the search execution step, the information processing system identifies a part where the operation is being performed from the contents of the operation video, and searches for a user who has expertise in the part.
4. The information processing system according to any one of claims 1 to 3,
   In the payment execution step, the information processing system executes a process in which the first user pays the second user compensation for writing the annotation and/or adding the text comment by the second user.
5. The information processing system according to any one of claims 1 to 4,
   In the display control step, the information processing system displays the evaluation regarding the second user so that the first user can view it.
6. The information processing system according to any one of claims 1 to 5,
   In the display control step, the annotation and the text comment are displayed in association with time information of the operation video.
7. The information processing system according to any one of claims 1 to 6,
   An information processing system, wherein the surgical operation video is a surgical operation video.
8. An information processing method,
   A method comprising each step of the information processing system according to any one of claims 1 to 7.
9. A program,
   A program that causes a computer to execute each step of the information processing system according to any one of claims 1 to 7.