JP6608402B2 - Program, control method, and terminal device - Google Patents

Description

  The present invention relates to a program, a control method, and a terminal device.

  Conventionally, as an operation manual for a terminal device such as a mobile phone, an operation manual using an electronic medium as well as an operation manual using a paper medium has been created. An operation manual using an electronic medium is stored in a memory or the like of the terminal device or managed by a server device that can communicate with the terminal device. The operation manual is displayed on the display device of the terminal device according to the operation of the terminal device by the user. Is done. In recent years, a terminal device in which an application program for practicing an operation method as well as displaying an operation manual is installed is known.

  For example, Patent Document 1 describes a mobile phone that displays operation practice information for practicing operations such as various application programs and setting functions for a user who is not familiar with the operation of a terminal device such as a child or an elderly person. Has been. In this mobile phone, operation practice information is displayed when a practice mode is set in response to a user pressing a predetermined practice key, and the user can practice operation keys and the like while viewing the operation practice information. Become.

JP 2006-54739 A

  However, in recent terminal devices, a pointer device such as a touch panel may be employed as an input device. In such a terminal device, it is necessary to display an operation practice screen that is linked to the operation of the input device by the user. there were. In addition, when unnecessary communication occurs due to an erroneous operation of the user during the operation of the user's terminal device, communication costs may be incurred, and the user may be distrusted with the operation practice application.

  The present invention has been made to solve such conventional problems, and provides a user interface for practice of operating a terminal device that can be easily operated even by a user who is unfamiliar with the operation of the terminal device. It is an object of the present invention to provide a program, a control method, and a terminal device that make it possible.

  A program according to the present invention is a program that includes a display unit, a storage unit, and a processor, and causes a terminal device that executes the application program to explain a screen operation method of the application program. An application that stores information on a series of images that are transitioned when operated in accordance with the operation procedure and the position and size of an operation target object that is an operation target included in the series of images in a storage unit and is displayed on a display unit The setting display magnification of the image related to the program is acquired, and a predetermined image of the series of images is displayed on the display unit based on the setting display magnification according to a user operation, and the operation target object included in the predetermined image The position of the operation target object in which the frame image corresponding to is changed based on the setting display magnification The image is displayed on the display unit so as to be superimposed on a predetermined image, and is switched from the predetermined image to the next image in the series according to the selection instruction of the operation target object from the user. To cause the processor to execute.

  In the program according to the present invention, the position and size of each operation target object included in the execution screen of the application program are stored in the storage unit in the order of a predetermined operation procedure, and a practice instruction for the application program from the user is acquired. If the application program is executed, the frame image corresponding to the operation target object included in the execution screen of the application program is displayed superimposed on the execution screen, and the operation target object selection instruction corresponding to the frame image is obtained from the user. In such a case, a predetermined process associated with the operation target object is executed, and a frame image corresponding to the operation target object included in the next execution screen is displayed superimposed on the next execution screen, and displayed on the next execution screen. Associated with the operation target object corresponding to the frame image included If the predetermined processing includes communication processing with another device, the predetermined processing associated with the operation target object is executed even if the operation target object selection instruction corresponding to the frame image is acquired from the user. It is preferable to execute a process of displaying a predetermined image without doing so.

  Further, in the program according to the present invention, when the home screen including the icon image of the application program is displayed on the display unit, when the presentation instruction of the icon image of the application program is acquired from the user, the image data of the home screen is acquired. Then, the position and size of the icon image of the application program is specified from the acquired image data, and the frame image corresponding to the specified position and size is displayed superimposed on the displayed home screen, and is displayed by the user. When an instruction to select an icon image corresponding to the frame image is acquired, it is preferable to execute processing for acquiring an instruction instruction for an application program.

  A control method according to the present invention is a control method for a terminal device that includes a display unit, a storage unit, and a processor, executes an application program, and explains a screen operation method of the application program. An application that stores information on a series of images that are transitioned when operated in accordance with the operation procedure and the position and size of an operation target object that is an operation target included in the series of images in a storage unit and is displayed on a display unit The setting display magnification of the image related to the program is acquired, and a predetermined image of the series of images is displayed on the display unit based on the setting display magnification according to a user operation, and the operation target object included in the predetermined image Is superimposed on a predetermined image by changing the position and size of the operation target object based on the set display magnification. It displayed on the display unit to way, in accordance with a selection instruction of the operation target object from a user, from a given image, to switch to the next image of the series of images, comprising.

  A terminal device according to the present invention is a terminal device that executes an application program and explains a screen operation method of the application program, and a series of transitions when operated in a predetermined operation procedure in the display unit and the application program A storage unit that stores image information and a position and size of an operation target object that is an operation target included in the series of images, and an acquisition unit that acquires a setting display magnification of an image related to an application program displayed on the display unit In response to a user operation, a predetermined image of the series of images is displayed on the display unit based on the set display magnification, and an operation target object included in the predetermined image is operated based on the set display magnification. Change the position and size of the target object and display it on the display unit so that it is superimposed on a predetermined image , In accordance with the selective operation of the operation target object from a user, comprising a predetermined image, and a display processing unit that performs switching to the next image of the series of images, a.

  With the program, the control method, and the terminal device according to the present invention, it is possible to provide a user interface for operating the terminal device that can be easily operated even by a user unfamiliar with the operation of the terminal device.

2 is a schematic diagram for explaining an outline of a terminal device 1; FIG. 2 is a schematic diagram for explaining an outline of a terminal device 1; FIG. 2 is a schematic diagram for explaining an outline of a terminal device 1; FIG. 2 is a diagram illustrating an example of a schematic configuration of a terminal device 1. FIG. It is a figure which shows an example of each screen displayed on the terminal display part 14 of the terminal device 1. FIG. It is a figure which shows an example of each screen displayed on the terminal display part 14 of the terminal device 1. FIG. It is a figure which shows an example of each screen displayed on the terminal display part 14 of the terminal device 1. FIG. It is a figure which shows an example of each screen displayed on the terminal display part 14 of the terminal device 1. FIG. It is a figure which shows an example of each screen displayed on the terminal display part 14 of the terminal device 1. FIG. It is a figure which shows an example of each screen displayed on the terminal display part 14 of the terminal device 1. FIG. It is a figure which shows an example of each screen displayed on the terminal display part 14 of the terminal device 1. FIG. (A) is a figure which shows an example of the data structure of 1st operation procedure table T1, (b) is a figure which shows an example of the data structure of 2nd operation procedure table T2. It is a figure which shows an example of the operation | movement flow of an icon image presentation process. It is a figure which shows an example of the operation | movement flow of practice information display processing.

  Hereinafter, various embodiments of the present invention will be described with reference to the drawings. However, it should be noted that the technical scope of the present invention is not limited to these embodiments, and extends to the invention described in the claims and equivalents thereof.

(Overview of terminal device 1)
1-3 is a schematic diagram for demonstrating the outline | summary of the terminal device 1 in this embodiment.

  The terminal device 1 according to the present embodiment is a multi-function mobile phone (so-called “smart phone”) or the like in which an application program is installed, and can execute the installed application program. Hereinafter, the application program installed in the terminal device 1 may be simply referred to as “application program”.

  The application program is an instruction sequence for causing the terminal device 1 to execute specific information processing, and is, for example, an e-mail program, a web browser program, a camera control program, an electronic game program, a schedule management program, or the like.

  Further, the terminal device 1 stores guide information for explaining each screen operation method of the plurality of application programs. The guide information is manual data for explaining how to operate various screens during execution of the application program, data for practicing various screen operating methods, or the like. Note that data for practicing various screen operation methods is described in a predetermined language such as HTML (HyperText Markup Language), XML (Extensible Markup Language), or JSON (JavaScript Object Notation). The guide information may be any data such as image data, document data, and audio data as long as it is information for explaining the screen operation method of the application program.

  Hereinafter, a case will be described as an example in which the terminal device 1 allows the user to practice a series of screen operation methods that transition when operated by a predetermined operation procedure during execution of the application program.

  The terminal device 1 stores information of an image obtained by capturing an execution screen of an application that allows a user to practice an operation method. In the example illustrated in FIG. 1, information of an image obtained by capturing each of a series of execution screens that transition when operated by a predetermined operation procedure during execution of an e-mail program is stored in the terminal device 1. That is, images of a plurality of execution screens of the e-mail program are stored in the order of a predetermined operation procedure. The execution screen image is created in advance by the creator of the guide information.

  Further, the terminal device 1 stores the position and size of the operation target object that is included in the stored series of images and that is the operation target in a predetermined operation procedure. The operation target object is a button object for executing a predetermined process, a text box for inputting a predetermined character string, or a text or image with link information for transitioning to another screen, etc. It is. Further, the position of the operation target object is two-dimensional coordinate information whose origin is a predetermined position on the display screen of the terminal device 1 (for example, the position of the upper end and the left end of the display screen). It is shown as a ratio to the vertical and horizontal widths of the display area. The size of the operation target object is a value indicating the height and width of the operation target object, and is indicated by, for example, a ratio to the vertical width and horizontal width of the display area of the display screen.

  In the example illustrated in FIG. 1, the terminal device 1 creates a frame image corresponding to the position and size of the operation target object “send button”, displays an image including the operation target object corresponding to the frame image, and creates the frame image. The frame image thus displayed is superimposed on the image and displayed ((1) in FIG. 1).

  When the display area of the operation target object included in the displayed image (that is, within the frame image) is instructed by the user's tap operation or the like ((2) in FIG. 1), the terminal device 1 stores the stored series Next, the next image after the currently displayed image is displayed ((3) and (4) in FIG. 1).

  As described above, the terminal device 1 allows the user to practice the operation method of the execution screen of the application program without starting the application program. Thereby, even if the user makes an operation mistake during the operation practice, the terminal device 1 does not perform a process that deviates from the practice of the operation method. As described above, the terminal device 1 can concentrate on learning the operation method even if the user is unfamiliar with the operation of the terminal device, and can provide a simple user interface.

  Next, the case where the terminal device 1 has set a predetermined display magnification for displaying each object (text, icon, button, etc.) included in the display screen will be described. Hereinafter, the set display magnification may be referred to as a set display magnification.

  In the example shown in FIG. 2, “1.2 times” is set as the setting display magnification by the user. In this case, the terminal device 1 displays a display screen 210 in which each object included in the normal display screen 200 (or a display image at the set display magnification “1”) is enlarged to “1.2 times”.

  The normal display screen 200 includes a first OS (Operating System) display area 201, a second OS display area 202, and an application display area 203. For example, in the first OS display area 201, status information of the terminal device 1 is displayed by an icon or the like. In the second OS display area 202, an operation menu for switching application programs of the terminal device 1 is displayed by icons or the like. An application program execution screen is displayed in the application display area 203.

  The enlarged display screen 210 includes a first OS display area 211, a second OS display area 212, and an application display area 213. For example, the same icons as the first OS display area 201 and the second OS display area 202 are enlarged and displayed in the first OS display area 211 and the second OS display area 212 by “1.2 times”. In the application display area 213, an application program execution screen is displayed.

  As shown in FIG. 2, the icons included in the first OS display area 211 and the second OS display area 212 are displayed larger (1.2 times) than the icons included in the first OS display area 201 and the second OS display area 202. Is done. Therefore, the vertical size of the first OS display area 211 and the second OS display area 212 is longer than that of the first OS display area 201 and the second OS display area 202. Therefore, the application display area 213 and the application display area 203 have different aspect ratios (aspect ratios).

  The display frame display process based on the set display magnification will be described below. In the example illustrated in FIG. 3, the terminal device 1 displays the display image 300 obtained by capturing the execution screen of the application on the enlarged display screen 210 described with reference to FIG. 2. The display image 300 is an image obtained by capturing an application program execution screen on the normal display screen 200.

  The terminal device 1 changes the position and size of the operation target object “send button” included in the display image 300 based on the set display magnification. In the position changing process, for example, first, the terminal device 1 specifies the positions of the upper end and the left end of the operation target object on the normal display screen 200. Next, the terminal device 1 calculates a position obtained by multiplying the position of the identified operation target object by the vertical reduction ratio of the application display area 203 changed based on the set display magnification as the relative position of the operation target object. Then, the terminal device 1 calculates the position of the frame image by adding the upper end position and the left end position of the display image 300 to the relative position.

  Further, in the size changing process, the terminal device 1 uses a numerical value obtained by multiplying the size (height and width) by the vertical reduction ratio of the application display area 203 to the frame image after the set display magnification is set. Size.

  For example, when the position of the operation target object is (10, 160) and the vertical reduction ratio of the application display area 203 is 90%, the relative position is 90% (application display area 203) of (10, 160). (9,144) of the vertical reduction ratio). Each position is indicated by coordinates in the vertical direction (downward direction) and the horizontal direction (rightward direction), with the upper end and the left end position of the normal display screen being the origin (0, 0). The position of the frame image is (9, 154) obtained by adding the upper end and left end positions (0, 10) of the display image 300 to the relative position (9, 144). When the size of the operation target object is (20, 30) and the vertical reduction ratio of the application display area 203 is 90%, the size of the frame image is (9, 27). Note that the reduction ratio is not limited to 90% described above. For example, the reduction ratio may be a ratio set so that the display image 300 obtained by capturing the execution screen of the application program on the normal display screen 200 is displayed in the application display area 213 based on the set display magnification. Other ratios may be used. The mobile terminal 1 may display a display image 300 obtained by multiplying the application display area 213 by a predetermined enlargement ratio. In this case, next, the terminal device 1 calculates a position obtained by multiplying the position of the specified operation target object by a predetermined enlargement ratio as a relative position of the operation target object. Then, the terminal device 1 calculates the position of the frame image by adding the upper end position and the left end position of the display image 300 to the relative position. In the size changing process, the terminal device 1 uses a value obtained by multiplying the size (height and width) by a predetermined enlargement ratio as the size of the frame image after the set display magnification is set.

  The terminal device 1 displays the frame image so as to be superimposed on the display image 300 based on the calculated position and size of the frame image. Accordingly, the terminal device 1 can display a frame image corresponding to the position and size of the operation target object included in the display image 300 even if the set display magnification is set.

  When the display area of the operation target object included in the image displayed based on the set display magnification (that is, within the frame image) is instructed to be selected by the user's tap operation or the like, the terminal device 1 stores the stored series. The next image after the currently displayed image is displayed based on the set display magnification.

  As described above, the terminal device 1 can guide the frame image corresponding to the operation target object of the displayed image to the user even if the set display magnification is set for the displayed image. Therefore, the terminal device 1 can provide a user interface for practice of operating the terminal device that can be easily operated even by a user who is unfamiliar with the operation of the terminal device.

  The description of FIGS. 1 to 3 described above is merely an explanation for deepening the understanding of the contents of the present invention. Specifically, the present invention may be implemented in various embodiments described below, and may be implemented in various modifications without substantially exceeding the principle of the present invention. All such variations are within the scope of the present disclosure and the specification.

(Terminal device 1)
FIG. 4 is a diagram illustrating an example of a schematic configuration of the terminal device 1.

  The terminal device 1 has a function of displaying guide information corresponding to an application program installed in the terminal device 1. For this purpose, the terminal device 1 includes a terminal communication unit 11, a terminal storage unit 12, a terminal operation unit 13, a terminal display unit 14, and a terminal processing unit 15.

  In the present embodiment, a multifunction mobile phone is assumed as the terminal device 1, but the present invention is not limited to this. The terminal device 1 only needs to be applicable to the present invention. For example, a mobile phone (so-called “feature phone”), a portable information terminal (Personal Digital Assistant, PDA), a portable game machine, a portable music player, a tablet terminal, a tablet A PC (Personal Computer), a notebook PC, or the like may be used.

  The terminal communication unit 11 includes a communication interface circuit including an antenna whose sensitivity band is a predetermined frequency band, and connects the terminal device 1 to a network such as the Internet. The terminal communication unit 11 establishes a radio signal line by a CDMA (Code Division Multiple Access) method or the like with a base station via a channel assigned by a base station (not shown), and communicates with the base station. Communicate. Then, the terminal communication unit 11 transmits the data supplied from the terminal processing unit 15 to an external device such as a server. In addition, the terminal communication unit 11 supplies data received from the external device to the terminal processing unit 15.

  The terminal storage unit 12 includes, for example, a semiconductor memory device. The terminal storage unit 12 stores an operating system program, a driver program, an application program, data, and the like used for processing in the terminal processing unit 15. For example, the terminal storage unit 12 stores, as a driver program, an input device driver program that controls the terminal operation unit 13, an output device driver program that controls the terminal display unit 14, and the like. The terminal storage unit 12 stores a program for acquiring and displaying display data as an application program. In addition, the terminal storage unit 12 includes, as data, guide information corresponding to each of a plurality of application programs (an image obtained by capturing an execution screen of an application program corresponding to each operation procedure), a setting display magnification set by a user, An icon image of the application program, the first operation procedure table T1, the second operation procedure table T2, and the like are stored. When the setting display magnification is not set by the user, the initial value (1 time) of the setting display magnification is stored. Further, when an image obtained by capturing an execution screen of an application program corresponding to each operation procedure is stored as guide information, an image ID is stored in association with each image. Further, the terminal storage unit 12 may temporarily store temporary data related to a predetermined process. Details of the first operation procedure table T1 and the second operation procedure table T2 will be described later.

  The terminal operation unit 13 may be any device as long as the terminal device 1 can be operated, for example, a touch panel. The user can input characters, numbers, symbols, and the like using the terminal operation unit 13. When operated by the user, the terminal operation unit 13 generates a signal corresponding to the operation. The generated signal is supplied to the terminal processing unit 15 as a user instruction.

  The terminal display unit 14 may be any device as long as it can output a moving image, a still image, and the like, such as a touch panel display device, a liquid crystal display, an organic EL (Electro-Luminescence) display, and the like. The terminal display unit 14 displays a moving image corresponding to the moving image data supplied from the terminal processing unit 15, a still image corresponding to the still image data, and the like.

  The terminal processing unit 15 includes one or a plurality of processors and their peripheral circuits. The terminal processing unit 15 controls the overall operation of the terminal device 1 and is, for example, a CPU (Central Processing Unit). The terminal processing unit 15 includes a terminal communication unit 11 and a terminal so that various processes of the terminal device 1 are executed in an appropriate procedure based on a program stored in the terminal storage unit 12, an operation of the terminal operation unit 13, and the like. The operation of the display unit 14 and the like is controlled. The terminal processing unit 15 executes processing based on programs (operating system program, driver program, application program, etc.) stored in the terminal storage unit 12. Further, the terminal processing unit 15 can execute a plurality of programs (such as application programs) in parallel.

  The terminal processing unit 15 includes a terminal control unit 151, an acquisition unit 152, and a display processing unit 153. Each of these units included in the terminal processing unit 15 is a functional module realized by a program executed by a processor included in the terminal processing unit 15. Alternatively, these units included in the terminal processing unit 15 may be implemented in the terminal device 1 as firmware.

  Hereinafter, an example of each screen displayed on the terminal display unit 14 of the terminal device 1 will be described with reference to FIGS.

  FIG. 5A is a diagram illustrating an example of a home screen 500 displayed on the terminal display unit 14 of the terminal device 1.

  On the home screen 500, an icon image 501 of an application program executable by the terminal device 1 is displayed. The home screen 500 is a screen displayed when the terminal device 1 is activated.

  When an application program for which an operation method is to be practiced is designated in accordance with the operation of the terminal operation unit 13 by the user, the position and size of the icon image 501a of the designated application program are displayed from the displayed icon images 501. A frame image to be displayed is displayed. For example, the terminal device 1 stores an icon image 502 for each application program. When an application program for which an operation method is desired to be practiced is specified, the icon image 502 of the specified application program is read from the terminal storage unit 12. Next, the terminal device 1 acquires an image obtained by capturing the current home screen 500. Next, the terminal device 1 extracts an icon image 501a that matches the read icon image 502 from the acquired images by a known similar image search process, and calculates the position and size of the extracted icon image 501a. . Then, the terminal device 1 displays a frame image corresponding to the calculated position and size superimposed on the home screen.

  FIG. 5B is a diagram illustrating an example of the home screen 510 displayed on the terminal display unit 14 of the terminal device 1 and displaying the frame image 511.

  Similar to the home screen 500, an icon image 501 of an application program is displayed on the home screen 510. A frame image 511 corresponding to the icon image 501a of the application program designated by the user is displayed superimposed on the home screen 510. Note that information for tapping the inside of the frame image 511 (for example, character string information such as “Let's tap an icon in the frame”) may be displayed on the home screen 510.

  FIG. 6A is a diagram illustrating an example of an execution screen 600 displayed on the terminal display unit 14 of the terminal device 1. The execution screen 600 is displayed when the application program of the icon image 501a is started when the user taps the icon image 501a displayed on the home screen 510.

  In the example shown in FIG. 6A, an execution screen when the e-mail program is executed is displayed as the execution screen 600. FIG. 6B is a diagram illustrating an example of a screen displayed by superimposing a frame image 610 on the execution screen 600 displayed on the terminal display unit 14 of the terminal device 1. On the execution screen 600, operation method information 611 is superimposed and displayed together with the frame image 610.

  The frame image 610 is a frame image corresponding to the position and size of the operation target object “new button” that is the operation target on the displayed execution screen 600. In the operation method information 611, character string information or the like that prompts the user to operate the operation target object corresponding to the displayed frame image 610 is displayed.

  FIG. 7A is a diagram illustrating an example of an execution screen 700 displayed on the terminal display unit 14 of the terminal device 1.

  The execution screen 700 is an execution screen for an e-mail program when the user taps an operation target object corresponding to the frame image 610 displayed on the execution screen 600. In the example illustrated in FIG. 6B and FIG. 7A, a new mail creation screen is displayed as the execution screen 700 in response to the operation target object “new button” being tapped by the user.

  A frame image 701 and operation method information 702 are superimposed on the execution screen 700 and displayed. The frame image 701 is a frame image corresponding to the position and size of the operation target object “destination input box” that is the operation target on the displayed execution screen 700. In the operation method information 702, character string information or the like that prompts the user to operate an operation target object corresponding to the displayed frame image 701 is displayed.

  FIG. 7B shows an example of a screen displayed by superimposing a frame image 710 corresponding to the operation target object that is the next operation target on the execution screen 700 displayed on the terminal display unit 14 of the terminal device 1. FIG. A new frame image 710 and operation method information 711 are superimposed on the execution screen 700 and displayed together with the frame image 701 shown in FIG. In the example shown in FIG. 7A and FIG. 7B, the cursor is moved in the “destination input box” in response to a tap operation performed on the operation target object “destination input box” corresponding to the frame image 701 by the user. The frame image 710 and the operation method information 711 are superimposed on the execution screen 700 and displayed.

  The frame image 710 is a frame image corresponding to the position and size of the operation target object “character input button” which is the next operation target on the displayed execution screen 700. The operation method information 711 displays character string information that prompts the user to operate an operation target object corresponding to the displayed frame image 710.

  FIG. 8A shows an example of a screen displayed by superimposing a frame image 800 corresponding to the operation target object that is the next operation target on the execution screen 700 displayed on the terminal display unit 14 of the terminal device 1. FIG. On the execution screen 700, a frame image 800 and operation method information 801 are newly superimposed and displayed together with the frame image 701 shown in FIG. In the example shown in FIG. 7B and FIG. 8A, the character “a” is changed to “destination” in response to the operation of the operation target object “character input button” corresponding to the frame image 710 being tapped by the user. The processing to be displayed in the “input box” is executed, and the frame image 800 and the operation method information 801 are superimposed on the execution screen 700 and displayed.

  The frame image 800 is a frame image corresponding to the position and size of the operation target object “character input button” that is the next operation target on the displayed execution screen 700. In the operation method information 801, character string information or the like that prompts the user to operate an operation target object corresponding to the displayed frame image 800 is displayed.

  FIG. 8B illustrates an example of a screen in which a frame image 810 corresponding to the operation target object that is the next operation target is superimposed on the execution screen 700 displayed on the terminal display unit 14 of the terminal device 1. FIG. On the execution screen 700, a frame image 810 and operation method information 811 are newly superimposed and displayed together with the frame image 701 shown in FIG. In the example shown in FIG. 8A and FIG. 8B, the character “e” is changed to “destination” in response to the operation of the operation target object “character input button” corresponding to the frame image 800 being tapped by the user. The processing to be displayed in the “input box” is executed, and the frame image 810 and the operation method information 811 are superimposed on the execution screen 700 and displayed.

  The frame image 810 is a frame image corresponding to the position and size of the operation target object “character input button” that is the next operation target on the displayed execution screen 700. In the operation method information 811, character string information or the like that prompts the user to operate an operation target object corresponding to the displayed frame image 810 is displayed.

  FIG. 9A illustrates the following operation on the execution screen 700 displayed on the terminal display unit 14 of the terminal device 1 after all the predetermined character strings are input to the “destination input box” corresponding to the frame image 701. It is a figure which shows an example of the screen which superimposed and displayed the frame image 900 corresponding to the operation target object "subject input box" and the frame image 901 corresponding to the operation target object "character switching button".

  When all the predetermined character strings are input to the “destination input box” corresponding to the frame image 701, the frame image 900 is displayed, and the cursor is displayed in the “subject input box” corresponding to the frame image 900 And the frame image 901 and the operation method information 902 are superimposed on the execution screen 700 and displayed.

  The frame image 901 is a frame image corresponding to the position and size of the operation target object “character switching button” which is the next operation target on the displayed execution screen 700. In the operation method information 902, character string information that prompts the user to operate the operation target object corresponding to the displayed frame image 901 is displayed.

  FIG. 9B shows an example of a screen displayed by superimposing a frame image 910 corresponding to the operation target object that is the next operation target on the execution screen 700 displayed on the terminal display unit 14 of the terminal device 1. FIG. On the execution screen 700, a frame image 910 and operation method information 911 are newly superimposed and displayed together with the frame image 901 shown in FIG. In the example shown in FIG. 9A and FIG. 9B, the input mode of the character input button is changed in response to the operation target object “character switching button” corresponding to the frame image 901 being tapped by the user. A process for changing from the alphanumeric input mode to the kana input mode is executed, and the frame image 910 and the operation method information 911 are superimposed on the execution screen 700 and displayed.

  The frame image 910 is a frame image corresponding to the position and size of the operation target object “character input button” that is the next operation target on the displayed execution screen 700. In the operation method information 911, character string information that prompts the user to operate the operation target object corresponding to the displayed frame image 910 is displayed.

  FIG. 10A shows the next operation on the execution screen 700 displayed on the terminal display unit 14 of the terminal device 1 after all predetermined character strings have been input in the “subject input box” corresponding to the frame image 901. It is a figure which shows an example of the screen which superimposed and displayed the frame image 1000 corresponding to the operation target object "text input box" which is the object, and the frame image 1001 corresponding to the operation target object "character input button".

  When all the predetermined character strings are input in the “subject input box” corresponding to the frame image 901, the frame image 1000 is displayed, and a process of displaying a cursor in the “text input box” corresponding to the frame image 1000 And the frame image 1001 and the operation method information 1002 are superimposed on the execution screen 700 and displayed.

  The frame image 1001 is a frame image corresponding to the position and size of the operation target object “character input button” that is the next operation target on the displayed execution screen 700. In the operation method information 1002, character string information or the like that prompts the user to operate an operation target object corresponding to the displayed frame image 1001 is displayed.

  FIG. 10B is a diagram illustrating an example of a screen for inputting the last character of a predetermined character string in the “text input box” corresponding to the frame image 1000. On the execution screen 700 displayed on the terminal display unit 14 of the terminal device 1, the frame image 1010 and the operation method information 1011 corresponding to the operation target object “text input box” that is the next operation target are superimposed and displayed.

  The frame image 1010 is a frame image corresponding to the position and size of the operation target object “character input button” which is the next operation target on the displayed execution screen 700. The operation method information 1011 displays character string information that prompts the user to operate the operation target object corresponding to the displayed frame image 1010.

  In response to selection of the operation target object corresponding to the frame image 1010 by the user, processing for displaying the character “.” In the “text entry box” is executed, and then the execution of the e-mail program is terminated. To do. Then, an image display screen 1100 described later is displayed.

  FIG. 11A is a diagram illustrating an example of an image display screen 1100 displayed on the terminal display unit 14 of the terminal device 1. On the image display screen 1100, an execution screen image 1101, a frame image 1102, operation method information 1103, and the like are displayed. The image display screen 1100 is displayed when an operation target object corresponding to the frame image 1010 is selected on the execution screen 700 shown in FIG.

  The execution screen image 1101 is an image obtained by capturing the execution screen displayed when the operation target object corresponding to the frame image 1010 is selected. The frame image 1102 is a frame image corresponding to the position and size of the image area of the operation target object that is the operation target in the image 1101 on the execution screen. In the operation method information 1103, character string information or the like that prompts the user to operate the image area of the operation target object corresponding to the displayed frame image 1102 is displayed.

  The execution screen corresponding to the image 1101 is an execution screen or the like when a predetermined process associated with the operation target object corresponding to the frame image 1010 included in the image 1101 includes a communication process with another apparatus. . Accordingly, even if the user makes an operation mistake during the operation practice, unnecessary communication costs are not incurred, so that the user can concentrate on learning the operation method.

  FIG. 11B is a diagram illustrating an example of an image display screen 1110 displayed on the terminal display unit 14 of the terminal device 1. On the image display screen 1110, an image 1111 of the execution screen, operation method information 1112, an end button 1113, and the like are displayed. The image display screen 1110 is displayed when the image area of the operation target object corresponding to the frame image 1102 is selected on the image display screen 1100 shown in FIG.

  The execution screen image 1111 is an image obtained by capturing the execution screen displayed when the operation target object corresponding to the frame image 1102 is selected. The operation method information 1112 displays character string information indicating that a predetermined operation procedure to be practiced by the user is completed. When the end button 1113 is selected by the user, the display of the image display screen 1110 is ended, and the practice of the operation method is ended.

(Various tables)
Hereinafter, various tables stored in the terminal storage unit 12 will be described. FIG. 12A is a diagram illustrating an example of a data structure of the first operation procedure table T1, and FIG. 12B is a diagram illustrating an example of a data structure of the second operation procedure table T2. Note that all or some of the various tables stored in the terminal storage unit 12 may be stored by one or a plurality of devices different from the terminal storage unit 12. In this case, the terminal device 1 is connected to one or a plurality of devices that store various tables via a wireless or wired communication network, and receives information included in the various tables from one or more devices, or Information stored in various tables is transmitted to one or a plurality of devices.

  The first operation procedure table T1 shown in FIG. 12A is a procedure for transitioning to the position and size of the frame image corresponding to the operation target object that is the operation target on the execution screen of the application program, and an image obtained by capturing the execution screen. It is a table for managing. The first operation procedure table T1 is created for each function that the user wants to practice in each application program. In the first operation procedure table T1, for each predetermined operation procedure to be practiced by the user, the procedure ID (Identification), position, size, image practice instruction, etc. of each operation procedure are associated with each other according to the order of the predetermined operation procedure. Remembered.

  The procedure ID is an example of identification information for uniquely identifying each operation procedure. Further, information indicating the position of the operation target object that is the operation target on the execution screen of the application program corresponding to each operation procedure is stored in the position. The size stores information indicating the size of the operation target object that is the operation target on the execution screen of the application program corresponding to each operation procedure. In the image practice instruction, a second operation procedure table T2 (to be described later) related to an image obtained by capturing an execution screen that transitions when an operation target object that is an operation target on the execution screen of the application program corresponding to each operation procedure is selected. The procedure ID is stored. If no information is stored in the image practice instruction, the execution screen does not transition to the captured image.

  The second operation procedure table T2 shown in FIG. 12B is a table for managing the position and size of the frame image corresponding to the operation target object included in the image captured from the execution screen of the application program. It is. The second operation procedure table T2 is created for each function that the user wants to practice in each application program. The second operation procedure table T2 stores the procedure ID, image ID, position, size, and the like of each operation procedure in association with each other according to the order of the predetermined operation procedure for each of the predetermined operation procedures to be practiced by the user.

  The procedure ID is an example of identification information for uniquely identifying each operation procedure. The image ID stores identification information for identifying an image obtained by capturing an execution screen of the application program corresponding to each operation procedure. Further, information indicating the position of the operation target object that is the operation target on the execution screen of the application program corresponding to each operation procedure is stored in the position. The size stores information indicating the size of the operation target object that is the operation target on the execution screen of the application program corresponding to each operation procedure.

(Operation flow of icon image presentation processing)
FIG. 13 is a diagram illustrating an example of an operation flow of guide screen display processing executed by the acquisition unit 152 and the display processing unit 153.

  First, the acquisition unit 152 acquires an icon image presentation instruction from the terminal control unit 151 (step S101). That is, the terminal control unit 151 selects an application program to be practiced by the user from among a plurality of application programs displayed on an unillustrated practice start screen, and the function to be practiced among the selected application programs is selected by the user. When the designation is detected, an icon image presentation instruction is supplied to the acquisition unit 152. Note that the icon image presentation instruction includes information indicating the selected application program and information indicating a function to be practiced.

  Next, the acquisition part 152 acquires the image which captured the present home screen of the terminal device 1 (step S102).

  Next, the display processing unit 153 reads the icon image of the application program from the terminal storage unit 12 based on information indicating the application program included in the icon image presentation instruction. Then, the display processing unit 153 extracts an icon image that matches the read icon image from the images acquired in step S102 (step S103).

  Next, the display processing unit 153 calculates the position and size of the extracted icon image (step S104). That is, the upper end and the left end of the captured image are set to the origin (0, 0), and the upper end and the left end of the extracted icon image are based on the vertical height and the horizontal size of the captured image. Calculate the position and size.

  Then, the display processing unit 153 generates a frame image corresponding to the calculated position and size of the frame image, displays the generated frame image so as to be superimposed on the home image (step S105), and performs a series of steps. finish.

  Note that steps S103 to S104 described above may be executed by an external device such as a server. In this case, for example, the acquisition unit 152 transmits an icon image presentation instruction and the acquired image to the external device via the terminal communication unit 11. The external device stores the icon image of each application program. When the external device receives the icon image presentation instruction and the image transmitted from the terminal device 1, each process of steps S103 to S104 is executed. Then, the external device transmits the execution results of the processes in steps S103 to S104 to the terminal device 1.

Thereby, the terminal device 1 can display the frame image on the home screen without consuming the storage capacity of the terminal storage unit 12.
(Practice information display process flow)
FIG. 14 is a diagram illustrating an example of an operation flow of practice information display processing executed by the terminal control unit 151, the acquisition unit 152, and the display processing unit 153. The practice information display process is executed when an icon image corresponding to the frame image displayed on the home screen is selected in response to an operation by the user through the terminal operation unit 13.

  First, the terminal control unit 151 activates the application program corresponding to the selected icon image, the display processing unit 153 displays the first execution screen of the application program, refers to the first operation procedure table T1, A frame image corresponding to the position and size related to the procedure ID of the operation procedure is displayed (step S201).

  Next, the display processing unit 153 determines whether a selection instruction has been acquired (step S202). When an operation target object to be operated on the execution screen displayed in step S201 is selected, the terminal control unit 151 supplies a selection instruction to the display processing unit 153, and the display processing unit 153 includes the terminal control unit 151. Earn selection instructions supplied by.

  If the display processing unit 153 determines that the selection instruction has not been acquired (step S202—No), the display processing unit 153 executes step S202 until the selection instruction is acquired from the terminal control unit 151.

  Next, when it is determined that the selection instruction has been acquired (step S202—Yes), the display processing unit 153 determines whether an image practice instruction is stored in the procedure ID of the current operation procedure (step S203). ).

  If it is determined by the display processing unit 153 that an image practice instruction is stored (step S203—Yes), the terminal control unit 151 ends the execution of the application program being executed (during practice), and will be described later in step S206. Proceed with the process.

  Further, when the display processing unit 153 determines that the image practice instruction image practice instruction is not stored (step S203—No), the terminal control unit 151 associates with the operation target object determined to be selected in step S202. The predetermined processing is executed (step S204). The predetermined process is a process for displaying a mail creation screen (an example of a “next execution screen”), for example, when the operation target object is a button for instructing a new mail creation. When the target object is a button for instructing character input, the character object is displayed in the input box where the cursor is displayed.

  Next, the display processing unit 153 specifies the procedure ID of the operation procedure next to the operation procedure corresponding to the operation target object determined to be selected in step S202, and specifies the procedure ID by referring to the first operation procedure table T1. The position and size associated with the performed procedure ID are extracted. Next, the display processing unit 153 displays a frame image corresponding to the extracted position and size, and returns the process to step S202 (step S205).

  The display processing unit 153 acquires the setting display magnification stored in the terminal storage unit 12 (step S206).

  Next, the display processing unit 153 extracts the procedure ID included in the image practice instruction determined to be stored in step S203, refers to the second operation procedure table T2, and associates the image associated with the extracted procedure ID. Extract ID, position and size. Next, the display processing unit 153 displays an image associated with the extracted image ID (an image obtained by capturing the execution screen) based on the setting display magnification, and sets a frame image corresponding to the extracted position and size as the setting display magnification. (Step S207).

  Next, the display processing unit 153 determines whether a selection instruction has been acquired (step S208). That is, when the image area of the operation target object included in the displayed image is selected, the terminal control unit 151 supplies a selection instruction to the display processing unit 153, and the display processing unit 153 The selection instruction supplied from the control unit 151 is obtained.

  When the display processing unit 153 does not acquire the selection instruction (No at Step S208), the display processing unit 153 executes Step S208 until the selection instruction is acquired from the terminal control unit 151.

  Next, if it is determined that the selection instruction is acquired in step S208 (step S208-Yes), the display processing unit 153 determines whether the operation target object when it is determined that the selection instruction is acquired is an end button. It is determined whether or not (step S209).

  When the display processing unit 153 determines that the operation target object is not the end button when it is determined in step S208 that the selection instruction has been acquired (No in step S209), the display processing unit 153 determines that the selection instruction has been acquired in step S208. The procedure ID of the operation procedure next to the operation procedure corresponding to the operation target object at that time is specified, and the specified image ID, position, and size are extracted with reference to the second operation procedure table T2. Next, the display processing unit 153 displays an image associated with the extracted image ID, displays a frame image corresponding to the extracted position and size (step S210), and returns the process to step S208.

  If the display processing unit 153 determines that the operation target object when it is determined in step S208 that the selection instruction has been acquired is an end button (step S209—Yes), the display processing unit 153 ends the series of steps.

(Modification 1)
In addition, this invention is not limited to embodiment mentioned above. For example, the operation method information 611 is not limited to character string information, and may be any information as long as it is information that prompts the user to operate the operation target object corresponding to the displayed frame image. It may be an arrow image pointing to an image), audio information, or the like.

(Modification 2)
In addition, when a frame image is displayed on each screen of the terminal device 1, when a display area other than the frame image is designated according to the operation of the terminal operation unit 13 by the user, the terminal control unit 151 performs the designation. The information may be invalidated. As a result, even if the user designates a display area other than the frame image by an erroneous operation, the erroneous operation is not reflected in the application being executed. It becomes possible to provide screen operation practice.

  Note that the first and second modifications described above may be combined into at least the two or more modifications and incorporated into the terminal device 1 of the present embodiment.

  It should be understood by those skilled in the art that various changes, substitutions, and modifications can be made thereto without departing from the spirit and scope of the present invention.

DESCRIPTION OF SYMBOLS 1 Terminal device 11 Terminal communication part 12 Terminal memory | storage part 13 Terminal operation part 14 Terminal display part 15 Terminal processing part 151 Terminal control part 152 Acquisition part 153 Display processing part

Claims (4)

A program having a display unit, a storage unit, and a processor for causing a terminal device that executes an application program to explain a screen operation method of the application program,
In the application program, information on a series of execution screens of the application program that transitions when operated by a predetermined operation procedure is stored in the storage unit, and is an operation target included in the execution screen of the series of application programs Storing the position and size of the operation target object in the storage unit in the order of the predetermined operation procedure ;
Obtaining a setting display magnification of the execution screen of the application program displayed on the display unit;
If you get the practice instruction of the application program from the user, to execute the application program, displays a predetermined execution screen of the execution screen of the set of application programs, the display unit based on the set display magnification And
A frame image corresponding to the operation target object included in the predetermined execution screen is created with the position and size of the operation target object changed based on the setting display magnification, and is superimposed on the predetermined execution screen. Displayed on the display unit,
When the process associated with the operation target object corresponding to the frame image includes a communication process with another device, the application program is selected according to a selection instruction of the operation target object corresponding to the frame image from the user. to execute the predetermined processing associated with the operation target object without executing the communication process, from the predetermined image, have the following line switching to the execution screen of the execution screen of the set of application programs, Displaying a frame image corresponding to the operation target object included in the next execution screen so as to be superimposed on the next execution screen;
A program that causes the processor to execute the above. When a home screen including an icon image of the application program is displayed on the display unit, when obtaining a presentation instruction for the icon image of the application program from a user, the image data of the home screen is acquired,
From the acquired image data, specify the position and size of the icon image of the application program,
A frame image corresponding to the specified position and size is displayed superimposed on the displayed home screen,
If you get the selection instruction of the icon image corresponding to the frame image from the user, it executes the process of acquiring the training instruction of the application program, the program according to claim 1. A control method for a terminal device that includes a display unit, a storage unit, and a processor, executes an application program, and explains a screen operation method of the application program,
In the application program, information on a series of execution screens of the application program that transitions when operated by a predetermined operation procedure is stored in the storage unit, and is an operation target included in the execution screen of the series of application programs Storing the position and size of the operation target object in the storage unit in the order of the predetermined operation procedure ;
Obtaining a setting display magnification of the execution screen of the application program displayed on the display unit;
If you get the practice instruction of the application program from the user, to execute the application program, displays a predetermined execution screen of the execution screen of the set of application programs, the display unit based on the set display magnification And
A frame image corresponding to the operation target object included in the predetermined execution screen is created with the position and size of the operation target object changed based on the setting display magnification, and is superimposed on the predetermined execution screen. Displayed on the display unit,
When the process associated with the operation target object corresponding to the frame image includes a communication process with another device, the application program is selected according to a selection instruction of the operation target object corresponding to the frame image from the user. to execute the predetermined processing associated with the operation target object without executing the communication process, from the predetermined image, have the following line switching to the execution screen of the execution screen of the set of application programs, Displaying a frame image corresponding to the operation target object included in the next execution screen so as to be superimposed on the next execution screen;
A control method comprising: A terminal device for executing an application program and explaining a screen operation method of the application program,
A display unit;
In the application program, information on a series of execution screens of the application program that transition when operated in a predetermined operation procedure is stored , and an operation target object that is an operation target included in the execution screen of the series of application programs is stored . A storage unit for storing the position and size in the order of the predetermined operation procedure ;
An acquisition unit for acquiring a setting display magnification of an execution screen of the application program displayed on the display unit;
If you get the practice instruction of the application program from the user, to execute the application program, displays a predetermined execution screen of the execution screen of the set of application programs, the display unit based on the set display magnification And
A frame image corresponding to the operation target object included in the predetermined execution screen is created with the position and size of the operation target object changed based on the setting display magnification, and is superimposed on the predetermined execution screen. Displayed on the display unit,
When the process associated with the operation target object corresponding to the frame image includes a communication process with another device, the application program receives the instruction to select the operation target object corresponding to the frame image from the user. to execute the predetermined processing associated with the operation target object without executing the communication process, from the predetermined image, have the following line switching to the execution screen of the execution screen of the set of application programs, A display processing unit for displaying a frame image corresponding to the operation target object included in the next execution screen in a superimposed manner on the next execution screen ;
A terminal device comprising:

Images