JP2018072563A - Image display unit, control method thereof and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable easy and thorough confirmation of a display order of all subregions in an image even if the number of subregions is large, and to enable an intuitive change of the display order of the subregions while carrying out such confirmation.SOLUTION: An image display unit for displaying an image includes storage means for storing subregion data of a plurality of subregions included in the image. The image display unit displays each of the plurality of subregions in a display order set by the subregion data in response to a user operation, and displays the image while superposing an object corresponding to the display order of each of the plurality of subregions onto the plurality of subregions.SELECTED DRAWING: Figure 11

Description

  The present invention relates to an image display device, a control method thereof, and a program.

  In general, presentations and product descriptions for face-to-face sales performed in a conference room are performed using a screen displayed using a presentation application or a document browsing application. In particular, in a presentation application, a plurality of objects can be arranged on an image of one page, an animation effect can be set for each object, and the objects can be displayed in animation in order according to the setting.

  In Patent Document 1, as a method for setting the order of animation display of objects, a display order displayed near the object in the image and a list indicating the display order displayed in another frame outside the image are used together. A method is described. According to this, before starting the presentation, the user confirms the display order by the numerical values displayed near the object, and displays the display of each object by changing the display order of items on the list outside the image as necessary. The order can be changed.

  On the other hand, there is a display method for displaying an image in consideration of displaying on a small screen of a mobile terminal such as a smartphone. For example, a partial area of an image is automatically analyzed based on components such as characters and photos included in the image, and the analyzed partial area is displayed on a small screen of a mobile terminal by a simple operation such as tapping a button. Some display in order at the enlargement ratio for each combined partial area. By projecting an image displayed on the mobile terminal by this display method onto a projector and giving a presentation, each recognized partial area can be displayed in order at an enlargement ratio for each partial area. As a result, the listener can easily determine where the speaker is explaining the slide, so that the contents can be easily understood. In addition, since projected images change one after another according to the explanation by the speaker, it is possible to suppress the tiredness of the listener.

Japanese Patent No. 5667090

  There are cases where materials used for product explanation for face-to-face sales differ between a user who creates materials and sets the display order, and a user who uses the materials to present to customers in face-to-face sales. In such a use case, a user who makes a presentation needs to check the display order of each set item before explaining the product. In addition, since a user presenting to a customer handles a large amount of explanatory material, it is difficult to memorize the display order of all pages even if the material has been handled once. For this reason, it is necessary to confirm the set display order before explaining the product. However, the above-described conventional technique has a problem that if the number of partial areas included in the page image increases, it takes time to check the set display order, and it is difficult to store all of them.

  For example, when the number of partial areas included in one page of an image is from several tens to several tens, it is necessary to check the display order displayed in the vicinity of the partial areas in the image for each page. It was. There is also a method of checking the display order of each object by referring to a list outside the image, but it is difficult to intuitively understand which partial area in the image corresponds to the displayed information. Therefore, it takes time to confirm all the display orders. In particular, the product description materials for face-to-face sales are generally composed of a plurality of pages of 10 pages or more, and it takes more time to confirm the display order of the partial areas in the images for all the pages. Furthermore, since it is difficult to confirm the display order of the partial areas in all the images on all the pages, the presentation may be performed while overlooking a portion where the display order is not as intended. In such a case, the user feels frustration such as temporarily unable to understand the explanation in the overlooked place.

  An object of the present invention is to solve the above-described problems of the prior art.

  An object of the present invention is to provide a technique capable of easily confirming the display order of partial areas included in an image.

  An object of the present invention is to provide a display means that can easily understand the order of all partial areas on an image without providing a list of different frames. Further, a simple order changing means using the display means is provided.

In order to achieve the above object, an image display apparatus according to an aspect of the present invention has the following arrangement. That is,
Storage means for storing partial area data of a plurality of partial areas included in the image;
A presentation means for displaying each of the plurality of partial areas in a display order set by the partial area data in response to a user operation;
And display control means for displaying the image by superimposing objects corresponding to the display order of the plurality of partial areas on the plurality of partial areas.

  According to the present invention, the display order of partial areas included in an image can be easily confirmed.

  Other features and advantages of the present invention will become apparent from the following description with reference to the accompanying drawings. In the accompanying drawings, the same or similar components are denoted by the same reference numerals.

The accompanying drawings are included in the specification, constitute a part thereof, show an embodiment of the present invention, and are used together with the description to explain the principle of the present invention.
FIG. 2A is a block diagram for explaining a hardware configuration of the image display apparatus according to the first embodiment, and FIG. 2B is a functional block diagram for explaining functions of an image processing program according to the first embodiment. FIG. 3 is a diagram for explaining an example of content processed by the image processing program according to the first embodiment and an image included in the content. 5 is a flowchart for explaining image structure analysis processing executed by a page structure analysis unit of the image display apparatus according to the first embodiment. FIG. 6 is a diagram for explaining the structural analysis of the image of the first page according to the first embodiment. FIG. 3 is a diagram illustrating an example of partial area data according to the first embodiment and a partial area management table for managing the partial area data. 5 is a flowchart for explaining mode control processing executed by a mode control unit of the image display apparatus according to the first embodiment. FIG. 3 is a diagram for explaining a content list displayed on a display unit of the image display apparatus according to the first embodiment. FIG. 4 is a diagram for explaining screen switching in presentation processing in the image display apparatus according to the first embodiment. 5 is a flowchart for explaining presentation processing and partial area display enlargement processing by a presentation processing unit of the image display apparatus according to the first embodiment; FIG. 6 is a diagram for explaining an example of screen transition of the display unit when the image display apparatus according to the first embodiment executes a presentation process. 7 is a flowchart for explaining a sequential flow line display process executed by the editing processing unit in step S613 of FIG. FIG. 5 is a diagram for explaining a specific example of sequential flow line display processing by the image display apparatus according to the first embodiment. 9 is a flowchart for explaining editing processing executed by an editing processing unit of the image display apparatus according to the second embodiment. 14 is a flowchart for explaining display setting change processing in S1302 of FIG. 13; FIG. 15 is a flowchart for explaining the sequential flow line display process in S1409 of FIG. 14; FIG. FIG. 10 is a diagram illustrating an example of screen transition when sequential flow line display processing is executed in the image display apparatus according to the second embodiment. FIG. 9 is a diagram for explaining partial area data in which display settings of a partial area management table are changed in the image display apparatus according to the second embodiment. 10 is a flowchart for describing processing for changing the display order of partial areas, which is executed by the editing processing unit of the image display apparatus according to the third embodiment. FIG. 10 is a diagram illustrating an example of screen transition of a display unit of an image display device according to a third embodiment. 9 is a flowchart for describing processing for changing the display order of partial areas, which is executed by an editing processing unit of an image display device according to a fourth embodiment. FIG. 10 is a diagram illustrating an example of screen transition of a display unit of an image display device according to a fourth embodiment. FIG. 10 is a diagram illustrating an example of a menu screen that is displayed on the display unit of the image display device according to the fifth embodiment and switches the order changing method. 10 is a flowchart for explaining a mode control process for reflecting a selected order changing method in an edit mode in the image display apparatus according to the fifth embodiment.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. The following embodiments do not limit the present invention according to the claims, and all combinations of features described in the embodiments are not necessarily essential to the solution means of the present invention. . In the present embodiment, a technique for easily confirming the display order of partial areas in a page image will be described.

[Embodiment 1]
FIG. 1A is a block diagram illustrating a hardware configuration of the image display apparatus 100 according to the first embodiment.

  The image display apparatus 100 includes a CPU 101, a RAM 102, a ROM 103, an HDD 104, and a display unit 105. When the power is turned on, the CPU 101 executes a boot program stored in the ROM 103, develops a program or OS installed in the HDD 104 on the RAM 102, and executes it to control the operation of the image display apparatus 100. The display unit 105 displays images such as various menu screens and page images under the control of the CPU 101. The display unit 105 includes a touch panel, and is configured by overlapping a touch panel on a display unit that can display an image. The CPU 101 can detect, for example, the following operations and states on the touch panel. This operation can be performed by touching the touch panel with a finger or pen (hereinafter, tap-in), without touching the touch panel with a finger or pen (hereinafter referred to as long tap), or by moving the touch panel while touching with a finger or pen ( In the following, drag). Furthermore, an operation of releasing a finger or pen that has been touching the touch panel (hereinafter, tap-out), an operation of moving the touch panel in a specific direction with the finger or pen (hereinafter, swiping), and the like are also included.

  These touch operations and their position coordinates are periodically notified from the display unit 105 to the CPU 101, and the CPU 101 determines what touch operation is performed on which position on the touch panel based on the notified information. . The moving direction of the finger or pen moving on the touch panel can also be determined for each vertical component / horizontal component on the touch panel based on the change in position coordinates. In addition, the touch panel uses any one of various types of touch panels such as a resistive film type, a capacitance type, a surface acoustic wave type, an infrared type, an electromagnetic induction type, an image recognition type, and an optical sensor type. Also good.

  A RAM 102 is a memory in which a program executed by the CPU 101 is expanded, and is also used as a work memory of the CPU 101. The HDD 104 stores a program executed by the CPU 101 and is used for storing image data and the like. Data processed by the CPU 101 executing the image processing program 110 is called content. The content and the image included in the content will be described later with reference to FIG.

  FIG. 1B is a functional block diagram illustrating functions of the image processing program 110 according to the first embodiment.

  The image processing program 110 includes a page structure analysis unit 111, a partial area management unit 112, a presentation processing unit 114, and an editing processing unit 115. The page structure analysis unit 111 executes a structure analysis process of an image (hereinafter referred to as a page image) printed on one page of paper during normal printing. The structural analysis is a process of automatically analyzing a plurality of partial areas (objects) in a page image in accordance with components such as text and diagrams included in the page image. This page image structure analysis processing will be described later with reference to the flowchart of FIG. The component processed by the page structure analysis unit 111 refers to each element laid out in a document such as a title, text, header, footer, figure, and background. This component will be described later with reference to FIG. Information on the partial area obtained after executing this structural analysis process is referred to as partial area data. The partial area data includes data related to the partial area, such as the display order, coordinates, width / height (hereinafter, width / height) of the partial areas in the page image, and area attributes. The region attribute corresponds to the classification of the document constituent elements related to the partial region data, and includes, for example, text, diagrams, tables, and the like. Details of the partial area data will be described later with reference to FIGS.

  The partial area management unit 112 manages partial area data. The partial area management table is a table for managing a plurality of partial area data. The partial area management table will be described later with reference to FIG.

  The mode control unit 113 controls the content display mode. Content display modes include a presentation mode and an edit mode. The presentation mode is a mode for explaining products etc. while displaying the above-mentioned partial areas, and the edit mode is a mode for changing the display order and display settings of the partial areas. This mode control process will be described with reference to FIG.

  The presentation processing unit 114 executes processing related to the presentation mode, and this processing is called presentation processing. The presentation process is a process for displaying the partial areas in the page image in order at the display magnification for each partial area in accordance with the user's operation. Information such as the display order, coordinates, and height of the partial areas necessary for display is specified by the partial area data. The presentation processing and the partial area enlargement display processing executed by the presentation processing unit 114 will be described later with reference to the flowchart of FIG.

  The edit processing unit 115 executes processing related to the edit mode. This process is called an editing process. In the edit mode, first, processing for displaying the display order of the partial areas in an easy-to-understand manner is executed. This is a flowchart of FIG. 11 and will be described later as a sequential flow line display process. The editing process is a process of changing the display order of partial areas in the page image and changing the display settings of the partial areas. These editing processes will be described later with reference to FIG. 13 of the second embodiment, and the change of the display setting of the partial area will be described later with reference to FIG. 14 (the second embodiment). Will be described later with reference to FIG. 18 (Embodiment 3) and FIG. 20 (Embodiment 4).

  FIG. 2 is a diagram illustrating an example of content processed by the image processing program 110 according to the first embodiment and a page image included in the content.

  As shown in FIG. 2, the content 200 includes a plurality of page images 201-1 to 201-N. Here, the number of content pages is N.

  The content 200 is a document file or the like handled by a general information processing apparatus, and the page images 201-1 to 201-N correspond to, for example, each page of the document file. Here, the page image format is a raster image such as a JPEG or TIFF file.

  FIG. 3 is a flowchart illustrating the page image structure analysis processing executed by the page structure analysis unit 111 of the image display apparatus 100 according to the first embodiment. Note that the processing shown in this flowchart is realized by the CPU 101 executing an image processing program developed from the HDD 104 to the RAM 102. Therefore, the processing subject will be described as the CPU 101 here. This page image structure analysis process is a process of automatically analyzing a plurality of partial areas in a page image in accordance with the document component of the page image. The page images to be processed here are page images 201-1 to 201-N included in the content 200 in FIG. As an example of the page image to be processed, the page image 201-1 of the first page of the content 200 shown in FIG. 2 will be described as an example.

  Before describing the flowchart of FIG. 3, the page image 201-1 will be described with reference to FIG.

  FIG. 4 is a diagram for explaining the structural analysis of the page image 201-1 according to the first embodiment.

  FIG. 4A illustrates the configuration of the page image 201-1.

  The page image 201-1 includes a background 400, texts 401 to 404, graphics 405, texts 406 and 407, graphics 408, text 409, table 410, graphics 411 to 413, table 414, texts 415 and 416 as document components. Is included.

  The page structure analysis unit 111 executes page image structure analysis processing in the order of pages for all pages of the content 200. That is, the process starts from the first page image, and when the process for the first page image ends, the process for the next page image is executed. Repeat until the last page.

  In S301, the CPU 101 sets a variable i for storing a page number to be processed in the RAM 102, and sets it to “1”. In step S <b> 302, the CPU 101 reads the page image of the i-th page (first page) of the content 200 stored in the HDD 104 of the image display apparatus 100. In step S303, the CPU 101 recognizes a partial area according to the document component of the page image read in step S302.

  FIG. 4B is a diagram illustrating a result of analyzing the partial region of the page image of FIG. 4A by the page structure analysis unit 111 according to the first embodiment. In FIG. 4B, a partial area as a result of the page structure analysis is indicated by a rectangular dotted line.

  4A and 4B, the background 400 is automatically analyzed as a partial area 420, and similarly, the text 401 is analyzed as a partial area 421. Similarly, the texts 402 to 404 are analyzed as partial areas 422 to 424, respectively. Further, the graphic 405 is analyzed as a partial area 425, and the texts 406 and 407 are analyzed as partial areas 426 and 427, respectively. The graphic 408 is analyzed as a partial area 428, the text 409 is analyzed as a partial area 429, and the table 410 is analyzed as a partial area 430. Furthermore, the figures 411 to 413 are analyzed as partial areas 431 to 433, respectively. The table 414 is analyzed as a partial area 434, and the texts 415 and 416 are analyzed as partial areas 435 and 436, respectively.

  In step S304, the CPU 101 recognizes the coordinates and width of each partial area.

  FIG. 4C is a diagram for explaining the coordinates and the height of the partial region 431.

  Here, the coordinates are expressed in a coordinate system having the origin at the upper left of the partial area 420 (area having the same range as the entire page image) whose area attribute is “background”. The coordinates of the partial area 431 are indicated by the X-axis direction distance X31 from the origin to the upper left coordinates of the partial area 431, and the Y-axis direction distance Y31 from the origin to the upper left coordinates of the partial area 431. That is, the coordinates of the partial area 431 are represented by (X31, Y31). The height is represented by the length W31 in the X-axis direction and the length H31 in the Y-axis direction of the partial region 431.

  In step S305, the CPU 101 executes an OCR process, and determines whether each partial area includes characters. If it is determined that the character is included, the process proceeds to S306. If it is determined that the character is not included, the process proceeds to S308. In step S306, the CPU 101 acquires the character size, and in step S307, the CPU 101 recognizes the character direction and proceeds to step S308. The character direction is the writing direction and indicates whether the character is written horizontally or vertically. In step S308, the CPU 101 recognizes the region attribute of each partial region. The area attribute indicates an attribute corresponding to a document component, and includes a title, text, header, footer, table, figure, background, and the like.

  In step S309, the CPU 101 analyzes the display order of the partial areas and performs ordering. This display order is determined according to the character direction, coordinates, and height of the partial areas acquired up to S308. Basically, when the character direction is horizontal writing, ordering is performed from the partial area located at the upper left in the page image to the partial area located at the lower right. When the character direction is vertical writing, ordering is performed from the partial area located at the upper right in the page image to the partial area located at the lower left.

  In the example of FIG. 4B, since the character direction of the text area is horizontal, the display order of the partial area 431 is “11”, the display order of the partial area 432 is “12”, and the display order of the partial area 433 is The display order is determined as “13”. Here, the display order is performed from the partial area 421 located at the upper left in the page image to the partial area located at the lower right. Hereinafter, the display order of the partial area 434 is “14”, the display order of the partial area 435 is “15”, and the display order of the partial area 436 is “16”. Here, since the partial area 420 as the background is displayed at the top of the page, the display order is “0”. Here, in the presentation mode, the partial area 420 recognized as the background is not enlarged for each partial area. Note that the display order of the partial areas can be changed by an order change process described with reference to FIG. 18 according to the third embodiment.

  In step S310, the CPU 101 stores information on the partial area recognized in the process so far as partial area data. In practice, the CPU 101 that functions as the page structure analysis unit 111 functions as the partial region management unit 112 and stores the partial region data. In step S311, the CPU 101 increments the variable i specifying the page to be processed by 1, and then advances to step S312. In step S <b> 312, the CPU 101 determines whether the variable i exceeds the number of pages of the content 200 being processed, that is, whether all pages of the content 200 have been processed. When all the pages of the content 200 have not been processed, the process returns to S302, the page image of the next page is acquired, and the subsequent processing is performed. When it is determined that the variable i exceeds the total number of pages and all the pages of the content 200 have been processed, the structure analysis process is terminated.

  FIG. 5 is a diagram illustrating an example of partial area data according to the first embodiment and a partial area management table for managing the partial area data. The partial area management unit 112 manages partial area data using a partial area management table. The partial area management unit 112 stores the partial area management table in the RAM 102 or the HDD 104 that is a storage device of the image display apparatus 100. 5B is referred to in the third embodiment, and FIG. 5C is referred to in the fourth embodiment.

  In FIG. 5, each column of the partial area management table indicates items of partial area data. The items actually managed as partial area data are 501 to 509, and the partial area 521, the frame 522, and the order label 523 are provided to be associated with the partial areas referred to in FIG. 4, FIG. 12, and FIG. . The item of partial area data includes page number 501, partial area number 502, coordinate 503, width 504, area attribute 505, display setting 506, character direction 507, character size 508, and display order 509. The page number 501 is the page number of the page image including the partial area. The partial area number 502 is a number for identifying the partial area. The “background” number of the page image is set to “0”, and “1” to “16” in the order of the partial areas 421 to 436 in FIG. Is given. With these page number 501 and partial area number 502, each partial area can be uniquely specified in the content. A coordinate 503 indicates the upper left coordinate of the partial area, and a width height 504 indicates the width and height of the partial area. The coordinates and the height of the partial area have been described with reference to FIG. An area attribute 505 indicates an attribute corresponding to the partial area, and includes title, text, header, footer, table, figure, and background as attributes. A display setting 506 indicates whether to display or hide the partial area in the presentation process. A partial area enlarged in the presentation process is set as “display”, and a partial area hidden in the presentation process is set as “non-display”. The partial area display setting change processing will be described in the third embodiment. The character direction 507 indicates the direction of the character when the character is included in the partial area (writing direction). When the character direction is horizontal writing, “horizontal” is displayed. When the character direction is vertical writing, “vertical” is displayed. Set. A character size 508 indicates a character size when a character is included in the partial area. A display order 509 indicates the display order of the partial areas related to the page image of one page. This display order editing process will be described later with reference to FIG. 18 (Embodiment 3) and FIG. 19 (Embodiment 3).

  Next, data of each row of the partial area management table will be described.

  Each row of the partial area management table corresponds to one partial area data. For example, in the page image of FIG. 4B, since there are 17 partial areas 420 to 436, the partial area table of the page image of 1 page in FIG. Similarly, this partial area management table manages all partial area data included in all page images 201-1 to 201-N of the content 200. These partial area data are used in a mode control process, a presentation process, an edit process, an order change process and the like which will be described later.

  Next, the partial area 521, the frame 522, and the order level 523 will be described.

  The partial area 521 corresponds to each partial area extracted by the page analysis according to FIG. A frame 522 indicates a frame that surrounds each partial area described with reference to FIGS. The process for displaying the frame of the partial area will be described later with reference to the flowchart of FIG. An order label 523 indicates an order label of each partial area described with reference to FIGS. The process of displaying the order label 523 will be described later with reference to the flowchart of FIG.

  The mode control process according to the first embodiment is a process of displaying content by selecting one from a plurality of display modes. In the first embodiment, at least two of the presentation mode and the editing mode are provided as display modes. The presentation mode is a mode for executing a presentation process for displaying a partial area of a page image at an enlargement ratio for each partial area in accordance with a set display order. For example, it is used when a presentation is given in a conference room or a product explanation is given for face-to-face business negotiations. In the edit mode, before displaying the page image in the presentation mode, the display order of the partial areas constituting the page image and the objects are displayed between the partial areas in which the order is continuous, and the display order is changed. Mode. The mode control process is executed when a desired content is selected from the content list and displayed by a user operation.

  FIG. 7 is a diagram illustrating a content list displayed on the display unit 105 of the image display apparatus 100 according to the first embodiment.

  FIG. 7A is a diagram showing an example of the content list 700 according to the first embodiment.

  This content list 700 shows a state in which a plurality of contents 701 to 703 stored in the HDD 104 of the image display device 100 are displayed as a list. The content selection in the content list 700 is performed in response to an instruction from the user, such as a user tap-in operation. As shown in FIG. 7A, when a tap-in operation is received for the content 1 (701), the mode control process for the content 1 is executed and the content 1 (701) is displayed.

  FIG. 6 is a flowchart illustrating the mode control process executed by the mode control unit 113 of the image display apparatus 100 according to the first embodiment. Note that the processing shown in this flowchart is realized by the CPU 101 executing an image processing program developed from the HDD 104 to the RAM 102. Therefore, the processing subject will be described as the CPU 101 here.

  First, in step S601, the CPU 101 receives a user operation. At this time, a content list as shown in FIG. The user operation accepted in S601 is one of two types of “presentation mode start instruction” and “edit mode start instruction”.

  In step S601, the CPU 101 proceeds to step S602 when receiving a “presentation mode start instruction”, and proceeds to step S611 when receiving an “edit mode start instruction”. Here, the “presentation mode start instruction” is a process of selecting a content that the user wants to display from the content list 700 by a tap-in operation. The “edit mode start instruction” is a process for selecting a content that the user wants to edit from the content list 700 by a long tap operation. In step S <b> 602, the CPU 101 reads partial area data of content for which a selection instruction has been accepted. For example, in the case of content 1 (701) shown in FIG. 7A, the partial area data shown in FIG. 5A is read. In step S603, the CPU 101 displays a page image based on the partial area data read in step S602. An example of a screen displayed on the display unit 105 at this time is shown in FIG. Here, when the “presentation mode start instruction” is received, the mode control unit 113 displays a page image and ends the mode control process. The subsequent presentation process will be described later with reference to the flowchart of FIG.

  On the other hand, in S611, the CPU 101 reads partial area data of the content for which the selection instruction has been accepted. In step S612, the CPU 101 displays a page image based on the partial area data read up to step S611. In step S613, the CPU 101 executes sequential flow line display processing based on the partial area data read up to step S612. An example of the screen of the display unit 105 at this time is shown in FIG. 12B, which will be described later with reference to FIG. At this time, when the “editing mode start instruction” is received, the mode control unit 113 displays the page image, executes the sequential flow line display process, and ends the mode control process. It should be noted that the edit process and the order change process can be continuously executed after this, and these will be described in the third and fourth embodiments.

  Note that the user operation accepted by the CPU 101 in step S601 is not limited to the above operation, and it is sufficient that the mode control unit 113 can recognize which mode has been selected.

  For example, FIG. 7B is a diagram showing a menu 711 that pops up when the user long taps the content 1 (701) in FIG. 7A. In this menu 711, when the presentation button 712 is touched, the presentation mode is selected, and when the edit button 713 is touched, the editing mode is selected. Thus, the user operation in S601 described above may be determined.

  In addition, when opening the content to be processed, the case of opening in the presentation mode and the case of opening in the editing mode are provided separately, thereby preventing the mistaken operation of the editing mode in the presentation mode and the user's operation error. Can be suppressed.

  Next, presentation processing for sequentially displaying the partial areas of the page image in the presentation mode will be described.

  FIG. 8 is a diagram for explaining screen switching in the presentation process in the image display apparatus 100 according to the first embodiment.

  FIG. 8A is a diagram illustrating a state where the first page image is displayed on the display unit 105 of the image display apparatus 100. Here, the screen size of the display unit 105 is W00 for the width and H00 for the height. A user operation accepted in the presentation mode is called a presentation operation. The presentation operation includes at least the following operations.

  The user operation for displaying the “next partial area” is a tap-in operation to the right half 801 of the screen of the display unit 105 in FIG. When this operation is accepted, the screen transitions to the display of the next partial area. The user operation for displaying the “previous partial region” is a tap-in operation to the left half 802 of the screen of the display unit 105 in FIG. When this operation is accepted, the screen transitions to the display of the next partial area.

  The user operation for displaying “next page” is a left swipe operation 811 on the screen of the display unit 105 in FIG. When this operation is accepted, the screen transitions to the next page image display. The user operation for displaying the “previous page” is a right swipe operation 812 on the screen of the display unit 105 in FIG. When this operation is accepted, the screen changes to the previous page image. The screen switching operation in this presentation operation is not limited to the above operation. For example, when the button for displaying “next display” or “previous display” is displayed on the screen, the screen displays the next partial area or the next page, or the previous partial area or the previous page. You may transition to.

  FIG. 9 is a flowchart for explaining the presentation processing and the partial region display enlargement processing by the presentation processing unit 114 of the image display apparatus 100 according to the first embodiment. Note that the processing shown in this flowchart is realized by the CPU 101 executing an image processing program developed from the HDD 104 to the RAM 102. Therefore, the processing subject will be described as the CPU 101 here.

  This presentation process is a display process that is sequentially executed on all partial areas included in all pages of the content 200. That is, the display is started from the partial area of the display order 0 on the first page of the first page, and the process is executed on each partial area of the first page according to the display order. Then, after the process for all the partial areas on the first page is completed, the display process for the partial area of the next page is executed, and this is repeated until the last page.

  First, in step S <b> 901 in the flowchart of FIG. 9A, the CPU 101 sets a variable i that refers to a page number to be processed in the RAM 102. This variable i is initially set to “1” indicating the first page number. In step S902, the CPU 101 acquires all partial area data included in the page image having the page number set in the variable i from the partial area management table. In step S <b> 903, the CPU 101 sets the display order 509 of the partial area to be processed to a variable j provided in the RAM 102. This variable j is an index for executing determination processing for all partial areas included in one page, and initially, “0” that is the top display order is set.

  Next, the processing advances to step S904, and the CPU 101 executes partial area enlargement display processing. This is a process of displaying the partial area at an appropriate display magnification according to the partial area data stored in the partial area management data of FIG. This partial area enlargement display processing will be described later with reference to the flowchart of FIG.

  In step S905, the CPU 101 receives a user operation. The user operation accepted here is the operation described with reference to FIGS. That is, any one of an operation for displaying “next partial area”, an operation for displaying “next page”, an operation for displaying “previous partial area”, and an operation for displaying “previous page”. It is. If an operation for displaying “next partial area” is accepted, the process proceeds to S906. If an operation for displaying “next page” is accepted, the process proceeds to S909. If an operation for displaying “Previous Partial Area” is accepted, the process proceeds to S911. If an operation for displaying “Previous Page” is accepted, the process proceeds to S914.

  First, a case where an operation for displaying “next partial area” is received will be described. In step S906, the CPU 101 increments the value of the variable j by 1 to process the next partial area. In step S907, the CPU 101 determines whether the value of the variable j exceeds the total number of partial areas of the page being processed. If it is determined that the value of the variable j does not exceed the total number of partial areas, the process advances to step S908, and the CPU 101 determines the display setting 506 for the partial area data in the display order indicated by the value of the variable j. If it is determined that the display setting 506 is “display”, the process advances to step S904 to display the partial area. On the other hand, if it is determined that the display setting 506 is “non-display”, the process proceeds to S906, and the variable j is incremented by 1, and the process for the partial area in the next display order is performed. In this way, if the value of the variable j exceeds the total number of partial areas in S907, the processing of all partial areas of the page being processed has been completed, and the process proceeds to S909. In step S909, the CPU 101 increments the value of the variable i for referring to the page by one, and proceeds to step S910. The CPU 101 determines whether the value of the variable i exceeds the number of pages of the content 200 being processed. If it is determined that the value of the variable i does not exceed the number of pages, the process returns to S902, and the process is executed from the acquisition of the partial area data of the next page. On the other hand, if it is determined in S910 that the value of the variable i has exceeded the number of pages, since the processing for all pages has been completed, this presentation processing is terminated.

  Next, a case where an operation for displaying the “previous partial area” is received will be described. In step S911, the CPU 101 decrements the value of the variable j that refers to the previous partial area. Next, proceeding to S912, the CPU 101 determines whether or not the value of the variable j has become less than 0, that is, whether or not the processing has been completed up to the top partial area of the page being processed. If so, the process proceeds to S914. If not, the process proceeds to S913 because there is a partial area to be processed for the page. In step S913, the CPU 101 determines the display setting 506 of the partial area data to be processed. If the display setting 506 is “display”, the process advances to step S904 to display the partial area, and if the display setting 506 is “non-display”, the process advances to step S911. The variable j is decremented by 1, and the process for the partial area in the next display order is performed. In step S <b> 914, the CPU 101 decrements the value of the variable i by 1 so as to process the previous page, and proceeds to step S <b> 915. In step S <b> 915, the CPU 101 determines whether the value of the variable i is less than 1, that is, whether the page before the first page, i.e., all the content pages have been checked. If it is determined that the variable i is not less than 1, that is, the first page or a page after that, the process returns to S902, and the process is executed from acquisition of all partial area data of the page. On the other hand, if it is determined that the value of the variable i is less than 1, since the processing up to the first page has been completed, this presentation processing is terminated.

  Next, the partial area enlargement display process executed in S905 of FIG. 9A will be described. Subsequently, screen transition of the display unit 105 of the image display apparatus when the presentation process is executed will be described.

  FIG. 9B is a flowchart for explaining the enlarged display process of the partial area in S904 of FIG. This process is a process of setting an appropriate display enlargement ratio and display coordinates for each partial area, and performing an enlarged display for each partial area so that the entire area fits within the screen of the display unit 105.

  In step S921, the CPU 101 determines the display enlargement ratio based on the width of the partial area of the partial area data corresponding to the partial area to be processed. For the display enlargement ratio, the enlargement ratios in the width direction and the height direction are once calculated, and the lower one is set as the display enlargement ratio. For example, in the case of the partial area data (page number = 1, partial area number = 0) in the first row of the partial area management table in FIG. 5A, the expansion ratio W10 / W00 in the width direction and the expansion ratio in the height direction. H10 / H00 are compared, and the lower value is set as the display magnification of the partial area. In step S922, the CPU 101 sets the center coordinates of the screen of the display unit 105 as the center coordinates of the partial area based on the coordinates specified by the partial area data to be processed. Thus, the partial area is displayed at the center of the screen of the display unit 105. Here, as described above, H00 and W00 indicate the height and width of the screen of the display unit 105, and H10 and W10 indicate the height and width of the partial area.

  For example, in the case of the partial area data (background) in the first row of the partial area management table shown in FIG. 5A, the display is displayed at the center coordinates (X10 + (W10 / 2), Y10 + (H10 / 2)) of the partial area. The central coordinates (W00 / 2, H00 / 2) of the screen of the unit 105 are set. In step S923, the CPU 101 displays the partial area on the display unit 105 in accordance with the enlargement ratio and display coordinates set in step S922, and ends the process.

  FIG. 10 is a diagram illustrating an example of screen transition of the display unit 105 when the image display apparatus 100 according to the first embodiment executes the presentation process. Here, as an example, partial area data (partial area numbers 0 to 0 in FIG. 5A) obtained by executing page structure analysis processing (FIG. 3) for the page image 201-1 of the first page of the content 200 is shown. 16 data). Further, the partial areas included in the page image 201-1 are 17 pieces of 420 to 436 in FIG.

  In the presentation process of FIG. 9A, the CPU 101 acquires all partial area data of the image data of the first page in S902. In step S <b> 903, the CPU 101 sets “0” to the variable j indicating the display order, and executes processing for the partial area (page number = 1, partial area number = 0) whose display order is 0. This data is the data in the first row of the partial area management table (FIG. 5A), and the area attribute is “background” and indicates the entire page image. In step S904, the CPU 101 executes enlargement display processing for the partial area 420. A display example of the display unit 105 at this time is shown in FIG.

  Thereafter, each partial area in the page image is displayed according to the user operation accepted in S905.

  Here, as an example for briefly explaining the processing according to the first embodiment, a screen transition when an operation for displaying “next partial area” as a user operation is received will be described as an example. First, the screen transition in this case is shown in the order of FIGS. 10 (A), (B), (C), (D), (E), and (F). This will be described in order below.

  In the state where the variable j = 0 in FIG. 10A, an operation for displaying the “next partial area” is accepted in S905, the process proceeds to S906, and the value of the variable j for display order is changed from 0 to 1 To do. Then, the process is executed for the partial area 421 (page number = 1, partial area number = 1) whose display order is “1”. In S907, it is determined that the variable j = 1 is less than 17 which is the total number of partial areas, and the process proceeds to S908. The display setting 506 of the partial area 421 whose display order is “1” is determined as “display”. The process proceeds to S904. In S904, the enlarged display process of the partial area 421 whose display order is “1” is executed. A display example of the display portion 105 at this time is shown in FIG. Here, a portion corresponding to the partial region 421 is highlighted.

  In the state where the variable j = 1 in FIG. 10B, the operation of “next partial area” is accepted in S905, the process proceeds to S906, and the value of the variable j for display order is incremented by 1 and changed from 1 to 2 To do. Then, the process is executed for the partial area 422 (page number = 1, partial area number = 2) whose display order is “2”. In step S907, the variable j = 2 is determined to be less than the total number of partial areas 17, and the process proceeds to step S908. The display setting 506 of the partial area 422 whose display order is “2” is determined to be “display”, and the process proceeds to step S904. In S904, an enlarged display process of the partial area 422 is executed. A display example of the display portion 105 at this time is shown in FIG. In FIG. 10C, a portion corresponding to the partial region 422 is highlighted. Hereinafter, similarly, the value of the variable j is sequentially incremented from 3 to 9, and the enlarged display processing of the partial areas 423 to 429 is sequentially executed (not shown).

  When the operation of “next partial area” is accepted in S905 when the variable j = 9, the process proceeds to S906, and the value of the variable j for display order is incremented by 1 to “10”. Then, the process is executed for the partial area 430 (page number = 1, partial area number = 10) whose display order is “10”. In step S907, since the variable j = 10 is less than the total number of partial areas 17, the process proceeds to step S908. Since the display setting 506 of the partial area 430 whose display order is “10” is “display”, the process proceeds to step S904. The enlarged display process is executed. A display example of the display portion 105 at this time is shown in FIG. In FIG. 10D, a portion corresponding to the partial region 430 is highlighted.

  When the operation of “next partial area” is accepted in S905 when the variable j = 10 in FIG. 10D, the process proceeds to S906, and the value of the variable j for display order is incremented by 1 to “11”. . Then, the process is executed for the partial area 431 (page number = 1, partial area number = 11) whose display order is “11”. In step S907, since the variable j = 11 is less than the total number of partial areas 17, the process advances to step S908. The display setting 506 of the partial area 431 whose display order is “11” is determined as “display”, and the process advances to step S904. In S904, an enlarged display process of the partial area 431 is executed. A display example of the display unit 105 at this time is shown in FIG. In FIG. 10E, a portion corresponding to the partial region 431 is highlighted. Similarly, when an operation of “next partial area” is accepted in FIG. 10E, an enlarged display process of the partial area 434 is executed as shown in FIG.

  Hereinafter, similarly, the value of the variable j is sequentially incremented from 13 to 16, and the enlarged display processing of the partial areas 433 to 436 is sequentially executed (not shown). In S907, when the value of the variable j is not less than “17”, the process proceeds to S909, and the process proceeds to the next page.

  As described above, a user who makes a presentation can make a presentation while enlarging and displaying each partial area according to the description.

  Next, sequential flow line display processing executed in the edit mode will be described. The sequential flow line display process is a process executed in S613 when the edit mode is selected in the mode control process described with reference to FIG. The order flow line display process is a process for superimposing and displaying on the page image the order flow lines connecting the ranges of the partial areas, the order labels indicating the display order, and the partial areas in which the order is continuous.

  FIG. 11 is a flowchart illustrating the sequential flow line display process executed by the editing processing unit 115 in step S613 of FIG.

  FIG. 12 is a diagram illustrating a specific example of sequential flow line display processing by the image display apparatus 100 according to the first embodiment.

  Here, FIGS. 12B and 12E show examples of screens displayed on the display unit 105 when the sequential flow line display process is executed.

  With reference to the flowchart of FIG. 11A, the sequential flow line display process of S613 of FIG. 6 will be described. In this order flow line display process, the order flow lines are displayed between the partial areas in which the display order is continuous, and the display order of all the partial areas in the page image can be easily combined with the positions of the partial areas. It is a process that makes it possible to grasp without omission.

  In step S1101, the CPU 101 sets a value of a variable i in the RAM 102 that indicates a page number to be processed. This variable i is an index for referring to the page, and the initial value is set to “1” indicating the first page number. In step S1102, the CPU 101 acquires all partial area data included in the page image having the page number set in the variable i from the partial area management table. In step S <b> 1103, the CPU 101 sets a variable j indicating the display order of the partial areas to be processed in the RAM 102. This variable j is a reference index for a partial area included in one page, and its initial value is set to “0” indicating the top display order.

  In step S1104, the CPU 101 superimposes and displays the frame of the partial area on the page image based on the coordinates 503 and the height 504 of the partial area. In step S1105, the CPU 101 sequentially displays the display order 509 of the partial areas to be processed as labels on the page image. In step S1106, the CPU 101 superimposes and displays an order flow line connecting the order label of the partial area corresponding to the variable j and the order label of the partial area corresponding to (variable j + 1) on the page image. In step S1107, the CPU 101 executes control processing for the display state of the sequential flow lines. The process for controlling the display state of the sequential flow lines will be described later with reference to FIG. The process advances to step S1108, and the CPU 101 increments the value of the variable j by 1, and advances to step S1109. In step S1109, the CPU 101 determines whether the value of the variable j is less than the total number of partial areas included in the page being processed, that is, whether the processing for all partial areas of the page has been completed. If not completed, there is an unprocessed partial area, and the process returns to S1106. If completed, the sequential flow line display process ends.

  FIG. 11B is a flowchart for explaining the display state control processing of the sequential flow line in S1107 of FIG. The order flow line display state control process switches the order flow line display state or notifies the user that the display order may not match the user's intention.

  First, in step S <b> 1111, the CPU 101 sets a display order (here, “0” as an initial value) in a variable k of the RAM 102. This variable k is an index for referring to all partial areas included in the page to be processed. The above-described order variable j and this variable k are different indexes. In step S1112, the CPU 101 determines whether the value of the variable k is equal to the value of the variable j or (variable j + 1). When it determines with it being equal here, it progresses to S1113, and when that is not right, it progresses to S1115. In step S <b> 1113, the CPU 101 determines whether the target flow line (the display order connects the variable j and the (variable j + 1) -th) and the partial region whose display order is the variable k-th. That is, here, it is determined whether or not the order flow line is included in the k-th partial area whose display order is the variable. That is, it is determined whether or not the sequential flow line straddles a partial area different from the connected partial areas. If the determination result is true, that is, if another determination is made, the process advances to step S1114 to increment the hit determination counter provided in the RAM 102. On the other hand, when this is not the case, that is, when not crossing a partial area different from the connected partial areas, S114 is skipped and the process proceeds to S1115. In step S1115, the CPU 101 increments the value of the variable k and proceeds to step S1116. In S1116, the CPU 101 determines whether the value of the variable k is less than the total number of partial areas included in the page being processed. If so, the process proceeds to S1112. If not, the process proceeds to S1117.

  In step S1117, the CPU 101 determines whether the value of the hit determination counter is 1 or more. This determines whether or not the sequential flow line straddles a partial area that is not connected to itself. If the value of the hit determination counter is not 1 or more, that is, if it does not straddle the partial area, the process proceeds to S1118. The sequence flow line is normally displayed, and this process is terminated. On the other hand, if the value of the hit determination counter is 1 or more, the process proceeds to S1119. At that time, the order flow line is highlighted, and this process ends. In the first embodiment, FIG. 12C shows a normal order sequential flow line, and FIG. 12D shows a highlighted sequential flow line.

  FIGS. 12B and 12E are diagrams illustrating screen examples when the sequential flow line display control process and the sequential flow line display state control process of FIG. 11 are executed. Note that FIG. 12B and FIG. 12E are the same. 12B and 12E, since there are many objects to be displayed, FIG. 12B is shown without reference numerals, and FIG. 12E is enlarged and shown with reference numerals. What is actually displayed on the display unit 105 of the image display device 100 is the image shown in FIG. Here, description will be made with reference to FIG.

  In FIG. 12E, reference numerals 1201 to 1216 denote partial area frames superimposed and displayed in the sequential flow line display processing S1104. “Order labels”, which are numbers indicating the display order of the partial areas superimposed and displayed in S1105 of FIG. The correspondence with the partial areas 421 to 436 in FIG. 4 is indicated by an item 522 in FIG. Further, the sequential flow lines superimposed and displayed in S1106 in FIG. 11A are indicated by 1241 to 1255 in FIG.

  As shown in FIG. 12B, the display order of the partial areas starts from the upper left and moves to the lower left, then moves from the lower left to the upper right, further moves from the upper right to the lower right, and finally moves to the middle right end. It can be easily grasped. Further, in FIG. 12E, the order of display of the order flow lines 1241 to 1255 is not as intended by properly using normal display (FIG. 12C) and highlighted display (FIG. 12D). It is easy to find potential partial areas. For example, in FIG. 4 (B), FIG. 12 (B) and (E), the partial areas 431 to 433 located in the middle right are desired to be displayed continuously.

  In such a case, the highlighted sequential flow line 1254 has no sequential flow line between the partial region 431 and the partial region 432, and there is a possibility that the display order of the sequential flow line may not be as intended. Is displayed to the user in an easy-to-understand manner. A method for changing the order will be described in the third and fourth embodiments. Note that, here, the highlighting of the sequential flow line is applied only to the sequential flow line across a plurality of partial regions.

  As described above, according to the first embodiment, the user can display all the partial areas in the page image by displaying the order flow lines between the partial areas in which the display order of the partial areas in the page is continuous. The order can be easily grasped together with the position of the partial area without omission.

  In addition, by displaying the order flow line using the normal display and the highlight display properly, the user can easily grasp the partial area whose display order is not as intended. Thereby, even when the user confirms the display order of all the partial areas for a plurality of pages, the work load required for the display can be reduced. Furthermore, it is possible to remove the frustration of the user due to the display order not being intended as it occurred after the presentation started.

[Embodiment 2]
Next, a second embodiment of the present invention will be described. In the second embodiment, a case where display settings for each partial area are added to the sequential flow line display processing described in the first embodiment will be described. The display setting changing process is a process of changing display or non-display for each partial area, and FIG. 14 is a flowchart for explaining the display setting changing process. The partial area for which “display” is set is an enlarged display target in the presentation process described with reference to the flowchart of FIG. 9, and the partial area for which “non-display” is set is not displayed in the presentation process. In the second embodiment, a case where sequential flow lines are displayed only between partial areas set to “display” in the display setting change processing will be described. Note that the hardware configuration of the image display apparatus according to the second embodiment is the same as that of the image display apparatus according to the above-described first embodiment, and a description thereof will be omitted. The difference between the second embodiment and the first embodiment is the display setting change process and the sequential flow line display process that accompanies the change process, and the rest is the same as the first embodiment.

  FIG. 13 is a flowchart for describing editing processing executed by the editing processing unit 115 of the image display apparatus 100 according to the second embodiment. Note that the processing shown in this flowchart is realized by the CPU 101 executing an image processing program developed from the HDD 104 to the RAM 102. Therefore, the processing subject will be described as the CPU 101 here.

  In this editing process, as described with reference to the flowchart of FIG. 6, after the content is displayed in the edit mode, a user operation is accepted and editing for the setting of the partial area is executed. In FIG. 13, the CPU 101 repeatedly executes the processes from S1301 to S1304 according to the user operation in the edit mode.

  First, in step S1301, the CPU 101 accepts a user operation on the display unit 105. If a tap-in operation is accepted, the process proceeds to step S1302, and if a long tap operation is accepted, the process proceeds to step S1303. In step S1302, the CPU 101 executes display setting change processing on the partial area to be processed. This display setting changing process will be described later with reference to the flowchart of FIG. In step S <b> 1303, the CPU 101 executes display order change processing for the partial area to be processed. This order change process will be described in the third embodiment.

  In the second embodiment, the display setting change process is performed when a tap operation is received, and the order change process is performed when a long tap operation is received. However, the present invention is not limited to this operation. That is, it is only necessary to recognize which operation has been performed on the partial area to be processed.

  The display setting changing process in S1302 is a process executed in the edit mode, and is a process for changing the display setting of the partial area. The display setting of the partial area is for setting data included in the partial area data (display setting 506 in FIG. 5A). In the presentation process, it is determined whether to display or not display the partial area. This is the setting. This setting is referred to in S908 and S913 in FIG.

  Further, the display setting of the partial area is expressed by, for example, a color within the frame of the partial area in the edit mode. As will be described later with reference to FIG. 16C, the frame of the partial area whose display setting is “display” is displayed transparently, and the frame of the partial area whose display setting is “non-display” is displayed semi-transparently. Therefore, the inside of the frame is grayed out.

  Explaining earlier, FIG. 16C shows a state in which the partial area data of FIG. 17C is displayed in the edit mode. That is, in the partial region data of FIG. 17C, partial regions whose display setting 506 is “non-display” are partial regions 421, 425, and 428. The frames of these partial areas are frames 1201, 1205, and 1208. Since these frames are displayed in a translucent manner, the frames are grayed out. Since the display setting of the other partial areas is “display”, the inside of the frame is displayed transparently.

  In the second embodiment, the partial area whose display setting is “display” and the partial area whose display setting is “non-display” are expressed by the difference in color in the frame, but this expression method is not limited to this. For example, the display density of the partial area with the display setting “display” and the partial area “non-display” may be changed, or the decoration at the time of display may be different. That is, it is only necessary that both are displayed so as to be identifiable.

  Next, a process of changing the display setting of each partial area set in this way will be described with reference to the flowchart of FIG.

  FIG. 14 is a flowchart for explaining the display setting change processing in S1302 of FIG.

  First, in S1401, the CPU 101 acquires the coordinates acquired by the user's tap-in operation in S1301 as the coordinates of the selection instruction. In step S1402, the CPU 101 determines whether there is a partial area in the area designated by the coordinates. This determination is executed for all partial areas other than “background” among the partial areas included in the page image. Since “background” is the entire page image and the display setting is always “display”, it is excluded from this determination target. If it is determined that the designated coordinates include a partial area, the process advances to step S1403. If not, the process ends. In step S <b> 1403, the CPU 101 obtains the display setting 506 for the partial area determined to include the designated coordinates from the partial area data and determines whether it is “display”. If the display setting 506 of the partial area is “display”, the process proceeds to S1404, and if it is “non-display”, the process proceeds to S1405.

  In step S1404, the CPU 101 changes the display setting 506 of the partial area from “display” to “non-display”, and the process advances to step S1405. In step S1405, the CPU 101 changes the color in the display frame of the partial area from transparent to translucent grayout, and the process advances to step S1408. Here, the display method is not limited to gray out, but may be changed to a display method indicating “non-display”. On the other hand, in step S1406, the CPU 101 changes the display setting 506 of the partial area from “non-display” to “display”, and the process advances to step S1407. In step S1407, the CPU 101 changes the color in the display frame of the partial area from translucent gray out to transparent, and the process advances to step S1408. Here, the display method is not limited to transparent, but may be changed to a display method indicating “display”.

  In step S <b> 1408, the CPU 101 stores the changed display settings in the partial area data display settings 506. In step S1409, the CPU 101 updates the display of the flow line order and the order label based on the updated display order. The contents of this process will be described with reference to the flowchart of FIG.

  FIG. 15 is a flowchart for explaining the sequential flow line display processing in S1409 of FIG. The difference between the sequential flow line display process shown in FIG. 15 and the sequential flow line display process (FIG. 11) according to the first embodiment is that the partial area display setting 506 described in FIG. 14 is considered. It is in. That is, in the second embodiment, the sequential flow line display process is performed in consideration of the display setting 506 included in the partial area data.

  In FIG. 15, the processing of S1501 to S1505 is the same as S1101 to S1105 of FIG.

  In step S1505, an order label indicating the partial region display order 509 is superimposed on the page image, and the process advances to step S1506. In step S1506, the CPU 101 determines whether the display setting of the partial area of the display order variable j to be processed is “display”. If “display” is determined here, the process proceeds to step S1507. If “not displayed”, the process proceeds to step S1521. In step S1521, the CPU 101 hides the order flow line connected to the partial region whose display order is the variable j, and proceeds to step S1514. As a result, the sequential flow lines coming out of the non-display partial area are deleted.

  In step S <b> 1507, the CPU 101 assigns (variable j + 1) to the variable k indicating the display order of the partial areas to be processed. The process advances to step S1508, and the CPU 101 determines whether the display setting of the partial area whose display order is the variable k is “display”. If it is determined as “display”, the process proceeds to S1511. If it is “not displayed”, the process proceeds to S1509. In step S1509, the CPU 101 increments the value of the variable k by 1 and proceeds to step S1510, in which the CPU 101 determines whether processing for all partial areas included in the page being processed has been completed. If processing for all partial areas is not completed, the process returns to S1508. If processing for all partial areas of the page is completed, the process proceeds to S1513.

  In step S <b> 1511, the CPU 101 superimposes and displays an order flow line connecting the order label of the variable j indicating the display order and the order label of the variable k as the display order on the page image. Next, proceeding to step S <b> 1512, the CPU 101 executes control processing for the display state of sequential flow lines. The process for controlling the display state of the sequential flow lines is as described with reference to the flowchart of FIG. The process advances to step S1513, and the CPU 101 increments the value of the variable j by 1 and advances to step S1514. In step S1514, the CPU 101 determines whether the value of the variable j is less than the total number of partial areas included in the page being processed. It is determined whether the processing for is completed. If the process for all partial areas has not been completed, the process returns to S1511. If the process for all partial areas has been completed, this process ends.

  As described above, according to this processing, sequential flow lines are displayed only between partial areas whose display setting is “display”.

  An example of screen transition when the display setting change process described with reference to the flowchart of FIG. 14 and the sequential flow line display process described with reference to the flowchart of FIG. 15 are executed will be described with reference to FIG. To do.

  FIG. 16 is a diagram illustrating an example of screen transition when the sequential flow line display process is executed in the image display apparatus 100 according to the second embodiment.

  Here, as an example, the page image 201-1 of the first page of the content 200 is used. The page structure analysis process described in the first embodiment is executed on the page image 201-1, and partial areas 421 to 436 shown in FIG. 4B are acquired. Here, FIG. 17A shows partial area data of the partial areas 421 to 430 among the partial area data of these partial areas.

  FIG. 16A shows an example in which the frame of each partial area, the order label, and the order flow line are superimposed on the partial area when the page image 201-1 is opened in the edit mode. This is the same as FIGS. 12A and 12E. In FIG. 16A, portions common to those in FIG. 12E are denoted by the same reference numerals.

  In FIG. 16A, when a tap operation on the partial region 421 (the region 1601 on which the order label 1221 is superimposed, the display order = 1) is received, it is determined in S1401 to S1402 that the partial region 421 has been tapped. In step S1403, it is determined whether the display setting of the partial area 421 is “display”. Here, from FIG. 17A, since the display setting of the partial area 421 is “display”, the process proceeds to S1404, and the display setting of the partial area is changed to “non-display”. In step S1405, the color in the display frame 1221 of the partial area 421 is changed to gray out. Thereafter, the process proceeds to S1409, and the display of the sequential flow line 1241 is updated. In FIG. 16B, the inside of the frame 1221 of the partial area 421 displayed in FIG. 16A is grayed out.

  If the display setting of the partial area 421 is determined to be “non-display” in S1506 of FIG. 15 showing the details of the processing of S1409, the process proceeds to S1521, and the sequential flow line 1241 that has come out of the partial area 421 is hidden. This display state is shown in FIG. In FIG. 16B, the sequential flow line 1241 displayed in FIG. 16A is deleted. The partial area data at this time is shown in FIG. In FIG. 17B, as indicated by 1701, the display setting of the partial area 421 is changed to “non-display”.

  Next, a case will be described in which the area 1602 corresponding to the partial area 425 and the area 1603 corresponding to the partial area 428 are tapped in the display state of FIG.

  In FIG. 16B, when a tap operation is received on the area 1602 (partial area including the order label 1225, display order = 5), it is determined in steps S1401 to S1402 that the partial area 425 is tapped. In step S1403, it is determined whether the display setting of the partial area 425 is “display”. Here, from FIG. 17B, since the display setting of the partial area 425 is “display”, the process proceeds to S1404, the display setting is changed to “non-display”, and the display area 1205 of the area 1602 is displayed in S1405. Change the color to grayed out. Thereafter, the process proceeds to S1409, and the display of the sequential flow line 1244 is updated. Here, when the variable j = 4, the display setting of the partial area 424 is determined as “display” in S1506, and when the variable k = 6, the display setting of the partial area 426 is determined as “display”. As a result, in step S <b> 1511, the order flow line 1602 connecting the order label of the partial area 424 and the order label of the partial area 426 is displayed. Next, when the variable j = 5, the display setting of the partial area 425 is determined as “non-display” in S1506, and both the sequential flow line 1244 and the sequential flow line 1245 connected to the partial area 425 are hidden in S1521. . As a result, as shown in FIG. 16C, the sequential flow line 1244 input to the area 1602 in FIG. 16B and the sequential flow line 1245 output from the area 1602 are both erased.

  Next, a case where a tap operation on an area 1603 corresponding to the partial area 428 (partial area displaying the order label 1228, display order = 8) is received will be described. At this time, in S1401 to S1402, it is determined that the area 1603 corresponding to the partial area 428 is tapped, and the process proceeds to S1403, where it is determined whether the display setting of the partial area 428 is “display”. At this time, as shown in FIG. 17B, since the display setting of the partial area 428 is “display”, the process proceeds to S1404, and the display setting of the partial area is changed to “non-display”. In step S1405, the color of the display frame 1208 of the partial area 428 is changed to gray out, and the process advances to step S1409.

  In S1409, when the variable j = 7, the display setting of the partial area 427 is determined as “display” in S1506, and when the variable k = 9, the display setting of the partial area 429 is determined as “display” in S1508. The process advances to step S1511 to display an order flow line 1604 (FIG. 16C) that connects regions before and after the region 1603 (corresponding to the partial region 427 and the partial region 429). If the display setting of the partial area 428 is determined to be “non-display” in S1506, the process proceeds to S1521, and the sequential flow line 1247 and the sequential flow line 1248 in FIG. 16B connected to the area 1603 corresponding to the partial area 428 are obtained. Hide. This display state is shown in FIG. 16C, and the partial area data at that time is shown in FIG. In FIG. 17C, the display setting 506 of the partial area 425 is changed to “non-display” in 1702, and the display setting 506 of the partial area 425 is changed to “non-display” in 1703.

  In the sequential flow line display processing according to the second embodiment, the count-up of the hit determination in S1117 in FIG. 11B is performed only in the partial area where the display setting of the partial area is “display” (not illustrated). ).

  As described above, according to the second embodiment, the display setting can be changed for each partial area, and the display of the sequential flow line between the partial areas can be changed in accordance with the changed display setting. it can. Thereby, the user can confirm the display order of only the partial areas set as display targets in the page image easily and without overlooking.

[Embodiment 3]
Next, Embodiment 3 of the present invention will be described. In the third embodiment, processing for changing the display order of partial areas will be described. The method for changing the display order of the partial areas according to the third embodiment is used together with the method for checking the display order of the partial areas using the order flow lines shown in the first and second embodiments. The hardware configuration of the image display apparatus according to the third embodiment is the same as that of the image display apparatus according to the above-described first embodiment, and thus the description thereof is omitted.

  In the method of changing the display order of partial areas according to the third embodiment, an arbitrary partial area is an operation (drag) target. Then, the partial area is moved (dropped) to an order flow line related to a desired display order. When this operation (drag and drop) is performed, the display order of the partial areas to be operated is inserted between the display orders of the partial areas connected by the selected order flow line. The display order changing process according to the third embodiment is shown in FIG. 18, and the screen transition of the display unit 105 of the image display apparatus 100 when it is executed will be described with reference to FIG.

  FIG. 19 is a diagram illustrating an example of screen transition of the display unit 105 of the image display apparatus 100 according to the third embodiment.

  FIG. 19A corresponds to the screen of FIG. 12E described in the first embodiment.

  FIG. 18 is a flowchart for describing processing for changing the display order of the partial areas executed by the editing processing unit 115 of the image display apparatus 100 according to the third embodiment. Note that the processing shown in this flowchart is realized by the CPU 101 executing an image processing program developed from the HDD 104 to the RAM 102. Therefore, the processing subject will be described as the CPU 101 here. In FIG. 18, S1800 and S1817 indicate the start and end of the iterative process, respectively.

  First, in step S <b> 1801, the CPU 101 receives a user operation on the screen of the display unit 105. If a “start instruction” is accepted, the process proceeds to S1802. If a “movement instruction” is accepted, the process proceeds to S1811. If a “confirmation instruction” is accepted, the process proceeds to S1821. Here, when the start of pressing the screen by the touch-in operation is detected, the CPU 101 determines that a “start instruction” has been received. When a touch-in operation is detected and a drag operation is further detected, the CPU 101 determines that a “move instruction” has been received. When the drag operation ends and a tap-out operation is detected, the CPU 101 determines that a “confirmation instruction” has been received.

  If it is determined that the “start instruction” has been received, the process advances to step S1802, and the CPU 101 acquires the coordinates when the tap-in operation that is the “start instruction” is performed and the drag operation is started. This is the start instruction coordinate. In step S1803, the CPU 101 determines whether the start instruction coordinates are included in the partial area. This determination is executed for all partial areas whose area attributes are other than “background” among the partial areas included in the page image. “Background” is excluded from this determination because the entire page image is not subject to order change. If the CPU 101 determines in step S1803 that the start instruction coordinates are included in the partial area, the process advances to step S1804. If not, the process advances to step S1817, and returns to step S1800, which is the start of the iterative process. In step S <b> 1804, the CPU 101 temporarily holds the partial area including the start instruction coordinates as the partial area A. In step S1805, the CPU 101 copies the image in the range of the partial area A. This is a duplicate image. The process advances to step S1806, the CPU 101 hides the sequential flow lines between the previous and subsequent areas connected to the partial area A, and advances to step S1817.

  Next, when a “move instruction” is received, the process advances to step S <b> 1811, and the CPU 101 acquires the coordinates of the position of the page image on which the drag operation has been performed according to the “move instruction”. This is the movement instruction coordinate. In step S1812, the CPU 101 sets the movement instruction coordinates as the center coordinates of the duplicate image, and moves the duplicate image there. At this time, the duplicate image is displayed with shading to show the moving state. This display state is shown in FIG. In FIG. 19B, a duplicate image 1902 of an area 1901 (partial area displaying the order label 1235) corresponding to the partial area 432 is displayed during movement.

  In step S1813, the CPU 101 determines whether the duplicate image 1902 being moved matches the sequential flow line. In other words, a hit determination between the moving duplicate image 1902 and one of the order flow lines is performed. If the hit determination is true, the process proceeds to S1815. Otherwise, the process proceeds to S1814. In step S <b> 1814, the CPU 101 normally displays a sequential flow line and proceeds to step S <b> 1817. This normal display is a display process similar to S1118 in FIG. On the other hand, in step S <b> 1815, the CPU 101 displays the sequential flow lines determined to be hit in an overlapping manner. The overlapping display of the sequential flow lines will be described later with reference to FIG.

  Next, processing when a “confirmation instruction” is received will be described. When the “confirmation instruction” is received, in step S1821, the CPU 101 acquires the coordinates when the pressing of the screen by the drag operation is terminated by the “confirmation instruction”, that is, the tap-out operation is detected. This is set as the fixed instruction coordinates. Next, proceeding to S1822, the CPU 101 performs a hit determination to determine whether or not the confirmed instruction coordinates match the order flow line. If the result of the hit determination is true, the process proceeds to S1823. If not, the process proceeds to S1817, and the process returns to the start of the repetition process.

  In step S <b> 1823, the CPU 101 temporarily holds the partial areas connected by the sequential flow lines that coincide with the confirmed instruction coordinates as the partial areas B and C. Here, the first display order is a partial area B, and the second display order is a partial area C. In step S1824, the CPU 101 compares the display order of the partial area A that has been moved (here, a) with the order of the partial area B (here, b). The display order of the partial area C (here, c) is the next display order of the partial area B (c = b + 1). If the display order a is smaller than the display order b, that is, if the display order of the partial area A is before the display order of the partial area B, the process proceeds to S1825. In step S1825, the CPU 101 decrements the display order of the partial area A to the next partial area B (a + 1 to b) by −1. The process advances to step S1826, and the CPU 101 sets the original display order (b) of the partial area B as the display order of the partial area A, and then advances to S1829.

  On the other hand, if the display order of the partial area A is later than the display order of the partial area B in S1824, the process advances to S1827, and the CPU 101 displays (c to (a-1)) in front of the partial areas C to A. The order is incremented by 1 (increment). The process advances to step S1828, and the CPU 101 sets the original display order (c) of the partial area C as the display order of the partial area A, and then advances to step S1829.

  In step S1829, the CPU 101 overwrites and saves the updated display order on the partial area management data. The process advances to step S1830, and the CPU 101 updates the display of the flow line order and the order label based on the display order updated in step S1829. This process is the same as the sequential flow line display process described in Embodiment 1 with reference to FIG.

  Next, screen transition of the display unit 105 when the display order changing process described in the flowchart of FIG. 18 is executed will be described with reference to FIG.

  Here, as an example, the page image 201-1 of the first page of the content 200 is used. There are 16 partial areas 421 to 436 in FIG. 4B, which are obtained by executing the page structure analysis process (FIG. 3) on the page image 201-1, and automatically analyzing it. Moreover, the partial area data of these partial areas correspond to 16 pieces of 1 to 16 rows in FIG. As an example of changing the display order, three partial areas 431, 432, and 433 in FIG. 4B are used. The display order of these partial areas is “11”, “15”, and “16” in order. An example in which the display order of these partial areas is changed to “11”, “12”, and “13” will be described.

  In FIG. 19A, as described with reference to FIGS. 6 and 7, when the content is opened in the edit mode, the frame of each partial area, the order label, and the order flow line are displayed on the page image 201-1. It is displayed superimposed. This is the same as FIG.

  In the state of FIG. 19A, it is assumed that a drag operation is received for an area 1901 corresponding to the partial area 432 (partial area where the order label 1235 is displayed, display order = 15). In this case, when the drag operation is started, the process proceeds from S1801 to S1802, and the coordinates of the start change instruction, which is the pressing start position of the screen by the drag operation, are acquired. In step S1803, a hit determination between the start change coordinates and the partial areas 421 to 436 excluding the background attribute in the page image (partial area numbers of page number = 1 = 1 to 16) is executed. Here, since the start instruction coordinates are in the area 1901 corresponding to the partial area 432, the process proceeds to S 1804, and this area 1901 is set as the partial area A. In step S1805, a duplicate image of the partial area A is generated. In FIG. 19A, the image surrounded by the partial region frame 1215 is duplicated. Here, a duplicate image is indicated by reference numeral 1902 in FIG. In step S1806, the sequential flow line connected to the partial area A is hidden. In FIG. 19B, sequential flow lines 1254 and 1255 connected to the area 1901 are not displayed.

  Subsequently, when the pressed position by the drag operation is moved by the user operation, the process proceeds to S1811. Then, the drag coordinates are acquired as the movement instruction coordinates. In step S1812, the movement instruction coordinates are set to the center coordinates of the duplicate image 1902 and moved. The display state at this time is shown in FIG. Thereafter, it is assumed that the pressing position is moved to the position of the sequential flow line 1251 by the user's drag operation. The display state at this time is shown in FIG. The process advances to step S1813, and the hit determination between the moving duplicate image 1902 and the sequential flow line 1251 is true. If there is no partial area at the position corresponding to the confirmation instruction coordinate in S1803 (all hit determinations with respect to the partial area excluding the background attribute are all false), the duplicate image 1902 is deleted. That is, the display state returns to FIG. In step S1815, the sequential flow lines 1251 are displayed in an overlapping manner. In the third embodiment, the overlap display is an enlarged display of the sequential flow lines.

  When the drag operation is terminated by the user operation, it is determined that a confirmation instruction is accepted in S1801, and the process proceeds to S1821. Then, the end coordinate of the drag operation is acquired as the confirmation instruction coordinate. Next, proceeding to S1822, a hit determination between the confirmed instruction coordinates and each sequential flow line is executed. Here, since the fixed instruction coordinates are on the sequential flow line 1251, the process proceeds to S1823, and the region 1903 corresponding to the partial region 431 connected by the sequential flow line 1251 and the region 1904 corresponding to the partial region 434 are respectively referred to as the partial region B. Set to C. Next, the process proceeds to S1824, where the partial area A (display order = 15) and the partial area B (display order = 11) are compared. Since the display order of the partial area A is large, the process proceeds to S1827. In S1827 and S1828, the display order of the partial area A (area 1901) is inserted between the partial area B (area 1903) and the partial area C (area 1904).

  Specifically, in S1827, the display order of the partial areas from the partial area C (area 1904) to the front of the partial area A (area 1906) is incremented by +1. Accordingly, the display order of the area 1901 corresponding to the partial area 432 is changed from “15” to “16”, the display order of the area 1905 corresponding to the partial area 433 is changed from “16” to “16”, and the partial order is corresponding to the partial area 434. The display order of the area 1904 is changed from “12” to “13”. In S1828, “12”, which is the original display order of the partial area C (area 1904), is set in the display order of the partial area A (area 1901). In step S1830, the order flow line is updated.

  FIG. 5B shows the partial area management table at this time, and FIG. 19D shows the display state. As shown in FIG. 19D, the display order of the area 1903 corresponding to the partial area 431, the area 1901 corresponding to the partial area 432, and the area 1905 corresponding to the partial area 433 is “11”, “12”, “16” in this order. " An order flow line 1911 in FIG. 19D indicates an updated order flow line.

  Next, in the state shown in FIG. 19D, it is assumed that a drag operation is received on the area 1905 corresponding to the partial area 433 and an instruction to move the duplicate image 1907 to the position of the sequential flow line 1912 is received. This state is shown in FIG. In FIG. 19E, a region 1905 that is a drag target is a partial region A. In S1813, the moving area 1905 and the sequential flow line 1912 coincide with each other and the hit determination becomes true. Therefore, the sequential flow line 1912 extending from the area 1905 in S1815 is displayed in an overlapping manner.

  When a confirmation instruction is further input, the areas 1905 and 1904 to which the sequential flow line 1912 is connected are set as partial areas B and C in S1823, respectively. In step S1824, the display order of the area A (area 1905) is compared with the display order of the partial area B (area 1904). Since the display order of the area A is large here, the process advances to step S1827. In S1827, the display order of the partial areas from the area C (area 1904) next to the area A (area 1905) is incremented by 1, respectively. Accordingly, the display order of the area 1904 corresponding to the partial area 434 is changed from “13” to “14”, the display order of the area corresponding to the partial area 435 is changed from “14” to “15”, and the area 1906 corresponding to the partial area 436 is changed. The display order is changed from “15” to “16”. In step S1826, “13”, which is the original display order of the region C (region 1904), is set as the display order of the region A (region 1905 corresponding to the partial region 433). In step S1830, the sequential flow line is updated.

  FIG. 5C shows the partial area management table in this state, and FIG. 19F shows the display state at that time. In FIG. 19F, the display order of the area 1901 corresponding to the partial area 431 (the partial area displaying the order label 1231), the area 1901 corresponding to the partial area 432, and the area 1905 corresponding to the partial area 433 is “11” in order. “12” and “13”. And according to these display orders, a new order flow line is displayed.

  As described above, according to the third embodiment, the display order of the dragged partial areas is changed by dragging the partial areas to the position of the order flow line between the partial areas in which the display order is continuous. can do. In this way, the user can change the display order of the desired partial areas with an intuitive operation. The result of changing the display order can be easily confirmed.

[Embodiment 4]
Next, a fourth embodiment of the present invention will be described. In the above-described third embodiment, the display order changing method is a drag operation on a partial region, whereas the order changing process described in the fourth embodiment targets any order flow line as an operation (drag) target. Then, the order flow line is moved (dropped) to a partial area indicating a desired display order. By performing this operation (drag and drop), the display order of the selected partial areas is inserted into the display order of one partial area connected by the sequence flow line to be operated. Note that the hardware configuration of the image display apparatus according to the fourth embodiment is the same as that of the image display apparatus according to the first embodiment described above, and thus the description thereof is omitted.

  FIG. 20 is a flowchart for describing processing for changing the display order of the partial areas executed by the editing processing unit 115 of the image display apparatus 100 according to the fourth embodiment. Note that the processing shown in this flowchart is realized by the CPU 101 executing the image processing program 110 developed from the HDD 104 to the RAM 102. Therefore, the processing subject will be described as the CPU 101 here. In FIG. 18, S1800 and S1817 indicate the start and end of the iterative process, respectively. S2000 and S2015 are the start and end of the iterative process.

  First, in step S2001, the CPU 101 receives a user operation on the display unit 105. Here, when the start of pressing the screen by the touch-in operation is detected, the CPU 101 determines that a “start instruction” has been received. When a touch-in operation is detected and a drag operation is further detected, the CPU 101 determines that a “move instruction” has been received. Then, when the drag operation ends and a tap-out operation is detected, the CPU 101 determines that a “confirmation instruction” has been received.

  If it is determined that the “start instruction” has been received, the process advances to step S2002, and the CPU 101 acquires the coordinates at which the tap operation that is the “start instruction” has been performed and the drag operation has started. This is the start instruction coordinate. Next, proceeding to S2003, the CPU 101 determines whether or not the start instruction coordinates include a sequential flow line. This hit determination is executed for all sequential flow lines included in the page image. If it is determined that the start instruction coordinates include the sequential flow line, the process proceeds to S2004. If not, the process proceeds to S2015, and the process returns to S2000, which is the start of the repetition process. In step S2004, the CPU 101 temporarily holds the sequential flow line including the start instruction coordinates as the sequential flow line A. In step S2005, the CPU 101 determines whether the start instruction coordinate is closer to the start point or the end point of the sequential flow line A. The end point is the tip of the arrow, and the start point is the opposite. If it is determined that the fixed instruction coordinates are close to the start point, the process proceeds to S2006. If it is determined that the confirmation instruction coordinates are close to the end point, the process proceeds to S2007. In S2006, the CPU 101 temporarily sets the start point of the sequential flow line A as a movement instruction target, and proceeds to S2008. On the other hand, in S2007, the CPU 101 temporarily sets the end point of the sequential flow line A as a movement instruction target, and proceeds to S2008. In S2008, the CPU 101 temporarily holds the partial areas connected with the sequential flow line A as the partial areas B and C, and proceeds to S2015. Here, the display order is the partial area B and the subsequent display order is the partial area C.

  Next, if it is determined that a “move instruction” has been received, the process advances to step S <b> 2011, and the CPU 101 acquires the coordinates of the position of the page image on which the drag operation has been performed according to the “move instruction”. This is the movement instruction coordinate. Next, proceeding to S2012, the CPU 101 sets the movement instruction coordinates in the movement instruction target part of the sequential movement line A set up to that point and moves the sequential movement line. At this time, the order flow line A is enlarged to show the moving state. This display state will be described later with reference to FIGS. In step S2013, the CPU 101 determines whether or not the movement instruction coordinates that move the sequential flow lines are included in the partial area. If it is determined that the movement instruction coordinates are included in the partial area, the process proceeds to S2014. If not, the display of the moving order flow line is deleted, the process proceeds to S2015, and the process returns to the start of the repetition process. At this time, if an overlap to be described later is being displayed, it is also deleted (not shown). In step S2014, the CPU 101 displays (not illustrated) the movement instruction coordinates in a state where the partial areas determined to be hit in the previous step are overlapped, and the process advances to step S2015.

  If it is determined that the “confirmation instruction” has been received, the process advances to step S2021, and the CPU 101 obtains the coordinates at which the end of the drag operation as the “confirmation instruction”, that is, the tap-out operation is detected. This is set as the fixed instruction coordinates. Next, proceeding to S2022, the CPU 101 performs a hit determination for determining whether or not the confirmed instruction coordinates are included in the partial area. If it is determined here that the fixed instruction coordinates are included in the partial area, the process proceeds to S2023. If not, the moving sequential flow line image is deleted, the process proceeds to S2015, and the process returns to the start of the repetition process.

  In step S <b> 2023, the CPU 101 temporarily holds a partial area including the confirmed instruction coordinates as a partial area D. Next, proceeding to S2024, the CPU 101 determines whether the movement instruction target of the sequential flow line A is the start point or the end point. If it is the start point, the process proceeds to S2025. If it is the end point, the process proceeds to S2026.

  If it is the start point, in S2025, the CPU 101 inserts the partial area D in the display order immediately before the partial area C, and proceeds to S2027. A specific example will be described later with reference to FIG. On the other hand, if it is the end point, the CPU 101 inserts the partial area D in the display order immediately after the partial area B in S2026, and proceeds to S2027. A specific example will be described later with reference to FIG. In step S2027, the CPU 101 overwrites and saves the updated display order in the partial area management data. In step S2028, the CPU 101 updates the display of the flow line order and the order label based on the updated display order. Since this process has the same contents as the sequential flow line display process described with reference to FIG. 11, the description thereof is omitted.

  Screen transition of the display unit 105 when the above-described order changing process is executed will be described with reference to FIG.

  FIG. 21 is a diagram illustrating an example of screen transition of the display unit 105 of the image display apparatus 100 according to the fourth embodiment.

  Here, as an example, the page image 201-1 of the first page of the content 200 is used as in the third embodiment. There are 16 partial areas 421 to 436 in FIG. 4B, which are obtained by executing the page structure analysis process (FIG. 3) on the page image 201-1, and automatically analyzing it. Further, the partial area data of these partial areas correspond to lines 1 to 16 in FIG.

  As an example of the order change, three partial areas 431 to 433 in FIG. 4 are used. The display order of these partial areas is “11”, “15”, and “16” in order. An example in which the display order of these partial areas is changed to “11”, “12”, and “13” will be described.

  As described above, when the content is opened in the edit mode, as shown in FIG. 21A, the frame of each partial area, the order label, and the order flow line are superimposed on the page image.

  In FIG. 21A, it is assumed that a drag operation is accepted for the order flow line 1251 (the area 2101 displaying the order label 1231 and the area 2102 displaying the order label 1232 are connected). In this case, when the drag operation is started, the process proceeds from S2001 to S2002, and the pressing start position of the screen by the drag operation is acquired as the start instruction coordinates. In steps S2003 to S2004, a hit determination between the start instruction coordinates and all the order flow lines in the page image is performed. Here, the order flow line 1251 is temporarily held as the win order flow line A. In S2005 to S2007, it is determined whether the start instruction coordinate is closer to the start point or the end point of the sequential flow line A. In the case of FIG. 21A, the end instruction is closer to the end point (the tip of the arrow). As temporarily hold. In S2008, the partial areas connected by the sequential flow lines 1251 are set as the partial area B and the partial area C, respectively. Here, the partial area B becomes the partial area 2101, and the partial area C becomes the partial area 2102.

  Subsequently, when the pressed position by the drag operation is moved by the user operation, the process proceeds to S2011. In step S2011, the drag coordinates are acquired as movement instruction coordinates. In step S2012, the movement instruction coordinates are set to an end point (point of an arrow) that is a movement instruction target of the order flow line A, and the end point of the order flow line A is moved. The display state at this time is shown in FIG. In FIG. 21B, it is assumed that the sequential flow line 1251 is enlarged and moved to the position of the area 2103 corresponding to the partial area 432.

  If the user presses the screen to stop the drag operation, the process advances to step S2021 because a confirmation instruction is accepted in step S2001. In step S2021, the end coordinates of the drag operation are acquired as confirmation instruction coordinates. Next, in S2022, a hit determination between the confirmed instruction coordinates and each partial area is executed. Here, since the fixed instruction coordinates are in the area 2103, the process proceeds to S2023, and the area 2103 (corresponding to the partial area 432) is set as the partial area D. In S2024 to S2026, the display order of the partial area D (area 2103) is inserted immediately after the partial area B (area 2101).

  Specifically, the display order of the partial areas (areas 2105 and 2106) from the partial area of the next display order (display order = 12) of the partial area B (area 2101) to the front of the partial area D is increased by one. . Accordingly, the display order of the area 2104 is changed from “12” to “13”, the display order of the area 2105 is changed to “14”, and the display order of the area 2106 is changed to “15”. Then, the display order of the partial area D (partial area 2103) is set to “12” which is the next display order of the partial area B. In step S2028, the sequential flow line is updated.

  The partial area management table at this time is shown in FIG. 5B, and the display state at this time is shown in FIG. In FIG. 21C, the display order of the partial area 431 (area 2101), partial area 432 (area 2103), and partial area 433 (area 2107) is “11”, “12”, and “16” in order. Accordingly, the sequential flow line 2110 extends from the partial region 431 (region 2101) to the partial region 432 (region 2103).

  Furthermore, it is assumed that a drag operation is received with respect to the sequential flow line 2111 (connecting the area 2103 and the area 2104) in the state of FIG. In this case, when the drag operation is started, the process proceeds from S2001 to S2002, and the pressing start position of the screen by the drag operation is acquired as the start instruction coordinates. In steps S2003 to S2004, a hit determination between the start instruction coordinates and all the sequential flow lines in the page image is performed, and the sequential flow line 2111 is temporarily held as the sequential flow line A. In S2005 to S207, it is determined whether the start instruction coordinate is closer to the start point or the end point of the sequential flow line A. In the case of FIG. Hold temporarily as a target. In S2008, the partial areas connected by the sequential flow lines 2111 are set as the partial area B and the partial area C, respectively. Here, the partial area B is an area 2103 corresponding to the partial area 432, and the partial area C is an area 2104 corresponding to the partial area 434.

  Subsequently, when the pressed position by the drag operation is moved by the user operation, the process proceeds to S2011. Then, the drag coordinates are acquired as the movement instruction coordinates. Next, in S2012, the movement instruction coordinates are set to the end point (the tip of the arrow) that is the movement instruction target of the order flow line A, and the end point of the order flow line A is moved. The display state at this time is shown in FIG.

  In FIG. 21E, it is assumed that the order flow line 2111 is enlarged and moved to the position of the partial area 433 (area 2107). Then, if the pressing of the screen by the drag operation is finished by the user operation, it is determined that the confirmation instruction is accepted in S2001, and the process proceeds to S2021. Then, the end coordinate of the drag operation is acquired as the confirmation instruction coordinate. In step S2022, a hit determination between the confirmed instruction coordinates and each partial area is executed. Here, since the fixed instruction coordinates are in the area 2107 corresponding to the partial area 433, the process proceeds to S2023, and the area 2107 is set as the partial area D. In S2024 to S2026, the display order of the partial area D (area 2107) is inserted immediately after the display order of the area 2103. That is, the display order is “13”.

  Specifically, the display of the partial areas (areas 2105 and 2106) from the next display order of partial area B (display order of partial area 434 (area 2104) = 12) to the front of partial area D (area 2107). Increase the order by one each. The display order of the area 2103 corresponding to the partial area 432 is “12”, the display order of the area 2107 corresponding to the partial area 433 is “13”, and the display order of the area 2104 corresponding to the partial area 434 is “14”. To do. Then, “16”, which is the next display order of the partial area B (area 2105), is set in the display order of the partial area 436 (area 2106). In step S2028, the sequential flow line is updated.

  The partial area management table at this time is shown in FIG. 5C, and the display state at that time is shown in FIG. In FIG. 21F, the display order of the partial area 431 (area 2101), partial area 432 (area 2103), and partial area 433 (area 2107) is “11”, “12”, and “13” in order, and the user intends They are displayed in the order they were displayed.

  As described above, according to the fourth embodiment, the user can change the display order of the partial areas to a desired order by dragging and moving the order flow lines that connect the partial areas that are in sequence. Can do. And the effect that the display order was changed can be confirmed easily.

[Embodiment 5]
Next, a fifth embodiment of the present invention will be described. In this fifth embodiment, an example will be described in which the user can select one of the order change processes described in the third and fourth embodiments. Note that the hardware configuration of the image display apparatus according to the fifth embodiment is the same as that of the image display apparatus according to the first embodiment, and a description thereof will be omitted.

  FIG. 22 is a diagram illustrating an example of a menu screen for switching the order changing method displayed on the display unit 105 of the image display apparatus 100 according to the fifth embodiment. This switching is executed by the mode control unit 113.

  In FIG. 22A, a content list 700 as shown in FIG. 7 is displayed, and a menu button 2201 is added to the content list 700. Portions in common with FIG. 7 are denoted by the same reference numerals.

  When the mode control unit 113 receives a user tap operation on the menu button 2201, the mode control unit 113 displays a menu screen 2202 as illustrated in FIG.

  In FIG. 22B, the menu screen 2202 includes menu items 2211 to 2213. The menu used in the fifth embodiment is the order change method menu 2211. The order change method menu 2211 displays the menu name “order change method” and “image” indicating that the current setting value is the order change by specifying “image” described in the third embodiment. Yes. If the current setting value is the order change by designating the “flow line” described in the fourth embodiment, “flow line” is displayed.

  When the user performs a tap operation on the order change method menu 2211 on the screen of FIG. 22B, an order change method selection menu 2220 as shown in FIG. 22C is displayed.

  FIG. 22C shows a state in which the order change method selection menu 2220 is displayed in a pop-up manner.

  This selection menu 2220 selects the “image” button 2221 for selecting the order change by specifying the “image” described in the third embodiment and the order change by specifying the “flow line” described in the fourth embodiment. A “flow line” button 2222 is included. Furthermore, a “cancel” button 2223 is included. When a tap operation on the “flow line” button 2221 is accepted, “flow line” is set as the order change method, and the order change method selection menu is closed. This state is shown in FIG. On the other hand, when a tap operation on the “image” button 2221 is accepted, “image” is set as the order change method, and the order change method selection menu is closed. That is, the state returns to the state of FIG. When a tap operation on the “Cancel” button 2221 is accepted, the order change method selection menu is closed and the screen returns to the screen of FIG.

  FIG. 23 is a flowchart for explaining mode control processing for reflecting the selected order changing method in the edit mode in the image display apparatus 100 according to the fifth embodiment. In the mode control process (FIG. 23) according to the fifth embodiment, the same process as the mode control process (FIG. 6) of the first embodiment is denoted by the same reference numeral, and the description thereof is omitted. Note that the processing shown in this flowchart is realized by the CPU 101 executing an image processing program developed from the HDD 104 to the RAM 102. Therefore, the processing subject will be described as the CPU 101 here.

  In step S2301, the CPU 101 determines the order change method selected in FIG. If the order change method is “image”, the process proceeds to S2302, and if it is “flow line”, the process proceeds to S2303. In step S2302, as described with reference to FIG. 18 described above, the CPU 101 changes the display order of the partial areas by designating “image”. On the other hand, in step S2303, as described with reference to FIG. 20, the CPU 101 performs processing for changing the display order of the partial areas by designating the “flow line”.

  As described above, according to the fifth embodiment, the user can select whether to change the display order of partial areas by specifying “image” or “flow line”. effective.

[Other Embodiments]
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors related to the computer of the system or apparatus read the program. It can also be realized by processing to be executed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

  The present invention is not limited to the above-described embodiment, and various changes and modifications can be made without departing from the spirit and scope of the present invention. Therefore, in order to make the scope of the present invention public, the following claims are attached.

  DESCRIPTION OF SYMBOLS 100 ... Image display apparatus, 101 ... CPU, 102 ... RAM, 105 ... Display part, 110 ... Image processing program, 111 ... Page structure analysis part, 112 ... Partial area management part, 113 ... Mode control part, 114 ... Presentation processing part 115 ... Editorial department

Claims (14)

Storage means for storing partial area data of a plurality of partial areas included in the image;
A presentation means for displaying each of the plurality of partial areas in a display order set by the partial area data in response to a user operation;
Display control means for displaying the image by superimposing objects corresponding to the display orders of the plurality of partial areas on the plurality of partial areas;
An image display device comprising:   The image display apparatus according to claim 1, wherein the object includes a number indicating the display order.   The image display apparatus according to claim 1, wherein the object includes a flow line that connects the partial areas in correspondence with the display order. The partial area data further includes a display setting for setting display or non-display of each partial area,
A first changing means for changing the display setting in response to a user operation;
5. The display control unit according to claim 1, wherein the display control unit changes the display of the partial area and the object in which the display setting is changed in accordance with the change of the display setting by the first changing unit. The image display device according to claim 1.   5. The image according to claim 4, wherein the display control unit grays out a partial area in which the display setting is changed to non-display by the first changing unit, and erases a flow line connected to the partial area. Display device.   The image display device according to claim 3, further comprising a second changing unit that changes a display order of the partial area data.   The image according to claim 6, wherein the second change unit changes the display order of the partial areas by moving the partial areas displayed by the display control unit to the position of the flow line. Display device.   The image display according to claim 7, wherein the second change unit changes the display order of the moved partial areas to a display order between two partial areas connected to the flow line. apparatus.   The second changing means changes the display order of the partial areas connected to the start point or end point of the flow line by an operation of moving the flow line displayed by the display control means. 6. The image display device according to 6.   In the operation of moving the flow line, when the position of the flow line instructed by the user is close to the end point of the flow line, the second changing means is a partial area instructed by the end point of the moved flow line The image display device according to claim 9, wherein the display order is changed to be next to the display order of the partial areas connected to the start point of the flow line.   In the operation of moving the flow line, when the position of the flow line instructed by the user is close to the start point of the flow line, the second changing means is a partial area indicated by the start point of the moved flow line The image display device according to claim 9, wherein the display order is changed to be before the display order of the partial areas connected to the end point of the flow line.     The second changing means changes the display order of the partial areas by moving the partial area displayed by the display control means to the position of the flow line, or is displayed by the display control means. It further comprises selection means for selecting whether to change the display order of the partial areas by changing the display order of the partial areas connected to the start point or the end point of the flow lines by the operation of moving the flow lines. The image display device according to claim 1, wherein the image display device is an image display device. A method for controlling an image display device that has storage means for storing partial area data of a plurality of partial areas included in an image and displays the image,
A presentation step of displaying each of the plurality of partial areas in a display order set by the partial area data in response to a user operation;
A display control step of displaying the image by superimposing an object corresponding to each display order of the plurality of partial areas on the plurality of partial areas;
A control method for an image display device, comprising:   The program for functioning a computer as each means of the image display apparatus of any one of Claims 1 thru | or 12.