JP5550247B2 - Image processing apparatus, control method and program thereof, and storage medium - Google Patents

Description

The present invention relates to an image processing apparatus for inserting or synthesizing display data such as a character string into a moving image, a control method and program thereof, and a storage medium .

JP 2006-108943 A JP 7-322140 A

  Conventionally, moving image editing application software installed in a personal computer inserts display data such as characters and icons that are effective for moving images, or gives a desired effect to display data. be able to. Thereby, the interestingness of a moving image can be improved further.

  Due to the widespread use of video cameras, general users are increasingly taking pictures of moving images, and a function for easily inserting display data into a moving image and a function for providing an effect are desired.

  An editing apparatus is known that can easily add an effect while confirming a moving image by converting a position specified by a user into a virtual coordinate of the moving image and storing the coordinate during reproduction of the moving image ( Patent Document 1). There is also known a technique that enables one of a plurality of effects to be selected for a character (Patent Document 2).

In the prior art, there is a problem that can not see the effect of string or grant the effect was inserted into the moving image in real time.

An object of the present invention is to present an image processing apparatus , a control method and program for solving such a problem, and a storage medium .

An image processing apparatus according to the present invention includes a touch detection unit capable of detecting touch operations at a plurality of points on a display unit, a display control unit that controls to reproduce a moving image and display the moving image on the display unit, and the display control In response to a touch operation at a plurality of points detected by the touch detection means when the moving image is reproduced and displayed by the means, the positional relationship between the input points of the touch operation at the plurality of points is set with respect to the video. And a control unit that controls the display data to be combined with the corresponding effect, and the control unit opens a distance between the two input points when a touch operation at two points is detected. The display data is added with the effect that the display data is opened in the direction in which the distance between the two input points is increased from the state in which the display data is not displayed. And the two Without even moving to open the distance between the points to scale the size of the elements included in the display data, and controls so as to synthesize the display data in the video.
An image processing apparatus according to the present invention includes a touch detection unit capable of detecting touch operations at a plurality of points on a display unit, a display control unit that controls to reproduce a moving image and display the moving image on the display unit, and the display control In response to a touch operation at a plurality of points detected by the touch detection means when the moving image is reproduced and displayed by the means, the positional relationship of each input point of the touch operation at the plurality of points is set with respect to the moving image. And a control means for controlling the display data to be synthesized with a corresponding effect, and the control means reduces the distance between the two input points when a touch operation at two points is detected. The display data is closed from the state where the display data is displayed without changing the size of the elements included in the display data. It synthesizes the display data Thereafter, and controlling so as to delete the display data.
The control method of the image processing apparatus according to the present invention includes a touch detection step capable of detecting a touch operation at a plurality of points on the display means, a display control step for controlling to reproduce and display a moving image on the display means, In response to the touch operation at a plurality of points detected in the touch detection step when the moving image is reproduced and displayed in the display control step, each input point of the touch operation at the plurality of points is applied to the video. A control step for controlling the display data to be combined with an effect corresponding to the positional relationship, and the control step is a distance between two input points when a touch operation at two points is detected. in response to moving to open, from said state where the display data is not displayed, the display denoted by the effects such as the display data in the direction in which distance is opened between the two input points is opened Day for As appears, and without even moving such that the distance between the two input points opens to scale the size of the elements included in the display data, synthesizes the display data on the moving It is characterized by controlling as follows.
The control method of the image processing apparatus according to the present invention includes a touch detection step capable of detecting a touch operation at a plurality of points on the display means, a display control step for controlling to reproduce and display a moving image on the display means, In response to the touch operation at a plurality of points detected in the touch detection step when the moving image is reproduced and displayed in the display control step, each input point of the touch operation at the plurality of points is applied to the video. A control step for controlling the display data to be combined with an effect corresponding to the positional relationship, and the control step is a distance between two input points when a touch operation at two points is detected. in response to the moves to be smaller, a state where the display data is displayed without scaling the size of the elements included in the display data, such as the display data is closed You are given the results by combining the data for the display, then and controls to turn off the display data.

According to the present invention, it is possible to insert display data with an effect corresponding to a touch operation performed on a touch panel by a user with respect to a moving image that is being reproduced, and synthesize it with the moving image.

1 shows a schematic block diagram of an embodiment of the present invention. The flowchart of the display data insertion process of a present Example is shown. An example of a list display screen of display data is shown. An example of a moving image playback screen is shown. An example of a character string insertion start screen is shown. Indicating the character string inserted exit screen example. The example of a locus | trajectory from an input start position to an input end position is shown. An example of the configuration and contents of management information is shown. It is an example of storage and management of character string information. The flowchart of the reproduction | regeneration processing after character string insertion operation is shown. The example of the reproduction | regeneration screen of the moving image which synthesize | combined the character string is shown. It is an example of a character string cancellation screen. The schematic block diagram of 2nd Example of this invention is shown. 14 is a flowchart of display data insertion processing of the embodiment shown in FIG. The example of a relationship between the output coordinate value of a coordinate detection part and the display / non-display of a character string is shown. An example of a screen in which a character string (“Arrived!”) Is synthesized and displayed on a moving image by inputting two points is shown. An example of the relationship between the input operation to the touch panel input unit and the effect is shown. The example of a change of the display of each effect effect is shown. An example of an input operation with two fingers is shown. The example of the character string insertion end screen after the input of the character string is shown. The change of the display content of the character string displayed along the locus | trajectory from an input start position to an input end position is shown. An example of a structure and content of the management information memorize | stored in an insertion information storage part is shown. An example of the structure and contents of an effect management table is shown. The flowchart of the reproduction | regeneration processing after a character string is inserted in the desired position by the user is shown. FIG. 3 shows an example of a synthesized moving image reproduction screen that reproduces a synthesized moving image in the flow shown in FIG. 2.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 shows a schematic block diagram of an image reproducing apparatus incorporating an embodiment of an image processing apparatus according to the present invention. The playback apparatus 10 includes a system control unit 12, a touch panel 14, a display data storage unit 16, an input unit 18, a moving image storage unit 20, and an insertion information storage unit 22. The system control unit 12 is composed of a microcomputer.

  The system control unit 12 includes a microcomputer having a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory). The CPU executes various kinds of control by executing a control program stored in the ROM using the RAM as a work area.

  The input unit 18 has operation means such as a jog dial and a dial switch. The user can input various instructions to the system control unit 12 via the input unit 18. The input control unit 26 transmits the input operation of the input unit 18 to an appropriate processing control unit. By sending a selection instruction from the input control unit 26 to the moving image control unit 28, a desired moving image can be selected from the moving image storage unit 20. Similarly, desired display data can be selected from the display data storage unit 16 by sending a selection instruction from the input control unit 26 to the display data reading unit 24.

  The display data reading unit 24 reads the display data specified by the input control unit 26 from the display data storage unit 16 and sends the read display data to the display data control unit 30. The display data storage unit 16 can store a plurality of display data. The display data generally consists of characters or character strings, icons, image data, or combinations thereof.

  The moving image storage unit 20 stores moving image data compressed based on a moving image compression format such as MPEG2 (Moving Picture Experts Group). The moving image control unit 28 reads the compressed moving image data instructed by the input control unit 26 from the moving image storage unit 20 and expands it, and supplies the moving image data restored by the expansion to the moving image combining unit 32. The moving image control unit 28 also compresses (encodes) the moving image data from the moving image combining unit 32 and writes the compressed moving image data into the moving image storage unit 20.

  The touch panel input unit 14 a of the touch panel 14 sends a signal corresponding to the input operation position to the coordinate detection unit 36. The coordinate detection unit 36 calculates the input position of the touch panel input unit 14 a as coordinates, and sends the calculated coordinates to the display data control unit 30. The moving image data transmitted from the display control unit 38 is displayed on the moving image display unit 14 b of the touch panel 14.

  The display data control unit 30 generates display image data from the display data read by the display data reading unit 24 and the input position coordinate information obtained by the coordinate detection unit 36, and the moving image combining unit 32. Send to. When generating the display image data, the arrangement of the display image data on the screen is adjusted as necessary. The moving image composition unit 32 selects one of the reproduced moving image data from the moving image control unit 28 and the display image data from the display data control unit 30 and sends it to the display control unit 38. It is also possible to send data to the display control unit 38. The moving image synthesizing unit 32 also receives the display data information and the input position coordinates used when generating the display data image from the display data control unit 30, and receives the moving image scene number and playback time (from the moving image control unit 28). Frame). The moving image composition unit 32 sends these pieces of information to the insertion information generation unit 34 and also sends them to the moving image control unit 28 as necessary.

  The insertion information generation unit 34 generates management information by associating information sent from the moving image synthesis unit 32. This management information includes at least a scene number of a moving image, a reproduction time (number of frames), display data, and input position coordinates, and is stored in the insertion information storage unit 22.

  The display control unit 38 converts the moving image data from the moving image combining unit 32 into a format that can be displayed on the moving image display unit 14b, and sends the converted data to the moving image display unit 14b.

  The insertion information analysis unit 39 analyzes the display data insertion information in the reproduction process after the character string insertion operation.

  The moving image storage unit 20, the display data storage unit 16, and the insertion information storage unit 22 are made of a random access storage medium such as an HDD (Hard Disk Drive) or a flash memory.

  FIG. 2 shows a flowchart of the display data insertion process of this embodiment. It is assumed that a desired moving image designated by the user using the jog dial or the like of the input unit 18 is displayed on the moving image display unit 14b. Here, it is assumed that the user desires to insert the character string “Arrived!”, Which is display data, at a desired position between 10 and 13 seconds of the first scene of the moving image.

  The flow shown in FIG. 2 is started by a predetermined operation of the user in a state where the moving image can be reproduced. In accordance with a user instruction, display data in the display data storage unit 16 is read and displayed on the moving image display unit 14b of the touch panel 14 (S1). The display data stored in the display data storage unit 16 can be rewritten and can be changed depending on the application. FIG. 3 shows an example of a list display screen 40 for display data displayed on the touch panel 14 in step S1. On the character string list display screen 40, a menu item display unit 42, a character string scroll unit 44, and a character string display field 46 are displayed. The user can select a character string to be inserted into the moving image from a character string list displayed in the character string display field 46. Further, the character string list display screen 40 can be displayed during playback of a moving image or before starting playback.

  Next, the user selects a character string to be inserted from the character strings displayed on the touch panel 14 using the input unit 18 (S2). Display data indicating the selected character string is temporarily stored in a RAM or the like.

  When the user selects the moving image into which the character string selected in step S2 is to be inserted using the input unit 18, the moving image is reproduced and displayed on the moving image display unit 14b of the touch panel 14 (S3). Further, when the character string to be inserted is determined after the moving image is reproduced, step S3 may be executed before step S1.

  FIG. 4 shows an example of a moving image playback screen 50 that plays back the moving image selected by the user in step S3. On the moving image reproduction screen 50, an operation state display icon 52, a reproduction time (frame) 54, a scene number 56, and a reproduction moving image 58 are displayed. The operation state display icon 52 may be one in which the icon display is changed according to the operation state such as reproduction, fast forward, or slow reproduction. In FIG. 4, the reproduction time 54 represents that the scene of scene number 1 that has passed 10 seconds is being reproduced.

  When a scene in which a character string is desired to be inserted in the moving image being reproduced, the user designates a position on the touch panel 14 where the character string is to be inserted (S4). When the moving image is paused, playback starts at the same time as position input on the touch panel 14 is started. In this example, it is assumed that the normal reproduction is performed, but slow reproduction or frame advance reproduction may be used.

  The touch panel input unit 14a of the touch panel 14 sends the input start position input to the coordinate detection unit 36, and the coordinate detection unit 36 converts the input position into coordinates (S5).

  At the same time that the user starts input on the touch panel 14, the character string is displayed on the moving image display unit 14b of the touch panel 14 so that the head of the character string is positioned at the input position (S6). The string is always displayed unless you stop typing. By doing so, the user can visually confirm the insertion effect of the character string in real time on the moving image being reproduced. The work of reproducing and confirming a moving image with a character string afterwards is unnecessary.

  FIG. 5 shows an example of the character string insertion start screen 60 when position input of the character string is started on the touch panel 14 in step S6. When position input is performed on the touch panel 14, the touch panel input unit 14a sends data indicating the input position to the coordinate detection unit 36, and the coordinate detection unit 36 converts the input position 64 into coordinates. Simultaneously with this conversion operation, the selected character string 62 is displayed at the input start position 64. Until the position input is completed, the character string 62 continues to be displayed as described above. The user can confirm the moving image in which the character string 62 is inserted on the spot.

  The presence / absence of user input to the touch panel input unit 14a is determined (S7). When a user input is detected (S7), the coordinate detection unit 36 converts the input position to the touch panel input unit 14a into coordinates (S8). The moving image combining unit 32 combines the previously selected display data with the moving image being reproduced at the position corresponding to the coordinates converted in step S8 (S9). An image obtained by the synthesis is displayed on the moving image display unit 14 b of the touch panel 14. The insertion information generation unit 34 temporarily associates the coordinates obtained in step S8, the scene number of the moving image, the reproduction time (frame), and display data indicating the selected character string with each other in the built-in RAM. (S10). It returns to step S7 and the presence or absence of the user input to the touch-panel input part 14a is determined.

  When the user input to the touch panel input unit 14a is not detected (S7), since the position input to the touch panel input unit 14a has been completed, the display of the character string on the touch panel 14 is terminated (S11). By doing in this way, the user can confirm the insertion effect of the character string with respect to the moving image being reproduced in real time. The touch panel input unit 14a sends the last position input by the user, that is, the input end position, to the coordinate detection unit 36, which converts the coordinate detection unit 36 into coordinates (S12). The insertion information generation unit 34 generates management information including the coordinates of the input end position, the scene number of the moving image, the reproduction time (frame), the selected character string, and the data temporarily stored in the RAM in step S10. (S13). The management information generated by the insertion information generation unit 34 is stored in the insertion information storage unit 22 (S14), and the display data insertion process ends.

Figure 6 shows an example of a string insertion exit screen 66 when the position input string is terminated at step S14. FIG. 6 shows that the input starts at 10 seconds of the first scene of the moving image (input start position 64) and ends at 13 seconds (input end position 70). A character string is displayed in real time along a locus 68 from the input start position to the input end position. While the position is input, the scene number, playback time (frame), and character string of the moving image are associated with each other and temporarily stored in a RAM or the like. When the position input is completed, it is stored in the insertion information storage unit 22 as management information. When the input of the character string is completed, the displayed character string disappears. Since the character string is inserted along the trajectory 68 from the input start position 64 to the input end position 70, the trajectory 74 from the input start position 76 to the input end position 78 is a curve as shown in FIG. But there is no problem. In that case, the display data control unit 30 adjusts the character string to a shape along the curve.

  FIG. 8 shows an example of the configuration and contents of management information stored in the insertion information storage unit 22. The management information includes a scene number, character string information, reproduction time information, and position information of a moving image. The display data storage unit 16 stores a character string management table associating character string numbers with character string data as shown in FIG. The character string information consists of the character string number of this character string management table. The playback time information stores the playback time of the moving image in units of hours, minutes and seconds or frames. The position information includes coordinates obtained by converting the position input by the user by the coordinate detection unit 36. The description of the management information is an example, and the selected display data, the input position information, and the moving image information at the time of reproduction may be stored in association with each other.

  Using this management information, it is possible to display a character string at a location designated by the user at the time of moving image reproduction. A character string can be synthesized at a location designated by the user for a moving image. In addition, by compressing (encoding) by the moving image control unit 28, a moving image in which character strings are combined can be created. Details will be described later.

  Next, processing for executing reproduction processing after a character string insertion operation by the user will be described. The insertion information analysis unit 39 reads display data insertion information from the insertion information storage unit 22 in accordance with an instruction from the input control unit 26. The insertion information analysis unit 39 further analyzes the read insertion information and sends the analysis result to the display data reading unit 24.

  When the moving image control unit 28 receives a moving image read request from the input control unit 26, the moving image control unit 28 instructs the insertion information analysis unit 39 to read display data from the insertion information storage unit 22.

  The display data reading unit 24 reads display data from the display data storage unit 16 based on the analysis result of the insertion information analysis unit 39, and sends the read display data to the display data control unit 30. The display data reading unit 24 also sends another information group included in the display data insertion information to the display data control unit 30, and the display data control unit 30 uses the information to create a display data image. To do.

  A reproduction process after a character string is inserted at a desired position by the user will be described with reference to a flowchart of FIG. Here, the display data is a character string, and the user inserts the character string “Arrive!” Between 10 seconds and 13 seconds in the first scene of the moving image at the position specified by the method described above. An example of this will be described.

  The process shown in FIG. 10 is started by a predetermined operation from the user in a state where a moving image can be reproduced. In response to an instruction input by the user, the input control unit 26 receives a moving image playback request (S21). In response to an instruction received by the input control unit 26, the insertion information analysis unit 39 reads display data insertion information (management information) from the insertion information storage unit 22 (S22).

  The insertion information analysis unit 39 analyzes the display data insertion information (management information) read from the insertion information storage unit 22 (S23). By analyzing the display data insertion information, the scene number, character string information, reproduction time information, and position information of the moving image can be acquired, and the acquired information is temporarily stored in the RAM. When the management information does not exist, the insertion information analysis unit 39 determines that there is no management information.

  It is determined whether the analysis result of the management information stored in the RAM matches the scene number and playback time of the moving image to be played back (S24). If they do not match (S24), the moving image read from the moving image storage unit 20 is reproduced and displayed (S25). At this time, since the management information does not match, the moving image combining unit 32 does not combine with the display image data.

  If they match (S24), the insertion information analysis unit 39 transmits the analysis result of the management information to the display data reading unit 24, the coordinate detection unit 36, and the moving image synthesis unit 32 (S26). Based on the analysis result of the management information by the insertion information analysis unit 39, the character string image data is generated by the display data reading unit 24 and the coordinate detection unit 36 (S27). Based on the character string image data generated in step S27 and the analysis result of the management information received by the moving image combining unit 32, the character string image data and the moving image are combined (S28). The moving image synthesized in step S28 is sent to the display control unit 38 (S29). The moving image sent to the display control unit 38 is displayed on the moving image display unit 14 b of the touch panel 14.

  FIG. 11 shows an example of a combined video playback screen that plays back the video combined in step S29. By analyzing the management information generated when the user performs an input operation on the touch panel input unit 14a on the touch panel 14, the character string selected by the user in a scene desired by the user can be synthesized and displayed with the moving image.

  After step S25 or S29, it is determined whether or not there has been a reproduction end request from the input unit 18 by the user, or whether or not the final position of the moving image has been reached (S30). If either condition is met (S30), the reproduction process after the display data is inserted is terminated. If neither condition is met (S30), the process returns to step S24.

  FIG. 12 shows an example of a character string cancellation screen for canceling the character string input by the touch panel input unit 14 a on the touch panel 14. In the example shown in FIG. 12, it is assumed that the character string “arrived!” Inserted between 10 seconds and 13 seconds in the first scene of the moving image is to be canceled.

  A screen confirmation button 80 is displayed on the character string cancellation screen shown in FIG. When the screen confirmation button 80 is selected by the input unit 18 after the character string is inserted by the touch panel 14, confirmation playback is performed retroactively to at least the scene where the character string is inserted. If an input from the touch panel input unit 14a is detected during the confirmation reproduction, the inserted character string is deleted. A corresponding portion of the management information stored in the insertion information storage unit 22 is reconstructed by the insertion information generation unit 34.

  FIG. 13 is a block diagram showing the schematic configuration of a playback apparatus incorporating a second embodiment of the image processing apparatus according to the present invention for applying effects to moving images. The playback apparatus 110 shown in FIG. 13 is mainly different from the playback apparatus 10 in that the system control unit 112 includes an effect determination unit 137. The functions of the means 112 to 136, 138, and 139 of the playback apparatus 110 are the same as those of the means 12, 18 to 36, 38, and 39 of the playback apparatus 10, respectively, except that the function related to the effect is added. . Below, the difference regarding an effect is mainly demonstrated.

  The display data storage unit 116 stores display data and effect data. The display data reading unit 124 can read a desired one from the display data and effect data stored in the display data storage unit 116.

  The coordinate detection unit 136 calculates coordinates indicating the input position by the touch panel input unit 114 a and sends the coordinates to the display data control unit 130 and the effect determination unit 137. The effect determination unit 137 determines an effect to be added to the display data from the coordinates sent from the coordinate detection unit 136 and its movement. The determined effect is notified to the display data control unit 130.

The display data controller 130, the display data / effect data read by the display data reading unit 1 24, the coordinate information from the coordinate detection unit 1 36, and in accordance with the effect information from the effect determination unit 137, a display Generate image data. The generated display image data is sent to the moving image composition unit 132. When generating the display image data, the arrangement of the display image data on the screen is adjusted as necessary.

The moving image combining unit 132, display data information used when generating the display data image from the display data controller 1 30 receives the input position coordinate and the effect information, scene number of the moving image from the moving image control unit 128 And receive playback time (frame). The moving image composition unit 132 sends these pieces of information to the insertion information generation unit 134 and also sends them to the moving image control unit 128 as necessary.

  The insertion information generation unit 134 generates management information in association with the information sent from the moving image synthesis unit 132. This management information includes at least a moving image scene number, a reproduction time (number of frames), display data, input position coordinates, and effect information, and is stored in the insertion information storage unit 122.

  FIG. 14 shows a flowchart of the display data insertion process of the embodiment shown in FIG. It is assumed that a desired moving image designated by the user using the jog dial or the like of the input unit 118 is displayed on the moving image display unit 114b. Here, the user inserts the character string “Arrived!”, Which is display data, at a desired position with a desired effect between 10 seconds and 13 seconds of the first scene of the moving image. Suppose you want.

  The flow shown in FIG. 14 is started by a predetermined operation of the user in a state where the moving image can be reproduced. In accordance with a user instruction, display data in the display data storage unit 116 is read and displayed on the moving image display unit 114b of the touch panel 114 (S41). The display data stored in the display data storage unit 116 is rewritable and can be changed depending on the application. The display data in the display data storage unit 116 is displayed in a list as shown in FIG. 3 for selection. This list of character strings can be displayed during playback of a moving image or before starting playback.

  The user uses the input unit 118 to select a character string to be inserted into the moving image from the character string list (S42). Display data indicating the selected character string is temporarily stored in a RAM or the like.

  When the user selects a moving image into which the character string selected in step S42 is to be inserted using the input unit 118, the moving image is reproduced and displayed on the moving image display unit 114b of the touch panel 114 as shown in FIG. 4 (S43). ). Further, when the character string to be inserted is determined after the moving image is reproduced, step S43 may be executed before step S41.

  When a scene in which a character string is desired to be inserted in the moving image being reproduced, the user designates a position where the character string is to be inserted on the touch panel 114 (S44). When the moving image is paused, playback starts at the same time as position input on the touch panel 114 is started. In this example, it is assumed that the normal reproduction is performed, but slow reproduction or frame advance reproduction may be used.

  The touch panel input unit 114a of the touch panel 114 sends the input start position input to the coordinate detection unit 136, and the coordinate detection unit 136 converts the input position into coordinates (S45).

  The display data control unit 130 determines the display position of the character string and the necessity of display from the position or number of coordinates sent from the coordinate detection unit 136 by the input operation to the touch panel 114 by the user (S46). FIG. 15 shows an example of the relationship between the output coordinate value of the coordinate detection unit 136 and the display / non-display of the character string. In the example shown in FIG. 15, display / non-display of a character string is determined by the number of input coordinates and the distance between two points in the case of two-point input. When the distance between the two input points is larger than the threshold, the center of the character string is displayed in the middle between the two points. When the distance between the two points is smaller than the threshold value, the character string is not displayed. The threshold value may be arbitrarily set by the user or may be a value determined internally. In the case of one-point input or three-point input or more, no character string is displayed.

  When displaying a character string (S47), the composition display of the selected character string is started with respect to the moving image being reproduced (S48). FIG. 16 shows an example of a screen in which a character string (“arrived!”) Is synthesized and displayed on a moving image by inputting two points. Two points are input with the thumb and forefinger, and the character string 160 is displayed so that the center is positioned between the two input points. Unless the character string is not displayed by the effect, the character string is always displayed unless the input is stopped. By doing in this way, the user can confirm the effect of the character string on the moving image being reproduced in real time in real time, and the operation of reproducing and confirming again becomes unnecessary.

  If the character string is not displayed (S47), step S48 is bypassed.

  The presence or absence of user input to the touch panel input unit 114a is determined (S49). When a user input is detected (S49), the coordinate detection unit 136 converts the input position to the touch panel input unit 114a into coordinates (S50). The effect determination unit 137 determines an effect based on the coordinates from the coordinate detection unit 136 (S51). The display data control unit 130 creates display image data to which the effect determined by the effect determination unit 137 is added to the selected character string, and supplies the display image data to the moving image composition unit 132. The moving image combining unit 132 combines the display image data with the moving image being reproduced (S52). The image obtained by the synthesis is displayed on the moving image display unit 114b of the touch panel 114. By doing so, the user can confirm the effect of the character string on the moving image being reproduced in real time, and the operation of reproducing and confirming again becomes unnecessary.

  The insertion information generation unit 134 correlates the coordinates obtained in step S50, the scene number of the moving image, the playback time (frame), the display data indicating the selected character string, and the display data indicating the effect. The data is temporarily stored in the built-in RAM (S53). It returns to step S49 and the presence or absence of the user input to the touch-panel input part 114a is determined.

  When the user input to the touch panel input unit 114a is not detected (S49), the position input to the touch panel input unit 114a is finished, and thus the display of the character string on the touch panel 114 is finished (S54). If the effect process is being executed, the character string display disappears after the effect process ends. That is, the character string does not disappear during the effect processing. By doing in this way, the user can confirm the insertion effect of the character string with respect to the moving image being reproduced in real time.

  The touch panel input unit 114a sends the last position input by the user, that is, the input end position, to the coordinate detection unit 136, which converts the coordinate detection unit 136 into coordinates (S55). The insertion information generation unit 134 stores management information including the coordinates of the input end position, the scene number of the moving image, the playback time (frame), the selected character string and effect, and the data temporarily stored in the RAM in step S53. Generate (S56). The management information generated by the insertion information generation unit 134 is stored in the insertion information storage unit 122 (S57), and the display data insertion process ends.

  FIG. 17 shows a relationship example between the input operation to the touch panel input unit 114a and the effect. Each effect is determined according to the movement of the finger after the input. A plurality of effects are not performed at the same time, and a closer effect process is selected depending on the movement of the finger. FIG. 18 shows a change example of the display of each effect in the example shown in FIG.

  In the case of one-point input, the character string is not displayed.

  If the distance between the two points is smaller than the set threshold with two points input, the character string is not displayed. If the finger moves in such a state, the character string is displayed in such a manner that the curtain opens from a state where nothing is displayed. In the example shown in FIG. 17, the finger moves left and right, but it may be up and down or diagonal. The display of the character string is controlled in the direction in which the finger moves. When the movement is such that the two points rotate, the character string is displayed in such a manner as to rotate from a state where nothing is displayed. The direction of rotation is determined by the direction in which the finger moves. In the example shown in FIG. 17, clockwise rotation is performed.

  If the distance between the two points is greater than the set threshold when the two points are input, the character string is displayed so that the center of the character string is located in the middle between the two points. When two points are moved in the same direction in this state, the displayed character string also moves in the direction in which the finger is moved. In the example shown in FIG. 17, it moves to the left, but it can move in any direction. In this state, if the finger is moved so as to reduce the distance between the two points, the character string disappears with a movement that closes the curtain. In the example shown in FIG. 17, it moves to the left and right, but it can move in any direction. The display of the character string is controlled in the direction in which the finger moves.

  In addition, when the movement is such that the two points rotate, the displayed character string rotates. The direction of rotation is specified by the direction in which the finger moves. In the example shown in FIG. 17, clockwise rotation is performed. When moving to increase the distance between two points, a character string is displayed. Controls the display of the character string so that the curtain closes. If this operation is performed when the character string has disappeared, the character string that has begun to disappear is displayed again so that the curtain opens. In the example shown in FIG. 17, it moves left and right, but it can move in any direction, and the display of the character string is controlled in the direction in which the finger moves.

  If there are three or more inputs, the character string is not displayed. Also, the character string is not displayed when the number of input points is 1 to 2 and the distance between the two points is small.

  When the number of input points changes from one point to two points and the distance between the two points is large, the character string moves in the direction from the point input earlier to the point input later. In the example shown in FIG. 17, it moves from right to left, but it can move in any direction, and the display of the character string is controlled in the direction between two input points that move back and forth in time.

  When the number of input points is changed from two to one and the distance between the two points before becoming one point is small, the character string is not displayed. When the number of input points changes from two to one and the distance between the two points before the point is large, the character string moves with respect to the input point. In the example shown in FIG. 17, the movement is from left to right, but the movement is possible in any direction, and the display of the character string is controlled with respect to the changing direction of the input point.

  FIG. 19 shows an example of an input operation with two fingers. The thumb and index finger are brought close to each other as shown by a locus 170 indicated by an arrow. As described above, in the case of such an operation, an effect has been selected in which the displayed character string disappears with a movement that closes the curtain.

  FIG. 20 shows an example of a character string insertion end screen after the input of the character string is completed. As shown in FIG. 19, the input is started at 10 seconds of the first scene of the moving image, and the input is completed at 13 seconds on the locus 170 indicated by the arrow, and the screen shown in FIG. 20 is obtained.

  FIG. 21 shows a change in the display content of the character string displayed along the trajectory 170 from the input start position to the input end position. As the distance between the two input points gradually decreases, an effect is added in which the character string disappears. While the position is input, the scene number of the moving image, the playback time (frame), the character string, and the effect are temporarily associated with each other and stored in the built-in RAM or the like. Is remembered.

  FIG. 22 shows an example of the configuration and contents of management information stored in the insertion information storage unit 122. The management information includes a moving image scene number, character string information, effect information, reproduction time information, and position information. The character string number used in the character string information is associated with character string data in a character string management table as shown in FIG. Similarly, the effect number used for the effect information is associated with the contents of the effect processing by the effect management table as shown in FIG. The display data storage unit 116 stores such a character string management table and effect management data.

  The reproduction time information stores the reproduction time of the moving image in units of hours, minutes, seconds and frames. The position information stores the position input by the user as coordinates converted by the coordinate detection unit 136. The above description regarding the management information is merely an example, and it is only necessary that the selected display data, input position information, moving image information at the time of reproduction, and effect information are associated with each other.

  Using this management information, when a moving image is played back, a character string is combined with the moving image with a specified effect and compressed (encoded) by the moving image control unit 128 to create a moving image with the character string added. . Details of this operation will be described below.

  The insertion information analysis unit 139 reads display data insertion information from the insertion information storage unit 122 in accordance with an instruction from the input control unit 126. The insertion information analysis unit 139 further analyzes the read insertion information and sends the analysis result to the display data reading unit 124.

  When the moving image control unit 128 receives a moving image read request from the input control unit 126, the moving image control unit 128 instructs the insertion information analysis unit 139 to read display data from the insertion information storage unit 122.

The display data reading unit 124 reads display data and effect information from the display data storage unit 116 based on the analysis result of the insertion information analysis unit 139, and sends the read display data and effect information to the display data control unit 130. . Display data reading unit 1 24 also sends other information group display data controller 1 30 included in the display data insertion information, the display data controller 1 30 uses this information, the display data Create an image.

  A reproduction process after a character string is inserted at a desired position by the user will be described with reference to a flowchart of FIG. Here, the display data is a character string, and the user specified the character string “Arrive!” At the position specified by the method described above between 10 seconds and 13 seconds in the first scene of the moving image. A case where an effect is added and inserted will be described as an example.

  The process shown in FIG. 24 is started by a predetermined operation from the user in a state where a moving image can be reproduced. In response to an instruction input by the user, the input control unit 126 receives a moving image playback request (S61). In response to the instruction received by the input control unit 126, the insertion information analysis unit 139 reads display data insertion information (management information) from the insertion information storage unit 122 (S62).

The insertion information analysis unit 139 analyzes the display data insertion information ( management information) read from the insertion information storage unit 122 (S63). By analyzing the display data insertion information, the scene number, character string information, reproduction time information, position information, and effect information of the moving image can be acquired, and the acquired information is temporarily stored in the RAM. When the management information does not exist, the insertion information analysis unit 139 determines that there is no management information.

  It is determined whether the analysis result of the management information stored in the RAM matches the scene number and playback time of the moving image to be played back (S64). If they do not match (S64), the moving image read from the moving image storage unit 120 is reproduced and displayed (S65). At this time, since the management information does not match, the moving image combining unit 132 does not combine with the display image data.

  When they match (S64), the insertion information analysis unit 139 transmits the analysis result of the management information to the display data reading unit 124, the coordinate detection unit 136, and the moving image synthesis unit 132 (S66). Based on the analysis result of the management information by the insertion information analysis unit 139, the display data reading unit 124, the coordinate detection unit 136, and the effect determination unit 137 generate character string image data to which the effect is added (S67). Based on the character string image data generated in step S67 and the analysis result of the management information received by the moving image combining unit 132, the character string image data and the moving image are combined (S68). The moving image synthesized in step S68 is sent to the display control unit 138 (S69). The moving image sent to the display control unit 138 is displayed on the moving image display unit 114b of the touch panel 114.

  FIG. 25 shows an example of a combined video playback screen that plays back the video combined in step S69. By analyzing the management information generated when the user performs an input operation on the touch panel input unit 114a on the touch panel 114, the character string selected by the user in a scene desired by the user is converted into a moving image with a specified effect. Composite display is possible.

  After step S65 or S69, it is determined whether or not there is a reproduction end request from the input unit 118 by the user, or whether or not the final position of the moving image has been reached (S70). If either condition is met (S70), the reproduction process after the display data is inserted is terminated. If neither condition is met (S70), the process returns to step S64.

  As described with reference to FIG. 12, the character string inserted by the above-described method can be deleted or canceled by using the character string cancel screen having the screen confirmation button 80. When the inserted character string is canceled, the related management information stored in the insertion information storage unit 122 is also updated by the insertion information generation unit 134 so as to match the content after cancellation.

  The embodiment in which the designated display data is synthesized with the moving images stored in the moving image storage units 20 and 120 has been described. However, it is obvious that the present invention can be applied to a case where designated display data is synthesized in real time with a moving image being shot by the imaging unit, that is, being recorded, in the camera.

  The above description is merely an example, and the present invention is not limited to the above-described embodiment.

DESCRIPTION OF SYMBOLS 10 ... Playback apparatus 12 ... System control part 14a ... Touch-panel input part 14b ... Moving image display part 16 ... Display data storage part 20 ... Moving image storage part 22 ... Insertion information storage part 24 ... Display Data reading section 30 ... Display data control section 32 ... Moving image composition section 34 ... Insertion information generation section 36 ... Coordinate detection section 39 ... Insertion information analysis section

Claims (12)

Touch detection means capable of detecting touch operations at a plurality of points on the display means;
Display control means for controlling to reproduce and display a moving image on the display means;
In response to a touch operation at a plurality of points detected by the touch detection unit while the moving image is reproduced and displayed by the display control unit, each input point of the touch operation at the plurality of points is applied to the video. Control means for controlling to synthesize display data with an effect according to the positional relationship,
When the control means detects a touch operation at two points, the control means moves the two input points from a state in which the display data is not displayed in response to movement so that the distance between the two input points is increased. Even if the display data appears so that the display data appears in the direction in which the distance between the points opens , and the distance between the two input points opens, the display An image processing apparatus that controls to combine the display data with the moving image without scaling the size of the elements included in the data . Touch detection means capable of detecting touch operations at a plurality of points on the display means;
Display control means for controlling to reproduce and display a moving image on the display means;
In response to a touch operation at a plurality of points detected by the touch detection unit while the moving image is reproduced and displayed by the display control unit, each input point of the touch operation at the plurality of points is applied to the video. Control means for controlling to synthesize display data with an effect according to the positional relationship,
When the control means detects a touch operation at two points, the control means moves the display data from a state in which the display data is displayed in response to movement so that the distance between the two input points becomes small. The display data is synthesized with the effect of closing the display data without changing the size of the elements included in the display, and then the display data is controlled to be erased. An image processing apparatus. It said control means further image according to any one of claims 1 or 2, characterized in that controls to synthesize the display data subjected to effect corresponding to the input points of the touch operation Processing equipment. The control means has an effect that when the touch operation at two points is detected, the display data moves in the moving direction of the two input points in response to the two input points moving in the same direction. the image processing apparatus according to any one of claims 1 to 3, wherein the controller controls so as to synthesize the data for the display are denoted by. Wherein, among the video, any one of claims 1 to 4, wherein the controller controls so as to synthesize the display data for the duration corresponding to from the start to the end of the touch operation 1 The image processing apparatus according to item. Wherein, among the video, any one of claims 1 to 5, wherein the controller controls so as to synthesize the display data with respect to the position corresponding to the touch position at the touch operation An image processing apparatus according to 1. The image processing apparatus according to any one of claims 1 to 6, wherein the display data is a character string. The image processing apparatus an image processing apparatus according to any one of claims 1 to 7, characterized in that an imaging apparatus having an imaging unit. A touch detection step capable of detecting touch operations at a plurality of points on the display means;
A display control step for controlling to reproduce and display a moving image on the display means;
In response to the touch operation at a plurality of points detected in the touch detection step when the moving image is reproduced and displayed in the display control step, each input point of the touch operation at the plurality of points is applied to the video. A control step for controlling to synthesize display data with an effect according to the positional relationship,
In the control step, when the touch operation at two points is detected, the two input points are displayed from the state where the display data is not displayed in response to the movement so that the distance between the two input points is increased. Even if the display data appears so that the display data appears in the direction in which the distance between the points opens , and the distance between the two input points opens, the display A control method for an image processing apparatus, wherein control is performed so as to synthesize the display data with the moving image without scaling the size of elements included in the data . A touch detection step capable of detecting touch operations at a plurality of points on the display means;
A display control step for controlling to reproduce and display a moving image on the display means;
In response to the touch operation at a plurality of points detected in the touch detection step when the moving image is reproduced and displayed in the display control step, each input point of the touch operation at the plurality of points is applied to the video. A control step for controlling to synthesize display data with an effect according to the positional relationship,
In the control step, when the touch operation at two points is detected, the display data is displayed from the state where the display data is displayed in response to the movement so that the distance between the two input points is reduced. The display data is synthesized with the effect of closing the display data without changing the size of the elements included in the display, and then the display data is controlled to be erased. For controlling an image processing apparatus. A program that causes a computer to function as each unit of the image processing apparatus according to any one of claims 1 to 8 .   A computer-readable storage medium storing a program that causes a computer to function as each unit of the image processing apparatus according to any one of claims 1 to 8.

Images