JP2002366272A - Portable terminal device and method for scrolling display image - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a portable terminal device with which a display image of received information can be scrolled without having to adding new hardware to the conventional portable terminal device and without performing key depression and jog operation by a user. SOLUTION: When the received information is displayed on a displaying part 9, a photographing part 3 inputs a first image and the image is stored in an image storing part 4. Next, a partial image extracting part 5 designates a partial image in the first image on condition of being optionally set. When a time measured after the input of the first image by a time measuring part 10 reaches an optional set time, the photographing part 3 inputs a second image, a movement information calculating part 6 calculates movement information of the partial image between the two images, and a scroll information deciding part 7 decides scroll information for scrolling the display image. An image display controlling part 8 scrolls the display picture on the basis of the decided information.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to a portable terminal device having a function of scrolling a screen for displaying received information.

[0002]

2. Description of the Related Art In recent years, the use of portable terminal devices for information communication, such as portable telephones and portable information terminals, has been increasing, and communication of character / image information is frequently performed. When the amount of text / image information or normal display data downloaded to the portable terminal device is large, the current screen of the portable terminal device is generally small, so that such information cannot be displayed on the screen at one time. Therefore, in order for the user of the conventional portable terminal device to view all the received information, the user has to perform the scroll operation of the display screen by repeatedly performing the key pressing operation and the jog operation.

[0003]

However, it is very troublesome and inconvenient for the user to frequently perform a key press operation or a jog operation. Further, portable terminal devices having a relatively large display screen have been developed, but the amount of information handled by the portable terminal devices has also been increasing. Therefore, the function of scrolling the screen for displaying the character / image information and displaying the screen is indispensable to the portable terminal device.

In order to solve such a problem, Japanese Patent Application Laid-Open Nos. 11-305918 and 7-64754 propose a portable terminal device equipped with a tilt sensor. In these methods, when the user tilts the mobile terminal device in a direction in which the screen is to be scrolled, the screen can be scrolled by detecting the degree of tilt by the tilt sensor. However, in this method, new hardware such as a tilt sensor needs to be newly added inside the small portable terminal device, and it is necessary to secure a space inside the device, which causes a new problem such as an increase in cost. Disadvantageously occurs.

The present invention has been made in view of such circumstances, and does not require the addition of new hardware to a conventional portable terminal device, and allows a user to press a key or perform a jog operation. It is an object of the present invention to provide a mobile terminal device capable of scrolling a display screen of received information without any need.

[0006]

According to a first aspect of the present invention, there is provided a portable terminal device having a display screen for displaying received information and an optical photographing means. Storage means for storing two images photographed at predetermined time intervals by the means;
Extracting means for extracting a common partial image in the images,
Calculating means for calculating movement information including a moving direction when the partial image extracted in one image moves to the position of the partial image extracted in the other image in the one image; Scroll control means for controlling scrolling of the display screen based on movement information.

According to the present invention, when the user carrying the portable terminal device displays the received information on the display screen, an image is taken by the operation of the optical photographing means. For example, this image is an image including the face of the user who looks at the display screen. The photographed image is stored in the storage unit.
Then, when an arbitrarily set time elapses after the image is taken, the next image is taken by the operation of the photographing means. The next image taken is also stored in the storage means in the same manner. For example, the next image is an image when the user's own face or the like in the first image moves in the horizontal direction or the like. Further, by the operation of the extracting means, a partial image common to the first image and the next image is extracted. As an example of the partial image, a skin color region common to both images, a contour of a face, and the like are given. Then, by the operation of the calculation means, movement information including a moving direction when the partial image extracted in the first image moves to the position of the partial image extracted in the next image in the first image is calculated. Is done. Further, by the operation of the scroll control means, the scroll of the display screen is controlled based on the calculated movement information, and the screen displaying the received information is scrolled. Therefore, it is not necessary to add new hardware to the conventional portable terminal device, and the display screen of the reception information can be easily scrolled without the user pressing the keys or performing the jog operation.

According to a second aspect of the present invention, in the first aspect, the movement information further includes a movement distance.
The scroll control means controls the scroll of the display screen based on a scroll amount according to the movement information including the calculated moving distance and the moving direction and scroll information including the scroll direction.

According to the present invention, when the user carrying the portable terminal device displays the received information on the display screen, an image is taken by the operation of the optical photographing means. For example, this image is an image including the face of the user who looks at the display screen. The photographed image is stored in the storage unit.
Then, when an arbitrarily set time elapses after the image is taken, the next image is taken by the operation of the photographing means. The next image taken is also stored in the storage means in the same manner. For example, the next image is an image when the user's own face or the like in the first image moves in the horizontal direction or the like. Further, by the operation of the extracting means, a partial image common to the first image and the next image is extracted. As an example of the partial image, a skin color region common to both images, a contour of a face, and the like are given. By the operation of the calculating means, the movement including the movement distance and movement direction when the partial image extracted in the first image moves to the position of the partial image extracted in the next image in the first image Information is calculated. Further, by the operation of the scroll control means, the scroll of the display screen is controlled by the scroll amount and the scroll direction according to the movement information including the calculated movement distance and movement direction, and the screen for displaying the reception information is scrolled. . Therefore, it is possible to easily scroll the display screen of the reception information according to the movement of the mobile terminal device.

According to a third aspect of the present invention, there is provided a method of scrolling a display screen in a portable terminal device having a display screen for displaying received information and an optical photographing means, wherein the first photographed by the photographing means is provided. A first step of storing an image, a second step of extracting a predetermined partial image in the first image, and a step of taking a picture by the photographing means after a lapse of a predetermined time after the taking of the first image. A third step of storing the second image obtained, a fourth step of extracting a partial image common to the partial image in the second image, and a partial image extracted in the first image. Is the second
A fifth step of calculating movement information including a movement distance and a movement direction when moving in the first image to the position of the partial image extracted in the image, and based on the calculated movement information, Sixth controlling scrolling of display screen
And a step of:

According to the present invention, when the user carrying the portable terminal device displays the received information on the display screen, the first image is photographed by the operation of the optical photographing means. For example, the first image is an image including the user's own face looking at the display screen. The photographed first image is stored in a first step, and an arbitrarily designated partial image in the first image is extracted in a second step. For example, a skin color region of the face, a contour of the face, and the like in the first image are extracted as the partial images. Then, when an arbitrarily set time elapses after the first image is photographed, the second image is photographed by the operation of the photographing means. Further, the captured second image is stored in the third process. For example, the second image is an image in which the user's own face or the like has moved in the horizontal direction or the like as compared to the first image. Further, in the fourth step, a partial image common to the partial image extracted in the first image is extracted in the second image. Then, by the fifth process, the moving distance and moving direction when the partial image extracted in the first image moves to the position of the partial image extracted in the second image in the first image Is calculated. Further, according to the sixth step, scrolling of the display screen is controlled based on the calculated movement information,
The screen displaying the received information is scrolled. Therefore, the same effect as the first aspect can be obtained.

[0012]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram of a portable terminal device for explaining an embodiment of the present invention. In FIG. 1, a receiving unit 1 is a device including an antenna for receiving character / image information such as e-mail and web information via a base station, and is connected to a received information storage unit 2. The reception information storage unit 2 is a device for storing the character / image information received by the reception unit 1, and is connected to the screen display control unit 8. Further, a time measuring unit 10 is connected to the photographing unit 3, and the elapsed time is measured every time an image is photographed.

The photographing unit 3 is an optical image pickup device such as a CCD camera capable of photographing an image around the portable terminal device, and is connected to the image storage unit 4. In addition,
The captured image can be transmitted from a portable terminal device to a third party as an attached file or the like. The image storage unit 4 is a device for storing an image photographed from the photographing unit 3, and is connected to the partial image extracting unit 5.

The partial image extracting unit 5 is a device for extracting a characteristic region in a stored image from an image input from the photographing unit 3 according to preset conditions. The partial image extraction unit 5 extracts, for example, a skin color region often used as a human face region, a region where edges of a face contour are continuous, and the like as a partial image. This partial image extraction unit 5 is connected to the movement information calculation unit 6.

The movement information calculation unit 6 extracts the first image by the operation of the partial image extraction unit 5 when the next image is photographed after a certain period of time has elapsed since the photographing unit 3 photographed the first image. The same partial image as the extracted partial image is extracted in the next image. Then, the movement distance and the movement direction assuming that the position of the partial image has moved on the first image up to the position at the time of extraction in the next image are calculated as movement information. The movement information calculator 6 is connected to the scroll information determiner 7.

The scroll information determining unit 7 determines a scroll direction when the display screen of the portable terminal device is actually scrolled based on the movement information including the movement distance and the movement direction calculated by the movement information calculation unit 6. This is an apparatus for determining scroll information including a scroll amount.
The scroll information determination unit 7 is connected to the screen display control unit 8. At this time, the scroll amount of the display screen can be set so as to be changed by the operation of the movement information calculation unit 6 according to the magnitude of the calculated movement distance.

The screen display control unit 8 is connected to a display unit 9 such as a liquid crystal screen, and displays characters and characters displayed on the display unit 9 based on the scroll information determined by the scroll information determination unit 7.
This is a device for performing scroll control on received information such as image information. In the present embodiment, scrollable directions are eight directions of left, right, upper, lower, upper left, lower left, upper right, and lower right.

Next, an example of the operation of the above-described embodiment will be described in detail with reference to FIG. FIG. 2 is a flowchart for explaining an operation procedure of the mobile terminal device according to the embodiment. First, when the user of the portable terminal device obtains information such as characters and images such as e-mail and web information or normal display data by downloading the information via the base station, the information is obtained from the portable terminal device. The data is received by the receiving unit 1 and stored in the received information storage unit 2. Hereinafter, information such as characters and images such as e-mails and web information received by the portable terminal device or normal display data is referred to as reception information. Then, the user starts an application for displaying the received information on the display unit 9 such as a liquid crystal screen (step S1). The display on the display unit 9 at this time is, for example, displayed such that the first character is the upper left of the display screen in the case of character information, and the upper left of the image is the upper left of the display screen in the case of image information. Is displayed as follows. Thus, the reception information stored in the reception information storage unit 2 is displayed on the display unit 9 (Step S2).

It is assumed that the photographing section 3 of the portable terminal device is set in the direction of photographing the face of the user looking at the display section 9. Therefore, when reception information such as e-mail and web information is displayed on the display unit 9 in step S2, an image including the user's face area is captured from the imaging unit 3, and the image including the user's face is displayed as an image. It is stored in the storage unit 4 (step S3). The image input first after the activation of this application is called a first image.

FIG. 3 shows an example of a screen display of the received information displayed on the display unit 9 and an example of an input image input from the image input unit 3 at that time. For example, as shown in FIG. 3, the display screen 9 of the portion including the upper left portion of the received image is displayed on the display unit 9 of the portable terminal device as the first image, and this screen is displayed in step S2. Screen. In this case, the directions in which the display screen a displayed on the display unit 9 can be scrolled are three directions: right, down, and lower right. In step S3, as described above, when the display screen a is displayed on the display unit 9, the image capturing unit 3 captures a first image including the user's face, but the first image in the present embodiment is , Is displayed as the input image d in FIG.

After the first image is photographed and stored in the image storage unit 4, a partial image is extracted under a preset condition by the operation of the partial image extracting unit 5. Here, the preset conditions include, for example, performing an edge extraction process on the input image d, forming a large area in the image among the extracted edges and a shape close to the contour of a human face. There is a method in which the edge is regarded as a partial image when it is recognized that the image is present. Further, the color information of the input image d may be examined to extract the skin color region, and the largest skin color region in the input image d may be used as the partial image. Alternatively, a region located at the center of the input image d and having a certain size or more may be used as the partial image. Furthermore, a partial image may be formed by using information on both the extracted edge and the skin color region.

When the first image is input from the photographing unit 3, the elapsed time after that is measured by the time measuring unit 10. Then, the second and third images are input from the imaging unit 3 at time intervals arbitrarily set by the user or the like thereafter. Here, after the first image is captured from the imaging unit 3 and stored in the image storage unit 4, if the user terminates the application before the elapse of a preset time (step S4: YES), the display unit The screen display of the received information displayed in 9 ends. On the other hand, if the application has not been terminated (step S4: NO), the process proceeds to step S5.

In step S5, the time measuring unit 10
The elapsed time measured at the first time by the operation of the photographing unit 3
When a set time elapses after the image is shot, a new image is input from the shooting unit 3 and the image storage unit 4
Is stored. An image input after the set time has elapsed is defined as a second image. The second image is displayed while the user is viewing the reception information displayed on the display unit 9.
Since the relative position between the user and the portable terminal device changes, for example, only the left half of the user is photographed as shown in the input image e in FIG. The input second image is the first image
The image is stored in the image storage unit 4 in the same manner as in the case of the image.

Further, in the same manner as in the first image, the operation of the partial image extracting unit 6 for the second image is the same as the partial image such as the flesh-color area and the edge extracted from the first image. A partial image is extracted from the second image. And
By the operation of the movement information calculation unit 6, a movement distance and a movement direction at the time when the partial image in the first image is moved to the position extracted in the second image on the first image and the home is calculated, It is used as movement information (step S
6). For example, in the input image d and the input image e in FIG. 3, the moving distance of the half face and the vector moved in the horizontal left direction correspond to the movement information.

Then, by the operation of the scroll information determining section 7, it is determined whether or not there is a sufficient moving component for scrolling the screen based on the calculated moving information (step S7). That is, it is determined that the display screen is scrolled when there is a certain moving component.
Thus, if the determination result is YES, that is, if there is a sufficient moving component, the process proceeds to step S8 to calculate scroll information. If the determination result is NO,
It returns to step S4.

Further, by the operation of the scroll information determining section 7, scroll information including a scroll amount and a scroll direction for scrolling the display screen of the portable terminal device is determined based on the calculated movement information of the partial image. (Step S8). That is, in the case of FIG. 3, the scroll direction of the display screen is set to the right direction, and the scroll amount is the amount calculated based on the size of the received information and the moving distance, etc., by the operation of the scroll information determination unit 7. The operation of the screen display control unit 8 causes the display unit 9 to operate with the determined scroll amount in the determined scroll direction.
Is scrolled (step S)
9). For example, as shown in FIG. 3, when the captured image changes from the input image d to the input image d, the display unit 9 is scrolled from the display screen a to the display screen b. After the screen display is scrolled as described above, the process proceeds to step S4.
Return to

Here, the time measuring unit 10 newly measures the elapsed time after the second image is photographed from the photographing unit 3 and stored in the image storage unit 4. Then, it is assumed that when the elapsed time reaches a preset time, the image captured by the image capturing unit 3 is the input image f in FIG.
That is, the user pays attention to the lower right direction of the display screen displayed on the display unit 9 and determines that an image such as the input image f has been captured from the capturing unit 3 because the relative position between the mobile terminal device and the user has changed. I do. At this time, the first display screen a is displayed by the operation of the movement information calculation unit 6 and the scroll information determination unit 7.
Scroll information for scrolling in the lower right direction is obtained. Thus, the display screen 9 is displayed on the display unit 9 by scrolling the display screen a in FIG.

After the input image e, which is the second image, is input and the display screen b is displayed on the display unit 9, the second image is displayed.
Is used as the first image, the same partial image in this image is extracted as a partial image by the operation of the partial image extracting unit 5, and when the input image f is newly input as the second image, as described above, Instead of using the input image d as a reference, the scroll information may be determined by obtaining movement information from the input image e that is a new first image.

A program for realizing the functions of the partial image extraction unit 5, the movement information calculation unit 6, and the scroll information determination unit 7 in FIG. 1 is recorded on a computer-readable recording medium, and is recorded on this recording medium. The construction management may be performed by causing the computer system to read the executed program and executing the program. Here, the “computer system” includes an OS and hardware such as peripheral devices. Also, if the “computer system” uses a WWW system,
It also includes the homepage providing environment (or display environment). The “computer-readable recording medium” is a flexible disk, a magneto-optical disk,
A portable medium such as an OM or a CD-ROM, or a storage device such as a hard disk built in a computer system. Further, the “computer-readable recording medium” refers to a volatile memory (RAM) inside a computer system that becomes a server or a client when a program is transmitted through a network such as the Internet or a communication line such as a telephone line. And those holding programs for a certain period of time.

The above program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the "transmission medium" for transmitting a program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. Further, the program may be for realizing a part of the functions described above. further,
What can realize the above-mentioned function in combination with a program already recorded in the computer system, that is, a so-called difference file (difference program) may be used.

[0031]

As described above, according to the first aspect of the present invention, there is provided a portable terminal device including a display screen for displaying received information and an optical photographing means, wherein the photographing means performs a predetermined time. Storage means for storing two images taken at intervals, extraction means for extracting a common partial image in the two stored images, and partial image extracted in one of the images Calculating means for calculating moving information including a moving direction when moving in one image to the position of the partial image extracted in the image, and scrolling for controlling scrolling of the display screen based on the calculated moving information With the control means, there is no need to add new hardware to the conventional portable terminal device, and the display screen of the received information can be easily screened without the user pressing a key or performing a jog operation. Effect that can be Lumpur.

According to the second aspect of the present invention, the movement information further includes a movement distance, and the scroll control means determines a scroll amount and a scroll amount according to the movement information including the calculated movement distance and movement direction. Since the scroll of the display screen is controlled based on the scroll information including the scroll direction, an effect is obtained that the display screen of the received information can be easily scrolled according to the movement of the mobile terminal device.

[Brief description of the drawings]

FIG. 1 is a block diagram of a portable terminal device for explaining an embodiment of the present invention.

FIG. 2 is a flowchart illustrating an operation procedure of the mobile terminal device according to the embodiment.

FIG. 3 shows a screen display example of received information displayed on a display unit 9 and an input image example input from the image input unit 3 at that time.

[Explanation of symbols]

 Reference Signs List 3 shooting unit 4 image storage unit 5 partial image extraction unit 6 movement information calculation unit 7 scroll information determination unit 8 screen display control unit 9 display unit 10 time measurement unit

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) H04M 1/247 H04M 1/247 5K067 H04Q 7/38 H04B 7/26 109T F-term (Reference) 5B087 AA09 AB14 BC11 BC26 DE06 5C082 AA27 BA20 BA27 CA54 CA72 CB01 DA87 MM09 5E501 AA02 BA05 CA02 FA14 FB03 FB32 FB46 5K023 AA07 BB11 MM00 5K027 AA11 BB02 FF00 FF22 HH29 5K067 AA34 BB04 EE02 FF02 FF02 FF02 FF02

Claims (3)

[Claims] 1. A portable terminal device comprising a display screen for displaying received information and an optical photographing means, wherein the storage means stores two images photographed at a predetermined time interval by the photographing means. Means for extracting a common partial image in the two stored images; and extracting the partial image extracted in one image to the position of the partial image extracted in the other image. A mobile terminal device comprising: calculation means for calculating movement information including a movement direction when moving within an image; and scroll control means for controlling scrolling of the display screen based on the calculated movement information. . 2. The moving information further includes a moving distance, and the scroll control unit determines a scroll amount according to the moving information including the calculated moving distance and the moving direction and a scroll information including a scroll direction. The mobile terminal device according to claim 1, wherein scrolling of the display screen is controlled. 3. A method for scrolling a display screen in a portable terminal device having a display screen for displaying received information and an optical photographing means, wherein a first image stored by the photographing means is stored. And a second step of extracting a predetermined partial image in the first image
And storing a second image photographed by the photographing means after a lapse of a predetermined time after the photographing of the first image.
And a fourth step of extracting a partial image common to the partial image in the second image, and a partial image extracted in the first image is extracted in the second image. A fifth step of calculating movement information including a moving distance and a moving direction when the image is moved to the position of the partial image in the first image; and controlling scrolling of the display screen based on the calculated movement information. A method of scrolling the display screen of the mobile terminal device, comprising: