JP4591501B2 - In-vehicle operation device - Google Patents

Description

  The present invention provides a direction indicating operation means for a user to indicate a direction, a force generating means capable of generating a force such as a reaction force or a thrust against the operation of the direction indicating operation means, and a display screen as a display means. The present invention relates to an in-vehicle operation device including a control means for displaying and an operation support method for the in-vehicle operation device.

  2. Description of the Related Art Conventionally, there is an in-vehicle operation device that allows a user to select a button on a display screen of an in-vehicle display by operating a joystick or a trackball in a vehicle interior. According to this, for example, if a selectable button on the display screen is arranged above the cursor, the joystick can be tilted forward, the trackball can be rotated forward, By moving the cursor over the desired button, you can select the desired button, such as adjusting the temperature of the air conditioner, adjusting the volume of the audio equipment, adjusting the angle of the electric mirror, etc. A desired operation can be performed.

By the way, in this kind of in-vehicle operation device, in order to improve the operability for the user, a force such as a reaction force or a thrust force is generated for the operation of the joystick or the trackball, and the joystick or the trackball is guided in an appropriate operation direction. It is the composition to do. That is, for example, when a selectable button on the display screen is arranged above the cursor, the tilt direction of the joystick is guided forward, the rotation direction of the trackball is guided forward, etc. As described above, a force such as a reaction force or a thrust is generated to give the user a feeling of operation (operation feeling) (see, for example, Patent Document 1).
JP 2002-189556 A

  By the way, the configuration that generates a reaction force or a thrust force in response to an operation of a joystick, a trackball, or the like in this way is the screen configuration data indicating the screen configuration of the display screen, that is, the position of the button on the display screen. Since the data shown is stored in advance in the non-volatile memory of the in-vehicle operation device, it is possible to read out the screen configuration data stored in advance in the non-volatile memory of the in-vehicle operation device.

  On the other hand, in recent years, even in the passenger compartment, by launching a browser, a web page can be downloaded from the server via the Internet and displayed on the in-vehicle display, or various application software can be activated and displayed on the in-vehicle display. It is possible to do.

  There are cases where URLs and buttons are described on the display screen of a Web page or application software. For these reasons, when a URL or button is described on the display screen of a web page or application software, the user operates the joystick or the trackball to describe the URL or button. It is conceivable that a desired URL or a desired button is selected and connected (linked).

  However, since the screen configuration data indicating the screen configuration of the display screen of the Web page or application software is not stored in advance in the nonvolatile memory of the in-vehicle operation device, the URL or the URL is displayed on the display screen of the Web page or application software. It is difficult to identify the location where the button is described. Therefore, when a desired URL or a desired button described on the display screen of a web page or application software is selected by operating a joystick, a trackball, or the like, reaction force against the operation of the joystick, the trackball, etc. It is difficult to generate a force such as a thrust, and as a result, there is a problem that the operability of the user is lowered.

  The present invention has been made in view of the above circumstances, and its purpose is to select a desired area described on a display page of a Web page or application software by operating a direction instruction operation means. An in-vehicle operation device and an operation support method that can give the user a feeling of operation by generating a force such as a reaction force or a thrust with respect to the operation of the direction instruction operation means, and can improve the user's operability. Is to provide.

  According to the in-vehicle operation device of the first aspect, the image analysis unit analyzes the image on the display screen, and the control unit sets the display area of the display unit to a plurality of grids based on the image analysis result by the image analysis unit. The grid cursor is displayed, and the grid cursor is displayed on any grid among the divided grids. When the direction indicating operation unit is operated, the control unit generates a force such as a reaction force or a thrust against the operation of the direction indicating operation unit by the force generating unit, and the grit cursor of the display unit is moved to the direction indicating operation unit. Move it on any other grit in response to the operation.

  As a result, when the user operates a direction indicating operation means such as a joystick or a trackball to select a desired URL or a desired button described on the display screen of the Web page or application software, the grid of the display means is displayed. Since the cursor is moved on any other grid according to the operation of the direction instruction operation means, the grid cursor is moved on the grid where a desired URL or a desired button is described. A desired URL and a desired button can be selected.

  At this time, when the user operates the direction indicating operation means, a force such as a reaction force or a thrust is generated with respect to the operation of the direction indicating operation means. The user operability can be improved. Further, by giving the user a feeling of operation, the user can be notified of the movement amount and movement locus of the grid cursor, and the user can operate the direction indicating operation means with a blind touch feeling. Furthermore, since the display area is configured to be divided into a plurality of grids based on the image analysis result of the display screen, the display area can be divided into a plurality of grids in an optimal form according to the URL and button arrangement. This can also improve user operability.

  According to the in-vehicle operation device described in claim 2, the image analysis means performs character recognition on the image on the display screen, and the control means determines the display area of the display means based on the character recognition result by the image analysis means. Divide into multiple grit. As a result, when characters and sentences are described on the display screen, by recognizing the characters and sentences, the display area can be divided into a plurality of grids in an optimal form for each character or sentence. Can reliably select a desired character or a desired sentence.

  According to the in-vehicle operation device of the third aspect, the image analysis means performs figure recognition on the image on the display screen, and the control means determines the display area of the display means based on the figure recognition result by the image analysis means. Divide into multiple grit. As a result, when a drawing is described on the display screen, by recognizing the figure, the display area can be divided into a plurality of grids in an optimal form by using the figure as a unit, and the user can reliably obtain the desired figure. Can be selected.

  An embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a functional block diagram showing the configuration of the in-vehicle operation device. The in-vehicle operation device 1 includes a CPU 2 (control means in the present invention), a manual operation device 3, a communication device 4, a main display 5 (display means in the present invention), a sub-display 6, and a volatile memory 7. And a nonvolatile memory 8 and an image analysis unit 9 (image analysis means in the present invention).

  The CPU 2 is mainly composed of a microcomputer, and executes a control program to control the operation of the entire apparatus. The manual operation device 3 is basically installed on the side of the driver's seat in the passenger compartment so that the driver can operate while driving. As shown in FIG. The direction instructing operation means) stands up from the housing 11. The joystick 10 can be tilted in all directions of 360 degrees with the sphere part 10a as a fulcrum by rotating the sphere part 10a at the lower end in all directions of 360 degrees. When the joystick 10 tilts, the manual operation device 3 outputs the tilt direction (operation direction) and tilt angle (operation amount) to the CPU 2 as an operation detection signal.

  Inside the housing 11, for example, a force generating mechanism 12 (force generating means in the present invention) configured by combining a plurality of components such as a small motor is disposed. When a force generation signal is input from the CPU 2 to the manual operation device 3, the force generation mechanism 12 generates a force such as a reaction force or a thrust against the operation of the joystick 10. A determination button 13 is disposed on the upper end of the joystick 10, and an enlargement button 14 and a reduction button 15 are disposed on the upper surface of the housing 11. The manual operation device 3 outputs an operation detection signal to the CPU 2 when any one of the determination button 13, the enlargement button 14, and the reduction button 15 is operated.

  The communication device 4 downloads a Web page from the server 16 through the Internet 17, and the CPU 2 starts the browser to download the Web page from the server 16 to the communication device 4 through the Internet 17, and the downloaded Web page. Is displayed on the main display 5 and the sub-display 6.

  The main display 5 is installed near the cockpit of the driver's seat and displays a display screen in the display area when a display command signal is input from the CPU 2. At this time, since the main display 5 is installed near the cockpit of the driver's seat, the driver needs to move his / her line of sight in order to view the display screen displayed on the main display 5 during driving. .

  The sub display 6 has a display area smaller than the display area of the main display 5 described above, and is installed on a dashboard behind the handle (back side), for example, and receives a display command signal from the CPU 2. Then, the display screen is displayed in the display area. At this time, since the sub display 6 is installed on the dashboard behind the steering wheel, the driver needs to move his / her line of sight in order to view the display screen displayed on the sub display 6 while driving. There is no.

  The volatile memory 7 stores an image of a Web page downloaded from the server 16 to the communication device 4 through the Internet 17. The non-volatile memory 8 is configured to be able to store screen configuration data of the display screen of the application software of the in-vehicle operation device 1. When a read signal is input from the CPU 2, the stored screen configuration data is transferred to the CPU 2. Output to. Note that the application software of the in-vehicle operation device 1 here is for adjusting the temperature of an air conditioner, adjusting the volume of an audio device, or adjusting the angle of an electric mirror, for example.

  When the image analysis signal is input from the CPU 2, the image analysis unit 9 analyzes the Web page image stored in the volatile memory 7 and outputs the analyzed result to the CPU 2 as an image analysis result.

Next, the operation of the above configuration will be described with reference to FIGS. Here, on the assumption that the display screen is displayed on the main display 5,
(1) When the display screen displayed on the main display 5 is the application software of the in-vehicle operation device 1 (2) The case where the display screen displayed on the main display 5 is not the application software of the in-vehicle operation device 1 will be described sequentially.

(1) When the display screen displayed on the main display 5 is application software of the in-vehicle operation device 1 First, “when the display screen displayed on the main display 5 is application software of the in-vehicle operation device 1” will be described.

  The CPU 2 determines whether or not the display screen to be displayed on the main display 5 is application software of the in-vehicle operation device 1 (step S1), and the display screen to be displayed on the main display 5 is application software of the in-vehicle operation device 1. If this is detected ("YES" in step S1), in this case, since the screen configuration data of the display screen of the application software of the in-vehicle operation device 1 is stored in the nonvolatile memory 8, the read signal is transmitted to the nonvolatile memory. The corresponding screen configuration data is read out from the nonvolatile memory 8 (step S2).

  After that, when the CPU 2 displays the display screen of the application software on the main display 5 and detects that the joystick 10 is operated, the CPU 2 outputs a force generation signal to the manual operation device 3 and based on the screen configuration data. The force generation mechanism 12 generates a reaction force or thrust against the operation of the joystick 10 or cancels the reaction force or thrust (step S3).

(2) When the display screen to be displayed on the main display 5 is not application software of the in-vehicle operation device 1 Next, “when the display screen to be displayed on the main display 5 is not application software of the in-vehicle operation device 1” will be described.

  When the image of the Web page is downloaded from the server 16 to the communication device 4 through the Internet 17, the CPU 2 stores the downloaded image of the Web page in the volatile memory 7. When the CPU 2 detects that the display screen to be displayed on the main display 5 is not application software of the in-vehicle operation device 1 (“NO” in step S1), in this case, the following process is performed.

  That is, the CPU 2 determines whether or not the display screen to be displayed on the main display 5 is application software that operates on the browser (step S4). When the CPU 2 detects that the display screen to be displayed on the main display 5 is application software that operates on the browser (“YES” in step S4), the CPU 2 analyzes the HTML tag and creates screen information. (Step S5).

  Next, the CPU 2 determines whether or not the display screen to be displayed on the main display 5 is application software described in Java (registered trademark), Flash (registered trademark), or the like (step S6). When the CPU 2 detects that the display screen to be displayed on the main display 5 is application software described in Java (registered trademark) or Flash (registered trademark) (“YES” in step S6), the application 2 The size of the software area is analyzed based on the tag, and the previously created screen information is detailed (step S7).

  Next, the CPU 2 determines whether or not the structure analysis of the application software that operates on the browser is possible (step S8). When the CPU 2 detects that the structure analysis of the application software running on the browser is possible (“YES” in step S8), the CPU 2 performs the structure analysis of the application software and details the screen information previously created. (Step S9), and the display area of the main display 5 is divided into a plurality of grids based on the screen information (step S10).

  On the other hand, when the CPU 2 detects that the structure analysis of the application software running on the browser is not possible (“NO” in step S8), it performs image recognition (to be described later in detail) (step S11), and the main display. The display area 5 is divided into a plurality of grids based on the screen information based on the image recognition result (step S10).

If the CPU 2 detects that the application software does not operate on the browser (“NO” in step S4), the CPU 2 determines whether the structure analysis of the application software is possible (step S12). When the CPU 2 detects that the structure analysis of the application software is possible (“YES” in step S12), the CPU 2 analyzes the structure of the application software and creates screen information (step S13). The display area is divided into a plurality of grids based on the screen information (step S10).

  On the other hand, when the CPU 2 detects that the structure analysis of the application software is not possible (“NO” in step S12), the CPU 2 also performs image recognition described later in detail (step S14), and the main display 5 Is divided into a plurality of grids based on the screen information based on the image recognition result (step S10).

  Next, image recognition processing performed by the CPU 2 will be described with reference to a flowchart shown in FIG. When the CPU 2 shifts to the image recognition process, the CPU 2 outputs an image analysis signal to the image analysis unit 9 and analyzes the image of the Web page stored in the volatile memory 7. First, the CPU 2 determines whether or not there is a character area in the image (step S41). When the CPU 2 detects that there is a character area in the image (“YES” in step S41), the CPU 2 performs character recognition and performs a character or sentence. A grit is generated in units of (step S42).

  Next, the CPU 2 determines whether or not there is a special character string in the image (step S43), and detects that there is a special character string in the image (“YES” in step S43), the special character string. A grit is generated in units of (step S44). Next, the CPU 2 determines whether or not there is a graphic area in the image (step S45). When the CPU 2 detects that there is a graphic area in the image (“YES” in step S45), the CPU 2 performs graphic recognition and unitizes the graphic. A grid is generated (step S46), followed by color recognition to correct the grid (step S47).

  As a result, as shown in FIG. 6, the CPU 2 can divide the display area of the main display 5 into a plurality of grids in units of characters and figures on the display screen. Thereafter, the CPU 2 outputs a display command signal to the main display 5 to display the web page downloaded from the server 16 on the main display 5 and also displays a grid cursor (see P in FIG. 6). It is displayed on the grid (step S15). More specifically, the CPU 2 displays a grit cursor (see P in FIG. 6) on a grit in which, for example, a sentence “Breaking News Mr. XX is a press conference” is described. Then, the CPU 2 waits for an operation detection signal to be input from the manual operation device 3 (step S16).

  When the operation detection signal is input from the manual operation device 3 by the user operating the joystick 10 (“YES” in step S16), the CPU 2 analyzes the input operation detection signal (step S17), and the joystick. 10 is detected ("YES" in step S18).

  At this time, when the CPU 2 detects that the joystick 10 has been operated, the CPU 2 outputs a force generation signal to the manual operation device 3 and causes the force generation mechanism 12 to generate a reaction force or a thrust against the operation of the joystick 10. (Step S19). More specifically, if the tilt direction of the joystick 10 is the left direction, the CPU 2 generates a reaction force or a thrust against the tilt of the joystick 10 in the left direction, and the tilt direction of the joystick 10 is the backward direction. If so, a reaction force or a thrust is generated with respect to the backward tilting of the joystick 10.

  Next, the CPU 2 outputs a display command signal to the main display 5 and moves the grid cursor according to the operation direction of the joystick 10 (step S20). More specifically, the CPU 2 moves the grit cursor to the left grit if the tilt direction of the joystick 10 is the left direction, and moves the grit cursor if the tilt direction of the joystick 10 is the rear direction. Move it back on the grid.

  Then, the CPU 2 releases the reaction force and the thrust immediately after moving the grid cursor (step S21), returns to the above step S16, and waits again for the operation detection signal to be input from the manual operation device 3. Thereby, when the user operates the joystick 10, the user can feel an operational feeling due to the generation of reaction force and thrust.

  Next, when the user moves the grid cursor onto the grid where the desired link destination (character or graphic) is described in this way, the grid where the grid cursor is located is determined as the selected grid. Then, the determination button 13 is operated. When the operation detection signal is input from the manual operation device 3 by the user operating the enter button 13 (“YES” in step S16), the CPU 2 analyzes the input operation detection signal (step S17). It is detected that the determination button 13 has been operated (“YES” in step S22).

  Next, the CPU 2 determines whether or not the grid where the grid cursor is located can be determined as the selected grid (step S23). Here, if the grit where the grit cursor is located can be determined as the selected grit ("YES" in step S23), the CPU 2 determines the grit where the grit cursor is located as the selected grit, and selects the selected grit. A process of connecting to the link destination described in the grid determined as is performed (step S24). Thereby, the user can download and browse a desired linked web page thereafter.

  Specifically, as shown in FIG. 6, when the grit cursor is positioned on the grit in which a sentence “breaking news ○○ is a press conference” is described, the user moves the joystick 10 to the left. By operating in the direction, as shown in FIG. 7, the grid cursor can be moved over the grid in which the characters “society” are described, and by subsequently operating the enter button 13, “ Web pages with contents related to “society” can be downloaded and viewed.

  Further, as shown in FIG. 6, the user operates the joystick 10 backward when the grit cursor is positioned on the grit in which the sentence “Breaking News XX is a press conference” is described, for example. Thus, as shown in FIG. 8, the grid cursor can be moved over the grid where the graphic (photo image) is described, and by operating the decision button 13 subsequently, for example, the corresponding graphic is displayed. It can be enlarged and viewed.

  Through the series of processes described above, the user operates the joystick 10 to move the grid cursor to a desired link when the link destination is described in the Web page downloaded from the server 16 and displayed on the main display 5. By moving on the grid where the destination is described, it is possible to connect to a desired link destination and download and browse a desired Web page. At this time, when the user operates the joystick 10, a reaction force or a thrust is generated, so that the user can feel an operational feeling.

  By the way, when the user operates the enlarge button 14, the CPU 2 is input when an operation detection signal is input from the manual operation device 3 due to the user operating the enlarge button 14 ("YES" in step S16). The detected operation detection signal is analyzed (step S17), and it is detected that the enlargement button 14 has been operated ("YES" in step S25). Next, the CPU 2 determines whether or not the grit where the grit cursor is located can be enlarged (step S26). If the grit can be enlarged ("YES" in step S26), the display area of the main display 5 is displayed. Is reduced to a plurality of grids by reducing the size of the grid (step S27), the grid cursor is displayed on any grid (step S28), the process returns to step S16, and the operation is detected from the manual operation device 3. Wait again for the signal to be input.

  When the user operates the reduction button 15, the CPU 2 inputs an operation detection signal from the manual operation device 3 due to the user operating the reduction button 15 ("YES" in step S16). The detected operation detection signal is analyzed (step S17), and it is detected that the reduction button 15 has been operated (“YES” in step S29). Next, the CPU 2 determines whether or not the grid where the grid cursor is located can be reduced (step S30). If the grid can be reduced (“YES” in step S30), the display area of the main display 5 is displayed. The size of the grid is increased and re-divided into a plurality of grids (step S31), the grid cursor is displayed on one of the grids (step S32), the process returns to step S16, and the operation is detected from the manual operation device 3. Wait again for the signal to be input.

  That is, the CPU 2 manages the screen information in a hierarchy, and on the condition that the hierarchy in which the screen information can be created is lower, the grid where the grid cursor is located is expanded. The grid on which the grid cursor is located is reduced on the condition that the hierarchy in which screen information can be created is higher.

  By the way, the above has been described on the assumption that the display screen is displayed on the main display 5. However, when the display screen is displayed on the sub display 6, the same processing is performed. Then, it is possible to select a desired URL and a desired button without moving the line of sight while driving.

  As described above, according to the present embodiment, the in-vehicle operation device 1 performs image analysis on the display screen of the Web page downloaded from the server 16, and displays the Web page by dividing the main display 5 into a plurality of grids. Since the screen is displayed on the main display 5 and the joystick 10 is operated, the grid cursor is moved. Therefore, by moving the grid cursor over the grid in which the desired URL is described, the desired URL is displayed. You can choose.

  At this time, when the user operates the joystick 10, a force such as a reaction force or a thrust is generated with respect to the operation of the joystick 10, so that the user can be given an operational feeling. Operability can be improved. Further, by giving the user a feeling of operation, the user can be notified of the amount of movement of the grit cursor, so that the user can operate the joystick 10 as if it were a blind touch. Furthermore, since the display area of the main display 5 is configured to be divided into a plurality of grids based on the image analysis result of the display screen, the display area is divided into a plurality of grids in an optimum form according to the URL and button arrangement. It can be divided, and this also improves the operability for the user.

  The present invention is not limited to the embodiments described above, and can be modified or expanded as follows.

  The direction instruction operation means is not limited to a joystick, but may be other things such as a trackball.

  The manual operation device may be provided with a control circuit (CPU) so that the CPU and the control circuit of the manual operation device cooperate to control the reaction force and the generation of thrust in the force generation mechanism.

  The shape of the grid is not limited to a rectangle, but may be a triangle, a hexagon, or a circle. You may apply when generating the force which pulls a joystick back to the center.

  The display area may be divided into grids based on word units or phrase units. Moreover, the structure which switches by a character unit, a word unit, a phrase unit, and a sentence unit, and divides | segments into a grid may be sufficient by operating an enlargement button or a reduction button.

Functional block diagram showing an embodiment of the present invention External perspective view of manual operation equipment flowchart 3 equivalent figure 3 equivalent figure The figure which shows an example in which the display area of the main display was divided into a plurality of grids The figure which shows the example of the aspect by which the grid cursor was moved The figure which shows the other example of the aspect to which the grid cursor was moved

Explanation of symbols

  In the drawings, 1 is an in-vehicle operation device, 2 is a CPU (control means), 5 is a main display (display means), 9 is an image analysis unit (image analysis means), 10 is a joystick (direction instruction operation means), and 12 is force. A generating mechanism (force generating means).

Claims (4)

Direction instruction operation means operated by the user to instruct the direction;
Force generating means capable of generating a force such as a reaction force or a thrust with respect to the operation of the direction indicating operation means;
An in-vehicle operation device comprising a control means for displaying a display screen on the display means,
When the structure analysis of the display screen is not possible, the image analysis means for analyzing the image of the display screen,
The control unit divides the display area of the display unit into a plurality of grids based on an image solution result by the image analysis unit, and causes a grid cursor to be displayed on any one of the divided grids. When the direction indicating operation unit is operated, the force generating unit generates a reaction force, a thrust, or the like with respect to the operation of the direction indicating operation unit, and the grit cursor of the display unit is moved to the direction indicating operation. A vehicle-mounted operation device that is moved on any other grit in accordance with the operation of the means.   The in-vehicle operation device according to claim 1, wherein the image analysis unit performs character recognition on an image on a display screen, and the control unit converts a display area of the display unit into a character recognition result by the image analysis unit. An in-vehicle operation device that is divided into a plurality of grids based on the above.   3. The in-vehicle operation device according to claim 1, wherein the image analysis unit performs graphic recognition on an image on a display screen, and the control unit recognizes a display area of the display unit by using the image analysis unit. An in-vehicle operation device that is divided into a plurality of grids based on the result.   4. The in-vehicle operation device according to claim 1, wherein the image analysis unit performs color recognition on an image on a display screen, and the control unit defines a display area of the display unit as the image analysis unit. An in-vehicle operation device that is divided into a plurality of grids based on a color recognition result obtained by the above.

Images