JP7193904B2 - Electronic devices and systems - Google Patents

Description

The present invention relates to an electronic device capable of outputting content data acquired from an information terminal, and more particularly to controlling the information terminal from the electronic device side.

In recent years, the use of multifunctional information terminals represented by smartphones has been increasing. When such an information terminal is brought into the vehicle, the functions of the information terminal can be used in the in-vehicle device by connecting and cooperating with the electronic device mounted in the vehicle (hereinafter referred to as in-vehicle device). is possible. For example, when an application for playing music is started on an information terminal, the in-vehicle device acquires content data such as application information and music being played through the connection, and based on the acquired data, the in-vehicle device can be played on the side. Also, the user can control the operation of the content being played back on the information terminal by performing a touch operation on an operation command displayed on the screen of the in-vehicle device.

As a technology related to cooperation between the in-vehicle device and the information terminal, when the external device is connected, the default screen for the external device is displayed so that the operability does not deteriorate even if the external device is switched. There is (Patent Document 1).

JP 2008-269482

Next, the subject concerning this invention is demonstrated. FIG. 1 shows a default operation screen stored in advance in the in-vehicle device, displayed on the display unit 160 of the in-vehicle device. Reference numerals 161 to 168 denote operation commands. Operation instructions corresponding to the selected operation commands are transmitted to the information terminal in conjunction with the user's touch operation or the like, and the information terminal executes the received operation instructions. Take control of your content. However, depending on the application running on the information terminal side, some of the operation commands 161 to 168 may not be available. For example, application A can use the operation command 161, but application B cannot use the operation command 161. Originally, it is possible to check whether the function of the operation command can be used at the time of connection and to display only the usable operation command, but depending on the application, this confirmation work may not go well and the response of the availability may not be possible. However, on the electronic device side, an operation command that does not respond when confirming whether or not it can be used is treated as usable, so that the unusable operation command is displayed on the operation screen as usable. Further, in the case of an in-vehicle device that does not perform the above operation command usability confirmation work, all the operation commands are displayed regardless of whether they can be used or not.

If such an unusable operation command is displayed on the operation screen, the user who does not know the fact recognizes that the displayed operation command can be used. If the corresponding function is not executed, the user will be confused as to why the function is not executed.

The present invention has been made to solve such problems. It is an object of the present invention to provide an electronic device and an electronic system capable of accurately presenting to a user.

An electronic device according to the present invention includes connection means for enabling connection with an information terminal that holds one or more applications and is capable of executing one or more applications, and a plurality of operation commands. display means for displaying an image; input means for receiving an instruction from a user; operation instruction transmission means for transmitting an operation instruction of an operation command input by the input means to an information terminal via the connection means; and display control means for changing a display mode of an operation command requesting the operation instruction on the image according to a response status from the information terminal to the instruction.

In one embodiment, the electronic device further includes acquisition means for acquiring a response result from the information terminal to the operation instruction, and the display control means, when the acquisition means fails to acquire the response result to the operation instruction, Secondly, the display mode of the operation command that issued the request is changed. In one embodiment, when there is no response from the information terminal to the operation instruction within a predetermined time, the display control means considers that the response result to the operation instruction could not be obtained, and issues the request. Change the display mode of operation commands. In one embodiment, the display control means changes the display mode of the operation command requesting the operation instruction when the acquisition means acquires a response result from the information terminal indicating that the operation according to the operation instruction cannot be executed. . In one embodiment, the electronic device further includes notifying means for notifying information, and the notifying means responds to the change in the display mode of the operation command by the display control means and the use of the operation command. Notifies information indicating that it is not possible to In one embodiment, the display control means restores the display mode of the operation command to the original display mode in response to the application being executed being changed to another application. In one embodiment, the electronic device further includes storage means for storing the display mode changed for each application, and when it is detected that the application stored in the storage means is activated by the connection means, The display control means changes the display mode to the stored operation command.

The system according to the present invention has an information terminal connected to the electronic device, and the information terminal transfers application information being executed to the electronic device when the electronic device is connected.

According to the present invention, since the display mode is changed according to the response status to the operation instruction, the user can use the operation command even when the operation command usability confirmation work is not performed when connecting to the information terminal. It is possible to accurately grasp whether or not it is possible.

It is an operation screen for an external device displayed on the electronic device. It is a figure which shows the structural example of the communication system which concerns on the Example of this invention. 1 is a block diagram showing a typical configuration of an in-vehicle device according to an embodiment of the present invention; FIG. 3 is a block diagram showing the functional configuration of a display control program according to an embodiment of the invention; FIG. 1 is a block diagram showing a typical configuration of an information terminal according to an embodiment of the present invention; FIG. 4 is a sequence diagram of an in-vehicle device and an information terminal for explaining the operation of the display control program according to the embodiment of the present invention; FIG. FIG. 10 is a diagram showing transition of operation screens by execution of a display control program; FIG. 1 shows an example of changing the display mode; FIG. FIG. 2 shows an example of changing the display mode; FIG. It is an example figure which shows the notification means at the time of a display mode change. FIG. 4 is a sequence diagram of an in-vehicle device and an information terminal when switching applications; FIG. 10 is a diagram for explaining an operation screen when switching between applications; FIG.

DESCRIPTION OF THE PREFERRED EMBODIMENTS An electronic device according to the present invention will be described in detail below based on a specific embodiment with reference to the drawings. In a preferred embodiment of the present invention, the electronic device is mounted on a vehicle such as a moving object. An electronic device is a computer device that integrates, for example, a navigation function, a function to reproduce audio/video data, a function to receive television/radio broadcasts, and a function to execute application software (hereinafter referred to as an application). can be prepared In addition, the information terminal is preferably a portable device capable of processing information, and performs call functions, navigation functions, functions to reproduce audio/video data, functions to receive television/radio broadcasts, and executes applications. It can be a multifunctional terminal that integrates functions, such as a mobile phone, a smart phone, a palmtop personal computer, a laptop computer, a notebook computer, a tablet terminal, and the like. In addition, the information terminal can be connected to an external network, store voice data and image data, and execute various functions according to installed applications. In the following examples, a system in which an in-vehicle device and an information terminal are connected is illustrated.

The in-vehicle device uses the communication function included in the information terminal, acquires various content data for using the functions of the information terminal such as audio data and video data possessed by the information terminal, and outputs it from the in-vehicle device. can. This includes displaying an icon image based on the application on the information terminal side on the screen of the in-vehicle device, executing the application selected by the operation unit (touch panel) of the in-vehicle device, may read out only the application corresponding to the in-vehicle device, and the in-vehicle device may select and execute the application. Further, in this embodiment, a default operation screen for operating an application running on the information terminal side on the in-vehicle device side is provided in advance. For example, when an application for music reproduction is executed on the information terminal, the in-vehicle device displays a screen set as an operation screen for music reproduction from among the operation screens held.

FIG. 2 is a diagram showing a configuration example of a communication system according to an embodiment of the present invention. The communication system 1 includes an in-vehicle device 10 and an information terminal 30 . The in-vehicle device 10 enables two-way data communication with the information terminal 30 via the connection means 50 . The connection means 50 is, for example, a wireless connection such as Bluetooth (registered trademark) or Wi-Fi (registered trademark), or a wired connection such as USB. The data communicated includes audio data, video data, control data, and the like. When the information terminal 30 is connected to the in-vehicle device 10, it becomes possible to operate in cooperation with each other. For example, the resources of the information terminal 30 can be output by an audio system provided in the in-vehicle device 10 . The resources of the information terminal 30 include not only data and applications provided by the information terminal itself, but also data acquired by the information terminal 30 communicating with the external network NW.

A typical information terminal 30 is a smart phone, and the information terminal 30 has a function of voice call or packet communication via a public wireless line such as 3G, 4G, LTE. Furthermore, it is possible to provide a hands-free function for making voice calls via the in-vehicle device 10, a headphone set, or the like.

Next, FIG. 3 is a block diagram showing a typical configuration of an in-vehicle device according to an embodiment of the invention. The in-vehicle device 10 according to the present invention includes an input unit 100, a content reproduction unit 110, a navigation unit 120, a communication unit 130, a connection unit 140, an audio output unit 150, a display unit 160, a storage unit 170, and a control unit 180. be done. The input unit 100 inputs instructions from the user. The input unit 100 receives input from the user through a touch panel, an input key device, voice input, or the like, and provides the input instructions to the control unit 180 . For example, on the operation screen as shown in FIG. 1, it is possible to transmit the operation instruction of the operation command selected by the operation commands 161 to 168 via the connection unit 140 . The operation commands in this embodiment are commands for controlling content operations such as "play", "stop", "pause", "fast forward", "rewind", "repeat", and "shuffle".

The content reproduction unit 110 receives audio data and video data recorded on CDs, DVDs, Blu-ray discs, memory media, etc., television broadcasts and radio broadcasts, and reproduces them via the audio output unit 150 and the display unit 160. . The content reproduction unit 110 also requests the connected information terminal 30 to reproduce content, receives content data reproduced by the information terminal 30 , and reproduces the data via the audio output unit 150 and the display unit 160 . The navigation unit 120 displays a road map around the vehicle position based on position information transmitted from GPS satellites and position information obtained from various sensors such as a gyro sensor and an acceleration sensor mounted on the vehicle. It searches and guides the route to the destination. The communication unit 130 enables data communication with an external network or the like.

The connection unit 140 enables connection with the information terminal 30 . The connection unit 140 enables data transmission/reception between the in-vehicle device 10 and the information terminal 30 by wired connection or wireless connection. Also, the connection unit 140 corresponds to the connection unit 50 that connects the in-vehicle device 10 and the information terminal 30 in FIG. The audio output unit 150 outputs audio data relating to playback information of various contents, navigation information, etc., from an acoustic system such as a speaker, such as the content playback unit 110 and the navigation unit 120 . The display unit 160 displays video data related to playback information of various contents, navigation information, etc. on a display such as the content playback unit 110 and the navigation unit 120 . Furthermore, the display unit 160 displays an operation screen for controlling operations of content, such as operation commands.

The storage unit 170 can store applications and programs executed by the in-vehicle device 10, road map data, content data such as music and video, and the like. In one embodiment, the storage unit 170 stores operation screens for operating applications and content data being executed on the information terminal 30 with the in-vehicle device 10 . As a result, for example, an operation screen for music reproduction for operating an application for music reproduction and content data executed on the information terminal 30 on the in-vehicle apparatus 10, and a video reproduction operation screen for video reproduction executed on the information terminal 30 It stores an operation screen for moving image reproduction for operating applications and content data on the in-vehicle device 10, and the like. Which operation screen is to be displayed can be set as appropriate, and may be set in advance or may be changed by the user.

Further, in one embodiment, the storage unit 170 may store identification information such as an ID for identifying an application held by the in-vehicle device 10 . application can be activated on the information terminal side. Further, when the information terminal 30 is connected to the in-vehicle device 10, if only the applications that can be executed by the in-vehicle device 10 can be selected from among the applications held by the information terminal 30 and selected on the in-vehicle device side, Since the application information sent from the information terminal 30 is read out and displayed on the in-vehicle device each time it is connected, it is not necessary to always store the identification information in the storage unit 170 .

The control unit 180 includes a microcontroller, a microprocessor, or the like, and controls each unit by executing a program stored in the ROM/RAM or storage unit 170 . In a preferred aspect, when the information terminal 30 is connected to the in-vehicle device 10 , the control unit 180 enables cooperative operation between the in-vehicle device 10 and the information terminal 30 . Thus, the in-vehicle device 10 can function as if it were an input/output device of the information terminal 30 , and audio data and video data generated by the information terminal 30 are output from the in-vehicle device 10 .

FIG. 4 is a block diagram showing the functional configuration of the display control program installed in the vehicle-mounted device according to the embodiment of the present invention. The display control program 200 includes a connection detection unit 210 , a data acquisition unit 220 , an operation screen display unit 230 , an operation instruction transmission unit 240 , a response detection unit 250 and a display mode change unit 260 .

The connection detection unit 210 detects whether or not the information terminal 30 is connected to the in-vehicle device 10 via the connection unit 140 . The connection means may be a wired connection or a wireless connection, and can be modified according to the embodiment. The data acquisition unit 220 acquires various data related to the content executed on the information terminal 30 . For example, when an application is activated, it acquires application identification information, audio data, video data, and the like to be reproduced.

The operation screen display unit 230 acquires the corresponding operation screen from the storage unit 170 based on the content being executed, and displays it on the display unit 160 . The operation screen includes various operation commands for operating content. In a preferred embodiment, different operation screens are displayed according to the content to be reproduced. For example, when adapted to music playback, operation commands unique to music playback such as "shuffle", "repeat", "next track", and "previous track" may be provided unlike the operation screen for video playback. Also, the menu screen containing the operation commands and the like on the operation screen may not always be displayed, and the manner of displaying the operation screen can be modified. Operation instruction transmission section 240 transmits an operation instruction for the selected operation command to information terminal 30 via connection section 140 . The operation instruction is linked to the operation command. For example, when "playback" is selected as the operation command, "playback" is transmitted to the information terminal 30 as the operation instruction.

Response detection unit 250 detects a response from information terminal 30 to the operation instruction transmitted by operation instruction transmission unit 240 . Preferably, the response detection unit 250 detects a response from the information terminal 30 within a predetermined time after transmitting the operation instruction to the information terminal 30 . As a result, a function that cannot respond or a function that takes a long time to respond can be regarded as unusable. The time interval of the predetermined time may be appropriately changed according to the embodiment. Then, when a response is detected by the response detection unit 250, the in-vehicle device 10 outputs content based on the detected response result. Further, the response detection unit 250 may detect whether the detected response result indicates that the operation instruction cannot be executed.

The display mode changing section 260 changes the display mode of the operation command on the operation screen based on the response result detected by the response detection section 250 . For example, by image processing, the display color of the corresponding operation command is changed, or a diagonal line is superimposed on the operation command. In one embodiment, a plurality of images having different display modes of operation commands on the operation screen are stored in advance in the storage unit 160, and the operation screen is switched based on the response result from the response detection unit 250, thereby enabling the operation command to be displayed. The command display mode may be changed.

Next, FIG. 5 is a block diagram showing a typical configuration of an information terminal according to an embodiment of the invention. The information terminal 30 according to the present invention includes an input unit 300 that receives input from a user, a communication unit 310 that enables communication via a public telephone line network or network communication, a display control unit 320 that controls display display, a speaker Audio output unit 330 that controls audio output from, connection unit 340 that establishes connection of connection means 50 with electronic devices such as in-vehicle device 10, microcontroller, microprocessor, etc. By executing a program, each unit , application software for executing various functions such as audio reproduction, radio reproduction, and navigation, program memory 360 for storing programs for cooperating with the in-vehicle device 10, and various data. A data memory 370 is provided.

Next, the control operation of the display control program 200 according to the embodiment of the present invention will be described with reference to the sequence diagram of the in-vehicle device and the information terminal in FIG. First, the connection detection unit 210 detects whether the in-vehicle device 10 is connected to the information terminal 30 via the connection unit 140, and establishes the connection (S500). When the in-vehicle device 10 and the information terminal 30 are in a connected state, the in-vehicle device 10 outputs a content reproduction request to the information terminal 30 (S502). Preferably, a request to reproduce an application selected by the user from among applications displayed as icons or the like on the display unit 160 is output to the information terminal 30 . Along with the reproduction request, identification information such as the ID of the application to be activated is output.

The information terminal 30 activates the application based on the reproduction request, and transmits content data such as audio data and video data associated with the execution of the application to the in-vehicle device 10 (S504). Here, for example, when a music reproduction application is executed, only audio data is transmitted to the in-vehicle device 10, and when a video reproduction application is executed, audio data and video data are transmitted to the in-vehicle device 10. Send. The in-vehicle device 10 acquires the content data transmitted from the information terminal 30 from the data acquisition unit 220 (S506).

Next, the operation screen display unit 230 displays an operation screen including operation commands for operating the content to be reproduced on the display unit 160 (S508). In one embodiment, an operation screen pre-stored in the storage unit 170 is automatically selected and displayed in response to activation of the application. The user selects a desired operation content (playback, stop, etc.) from among the operation commands displayed on the screen, and transmits it as an operation instruction to the information terminal 30 by the operation instruction transmission unit 240 (S510).

The information terminal 30 controls the operation of the content based on the transmitted operation instruction (S512). For example, when the in-vehicle device 10 requests a "stop" operation instruction, the information terminal 30 controls to stop the content being reproduced. Then, the execution result for the operation instruction is transmitted to the in-vehicle device 10 . At this time, if the target operation instruction does not correspond to the application currently being executed, the information terminal 30 cannot execute the operation instruction and does not respond. Alternatively, in such a case, an execution result indicating that the operation instruction cannot be executed may be transmitted to the in-vehicle device 10 .

Next, the response detection unit 250 detects whether or not there is a response to the operation instruction from the information terminal 30 (S514). When the response from the information terminal 30 is detected, the process proceeds to S516, and when the response is not detected, the process proceeds to S518. Here, if a response from the information terminal 30 is detected in S514, it is detected whether or not the response result indicates that the requested operation instruction cannot be executed (S516). . If no response from the information terminal 30 is detected in S514, or if the response result detected in S516 indicates that the requested operation instruction cannot be executed, the display The display mode of the operation command is changed by the mode changing unit 260 (S518).

Here, the change of the display mode will be described in detail with reference to FIG. FIG. 7 shows an operation screen displayed on the display unit 160 of the in-vehicle device 10 when the music reproduction application is started on the information terminal 30. As shown in FIG. First, as shown in FIG. 7A, when the operation command 167 “shuffle” is selected, the operation instruction “shuffle” is transmitted to the information terminal 30 . Here, since the application being executed does not support the operation instruction "shuffle", the display mode of the operation command 167 is grayed out. Next, when the operation command 161 "previous track" is selected as shown in (b) while maintaining the display state, the operation instruction "previous track" is transmitted to the information terminal 30 . Here, since the application being executed does not support the operation instruction "previous track", the display mode of the operation command 161 is grayed out as in the case of "shuffle".

Next, while maintaining the display state, as shown in (c), when the operation command 168 “repeat” is selected, the operation instruction “repeat” is transmitted to the information terminal 30 . Here, since the application being executed corresponds to the operation instruction "repeat", the display mode is not changed by executing "repeat". Then, while maintaining the display state, as shown in (d), when the operation command 166 "next track" is selected, the operation instruction "next track" is transmitted to the information terminal 30 . Here, since the application being executed does not support the operation instruction "next track", the display mode of the operation command 166 is grayed out in the same manner as "shuffle" and "previous track". As described above, by making the display mode different from other operation commands, it is possible to show the user unusable operation commands.

The modification of the display mode is not limited to graying out the target operation command. For example, as shown in FIG. As shown in 9, the target operation command may be hidden. Also, they may be combined, for example, the target operation command may be shaded and grayed out. In addition to the examples shown above, the display mode after the change can be appropriately changed according to the embodiment as long as it is changed to a display mode different from the normal display mode.

At the same time when the display mode of the operation command that cannot be used is changed by the display mode changing unit 260, the reason for changing the display mode may be notified to the user on the display screen or by voice. For example, as shown in FIG. 10, when the operation command 167 "shuffle" cannot be used, the operation command 167 is grayed out and a comment 169 such as "this command cannot be used" is displayed on the display unit 160. do. As a result, the user can recognize the reason why the display mode was changed, so that the feeling of anxiety caused by the sudden change of the display mode can be eliminated. Also, the notification may be made every time the display mode is changed, or may be made only once for the first time.

Returning to S514, when the response detection unit 250 detects the response result from the information terminal 30 and the detected response result does not indicate that the information terminal 30 cannot execute the operation instruction, the information The execution result of the terminal 30 is output and the processing is continued, and when there is an operation instruction by the operation command again, the processing from S510 to S518 is performed.

According to the embodiments described above, when an operation command that cannot be executed on the information terminal side is displayed on the operation screen, the presence or absence of a response result to the operation instruction, or a response indicating that the operation command could not be executed Since the display mode of the target operation command is changed based on the detection of the result, the user can clearly understand that the target operation command cannot be used.

Next, regarding the control of the operation screen when switching the application being executed and executing another application, FIG. will be described with reference to FIG. 12 showing the variation of . First, the in-vehicle device 10 and the information terminal 30 are in a connected state via the connection means 50, the application A for music reproduction is executed on the information terminal side, and the content data of the application A is acquired on the in-vehicle device side. is output (S600). It is also assumed that the display mode of the operation screen at this time has already been changed by the display mode changing unit 260, as shown in FIG. 12(a). Here, the application A is stopped by the user's operation, and a request to start the application B is newly made (S602). In the embodiment, the application is switched from the in-vehicle device side, but it is also applicable to switching the application by operating the information terminal side.

In response to the activation request, the information terminal 30 stops the application A and newly activates the music reproduction application B (S604). Then, the activation information of the activated application B, content data, and the like are transmitted to the in-vehicle device 10 . In the in-vehicle device 10, the content data acquisition unit 220 acquires information about the application B (S606). In response to the execution of the new application, the in-vehicle device 10 restores the display mode of the previously deformed operation command on the operation screen to its original state as shown in FIG. 12(b) (S608). Then, when an input operation is performed using an operation command, the processes from S510 to S518 in FIG. 6 described above are executed.

As described above, by returning the operation screen to the original state each time the application is switched, an operation command that could not be used in application A but could be used in application B becomes effective. Conversely, it is also effective when there is an operation command that can be used in application A but cannot be used in application B.

In one embodiment, the in-vehicle device 10 stores the information of the operation screen transformed by the display mode changing unit 260 in the storage unit 170 for each application, and when the application is activated, the stored operation screen is displayed. may be automatically changed to As a result, since the display mode of the operation command that cannot be used is changed from the beginning, there is no need to detect the response result to the operation instruction each time. Furthermore, the user can grasp which operation command cannot be used when the operation screen is displayed.

Although preferred embodiments of the present invention have been described in detail, the present invention is not limited to specific embodiments, and various modifications and changes can be made within the scope of the gist of the invention described in the claims. It is possible.

1 communication system 10 in-vehicle device 30 information terminal 50 connection means NW network 160 display units 161 to 168 operation command 169 comment

Claims (7)

connection means for enabling connection with an information terminal that holds one or more applications and is capable of executing one or more applications;
display means for displaying an image including a plurality of operation commands;
input means for receiving a selection input from a user for an operation command on the image;
operation instruction transmission means for transmitting an operation instruction of the operation command input by the input means to the information terminal via the connection means;
Acquisition means for acquiring a response result from the information terminal to the operation instruction;
display control means for changing a display mode of the operation command requesting the operation instruction on the image according to the response result from the information terminal to the operation instruction ;
The electronic device, wherein the display control means changes a display mode of the operation command requesting the operation instruction when the acquisition means fails to acquire the response result to the operation instruction. If there is no response from the information terminal to the operation instruction within a predetermined time, the display control means considers that the response result to the operation instruction could not be obtained, and the operation requesting the operation instruction is performed. 2. The electronic device according to claim 1 , wherein the display mode of commands is changed. connection means for enabling connection with an information terminal that holds one or more applications and is capable of executing one or more applications;
display means for displaying an image including a plurality of operation commands;
input means for receiving a selection input from a user for an operation command on the image;
operation instruction transmission means for transmitting an operation instruction of the operation command input by the input means to the information terminal via the connection means;
Acquisition means for acquiring a response result from the information terminal to the operation instruction;
display control means for changing a display mode of the operation command requesting the operation instruction on the image according to the response result from the information terminal to the operation instruction;
The display control means changes the display mode of the operation command requesting the operation instruction when the acquisition means acquires the response result indicating that the operation according to the operation instruction cannot be executed from the information terminal. The electronic device further includes notification means for notifying information,
4. Any one of claims 1 to 3 , wherein said notification means notifies information indicating that said operation command cannot be used in response to said display control means changing the display mode of said operation command. The electronic device described in . 4. The display control means according to any one of claims 1 to 3 , wherein in response to the application being executed being changed to another application, the display mode of the operation command is returned to the original display mode. electronic device. The electronic device further includes storage means for storing the display mode changed for each application,
6. When the connection means detects that the application stored in the storage means is activated, the display control means changes the display mode to the stored operation command. An electronic device according to any one of the preceding claims. an electronic device according to any one of claims 1 to 6 ;
an information terminal connected to the electronic device;
An electronic system, wherein an information terminal transfers application information being executed to the electronic device when the electronic device is connected.

Images