JP4930407B2 - Information providing apparatus, information display method, and navigation apparatus - Google Patents

Description

  The present invention relates to an information providing device that displays information on a display device, and more particularly, to an information providing device information display method and a navigation device provided with voice recognition means for recognizing external voice.

  While driving the vehicle, various sounds are generated inside the vehicle, and various sounds and sounds (hereinafter referred to as audio information) generated outside the vehicle enter. The occupant discards information necessary for vehicle operation from the voice information, but may miss the voice information to be heard due to noise inside and outside the vehicle. In addition, a driver's license is issued if the prescribed conditions are met, but this condition does not include mild hearing impairment, and driving is prohibited even if hearing impairment occurs after issuing a driving certificate. There is no. A driver's license is often issued even if the language of the country to which it is issued is not fully understood.

  However, while driving, public services are provided by voice information, and communication with external people may be required in some cases, so drivers may feel inconvenient if their hearing and language skills are not sufficient. is there. Moreover, it can be said that such an inconvenience causes unfairness to a passenger using an environment having high publicity such as road and parking information.

With respect to this point, a technique for collecting external voice and assisting a passenger in the vehicle has been proposed (see, for example, Patent Document 1). Patent Document 1 describes a navigation device that recognizes voice information and displays the content on a display. Therefore, the occupant can visually grasp the audio information that could not be heard.
JP 2003-329475 A

  However, since the navigation device described in Patent Document 1 merely recognizes voice and displays it, there is a problem that it does not make sense for foreigners who have poor language skills, for example.

  In view of the above problems, an object of the present invention is to provide an information providing device, an information display method, and a navigation device that convey audio information outside a vehicle in a manner that is easy for a passenger to understand.

  In view of the above problems, the present invention provides an information providing device for displaying a service screen corresponding to a service on a display unit, a sound collecting means for collecting sound information outside the vehicle, and a sound information collected by the sound collecting means. Voice recognition means for voice recognition and conversion to text data; and screen change means for changing the display mode of the service screen based on the text data (for example, the voice information change unit 45 and the service screen generation unit 47). Features.

  According to the present invention, since the display mode of the service screen is changed based on the text data that recognizes the voice, the service screen can be optimized according to the voice information.

  In one embodiment of the present invention, when the service screen is a parking lot map of the service that provides a parking lot, the screen changing unit corresponds to the position information included in the text data, and a predetermined position of the parking lot map. The display mode is changed.

  According to the present invention, when an attendant of a parking lot instructs the parking position by voice, the position is emphasized on the parking lot map, for example, so that the occupant can easily grasp the parking position.

  In one embodiment of the present invention, when the service screen is a product menu of the service that provides food and drink in a drive-through, the screen changing unit is configured to purchase a product that can be purchased or cannot be purchased included in the text data. The display mode of the product in the product menu corresponding to the information is changed.

  According to the present invention, whether or not the product can be selected by the voice of the store clerk can be reflected in the displayed product menu, so that it is possible to make it easier for the occupant to know whether or not the product can be purchased.

  It is possible to provide an information providing device, an information display method, and a navigation device that convey audio information outside the vehicle in a manner that is easy to understand to the occupant.

  Hereinafter, the best mode for carrying out the present invention will be described with reference to the drawings.

  FIG. 1 is a diagram for explaining the outline of the information providing apparatus according to the present embodiment. For example, when parking in a parking lot, the vehicle receives parking lot information at the entrance of the parking lot as shown in FIG. The parking lot information is, for example, a parking lot map, congestion status, fee information, and the like. When the parking lot information is received, the in-vehicle device displays the parking lot information on the display as shown in FIG. 1B (hereinafter, a screen on which information specific to the service is displayed is referred to as a service screen). In FIG.1 (b), a parking lot map is displayed, and it turns out that this parking lot is 3F and 1F part is full.

  The occupant sees the service screen and can determine that the vehicle can park on the 2nd or 3rd floor, but the parking lot may be designated by voice to the parking staff or guards. In FIG. 1 (b), the staff member is designated "Please use A on the third floor". The information providing apparatus according to the present embodiment recognizes the voice and reflects it on the service screen. For example, the expressions “3rd floor” and “A” (reflected expressions described later) are extracted from the voice, and the display mode is changed so that the parking place A of 3F on the parking lot map is emphasized. In FIG.1 (c), it highlighted by enclosing with a circle | round | yen.

  Therefore, since the voice information is reflected on the service screen, it is easy for the occupant to grasp the voice instruction. Further, even when the occupant is a hearing impaired person or a foreigner, or when the clerk's voice cannot be heard, the voice information is reflected on the service screen, so that parking according to the instruction is possible. That is, the information providing apparatus according to the present embodiment makes it easier to grasp voice information, and can provide a voice service fairly regardless of hearing or language ability.

  In addition, although a service is provision of a parking lot, provision of food and drinks, provision of fuel, etc. as shown in FIG. 1, hereinafter, it will be referred to as a service including public services that do not have direct benefits, such as a check conducted by the police. Such a service type is hereinafter referred to as a service mode. The term “service provider” includes facilities and persons that provide such services.

  FIG. 2 shows a system configuration diagram of the information providing apparatus 100. Since the information providing apparatus 100 displays information using the display 52, the information providing apparatus 100 can be configured by, for example, a navigation system. The information providing apparatus 100 is controlled by the control unit 20, and includes a navigation function unit 200 that provides guidance information to the current location and destination to the occupant, a voice recognition unit 300 that recognizes voice from voice information, and a service that reflects voice information. A service screen display unit 400 for changing the screen. Note that all or part of the navigation function unit 200, the voice recognition unit 300, and the service screen display unit 400 may be configured separately.

  The information providing apparatus 100 includes an operation unit 51, a display 52, and a speaker 53. The operation unit 51 is a user interface for a user to input an operation to the information providing apparatus 100 such as a keyboard, a remote controller, a voice input device, or a touch panel integrated with the display 52. The display 52 is a display means such as a liquid crystal or an organic EL, and is used for displaying a road map and a service screen as will be described later. The speaker 53 is used to output voice information that prompts the user to drive the vehicle before the right or left turn on the route to the destination and guides the user to the destination.

[Navi function unit 200]
The control unit 20 is connected to a GPS receiver 21, a vehicle speed sensor 22, a gyro sensor 23, and a map DB (Data Base) 24 through a CAN (controller area network) or a dedicated line. The control unit 20 is a computer in which a CPU, a RAM, a ROM, a non-volatile memory, and an availability interface are connected by an internal bus, and the CPU executes a program or is a position detection unit realized by a predetermined arithmetic circuit 25, a route search unit 26 and a map drawing unit 27.

  The GPS receiver 21 supplements preferably four or more GPS satellites orbiting the earth, and detects the position (latitude / longitude / altitude) of the host vehicle based on the arrival time of radio waves from each GPS satellite. To do. Further, the vehicle speed sensor 22 shapes the change in magnetic flux when the convex portions installed at regular intervals pass on the circumference of the rotor provided in each wheel of the vehicle into a pulse waveform, and the pulse per unit time The wheel speed is measured for each wheel based on the number. The vehicle speed can be obtained by multiplying the wheel speed by a correction value determined according to the outer diameter of the tire and the interval between the convex portions. The gyro sensor 23 is, for example, a vibration piece type gyro sensor formed by micromachining. The acceleration when the vehicle rotates in the yawing direction with respect to the road surface is taken out as a voltage signal due to the change between the electrodes, and this is integrated. Convert to angle, that is, direction of travel.

  The map DB 24 stores road map information in association with position information such as latitude and longitude. The road map information is a table-like database in which link information of links constituting a road is associated with node information of nodes (intersections) connecting the links. Since the link information includes the link length, width, connection node, connection direction, and the like, the road shape can be detected from the road map information. The map DB 24 stores road types such as automobile roads and general roads for each link, and location information of facilities such as parking lots, restaurants, restaurants with drive-through, gas stations, and stations. Yes.

  The position detection unit 25 accumulates the travel distance detected by the vehicle speed sensor 22 in the direction detected by the gyro sensor 23 with the vehicle position information detected by the GPS receiver 21 as a starting point, thereby accurately determining the position of the vehicle. presume. In addition, you may correct | amend the position of the own vehicle by the map matching which matches the position of a vehicle with the most probable road with respect to the estimated position using the road map around a vehicle.

  The route search unit 26 searches for a route from the current position to the destination based on the position of the host vehicle (current position) and the destination input from the operation unit 51. The route search unit 26 extracts a link that can be a route to the destination, replaces each link with a cost according to the link length, right / left turn, width, traffic regulation, etc., and the route with the smallest integrated value of these costs. Is determined as the route to the destination.

  The map drawing unit 27 generates map screen information to be displayed on the display 52. The map drawing unit 27 extracts road map information including the current position and destination from the map DB 24, forms a road network according to the link length, the connection direction between the links, the width, the road type, and the like. Map screen information is generated by arranging a car icon, a destination icon at the destination, and an icon according to the facility. Predetermined color information is set for urban areas, rivers, seas, mountains, etc. other than links and roads.

[Voice recognition unit 300]
The voice recognition unit 300 will be described. A known technique can be used for voice recognition. Speech recognition is performed by a procedure of referring to various dictionaries and extracting text data from feature amounts obtained by performing acoustic analysis on speech information. The speech recognition unit 300 includes a text conversion unit 32 and a text analysis unit 33 that are executed by a CPU or implemented by a predetermined arithmetic circuit.

  A plurality of sound collecting microphones 31 are preferably arranged on the front bumper, the rear bumper, the left and right door mirrors, etc. to collect sound information outside the vehicle. The sound information collected by the sound collection microphone 31 is converted into an electrical signal and input to the text conversion unit 32. The text converter 32 performs, for example, Fourier transform on the electric signal to extract a feature amount effective for speech recognition. This feature value is obtained by extracting a sound effective for human hearing to understand voice information.

  The text analysis unit 33 is an acoustic model DB that indicates the correspondence between feature quantities and phonetic symbols by probability, a word DB that stores words that can be recognized in association with phonetic symbols, and a series of conversations from several word candidates. It has a language model DB that stores words that are often used in the database.

  The text conversion unit 32 refers to the acoustic model DB based on the feature amount obtained by acoustic analysis, and extracts a series of phonetic symbols that can be candidates. Since there are usually a plurality of candidate phonetic symbols, the text conversion unit 32 refers to the word DB and extracts words associated with the plurality of phonetic symbols. Therefore, even if the voice information has a similar pronunciation, a word in which no word is registered (it does not exist and the frequency of use is extremely low) is not extracted.

  Next, the text conversion unit 32 extracts words before and after the voice information includes and refers to the language model DB. Since the voice information has a number of words in time series, and each has a plurality of candidates, a combination between words in the time series is extracted, and candidate words that are not used in a series of conversations are excluded. Through the processing as described above, the text conversion unit 32 converts the voice information collected by the sound collection microphone 31 into text data. The text data is sent to the text analysis unit 33.

  In addition, when the communication apparatus 54 receives service screen information, if the service mode is transmitted together with the service screen information, words to be recognized can be narrowed down based on the service mode.

  The text analysis unit 33 detects the dependency relationship from the text data based on the analysis result of the morphological analysis for the input text data. In the morphological analysis, for example, an independent word and an attached word are extracted from words included in the text data with reference to the dictionary. In dependency analysis, clauses included in text data are analyzed, and the relationship between clauses in the relationship between dependency and dependency is extracted. As a result, words included in the text data and dependency information between the words are obtained. Part of speech can also be specified in general text analysis.

  As described above, words (particularly independent words) and dependency relationships can be acquired from the voice information collected by the sound collecting microphone 31 (hereinafter, data obtained by recognizing the voice information is referred to as text data, The text data shall include dependency-related information.)

[Service screen display section 400]
The service screen display unit 400 will be described. The service screen display unit 400 has a phase in which the service screen is displayed as it is on the display 52 based on the service screen information received by the communication device 54 and a phase in which the service screen reflecting the contents of the text data is displayed. The service screen display unit 400 includes a service determination unit 41, a reflection method determination unit 43, a voice information reflection unit 45, a selection prohibition unit 46, and a service screen generation that are executed by a CPU or implemented by a predetermined arithmetic circuit. Unit 47 and screen control unit 48. In addition, a service text DB 42 and a reflection information DB 44 are stored in the memory.

<Phase in which the service screen is displayed as it is on the display 52>
For example, the communication device 54 communicates with a communication device on the other side by a relatively narrow narrow range communication (DSRC (Dedicated Short Range Communication)) within a predetermined distance. For example, when a vehicle is equipped with an ETC (Electronic Toll Collection System) device for communicating with a communication device provided at a toll booth on a motorway, the communication device 54 can also be used. The communication device 54 automatically starts communication when entering the communication area of the service provider, performs ASK modulation or QPSK modulation on the 5.8 GHz band carrier wave, extracts the baseband signal, decodes it, and then converts the digital data. get. Then, a predetermined protocol (for example, TCP / IP) process is performed on the digital data, and service screen information is received. Note that any communication method such as wireless LAN, Bluetooth, and wireless USB may be used.

Service screen information The service screen information is preferably transmitted in a versatile format such as an HTML or XML file. If the service screen information is transmitted in such a format, the service screen generation unit 47 can generate a service screen by a browser engine used when displaying a homepage or the like on the display 52, and thus the cost can be suppressed.

  The service screen may be transmitted in a format such as a text format, bitmap data, JPEG data, or MPEG data. The text format or bitmap data which is a general format can be displayed on the display 52 as it is, and even if it is JPEG data or MPEG data, it is easy to obtain a decoder. It is possible to prevent the display from being impossible.

  Incidentally, the communication device 54 may receive the service screen information and the position information on the service screen and the operation input information associated with the position information. This is because when the display 52 is a touch panel, the operation input information is detected according to the position information touched by the passenger. The service screen generation unit 47 specifies operation input information based on the position information on the display 52 touched by the user, and the operation input information is passed through the communication device 54 according to the user's instruction (for example, touching the “confirm button”). To the service provider. Therefore, when a product or food or drink is displayed on the display 52 by the service screen and the occupant touches the screen to select it, operation input information of the product or food or drink selected by the occupant can be transmitted to the service provider.

Screen Switching The screen control unit 48 switches the screen displayed on the display 52 between the map screen generated by the map drawing unit 27 and the service screen generated by the service screen generation unit 47. When the VICS receiver receives traffic information from, for example, VICS (Vehicle Information and Communication System), the screen control unit 48 preferentially switches the screen displayed on the display 52 to traffic information, or is used for occupant operations. The screen is switched according to the usage mode of the information providing apparatus 100 such as a menu screen, a music or video playback screen, or the like.

  In the case of the map screen and the service screen, the screen control unit 48 displays the map screen on the display 52 in the initial state, and when the communication device 54 receives the service screen information, the screen control unit 48 preferentially switches to the service screen information. After that, when the occupant inputs an operation to return to the map screen from the operation unit 51 or a predetermined time elapses, the map screen is automatically returned. Note that the service screen may be superimposed on the map screen instead of completely switching between the map screen and the service screen. If the service screen is not so large or can be reduced, the user can view the map screen while checking the service screen. In addition, when displaying in a superimposed manner, the screen on the front side (for example, a service screen) may be translucently color-converted.

<Phase to display the service screen reflecting the contents of text data>
Service Mode First, the service mode determination will be described. As described above, when the service mode is transmitted together with the service screen information when the communication device 54 receives the service screen information, it can be used as the service mode as it is.

  If the service mode is not obtained, it is possible to estimate how the text data should be reflected on the service screen by several methods. For example, referring to the map DB 24 based on the position information detected by the position detection unit 25, if it is a parking lot, the service mode is providing a parking lot, and if it is a gas station, the service mode is providing fuel. If it is a restaurant, it can be detected that the service mode is provision of food and drink.

  When the service mode is not transmitted together with the service screen information or the service mode cannot be detected from the position information, the service determination unit 41 determines the service mode with reference to the service text DB 42 storing typical service terms for each service. . FIG. 3 shows an example of service terms associated with service modes. The service determination unit 41 searches the service text DB 42 for words included in the text data, and determines the service mode. For example, if the word “parking” is included in the text data, the service mode is provision of a parking lot, and if the word “welcome” is included, the service mode is provision of food and drinks, If the word “regular” is included, the service mode is provision of fuel, and if the word “license” is included, the service mode is check.

  Thus, by accurately detecting the service mode, i) the display mode of the service screen can be easily changed, and ii) the contents of the text data can be reflected on the service screen according to the service mode. Become.

  For example, when it is determined that the service mode is provided as a parking lot, the reflected expression relating to the place such as “3rd floor” and “A” is detected from the voice information “Please park at A on the third floor”. If there is the same text on the screen, “3rd floor” and “A” can be emphasized. In addition, when the service mode is determined to be provision of food and drink, the reflected expression related to the product name “coffee” is detected from the audio information “How about coffee?” Can emphasize “coffee”. In this case, the reflected expression is registered in the service text DB 42 for each service mode, for example.

  For example, if the service mode is determined, the information providing apparatus 100 can change the service screen more appropriately. For example, the service mode may be simplified when providing a parking lot, and the service mode may be changed to a more conspicuous display mode.

-Change the display mode of the service screen (decide how to reflect)
When the reflected expression is extracted based on the service mode, the reflection method determination unit 42 may determine the reflection method according to the reflection method registered in the service text DB 42, for example. However, if the reflection method is determined based only on the service mode, it may be difficult to appropriately reflect. For example, in the provision of a parking lot, both the voice information “Please park on the third floor” and the voice information “3rd floor is full” emphasize “3rd floor”. It is considered that “3rd floor” on the service screen should not be emphasized when the vehicle is full. The same applies to the provision of food and drinks. In the audio information "How about coffee?" And the audio information "Toy set is out of stock", "Coffee" should be emphasized, but "Toy set" Should not be stressed.

  Therefore, in this embodiment, text data is analyzed to determine a more appropriate display mode changing method. The reflection method determination unit 43 determines a reflection method based on text data including dependency information analyzed by the text analysis unit 33.

  The reflection method determination unit 43 refers to the reflection method registered in the reflection information DB 44 illustrated in FIG. 4 and determines the reflection method. In FIG. 4, a reflection instruction expression, a reflected expression, and a reflection method are registered for each service mode. The reflection instruction expression is an expression for transmitting a request, an instruction, a request, information transmission, and the like included in the text data. For example, “please” corresponds to a request, an instruction, and a request, “out of stock” and “deal” are equivalent to information transmission, and “how about” corresponds to a recommendation. The reflection method determination unit 43 first detects that such a reflection instruction expression is included in the text data.

  When there is a reflection instruction expression, the reflection method determination unit 43 determines the reflected expression using the dependency relationship detected by the text analysis unit 33. The reflected expression is a phrase related to the reflection instruction expression or an independent word in the phrase. For example, in the example sentence “please use the parking lot A on the third floor”, the phrase “use parking” is added to the reflection instruction expression “use it”. Therefore, “parking lot” which is an independent word of “parking lot” is a reflected expression. Also, “A parking” takes “A” and “A” takes “3F”. When the dependency relationships are chained in this way, all of them are extracted. Therefore, “parking lot”, “A”, and “3rd floor” are reflected expressions.

  Then, the reflection method determination unit 43 determines a reflection method based on the reflection instruction expression. For example, in a reflection instruction expression for a request such as “please”, the reflected expression is important, so the reflected expression is emphasized. As shown in FIG. 4, since reflection methods for emphasis, recommendation, deletion, inability to select, and information transmission are registered in association with the reflection instruction expression, the display form of the reflected expression in a method appropriate for the reflection instruction expression Can be changed.

Reflecting the contents of text data on the service screen The voice information reflecting unit 45 reflects the contents of the text data on the service screen. FIG. 5 is a diagram illustrating an example of reflection in the reflected expression. The voice information reflecting unit 45 acquires the reflected expression and the reflecting method from the reflecting method determining unit 43, changes the reflected expression to be emphasized, deletes the reflected expression, or recommends the reflected expression. Change the reflected expression. When there are a plurality of reflected expressions, all reflected expressions may be targeted, or a part thereof may be targeted.

  In FIG. 5, it is assumed that all service screens include the reflected expression “◯ △ □”. “◯ △ □” is a part of text data. As shown in FIG. 5A, if the text data is “please go to ○ Δ □” and the reflection method is emphasized, the voice information reflecting unit 45 underlines “○ Δ □”, for example. Also, for example, as shown in FIG. 5B, when the text data is “○ △ □ is recommended” and the reflection method is recommended, the voice information reflection unit 45 surrounds “○ △ □” with □. .

  Also, as shown in FIG. 5C, when the text data is “○ △ □ is limited to women” and the reflection method is strong emphasis, the voice information reflection unit 45 changes the color of “○ △ □”. . It may be displayed in feminine colors. Further, when the reflection method is strong emphasis, as shown in FIG. 5D, “◯ Δ □” is blinked to make “○ Δ □” stand out. In the case of emphasis or recommendation, the reflected expression may be thickened or the font may be changed.

  Further, as shown in FIG. 5E, when the text data is “○ △ □ is out of stock” and the reflection method cannot be deleted or selected, the voice information reflection unit 45 sets “○ △ □”. Delete from the service screen or disable selection. As shown in FIG. 5 (f), when the text data is “○ △ □ is 10% discount” and the reflection method is information transmission, the voice information reflection unit 45 displays “ A supplementary explanation is added and displayed as “10% discount”.

  By the way, in order for the voice information reflecting unit 45 to reflect the contents of the text data, it is necessary to detect the location of the reflected expression “◯ Δ □” on the service screen. In this case, if the service screen information is HTML or XML, the text “◯ Δ □” is included in these files as they are, so that the reflected expression can be easily detected.

  In addition, in order for the voice information reflecting unit 45 to reflect the contents of the text data in the reflected expression, it is necessary to apply a reflection process to the detected reflection target “◯ Δ □” or its surroundings. An example of the reflection is as shown in FIG. 5, but also in this case, when the service screen information is HTML or XML, the reflection process can be easily performed.

  For example, if the reflected expression is underlined, the U element is supported in XML, so <u> ○ △ □ </ u> is used to underline “○ △ □” on the service screen. be able to.

  For example, to surround the reflected expression (change the background color), in HTML and XML, write <SPAN STYLE = "background-color: #XXXX"> ○ △ □ </ SPAN>. The background color of Δ □ can be specified, and can be displayed as surrounded by ○ Δ □ (XXXX is a hexadecimal number specifying the color).

  For example, when changing the color of the reflected expression, the color of “○ △ □” on the service screen can be changed by describing <font color = # XXXX> ○ △ □ </ font> in HTML and XML. it can.

  For example, in the case where the reflected expression is blinked, “○ Δ □” on the service screen can be blinked by describing <blink> ○ Δ □ </ blink> in XML.

  Also, when deleting, the corresponding description of HTML or XML is deleted or commented, and when adding supplementary explanation, description is added. When the selection is disabled, the display may be left as it is or may be deleted from the service screen. However, when the selection is disabled in the displayed state, it is preferable to change the reflected expression to an inconspicuous color or the like.

  On the other hand, when the service screen information is transmitted in bitmap data, JPEG, MPEG, or the like, the reflected expression “◯ Δ □” is detected from the bitmap data, JPEG, or MPEG image data. For example, an OCR (Optical Character Recognition) technique is used for detection. In the case of JPEG or MPEG, first, it is converted into bitmap data. Then, edge processing is performed on the bitmap data, and OCR is performed on the edge image in which the outline of the reflected expression is emphasized. As a result, characters and numbers in the service screen can be extracted. For example, “◯ △ □” in the service screen can be detected by searching for “◯ △ □”.

  Further, when the service screen information is bitmap data, JPEG or MPEG, the audio information reflecting unit 45 cuts out the circumscribed rectangle of △ □□ with a predetermined margin in order to reflect the contents of the text data, A desired reflection process is performed on the cut-out area, and it is arranged at the original position on the service screen.

  For example, in the case of underlining, a pixel having a predetermined width at a position away from the lowermost pixel constituting “◯ Δ □” by a certain distance is replaced with, for example, black. As a result, “◯ Δ □” can be underlined. Since the color of the underline is preferably different from the background color, the color of the underline is, for example, a color separated from the background color by a predetermined distance in the XYZ color system.

  Similarly, if the color of the pixel surrounding “○ △ □” is replaced, “○ △ □” can be surrounded, and if the color of the pixel constituting “○ △ □” is changed, the color of “○ △ □” is changed. It is possible to blink “○ △ □” by switching and displaying the cutout area and the original cutout area with the same color as the background color and the original cutout area every predetermined time. Can do. When deleting, change the color of the pixels that make up “○ △ □” to the same color as the background color, and when adding supplementary explanations, display a character such as “deals” at the bottom of “○ △ □” as a bitmap. To do.

-Prohibition of operation When the service screen generation unit 47 acquires the operation input information together with the service screen information and the reflection method cannot be selected, the selection prohibition unit 46 informs the service screen generation unit 47 of the reflection. Even if there is operation input information for operating the expression (that is, input of position information), it is requested to prohibit the reception of the operation. Therefore, selection of a menu informed of “out of stock” or the like by voice information can be prohibited.

[Parking service screen]
The parking lot service screen will be described more specifically using the above configuration. FIG. 6 is a flowchart showing a procedure for reflecting the contents of the text data on the displayed service screen. The flowchart of FIG. 6 starts when the communication device 54 communicates with a service provider, for example.

  First, the service screen generation unit 47 acquires service screen information from the service provider via the communication device 54 (S10). Upon acquisition, the service screen generation unit 47 interprets the service screen information and sends it to the screen control unit 48, and the screen control unit 48 displays the service screen on the display 52 (S20). When acquiring subsequent service screen information, the service screen information is sequentially sent to the screen control unit 48. If it is a parking lot, a parking lot map or the like as shown in FIG.

  Next, the sound information reflecting unit 45 determines whether or not the sound collecting microphone 31 has collected sound information from the outside (S30).

  For example, when the sound collection microphone 31 receives voice information “Please use A on the third floor” (Yes in S30), the voice recognition unit 300 converts the voice information into text data (S40). Then, the reflection method determination unit 43 refers to the reflection information DB 44 as shown in FIG. 4 and determines whether there is a reflection instruction expression in the text data (S50). If there is a reflection instruction expression, the reflection method determination unit 43 sends the reflected expression and the reflection method to the voice information reflection unit 45.

When the reflection instruction expression is included in the text data (Yes in S50), the voice information reflection unit 45 reflects the reflection expression emphasis on the service screen based on the reflection expression and the reflection method (S60).
Reflection instruction: “Please use”
Reflected expressions: “Parking” “3rd floor” “A”
Reflection Method: Emphasis In FIG. 1C, the reflected expression “A” on the service screen is emphasized. That is, since “parking” and “3rd floor” are not on the service screen, “A” on the service screen is emphasized.

  The occupant can easily grasp the position of the parking lot by visually recognizing the display 52 on which the audio information is reflected as shown in FIG.

  Thereafter, the occupant visually views the service screen on which the voice information is reflected or the service screen on which the voice information is not reflected, for example, parks in the target parking area while viewing the parking lot map, or reports the parking position to the service provider, for example. (S70).

  When the service provider detects parking in the parking area and stops providing the service screen, or when the occupant operates to end the service screen (S80), the processing in FIG. 6 ends.

  As described above, since the information providing apparatus 100 according to the present embodiment can display the audio information that cannot be covered only by the information received by the navigation system or the like on the display 52, the amount of information to be displayed can be improved. In addition, a fair service can be provided to a hearing-impaired person or a passenger with poor language skills. Since the voice-recognized text data is not displayed as it is, information can be provided to foreigners fairly.

[Drive-through service screen]
The drive-through service screen will be specifically described based on the screen transition of FIG. The flowchart is the same as FIG. Service screens in drive-through have many menu screens, and menus can be selected from the service screen.

  When the vehicle arrives at the drive-through, the communication device 54 receives service screen information from a communication unit provided near the entrance (S10). For example, the screen control unit 48 switches the road map screen displayed so far to the service screen. As a result, a service screen is displayed as shown in FIG. 7A (S20). In FIG. 7A, a part of the hamburger shop menu is displayed, and the occupant selects a desired product from the operation unit 51 such as a touch panel.

Before and after the display of the service screen, a clerk of the hamburger shop may utter a voice (S30). Accordingly, the passenger can grasp information such as menus that are not displayed because the limited service or system renovation that is available only in this store has not caught up, through hearing. For example, when the store clerk utters “Toy set is out of stock”, the voice information reflection unit 45 detects the reflection instruction expression “out of stock” from the text data and refers to the reflection information DB 44 to reflect the reflected expression and the reflection method. Is read (S40, S50).
Reflection instruction expression: “Out of stock”
Reflected expression: "Toy set"
Reflection method: "Cannot be selected"
When the selection is disabled, the audio information reflecting unit 45 first changes the reflected expression or the color of a predetermined area surrounding the reflected expression to an inconspicuous color (S60). The inconspicuous color is, for example, the same color as the background color (for example, the background is lighter than gray). As a result, as shown in FIG. 7B, the products of the “toy set” on the service screen are displayed in different colors.

  Therefore, whether or not the product can be selected based on the voice of the store clerk can be reflected in the displayed product, which makes it easier for the occupant to grasp whether or not the product can be purchased.

  Further, the selection prohibition unit 46 prohibits the service screen generation unit 47 from selecting “toy set”. Since the service screen generation unit 47 stores the position information of “toy set”, even if the occupant selects “toy set” from the touch panel or the like, it is ignored. That is, it is not treated as selected. When a product that cannot be selected is selected by the occupant, audio information such as “Toy set is out of stock” may be output from the speaker 53 or may be displayed on the display 52. The occupant can grasp the reason why it cannot be selected again.

  When the occupant selects a desired product from the selectable products (S70), the service screen generation unit 47 detects the menu selected by the occupant from the position information on the service screen, and notifies the service provider via the communication device 54. Send.

  In this way, the contents of the audio information can be reflected also in the display information that is transmitted and received bidirectionally between the service provider and the vehicle.

  As described above, the information providing apparatus 100 according to the present embodiment can make it easier for an occupant to grasp information by voice because the voice information is reflected on the service screen.

It is a figure explaining the outline of an information service device. It is an example of the system block diagram of an information provision apparatus. It is a figure which shows an example of the service term matched with the service aspect. It is a figure which shows an example of reflection information DB. It is a figure which shows the example of reflection to a reflected expression It is a flowchart figure which shows the procedure which reflects the content of text data on the displayed service screen. It is a figure which shows the example of a transition of the service screen in a drive through.

Explanation of symbols

20 Control Unit 31 Sound Collecting Microphone 32 Text Conversion Unit 33 Text Analysis Unit 41 Service Determination Unit 42 Service Text DB
43 Reflection method determination unit 44 Reflection information DB
45 voice information reflecting unit 46 selection prohibiting unit 47 service screen generating unit 48 screen control unit 100 information providing apparatus

Claims (10)

In the information providing apparatus for displaying the service screen corresponding to the service on the display unit,
Sound collecting means for collecting sound information outside the vehicle;
Voice recognition means for recognizing and converting the voice information collected by the sound collection means into text data;
Screen changing means for changing a display mode of the service screen based on the text data;
An information providing apparatus comprising: Service determining means for determining a mode of the service based on current location information;
The screen change means changes a display mode of the service screen based on the service mode.
The information providing apparatus according to claim 1. The screen changing means is a predetermined expression included in the text data, and changes the same predetermined expression included in the service screen or a display mode around it.
The information providing apparatus according to claim 1. The screen changing means is a predetermined expression included in the text data, and deletes the same predetermined expression included in the service screen from the service screen;
The information providing apparatus according to claim 1. The screen changing means changes the predetermined expression or its surrounding display mode to emphasize the predetermined expression.
The information providing apparatus according to claim 3. When the service screen is a parking lot map of the service providing a parking lot,
The screen changing means changes the display mode of the predetermined position of the parking lot map corresponding to the position information included in the text data.
The information providing apparatus according to claim 1. When the service screen is a product menu of the service that provides food and drink in drive-through,
The screen changing means changes the display mode of the product in the product menu corresponding to the information of the product that can be purchased or the product that cannot be purchased included in the text data.
The information providing apparatus according to claim 1. A structure analysis unit for detecting a dependency relationship of the text data;
A change method determining means for detecting a change instruction expression relating to the predetermined expression from the text data based on a dependency relationship;
Change method storage means for storing a change method of a display mode of the predetermined expression in association with the change instruction expression;
The change method determination means reads the change method from the change method storage means based on the change instruction expression detected from the text data,
The screen changing means changes the predetermined expression or its surrounding display mode based on the read change method.
The information providing apparatus according to claim 3. In the information display method of the information providing apparatus for displaying the service screen corresponding to the service on the display unit,
A display unit displaying the service screen;
A step of collecting sound information outside the vehicle;
A step of recognizing the voice information collected by the sound collecting means and converting the voice information into text data;
Screen changing means for changing the service screen based on the text data;
An information display method characterized by comprising: The information providing device according to any one of claims 1 to 8,
Position detection means for detecting the current position;
Route search means for searching for a route from the current position to a predetermined position;
Road map information storage means for storing road map information;
A navigation device comprising:

Images