JP7155195B2 - Information processing device, information processing method and information processing program - Google Patents

Description

The present invention relates to an information processing device, an information processing method, and an information processing program.

A technique is disclosed for constructing a sound statistical database in which sounds in a predetermined space are accumulated for each type.

JP 2016-180791 A

However, the above conventional technology assumes fixed-point observation using a recording device or the like installed in a predetermined space. Therefore, even if it is possible to analyze the environment of a specific place such as the inside of a building, it is not possible to correct the user's position on the map based on the environmental information.

The present application has been made in view of the above, and aims to correct the user's position on the map based on the environmental information.

An information processing apparatus according to the present application includes an acquisition unit that acquires sensor information and position information simultaneously acquired by a user's terminal device having a sensor, and an estimation unit that estimates the context of the terminal device based on the sensor information. a comparison unit that compares the context on the map specified from the location information and the context of the terminal device; a correction unit that corrects the location of the user on the map according to the comparison result ; and , if the result of the comparison is that the context on the map and the context of the terminal device do not match, the comparison unit is configured to determine whether the location information is incorrect or the map information is incorrect. It is characterized by determining whether there is

According to one aspect of an embodiment, the user's position on the map can be corrected based on the environmental information.

FIG. 1 is an explanatory diagram showing an outline of an information processing method according to an embodiment. FIG. 2 is a diagram illustrating a configuration example of an information processing system according to the embodiment; FIG. 3 is a diagram illustrating a configuration example of a terminal device according to the embodiment; FIG. 4 is a diagram illustrating a configuration example of an information providing apparatus according to the embodiment; FIG. 5 is a diagram showing an example of the user information database. FIG. 6 is a diagram showing an example of a sensor information database. FIG. 7 is a diagram showing an example of a map information database. FIG. 8 is a flow chart showing a processing procedure according to the embodiment. FIG. 9 is a diagram illustrating an example of a hardware configuration;

Hereinafter, modes for implementing an information processing apparatus, an information processing method, and an information processing program according to the present application (hereinafter referred to as "embodiments") will be described in detail with reference to the drawings. The information processing apparatus, information processing method, and information processing program according to the present application are not limited to this embodiment. Also, in the following embodiments, the same parts are denoted by the same reference numerals, and overlapping descriptions are omitted.

[1. Outline of information processing method]
First, an outline of an information processing method performed by an information processing apparatus according to an embodiment will be described with reference to FIG. FIG. 1 is an explanatory diagram showing an outline of an information processing method according to an embodiment. In addition, in FIG. 1, the case where the position on a map is corrected based on environmental information is mentioned as an example, and is demonstrated.

As shown in FIG. 1 , the information processing system 1 includes a terminal device 10 and an information providing device 100 . The terminal device 10 and the information providing device 100 are communicably connected to each other by wire or wirelessly via a network N (see FIG. 2).

The terminal device 10 is a smart device such as a smartphone or tablet used by the user U, and can communicate with any server device via a wireless communication network such as 4G (Generation) or LTE (Long Term Evolution). It is a portable terminal device that can The terminal device 10 has a screen such as a liquid crystal display, which has a touch panel function. accepts the operation of An operation performed on an area where content is displayed on the screen may be an operation on the content. In addition, the terminal device 10 may be an information processing device such as a desktop PC (Personal Computer) or a notebook PC as well as a smart device. Here, a smartphone is used as an example.

The information providing device 100 acquires sensor information (sensor data) output from various sensors (sensors) mounted on the terminal device 10, user information, and position information from the terminal device 10 of the user U. It is a processing device, and is realized by a server device, a cloud system, or the like. Here, the information providing device 100 acquires sensor information, user information, and location information from each of an unspecified number of terminal devices 10, associates the sensor information, user information, and location information, and stores them in a database. . Then, the information providing device 100 corrects the position of the user U on the map based on the sensor information and the position information.

The sensor information is, for example, environmental sounds, atmospheric pressure, illuminance, and the like. User information is identification information or the like for uniquely identifying the user U or the terminal device 10 . The information providing apparatus 100 does not need to acquire user information when it is not necessary to specify a user and only sensor information and position information are sufficient.

For example, the information providing device 100 compares the context on the map specified from the position information of the user U (position information on the terminal device 10) and the context of the terminal device 10 based on the sensor information, and compares the context on the map. If the context of the terminal device 10 does not match, the position information of the user U on the map is corrected to a location where the context of the terminal device 10 and the context on the map match. Here, the context is the surrounding environment, situation, or the like. Also, the position information of the user U on the map may be displayed on the screen of the terminal device 10 .

Specifically, although the context on the map specified from the location information of the user U is "a place where you should not hear the flow of the river", the information providing apparatus 100 detects the terminal device based on the sensor information. In context 10, if "you can hear the river flowing", then the location information of the user U is corrected to the place where you can "hear the river flowing" on the map.

As a specific correction method, the information providing apparatus 100 estimates from the sound volume of "river flow" that it is "a place 3 m away from both edges (both banks) of the river", and "from the current position to the nearest river". The position information of the user U is corrected to the point of contact between the line connecting the two with a straight line distance and the place 3 m from the nearest bank of both banks of the river. Consistency with physical constraints such as elevation (cliffs, valleys, etc.) may also be reflected in the correction. Also, based on the ``sound of the river'', it is simply estimated to be ``a place within 3m from both edges (both banks) of the river'', and ``a straight line connecting the current position to the nearest river'' and ``both banks of the river''. The location information of the user U may be corrected to the point of contact with "the place 3 m from the nearest shore".

Rivers are just one example. In reality, the seaside and lakes where you can hear the sound of waves, valleys with strong wind blowing sounds, roads (general roads, highways, etc.) and railways where vehicles are running, construction sites under construction, background music (BGM) ) or inside a store where announcements are being made, as long as it is a place or object that generates a characteristic sound. In addition to or instead of environmental sounds, it is also possible to estimate "inside a forest", "inside a tunnel", "inside a building", "inside a moving body", etc. from the illuminance and luminosity.

[1-1. Outline of information processing method from data collection to correction of position on the map]
First, with reference to FIG. 1, an outline of an information processing method from data collection to correction of a position on a map will be described.

As shown in FIG. 1, the information providing device 100 acquires sensor information, user information, and location information from each of the terminal devices 10 of a plurality of users U located in various places (step S1).

At this time, the terminal device 10 may be moving or staying at one place. Also, the information providing apparatus 100 may receive sensor information, user information, and location information from the same terminal apparatus 10 that moves from place to place. The sensor information is, for example, environmental sounds, atmospheric pressure, illuminance, and the like.

Here, the location information is the location information of the user U, and is the location information when the terminal device 10 acquires the sensor information. Position information also corresponds to sensor information in a broad sense. The sensor information and location information also include information about the date and time when the terminal device 10 acquired the sensor information and location information.

Then, the information providing apparatus 100 classifies (categorizes) the sensor information by type (step S2).

For example, the information providing apparatus 100 extracts environmental sounds from the sounds included in the sensor information, and classifies (categorizes) the environmental sounds by sound type. The types of sound are, for example, the sound of river flow, the sound of waves, the running sound of trains and automobiles, and the like.

Then, the information providing apparatus 100 associates and stores the sensor information, the user information, and the position information received from the terminal device 10 (step S3).

For example, the information providing apparatus 100 associates sensor information, user information, and location information, accumulates them in a database, and holds them for a predetermined period (for example, one week/one month/half a year/one year). Note that, in practice, accumulation may be continued without specifying a period.

When the environmental sounds are classified by sound type, the information providing apparatus 100 stores the user information and the position information in association with each sound type.

The information providing device 100 then analyzes the sensor information and estimates the context of the terminal device 10 (step S4).

For example, when the environmental sound included in the sensor information includes the sound of an object R that emits a sound, the information providing apparatus 100 sets the context of the terminal device 10 to an environment or situation in which "the sound of R can be heard." Assume there is. Note that the object R that emits sound is, for example, a river, a road, a railroad, or the like.

Then, the information providing device 100 compares the context on the map specified from the position information of the user U and the context of the terminal device 10 based on the sensor information (step S5).

Then, when the context on the map does not match the context of the terminal device 10 as a result of the comparison, the information providing device 100 corrects the positional information of the user U on the map (step S6).

For example, when the context on the map and the context of the terminal device 10 do not match, the information providing device 100 sets the location information of the user U on the map to Correct for location.

Specifically, although the context on the map specified from the position information of the user U is "a place where the sound of R should not be heard", the information providing apparatus 100 detects the terminal device based on the sensor information. In context 10, if "the sound of R is heard", the position information of the user U is corrected to the estimated actual position P (estimated position), which is the place where the "sound of R is heard".

Here, the information providing apparatus 100 estimates "a place at a distance D from R" from "the sound of R", and "a line connecting the current position to R with a straight line distance" and "a place at a distance D from R". The actual position P is estimated to be the point of contact with the place where the user U is located, and the position information of the user U is corrected to the estimated actual position P. The distance D can be set arbitrarily. For example, the distance D may be 3m.

In addition, although the context on the map specified from the position information of the user U is "a place where echoes should not be heard", the information providing apparatus 100 determines that the context of the terminal device 10 based on the sensor information is If "the reverberating sound is heard", the positional information of the user U is corrected to the inside of the nearest tunnel, cave, structure, or the like from the current position.

Then, the information providing device 100 provides the corrected position information to the terminal device 10 of the user U whose position information has been corrected (step S7).

Alternatively, the information providing apparatus 100 corrects the map information as necessary when the corresponding location or object does not exist on the map as a result of the comparison (step S8).

For example, although the context on the map specified from the position information of the moving user U is "a place where R does not exist on the map", the information providing apparatus 100 determines whether the terminal device 10 based on the sensor information In the context, if the situation is "the sound of R is heard", the map information is corrected, and an icon or the like indicating the object R is added at a position of the distance D from the position information of the user U on the map. Alternatively, the information providing apparatus 100 changes the current map information to map information including the object R on the map, and corrects the position information of the user U to the position of the distance D from the object R on the map. . The information providing apparatus 100 may correct map information provided on a map search site or the like.

In this case, the position information of the user U on the map is corrected from "a place where R does not exist on the map" to "a place where R exists on the map". That is, the information providing apparatus 100 relatively corrects the location information of the user U by correcting the map information.

Then, the information providing device 100 provides the corrected map information to the terminal device 10 of the user U whose position information is displayed on the map (step S9).

[2. Configuration example of information processing system]
Next, the configuration of the information processing system 1 including the information providing device 100 according to the embodiment will be described with reference to FIG. FIG. 2 is a diagram showing a configuration example of the information processing system 1 according to the embodiment. As shown in FIG. 2, the information processing system 1 according to the embodiment includes a terminal device 10 and an information providing device 100. As shown in FIG. These various devices are communicatively connected via a network N by wire or wirelessly. The network N is, for example, a LAN (Local Area Network) or a WAN (Wide Area Network) such as the Internet.

Further, the number and types of devices included in the information processing system 1 shown in FIG. 2 are not limited to those illustrated. For example, in FIG. 2, only one type of terminal device 10 is shown for simplification of illustration, but this is only an example and is not limited, and two or more types may be used.

The terminal device 10 is an information processing device used by the user U, and is a small portable or mobile device that has various sensors and can be carried or moved. For example, the terminal device 10 is a smart device such as a smart phone or a tablet terminal. In fact, feature phones, PCs (Personal Computers), PDAs (Personal Digital Assistants), car navigation systems, wearable devices such as smart watches and head-mounted displays, smart glasses, smart speakers, 360-degree cameras etc.

In addition, the terminal device 10 is compatible with wireless communication networks such as LTE (Long Term Evolution), 4G (4th Generation), 5G (5th Generation: fifth generation mobile communication system), Bluetooth (registered trademark), wireless LAN (Local It is possible to communicate with the information providing apparatus 100 by connecting to the network N via short-range wireless communication such as Area Network).

The information providing device 100 is, for example, a PC, a server device, a mainframe, a workstation, or the like. Note that the information providing apparatus 100 may be realized by cloud computing.

[3. Configuration example of terminal device]
Next, the configuration of the terminal device 10 will be described using FIG. FIG. 3 is a diagram showing a configuration example of the terminal device 10. As shown in FIG. As shown in FIG. 3, the terminal device 10 includes a communication unit 11, a display unit 12, an input unit 13, a positioning unit 14, a sensor unit 20, a control unit 30 (controller), and a storage unit 40. Prepare.

(Communication unit 11)
The communication unit 11 is connected to the network N (see FIG. 2) by wire or wirelessly, and transmits and receives information to and from the information providing apparatus 100 via the network N. FIG. For example, the communication unit 11 is implemented by a NIC (Network Interface Card), an antenna, or the like.

(Display unit 12)
The display unit 12 is a display device that displays various information such as position information. For example, the display unit 12 is a liquid crystal display (LCD) or an organic EL display (Organic Electro-Luminescent Display). Also, the display unit 12 is a touch panel display, but is not limited to this.

(Input unit 13)
The input unit 13 is an input device that receives various operations from the user U. FIG. The input unit 13 has, for example, buttons and the like for inputting characters, numbers, and the like. Moreover, when the display unit 12 is a touch panel display, a part of the display unit 12 functions as the input unit 13 . Note that the input unit 13 may be a microphone or the like that receives voice input from the user U. FIG. The microphone may be wireless.

(Positioning unit 14)
The positioning unit 14 receives signals (radio waves) transmitted from GPS (Global Positioning System) satellites, and based on the received signals, position information (for example, latitude and longitude). That is, the positioning unit 14 positions the position of the terminal device 10 . GPS is merely an example of GNSS (Global Navigation Satellite System).

Also, the positioning unit 14 can measure the position by various methods other than GPS. For example, the positioning unit 14 may measure the position using various communication functions of the terminal device 10 as described below as auxiliary positioning means for position correction and the like.

(Wi-Fi positioning)
For example, the positioning unit 14 measures the position of the terminal device 10 using the Wi-Fi (registered trademark) communication function of the terminal device 10 or the communication network provided by each communication company. Specifically, the positioning unit 14 performs Wi-Fi communication or the like and measures the position of the terminal device 10 by measuring the distance to a nearby base station or access point.

(beacon positioning)
The positioning unit 14 may also use the Bluetooth (registered trademark) function of the terminal device 10 to measure the position. For example, the positioning unit 14 measures the position of the terminal device 10 by connecting with a beacon transmitter connected by the Bluetooth function.

(geomagnetic positioning)
Further, the positioning unit 14 positions the position of the terminal device 10 based on the geomagnetism pattern of the structure measured in advance and the geomagnetic sensor provided in the terminal device 10 .

(RFID positioning)
Further, for example, if the terminal device 10 has an RFID (Radio Frequency Identification) tag function equivalent to a contactless IC card used at station ticket gates, stores, etc., or has a function of reading an RFID tag In this case, the location used is recorded together with the information that the payment was made by the terminal device 10 . The positioning unit 14 may measure the position of the terminal device 10 by acquiring such information. Also, the position may be measured by an optical sensor provided in the terminal device 10, an infrared sensor, or the like.

The positioning unit 14 may measure the position of the terminal device 10 using one or a combination of the positioning means described above, if necessary.

(Sensor unit 20)
The sensor unit 20 includes various sensors mounted on the terminal device 10 . In the example shown in FIG. 3, the sensor unit 20 includes an acceleration sensor 21, a gyro sensor 22, an atmospheric pressure sensor 23, an air temperature sensor 24, a sound sensor 25, an optical sensor 26, a magnetic sensor 27, and an image sensor ( camera) 28.

The sensors 21 to 28 described above are only examples and are not limited. That is, the sensor unit 20 may be configured to include a part of the sensors 21 to 28, or may include other sensors such as a humidity sensor in addition to or instead of the sensors 21 to 28. .

The acceleration sensor 21 is, for example, a three-axis acceleration sensor, and detects physical movements of the terminal device 10 such as movement direction, speed, and acceleration of the terminal device 10 . The gyro sensor 22 detects physical movements of the terminal device 10 such as inclination in three axial directions based on the angular velocity of the terminal device 10 and the like. The atmospheric pressure sensor 23 detects the atmospheric pressure around the terminal device 10, for example.

Since the terminal device 10 includes the above-described acceleration sensor 21, gyro sensor 22, barometric pressure sensor 23, etc., techniques such as pedestrian dead-reckoning (PDR: Pedestrian Dead-Reckoning) using these sensors 21 to 23, etc. , the position of the terminal device 10 can be determined. This makes it possible to acquire indoor position information that is difficult to acquire with a positioning system such as GPS.

For example, a pedometer using the acceleration sensor 21 can calculate the number of steps, walking speed, and distance walked. In addition, by using the gyro sensor 22, it is possible to know the traveling direction, the direction of the line of sight, and the inclination of the body of the user U. Also, from the atmospheric pressure detected by the atmospheric pressure sensor 23, the altitude at which the terminal device 10 of the user U is present and the number of floors can be known.

The temperature sensor 24 detects the temperature around the terminal device 10, for example. The sound sensor 25 detects sounds around the terminal device 10, for example. The optical sensor 26 detects the illuminance around the terminal device 10 . The magnetic sensor 27 detects, for example, geomagnetism around the terminal device 10 . The image sensor 28 captures an image around the terminal device 10 .

The atmospheric pressure sensor 23, the temperature sensor 24, the sound sensor 25, the optical sensor 26, and the image sensor 28 described above detect the atmospheric pressure, temperature, sound, and illuminance, respectively, or capture an image of the surroundings to detect the terminal device 10. It is possible to detect the surrounding environment and situations. In addition, it is possible to improve the accuracy of the location information of the terminal device 10 based on the surrounding environment and situation of the terminal device 10 .

(control unit 30)
The control unit 30 includes, for example, a microcomputer having a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM, an input/output port, and various circuits. Also, the control unit 30 may be configured by hardware such as an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). The control unit 30 includes a transmission unit 31 , a reception unit 32 and a processing unit 33 .

(Sending unit 31)
The transmission unit 31 receives, for example, various information input by the user U using the input unit 13, various information detected by the sensors 21 to 28 mounted on the terminal device 10, and the terminal device positioned by the positioning unit 14. 10 can be transmitted to the information providing device 100 via the communication unit 11 .

(Receiver 32)
The receiving unit 32 can receive various types of information provided by the information providing device 100 and requests for sensor information from the information providing device 100 via the communication unit 11 .

(Processing unit 33)
The processing unit 33 controls the entire terminal device 10 including the display unit 12 and the like. For example, the processing unit 33 can output various types of information transmitted by the transmitting unit 31 and various types of information received by the receiving unit 32 from the information providing apparatus 100 to the display unit 12 for display.

(storage unit 40)
The storage unit 40 is realized by, for example, a semiconductor memory device such as RAM (Random Access Memory) or flash memory, or a storage device such as HDD (Hard Disk Drive), SSD (Solid State Drive), or optical disk. be. Various programs, various data, and the like are stored in the storage unit 40 .

[4. Configuration example of information providing device]
Next, the configuration of the information providing device 100 according to the embodiment will be described using FIG. FIG. 4 is a diagram showing a configuration example of the information providing device 100 according to the embodiment. As shown in FIG. 4, the information providing device 100 has a communication section 110, a storage section 120, and a control section .

(Communication unit 110)
The communication unit 110 is realized by, for example, a NIC (Network Interface Card) or the like. Also, the communication unit 110 is connected to the network N (see FIG. 2) by wire or wirelessly.

(storage unit 120)
The storage unit 120 is realized by, for example, a semiconductor memory device such as a RAM (Random Access Memory) or flash memory, or a storage device such as a hard disk or an optical disk. As shown in FIG. 4 , storage unit 120 has user information database 121 , sensor information database 122 , and map information database 123 .

(User information database 121)
The user information database 121 stores various information about the user U. FIG. For example, the user information database 121 stores various information such as user U attributes. FIG. 5 is a diagram showing an example of the user information database 121. As shown in FIG. In the example shown in FIG. 5, the user information database 121 has items such as "user ID (Identifier)", "age", "sex", "home", "work place", and "terminal ID".

“User ID” indicates identification information for identifying the user U. FIG. "Age" indicates the age of the user U identified by the user ID. Note that the "age" may be a specific age of the user U identified by the user ID, such as 35 years old. "Gender" indicates the gender of the user U identified by the user ID.

"Home" indicates location information of the home of the user U identified by the user ID. In the example shown in FIG. 5, "home" is represented by an abstract code such as "LC11", but may be latitude/longitude information or the like. Also, for example, "home" may be an area name or an address.

Further, "place of work" indicates location information of the place of work of the user U identified by the user ID (school in the case of a student). In the example shown in FIG. 5, the "place of work" is illustrated as an abstract code such as "LC12", but may be latitude/longitude information or the like. Also, for example, the "place of work" may be an area name or an address.

"Terminal ID" indicates identification information for identifying the terminal device 10 currently used by the user U identified by the user ID (terminal device in operation). It should be noted that the number of "terminal IDs" may be plural. That is, one user U may use a plurality of terminal devices at the same time. Also, the “terminal ID” may include information indicating the type of the terminal device 10 .

For example, in the example shown in FIG. 5, the age of the user U identified by the user ID "U1" is "twenties" and the gender is "male". Also, for example, user U identified by user ID "U1" indicates that home is "LC11". Also, for example, the user U identified by the user ID "U1" indicates that the place of work is "LC12". Also, for example, it indicates that the user U identified by the user ID "U1" uses the terminal device 10 identified by the terminal ID "D1".

Here, in the example shown in FIG. 5, abstract values such as "U1", "LC11", "LC12" and "D1" are used, but "U1", "LC11", "LC12" and " D1” stores information such as specific character strings and numerical values. Hereinafter, abstract values may also be illustrated in diagrams relating to other information.

The user information database 121 is not limited to the above, and may store various types of information depending on the purpose. For example, the user information database 121 may store various information about the user U's terminal device 10 . In addition, the user information database 121 is related to attributes of the user U, such as demographics (demographic attributes), psychographics (psychological attributes), geographic (geographical attributes), and behavioral (behavioral attributes). Information may be stored. For example, the user information database 121 includes name, family structure, occupation, job title, income, qualification, residence type (detached house, condominium, etc.), presence or absence of car, commute time, commute route, commuter pass section (station , routes, etc.), stations with high frequency of use (other than the nearest station to home/work place), behavior history (location information with high frequency of use), lessons (place, time of day, etc.), hobbies, interests, lifestyle, etc. Information may be stored. The user information database 121 may also store information on search queries (search keywords) that the user U has entered into a search engine or the like.

(Sensor information database 122)
The sensor information database 122 stores various information regarding sensor outputs. For example, the sensor information database 122 associates and stores sensor information, user information, and position information from the user U's terminal device 10 . FIG. 6 is a diagram showing an example of the sensor information database 122. As shown in FIG. In the example shown in FIG. 6, the sensor information database 122 has items such as "user ID", "acquisition date and time", "sensor information", "position information", and "context".

“User ID” is user information acquired from the terminal device 10 and indicates identification information for identifying the user U who uses the terminal device 10 . Note that the “user ID” may be identification information for identifying the terminal device 10 . That is, the "user ID" may be the "terminal ID".

“Acquisition date and time” indicates the date and time when the terminal device 10 acquired the sensor information, the user information, and the position information. Note that when the information providing apparatus 100 acquires the sensor information, the user information, and the location information from the terminal device 10 in real time, the information providing apparatus 100 acquires the sensor information, the user information, and the location information from the terminal device 10. It may be the date and time of acquisition. Also, the "acquisition date and time" is not limited to the year, month, day, hour, minute, and second, and may include information related to the day of the week, daytime/nighttime, and the like.

“Sensor information” indicates outputs (detection results) of various sensors of the terminal device 10 . "Sensor information" is, for example, environmental sound, air pressure, illuminance, and the like. The environmental sounds are classified (categorized) by sound type and stored.

“Positional information” indicates positional information at the time when the terminal device 10 acquires the sensor information. "Positional information" is positional information determined by GPS, PDR, or the like, for example.

“Context” indicates the environment and situation around the terminal device 10 estimated based on the sensor information. The "context" indicates, for example, "I can hear the flow of the river", "I can hear echoes", and "I can hear the sound of passing trains".

For example, in the example shown in FIG. 6, the terminal device 10 of the user U identified by the user ID "U1" has the sensor information "sensor information #11" and the location information "location information" in the acquisition date "acquisition date #A". #11” is acquired. Also, it indicates that the user U identified by the user ID "U1" is placed in the environment and situation indicated by the context "context #11".

Note that the sensor information database 122 may store various types of information, not limited to the above, depending on the purpose. For example, the sensor information database 122 may store information regarding expiration dates for storing sensor information. In addition, the sensor information database 122 may store information related to sensor information analysis results (environmental evaluation results). Further, the sensor information database 122 may store information related to comments about the situation at the time of sensor information acquisition input by the user U of the terminal device 10 . The comment may be text or voice.

(Map information database 123)
The map information database 123 stores various types of map-related information. FIG. 7 is a diagram showing an example of the map information database 123. As shown in FIG. In the example shown in FIG. 7, the map information database 123 has items such as "map ID", "map information", "user ID", "user position", and "context".

"Map ID" indicates identification information for identifying a map. Note that the map may be a map or a newly generated map.

"Map information" indicates map information provided to the user U in response to a request from the user U. Here, “map information” indicates map information in which user U's position information is displayed.

"User ID" indicates identification information for identifying the user U whose current position is displayed on the map. Note that the “user ID” may be identification information for identifying the terminal device 10 . That is, the "user ID" may be the "terminal ID".

"User position" indicates position information of the user U displayed on the map. The “user position” is position information obtained by positioning using GPS, PDR, or the like, for example.

"Context" indicates the surrounding environment and situation of the user U's current position on the map. The 'context' indicates, for example, 'a place where the flow of the river should not be heard', 'a place where echoes should not be heard', 'a place where the sound of passing trains can be heard', and the like.

For example, in the example shown in FIG. 7, the map identified by the map ID "U1" is the user position "U1" indicating the current position of the user U identified by the user ID "U1" in the map information "map information #1". Position information #1” is displayed, indicating that the environment and situation are indicated by the context “context #11”.

It should be noted that the map information database 123 may store various types of information, not limited to the above, depending on the purpose. For example, the map information database 123 may store area names indicated by the map, information on topography and geography on the map, other detailed information, supplementary information, and the like. In addition, the map information database 123 may store comments on the map input by the user U, and the like.

(control unit 130)
Returning to FIG. 4, the description is continued. The control unit 130 is a controller, and for example, a CPU (Central Processing Unit), an MPU (Micro Processing Unit), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or the like controls the information providing apparatus 100. Various programs (corresponding to an example of an information processing program) stored in the internal storage device are executed by using a storage area such as a RAM as a work area. In the example shown in FIG. 4 , the control section 130 has an acquisition section 131 , an accumulation section 132 , an estimation section 133 , a comparison section 134 , a correction section 135 and an output section 136 .

(Acquisition unit 131)
The acquisition unit 131 acquires sensor information (sensor data), user information, and location information from each of the terminal devices 10 of multiple users U located in various locations via the communication unit 110 . Note that if there is no need to specify the user and only the sensor information and the position information are sufficient, the user information does not need to be acquired.

(Collecting unit 132)
The accumulation unit 132 associates the sensor information, the user information, and the location information received from the terminal device 10 and accumulates them in a database. Note that the accumulation unit 132 may be part of the acquisition unit 131 .

Further, when the sensor information received from the terminal device 10 includes sound, the accumulation unit 132 extracts only the environmental sound from the sound. For example, the accumulation unit 132 may filter sounds included in sensor information to extract only environmental sounds. The accumulation unit 132 also classifies (categorizes) the environmental sounds by sound type. That is, the accumulation unit 132 also functions as an environmental sound extraction unit and a classification unit.

Note that the accumulation unit 132 may extract environmental sounds from sounds included in the sensor information received from the terminal device 10 using a model based on machine learning. In this case, the accumulation unit 132 may cause the model to learn sets of sounds included in the sensor information received from the terminal device 10 and environmental sounds categorized by type of sound, for example, as correct data. Then, the accumulation unit 132 extracts environmental sounds classified by sound type by inputting sounds included in the sensor information to the trained model. That is, the accumulation unit 132 may use the environmental sound output by the model when the sound included in the sensor information is input as the extraction result.

Any type of model can be adopted as the model. For example, the accumulation unit 132 may adopt SVM (Support Vector Machine) or DNN (Deep Neural Network) as a model. Here, the DNN may be a CNN (Convolutional Neural Network) or an RNN (Recurrent Neural Network). Also, the RNN may be LSTM (Long short-term memory) or the like. That is, any model can be adopted as the model. The model may also be a model realized by combining a plurality of models, such as a model combining CNN and RNN.

Learning is, for example, deep learning using a deep neural network (DNN: Deep Neural Network). Data mining and other machine learning algorithms may also be used. The accumulation unit 132 learns the model using the various learning methods described above.

(Estimation unit 133)
The estimation unit 133 analyzes sensor information and estimates the context of the terminal device 10 . For example, when the environmental sound included in the sensor information includes the sound of a river flowing, the estimating unit 133 estimates that the context of the terminal device 10 is an environment or situation in which "a river can be heard". .

Note that, basically, the context on the map is identified from the map information and the user's U location information. However, the estimation unit 133 may analyze the map information and estimate the context on the map from the user U's location information.

(Comparator 134)
The comparison unit 134 compares the context on the map specified from the position information of the user U and the context of the terminal device 10 based on the sensor information.

If the comparison result shows that the context on the map matches the context of the terminal device 10, the comparison unit 134 determines that the position information of the user U on the map is correct, and maintains the current status.

Further, when the comparison result shows that the context on the map and the context of the terminal device 10 do not match, the comparison unit 134 determines whether or not it is necessary to correct the position information of the user U on the map. Specifically, when the context on the map and the context of the terminal device 10 do not match, the comparison unit 134 determines that the location information of the user U or the map information is incorrect. to judge whether

Note that when the comparison unit 134 determines whether the error is in the position information of the user U or in the map information, the comparison unit 134 determines whether the position information of the user U or the map information is incorrect. may be determined based on a set of sensor information and position information.

(Correction unit 135)
When it is determined as a result of the comparison by the comparison unit 134 that the position information of the user U is incorrect, the correction unit 135 corrects the position information of the user U on the map. For example, the correction unit 135 corrects the position information of the user U on the map to a place where the context of the terminal device 10 and the context on the map match.

Further, when it is determined as a result of the comparison by the comparison unit 134 that the map information is erroneous, the correction unit 135 corrects the map information. For example, the correction unit 135 corrects the context of the terminal device 10 based on the sensor information even though the context on the map specified from the position information of the moving user U is “a place where no river exists on the map”. Then, if the situation is such that "the flow of the river can be heard", the map information is corrected, and an icon or the like indicating the river is added within a predetermined range from the position information of the user U on the map. Alternatively, the information providing apparatus 100 changes the current map information to map information including the river on the map, and corrects the position information of the user U to a predetermined distance from the river on the map. In this case, the correction unit 135 may generate a new map or update an existing map. For example, the correction unit 135 may correct map information provided on a map search site or the like. Also, the correction unit 135 may be a part of the correction unit 135 .

(Output unit 136)
The output unit 136 transmits the position information of the user U on the map corrected by the correction unit 135 to the terminal device 10 of the user U via the communication unit 110 . Also, the output unit 136 transmits the map information corrected by the correction unit 135 to the user U's terminal device 10 .

[5. Processing procedure]
Next, a processing procedure performed by the information providing apparatus 100 according to the embodiment will be described with reference to FIG. FIG. 8 is a flow chart showing a processing procedure according to the embodiment. Note that the processing procedure described below is repeatedly executed by the control unit 130 of the information providing apparatus 100 .

As shown in FIG. 8, the information providing device 100 acquires sensor information (sensor data), user information, and location information from each of the terminal devices 10 of a plurality of users U located in various places (step S101).

Then, the information providing apparatus 100 classifies (categorizes) the sensor information by type (step S102).

Then, the information providing apparatus 100 associates the sensor information, the user information, and the position information and stores them in the database (step S103).

For example, the information providing apparatus 100 extracts environmental sounds from the sounds included in the sensor information, classifies (categorizes) the extracted environmental sounds for each sound type, and classifies the classified environmental sounds for each sound type for the user. Information and location information are linked and stored.

The information providing device 100 then analyzes the sensor information and estimates the context of the terminal device 10 (step S104).

Then, the information providing device 100 compares the context on the map specified from the position information of the user U and the context of the terminal device 10 based on the sensor information (step S105).

At this time, if the result of the comparison shows that the context on the map matches the context of the terminal device 10 (step S105; Yes), the information providing device 100 maintains the current state and terminates the process.

When the context on the map and the context of the terminal device 10 do not match as a result of the comparison (step S105; No), the information providing device 100 determines whether it is necessary to correct the position information of the user U on the map. (step S106).

That is, when the context on the map and the context of the terminal device 10 do not match, the information providing device 100 determines whether the location information of the user U or the map information is incorrect. to decide.

Then, when the information providing apparatus 100 determines that there is an error in the position information of the user U and that it is necessary to correct the position information of the user U on the map (step S106; Yes), the position information of the user U on the map is corrected (step S107).

Then, the information providing device 100 provides the corrected position information to the terminal device 10 of the user U whose position information has been corrected (step SS108).

Further, when the information providing apparatus 100 determines that there is an error in the map information and that it is not necessary to correct the position information of the user U on the map (step S106; No), it corrects the map information ( step S109).

Then, the information providing device 100 provides the corrected map information to the terminal device 10 of the user U whose position information is displayed on the map (step SS110).

[6. Modification]
The terminal device 10 and the information providing device 100 described above may be implemented in various different forms other than the above embodiments. So, below, the modification of embodiment is demonstrated.

In the above-described embodiment, the terminal device 10 may be put away in the pocket of the user U's clothes, bag (bag, rucksack, pouch), or the like. Therefore, the terminal device 10 may transmit sensor information to the information providing device 100 only when it is being used by the user U (operation, video viewing, etc.). For example, the information providing apparatus 100 acquires sensor information when the terminal device 10 is running some application program (app), and does not acquire sensor information when no app is running. to Alternatively, the information providing apparatus 100 acquires sensor information when the terminal device 10 is in a non-sleep state, and does not acquire sensor information when the terminal device 10 is in a sleep state. Note that "does not acquire sensor information" includes "does not store sensor information even if acquired".

In addition, the information providing apparatus 100 acquires the operation history of the terminal device 10 and the like together with the sensor information. is stored in the pocket of user U's clothes, bag, or the like, and the sensor information from the terminal device 10 may not be acquired.

Further, the information providing apparatus 100 presumes that the terminal device 10 is stored in the user U's clothing pocket, bag, or the like, based on sensor information (environmental sound, air pressure, illuminance, etc.) or changes thereof. In some cases, the sensor information from the terminal device 10 may not be acquired.

Further, in the above embodiment, the user's current position on the map is corrected. good. For example, by analyzing the sensor information at the time of acquisition of each location information included in the user's past action history, if an environmental sound that is impossible at that location is detected at a certain location information, The position information is corrected to the position where the environmental sound or the like is detected.

Further, in the above embodiment, the information providing apparatus 100 may perform indoor positioning calibration based on sensor information (environmental sound, atmospheric pressure, illuminance, etc.) or changes thereof. For example, in indoor positioning, if the information providing apparatus 100 determines that such a sound cannot be heard in this place, it performs calibration. For example, redo indoor positioning. Alternatively, the sensor information at that location is reacquired (remeasured). Alternatively, since the sound should be heard in front of the store A, the information providing apparatus 100 determines that the actual position is in front of the store A, and corrects the radio positioning map using the calibration data.

Note that the information providing device 100 may be implemented as one function in the user U's terminal device 10 . For example, the information providing device 100 may be one of the user U's terminal devices 10 .

[7. effect〕
As described above, the information providing device 100 according to the present application includes the acquisition section 131 and the correction section 135 . The acquisition unit 131 acquires the sensor information and the position information simultaneously acquired by the terminal device 10 of the user U having the sensor. The correction unit 135 corrects the position of the user U on the map based on the sensor information and position information. Thus, the information providing apparatus 100 according to the present application can utilize HaaS (Hardware as a Service) to correct the user's position on the map based on the environment information.

[8. Hardware configuration]
Also, the terminal device 10 and the information providing device 100 according to the above-described embodiments are implemented by a computer 1000 configured as shown in FIG. 9, for example. The information providing apparatus 100 will be described below as an example. FIG. 9 is a diagram illustrating an example of a hardware configuration; The computer 1000 is connected to an output device 1010 and an input device 1020, and an arithmetic device 1030, a primary storage device 1040, a secondary storage device 1050, an output I/F (Interface) 1060, an input I/F 1070, and a network I/F 1080 are buses. It has a form connected by 1090.

The arithmetic device 1030 operates based on programs stored in the primary storage device 1040 and the secondary storage device 1050, programs read from the input device 1020, and the like, and executes various processes. The arithmetic unit 1030 is implemented by, for example, a CPU (Central Processing Unit), MPU (Micro Processing Unit), ASIC (Application Specific Integrated Circuit), FPGA (Field Programmable Gate Array), or the like.

The primary storage device 1040 is a memory device such as a RAM (Random Access Memory) that temporarily stores data used for various calculations by the arithmetic device 1030 . The secondary storage device 1050 is a storage device in which data used for various calculations by the arithmetic device 1030 and various databases are registered. State Drive), flash memory, or the like. The secondary storage device 1050 may be an internal storage or an external storage. Also, the secondary storage device 1050 may be a removable storage medium such as a USB memory or an SD (Secure Digital) memory card. Also, the secondary storage device 1050 may be a cloud storage (online storage), a NAS (Network Attached Storage), a file server, or the like.

The output I/F 1060 is an interface for transmitting information to be output to the output device 1010 that outputs various information such as a display, a projector, and a printer. (Digital Visual Interface), HDMI (registered trademark) (High Definition Multimedia Interface), and other standardized connectors. Also, the input I/F 1070 is an interface for receiving information from various input devices 1020 such as a mouse, keyboard, keypad, buttons, scanner, etc., and is realized by, for example, USB.

Also, the output I/F 1060 and the input I/F 1070 may be wirelessly connected to the output device 1010 and the input device 1020, respectively. That is, output device 1010 and input device 1020 may be wireless devices.

Also, the output device 1010 and the input device 1020 may be integrated like a touch panel. In this case, the output I/F 1060 and the input I/F 1070 may also be integrated as an input/output I/F.

Note that the input device 1020 includes, for example, optical recording media such as CDs (Compact Discs), DVDs (Digital Versatile Discs), PDs (Phase change rewritable discs), magneto-optical recording media such as MOs (Magneto-Optical discs), and tapes. It may be a device that reads information from a medium, a magnetic recording medium, a semiconductor memory, or the like.

Network I/F 1080 receives data from other devices via network N and sends the data to arithmetic device 1030, and also transmits data generated by arithmetic device 1030 via network N to other devices.

Arithmetic device 1030 controls output device 1010 and input device 1020 via output I/F 1060 and input I/F 1070 . For example, arithmetic device 1030 loads a program from input device 1020 or secondary storage device 1050 onto primary storage device 1040 and executes the loaded program.

For example, when the computer 1000 functions as the information providing device 100 , the arithmetic device 1030 of the computer 1000 implements the functions of the control unit 130 by executing a program loaded on the primary storage device 1040 . Further, arithmetic device 1030 of computer 1000 may load a program acquired from another device via network I/F 1080 onto primary storage device 1040 and execute the loaded program. Further, the arithmetic unit 1030 of the computer 1000 may cooperate with another device via the network I/F 1080, and call functions, data, etc. of the program from another program of the other device for use.

[9. others〕
Although the embodiments of the present application have been described above, the present invention is not limited by the contents of these embodiments. In addition, the components described above include those that can be easily assumed by those skilled in the art, those that are substantially the same, and those within the so-called equivalent range. Furthermore, the components described above can be combined as appropriate. Furthermore, various omissions, replacements, or modifications of components can be made without departing from the gist of the above-described embodiments.

Further, among the processes described in the above embodiments, all or part of the processes described as being automatically performed can be manually performed, or the processes described as being performed manually can be performed manually. All or part of this can also be done automatically by known methods. In addition, information including processing procedures, specific names, various data and parameters shown in the above documents and drawings can be arbitrarily changed unless otherwise specified. For example, the various information shown in each drawing is not limited to the illustrated information.

Also, each component of each device illustrated is functionally conceptual, and does not necessarily need to be physically configured as illustrated. In other words, the specific form of distribution and integration of each device is not limited to the illustrated one, and all or part of them can be functionally or physically distributed and integrated in arbitrary units according to various loads and usage conditions. Can be integrated and configured.

For example, the information providing apparatus 100 described above may be implemented by a plurality of server computers, and depending on the function, may be implemented by calling an external platform or the like using an API (Application Programming Interface), network computing, or the like. can be changed flexibly.

Also, the above-described embodiments and modifications can be appropriately combined within a range that does not contradict the processing content.

Also, the above-mentioned "section, module, unit" can be read as "means", "circuit", or the like. For example, the acquisition unit can be read as acquisition means or an acquisition circuit.

1 information processing system 10 terminal device 14 positioning unit 20 sensor unit 100 information providing device 110 communication unit 120 storage unit 121 user information database 122 sensor information database 123 map information database 130 control unit 131 acquisition unit 132 accumulation unit 133 estimation unit 134 comparison unit 135 correction unit 136 output unit

Claims (8)

an acquisition unit that acquires sensor information and location information acquired at the same time by a user's terminal device having a sensor;
an estimation unit that estimates the context of the terminal device based on the sensor information;
a comparison unit that compares the context on the map specified from the location information and the context of the terminal device;
a correction unit that corrects the position of the user on the map according to the result of the comparison ;
with
If the comparison result shows that the context on the map and the context of the terminal device do not match, the comparison unit determines whether the location information is incorrect or the map information is incorrect. to decide
An information processing device characterized by: an extraction unit that extracts an environmental sound from the sound included in the sensor information;
further comprising
The information processing device according to claim 1 , wherein the estimation unit estimates the context of the terminal device based on the environmental sound. When the position of the object included in the context on the map and the position of the object included in the context of the terminal device do not match, the comparison unit determines that the position information is erroneous. 3. The information processing apparatus according to claim 1, wherein the determination is made. 4. The correcting unit corrects the position of the user on the map when it is determined that the position information is erroneous. The information processing device according to . The comparison unit determines that the map information is erroneous when the object included in the context of the terminal device is not included in the context on the map. The information processing apparatus according to any one of claims 1 to 4 . 2. The correction unit corrects the map and relatively corrects the position of the user on the map when it is determined that the map information is erroneous. 6. The information processing device according to any one of 1 to 5 . An information processing method executed by an information processing device,
an acquisition step of acquiring sensor information and location information simultaneously acquired by a terminal device of a user having a sensor;
an estimation step of estimating the context of the terminal device based on the sensor information;
a comparing step of comparing the context on the map identified from the location information and the context of the terminal device;
a correction step of correcting the position of the user on the map according to the result of the comparison ;
including
In the comparison step, if the result of the comparison is that the context on the map and the context of the terminal device do not match, it is determined whether the location information or the map information is erroneous. to decide
An information processing method characterized by: an acquisition procedure for acquiring sensor information and location information simultaneously acquired by a terminal device of a user having a sensor;
An estimation procedure for estimating the context of the terminal device based on the sensor information;
a comparison procedure for comparing the context on the map identified from the location information and the context of the terminal device;
a correction procedure for correcting the position of the user on the map according to the result of the comparison ;
on the computer, and
In the comparison procedure, if the result of the comparison is that the context on the map and the context of the terminal device do not match, it is determined whether the location information or the map information is erroneous. to decide
An information processing program characterized by:

Images