KR20220106643A - Electronic device controlled based on sound data and method of controlling electronic device based on sound data - Google Patents

Abstract

According to various embodiments, an electronic device includes a communication circuit, a display, at least one sensor, and at least one processor. The at least one processor may be configured to obtain sound data regarding a surrounding environment from a wearable electronic device through the communication circuit, obtain sensor data through the at least one sensor, identify whether the surrounding environment matches one of a plurality of preset contexts, based on the sound data and the sensor data, and perform at least one of transmission, to the wearable electronic device, of a signal for controlling the wearable electronic device through the communication circuit or display of a notification message on the display, based on the matching context, a state of the display, or a state whether the wearable electronic device outputs a sound, in response to identifying that the surrounding environment matches one of the plurality of preset contexts. Various embodiments are possible.

Description

An electronic device controlled based on sound data and a method for controlling an electronic device based on sound data

Various embodiments relate to an electronic device controlled based on sound data and a method for controlling an electronic device based on sound data.

Recently, wearable electronic devices such as wireless Bluetooth earphones have been widely used. The wearable electronic device may include a sound input device such as a microphone, sense an external sound through the sound input device, and control an operation based on the sensed external sound. For example, the wearable electronic device activates or deactivates a noise canceling function that prevents external noise from being heard by the user based on the detected external sound, adjusts the volume of the sound being output, or enables or disables the external sound listening function. Alternatively, a notification sound to draw the user's attention may be output.

The wearable electronic device may determine a situation based on the sensed external sound and output a notification sound to draw the user's attention according to the determined situation. The printed effect may be reduced.

In addition, when the wearable electronic device controls an operation based on a sensed external sound, other parameters that may indicate the user's situation are not considered and the wearable electronic device determines the situation only based on the external sound, thereby limiting the accuracy of situation determination there may be

The electronic device according to various embodiments determines a situation of a user by considering sensor data together with sound data obtained from the wearable electronic device, a matching situation, a state of a display, and whether the wearable electronic device is outputting a sound In consideration of this, a notification message may be displayed on the display or the wearable electronic device may be controlled.

An electronic device according to various embodiments of the present disclosure includes a communication circuit, a display, at least one sensor, and at least one processor, wherein the at least one processor receives a sound related to the surrounding environment from the wearable electronic device through the communication circuit. acquiring data, acquiring sensor data through the at least one sensor, and determining whether the surrounding environment matches one of a plurality of preset situations based on the sound data and the sensor data, and In response to determining that the environment matches one of a plurality of preset situations, controlling the wearable electronic device based on the matching situation, the state of the display, and whether the wearable electronic device is outputting a sound It may be configured to perform at least one of transmitting a signal to the wearable electronic device through the communication circuit and displaying a notification message on the display.

According to various embodiments of the present disclosure, a method performed in an electronic device includes an operation of acquiring sound data about a surrounding environment from a wearable electronic device, an operation of acquiring sensor data through at least one sensor of the electronic device, the sound data and determining, based on the sensor data, whether the surrounding environment matches one of a plurality of preset situations; and in response to determining that the surrounding environment matches one of a plurality of preset situations, the matching transmitting a signal for controlling the wearable electronic device to the wearable electronic device, or on the display, based on a situation in which the electronic device is outputting a sound; It may include an operation of performing at least one of displaying a notification message.

A wearable electronic device according to various embodiments of the present disclosure includes a communication circuit, a sound input device, a sound output device, at least one sensor, and at least one processor, wherein the at least one processor includes an external electronic device through the communication circuit. establish a communication connection with a device, obtain sound data about the surrounding environment through the sound input device, obtain sensor data through the at least one sensor, based on the sound data and the sensor data, It is checked whether an environment matches one of a plurality of preset situations, and in response to determining that the surrounding environment matches one of a plurality of preset situations, the matching situation, a state of a display of the external electronic device and, based on whether the sound output device is outputting sound, at least one of controlling the sound output device or transmitting a signal to the external electronic device to display a notification on the display can be set.

According to various embodiments, an electronic device controlled based on sound data and a method for controlling an electronic device based on sound data are provided. The electronic device according to various embodiments determines a user's situation by considering the sensor data together with the sound data obtained from the wearable electronic device, and considers the situation, the state of the display, and whether the wearable electronic device is outputting sound Accordingly, a notification message may be displayed on the display or the wearable electronic device may be controlled.

Since the electronic device according to various embodiments may display a notification message on the display, even if the user inadvertently does not pay attention to the notification sound output from the wearable electronic device, the notification message displayed on the display may be recognized, It can effectively attract the user's attention.

Since the electronic device according to various embodiments determines the user's situation in consideration of the sensor data and the sound data obtained from the wearable electronic device, it is possible to accurately determine the user's situation as compared to relying only on the sound data.

1 is a block diagram of an electronic device in a network environment, according to various embodiments of the present disclosure;
2 is a block diagram of a wearable electronic device according to various embodiments of the present disclosure;
3 illustrates operations performed by an electronic device according to various embodiments.
4 illustrates operations performed by an electronic device according to various embodiments.
5 illustrates operations performed by an electronic device according to various embodiments.
6A to 6C illustrate exemplary screens displayed on an electronic device according to various embodiments.
7 is a block diagram of a wearable electronic device according to various embodiments of the present disclosure;
8 illustrates operations performed by a wearable electronic device according to various embodiments of the present disclosure.
9 illustrates operations performed by a wearable electronic device according to various embodiments of the present disclosure.
10 illustrates operations performed by a wearable electronic device according to various embodiments of the present disclosure.
11 is a front view and a rear view of a first wireless audio device that may be included in a wearable electronic device according to an exemplary embodiment.

1 is a block diagram of an electronic device 101 in a network environment 100, according to various embodiments. Referring to FIG. 1 , in a network environment 100 , an electronic device 101 communicates with an electronic device 102 through a first network 198 (eg, a short-range wireless communication network) or a second network 199 . It may communicate with the electronic device 104 or the server 108 through (eg, a long-distance wireless communication network). According to an embodiment, the electronic device 101 may communicate with the electronic device 104 through the server 108 . According to an embodiment, the electronic device 101 includes a processor 120 , a memory 130 , an input module 150 , a sound output module 155 , a display module 160 , an audio module 170 , and a sensor module ( 176), interface 177, connection terminal 178, haptic module 179, camera module 180, power management module 188, battery 189, communication module 190, subscriber identification module 196 , or an antenna module 197 . In some embodiments, at least one of these components (eg, the connection terminal 178 ) may be omitted or one or more other components may be added to the electronic device 101 . In some embodiments, some of these components (eg, sensor module 176 , camera module 180 , or antenna module 197 ) are integrated into one component (eg, display module 160 ). can be

The processor 120, for example, executes software (eg, a program 140) to execute at least one other component (eg, a hardware or software component) of the electronic device 101 connected to the processor 120. It can control and perform various data processing or operations. According to one embodiment, as at least part of data processing or operation, the processor 120 converts commands or data received from other components (eg, the sensor module 176 or the communication module 190 ) to the volatile memory 132 . may be stored in , process commands or data stored in the volatile memory 132 , and store the result data in the non-volatile memory 134 . According to an embodiment, the processor 120 is the main processor 121 (eg, a central processing unit or an application processor) or a secondary processor 123 (eg, a graphic processing unit, a neural network processing unit (eg, a graphic processing unit, a neural network processing unit) a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor). For example, when the electronic device 101 includes the main processor 121 and the sub-processor 123 , the sub-processor 123 uses less power than the main processor 121 or is set to be specialized for a specified function. can The auxiliary processor 123 may be implemented separately from or as a part of the main processor 121 .

The secondary processor 123 may, for example, act on behalf of the main processor 121 while the main processor 121 is in an inactive (eg, sleep) state, or when the main processor 121 is active (eg, executing an application). ), together with the main processor 121, at least one of the components of the electronic device 101 (eg, the display module 160, the sensor module 176, or the communication module 190) It is possible to control at least some of the related functions or states. According to an embodiment, the coprocessor 123 (eg, an image signal processor or a communication processor) may be implemented as part of another functionally related component (eg, the camera module 180 or the communication module 190 ). have. According to an embodiment, the auxiliary processor 123 (eg, a neural network processing device) may include a hardware structure specialized for processing an artificial intelligence model. Artificial intelligence models can be created through machine learning. Such learning may be performed, for example, in the electronic device 101 itself on which artificial intelligence is performed, or may be performed through a separate server (eg, the server 108). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but in the above example not limited The artificial intelligence model may include a plurality of artificial neural network layers. Artificial neural networks include deep neural networks (DNNs), convolutional neural networks (CNNs), recurrent neural networks (RNNs), restricted boltzmann machines (RBMs), deep belief networks (DBNs), bidirectional recurrent deep neural networks (BRDNNs), It may be one of deep Q-networks or a combination of two or more of the above, but is not limited to the above example. The artificial intelligence model may include, in addition to, or alternatively, a software structure in addition to the hardware structure.

The memory 130 may store various data used by at least one component (eg, the processor 120 or the sensor module 176 ) of the electronic device 101 . The data may include, for example, input data or output data for software (eg, the program 140 ) and instructions related thereto. The memory 130 may include a volatile memory 132 or a non-volatile memory 134 .

The program 140 may be stored as software in the memory 130 , and may include, for example, an operating system 142 , middleware 144 , or an application 146 .

The input module 150 may receive a command or data to be used by a component (eg, the processor 120 ) of the electronic device 101 from the outside (eg, a user) of the electronic device 101 . The input module 150 may include, for example, a microphone, a mouse, a keyboard, a key (eg, a button), or a digital pen (eg, a stylus pen).

The sound output module 155 may output a sound signal to the outside of the electronic device 101 . The sound output module 155 may include, for example, a speaker or a receiver. The speaker can be used for general purposes such as multimedia playback or recording playback. The receiver can be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from or as part of the speaker.

The display module 160 may visually provide information to the outside (eg, a user) of the electronic device 101 . The display module 160 may include, for example, a control circuit for controlling a display, a hologram device, or a projector and a corresponding device. According to an embodiment, the display module 160 may include a touch sensor configured to sense a touch or a pressure sensor configured to measure the intensity of a force generated by the touch.

The audio module 170 may convert a sound into an electric signal or, conversely, convert an electric signal into a sound. According to an embodiment, the audio module 170 acquires a sound through the input module 150 , or an external electronic device (eg, a sound output module 155 ) connected directly or wirelessly with the electronic device 101 . The electronic device 102) (eg, a speaker or headphones) may output a sound.

The sensor module 176 detects an operating state (eg, power or temperature) of the electronic device 101 or an external environmental state (eg, a user state), and generates an electrical signal or data value corresponding to the sensed state. can do. According to an embodiment, the sensor module 176 may include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, It may include a temperature sensor, a humidity sensor, or an illuminance sensor.

The interface 177 may support one or more specified protocols that may be used by the electronic device 101 to directly or wirelessly connect with an external electronic device (eg, the electronic device 102 ). According to an embodiment, the interface 177 may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

The connection terminal 178 may include a connector through which the electronic device 101 can be physically connected to an external electronic device (eg, the electronic device 102 ). According to an embodiment, the connection terminal 178 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (eg, a headphone connector).

The haptic module 179 may convert an electrical signal into a mechanical stimulus (eg, vibration or movement) or an electrical stimulus that the user can perceive through tactile or kinesthetic sense. According to an embodiment, the haptic module 179 may include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

The camera module 180 may capture still images and moving images. According to an embodiment, the camera module 180 may include one or more lenses, image sensors, image signal processors, or flashes.

The power management module 188 may manage power supplied to the electronic device 101 . According to an embodiment, the power management module 188 may be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

The battery 189 may supply power to at least one component of the electronic device 101 . According to one embodiment, battery 189 may include, for example, a non-rechargeable primary cell, a rechargeable secondary cell, or a fuel cell.

The communication module 190 is a direct (eg, wired) communication channel or a wireless communication channel between the electronic device 101 and an external electronic device (eg, the electronic device 102, the electronic device 104, or the server 108). It can support establishment and communication performance through the established communication channel. The communication module 190 may include one or more communication processors that operate independently of the processor 120 (eg, an application processor) and support direct (eg, wired) communication or wireless communication. According to one embodiment, the communication module 190 is a wireless communication module 192 (eg, a cellular communication module, a short-range communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module 194 (eg, : It may include a local area network (LAN) communication module, or a power line communication module). A corresponding communication module among these communication modules is a first network 198 (eg, a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network 199 (eg, legacy It may communicate with the external electronic device 104 through a cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (eg, a telecommunication network such as a LAN or a WAN). These various types of communication modules may be integrated into one component (eg, a single chip) or may be implemented as a plurality of components (eg, multiple chips) separate from each other. The wireless communication module 192 uses subscriber information (eg, International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module 196 within a communication network such as the first network 198 or the second network 199 . The electronic device 101 may be identified or authenticated.

The wireless communication module 192 may support a 5G network after a 4G network and a next-generation communication technology, for example, a new radio access technology (NR). NR access technology includes high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and access to multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency) -latency communications)). The wireless communication module 192 may support a high frequency band (eg, mmWave band) to achieve a high data rate, for example. The wireless communication module 192 uses various techniques for securing performance in a high-frequency band, for example, beamforming, massive multiple-input and multiple-output (MIMO), all-dimensional multiplexing. It may support technologies such as full dimensional MIMO (FD-MIMO), an array antenna, analog beam-forming, or a large scale antenna. The wireless communication module 192 may support various requirements defined in the electronic device 101 , an external electronic device (eg, the electronic device 104 ), or a network system (eg, the second network 199 ). According to an embodiment, the wireless communication module 192 may include a peak data rate (eg, 20 Gbps or more) for realizing eMBB, loss coverage (eg, 164 dB or less) for realizing mMTC, or U-plane latency for realizing URLLC ( Example: Downlink (DL) and uplink (UL) each 0.5 ms or less, or round trip 1 ms or less) can be supported.

The antenna module 197 may transmit or receive a signal or power to the outside (eg, an external electronic device). According to an embodiment, the antenna module 197 may include an antenna including a conductor formed on a substrate (eg, a PCB) or a radiator formed of a conductive pattern. According to an embodiment, the antenna module 197 may include a plurality of antennas (eg, an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network such as the first network 198 or the second network 199 is connected from the plurality of antennas by, for example, the communication module 190 . can be selected. A signal or power may be transmitted or received between the communication module 190 and an external electronic device through the selected at least one antenna. According to some embodiments, other components (eg, a radio frequency integrated circuit (RFIC)) other than the radiator may be additionally formed as a part of the antenna module 197 .

According to various embodiments, the antenna module 197 may form a mmWave antenna module. According to one embodiment, the mmWave antenna module comprises a printed circuit board, an RFIC disposed on or adjacent to a first side (eg, bottom side) of the printed circuit board and capable of supporting a designated high frequency band (eg, mmWave band); and a plurality of antennas (eg, an array antenna) disposed on or adjacent to a second side (eg, top or side) of the printed circuit board and capable of transmitting or receiving signals of the designated high frequency band. can do.

At least some of the components are connected to each other through a communication method between peripheral devices (eg, a bus, general purpose input and output (GPIO), serial peripheral interface (SPI), or mobile industry processor interface (MIPI)) and a signal ( e.g. commands or data) can be exchanged with each other.

According to an embodiment, the command or data may be transmitted or received between the electronic device 101 and the external electronic device 104 through the server 108 connected to the second network 199 . Each of the external electronic devices 102 or 104 may be the same as or different from the electronic device 101 . According to an embodiment, all or a part of operations executed in the electronic device 101 may be executed in one or more external electronic devices 102 , 104 , or 108 . For example, when the electronic device 101 needs to perform a function or service automatically or in response to a request from a user or other device, the electronic device 101 may perform the function or service itself instead of executing the function or service itself. Alternatively or additionally, one or more external electronic devices may be requested to perform at least a part of the function or the service. One or more external electronic devices that have received the request may execute at least a part of the requested function or service, or an additional function or service related to the request, and transmit a result of the execution to the electronic device 101 . The electronic device 101 may process the result as it is or additionally and provide it as at least a part of a response to the request. For this purpose, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. The electronic device 101 may provide an ultra-low latency service using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device 104 may include an Internet of things (IoT) device. The server 108 may be an intelligent server using machine learning and/or neural networks. According to an embodiment, the external electronic device 104 or the server 108 may be included in the second network 199 . The electronic device 101 may be applied to an intelligent service (eg, smart home, smart city, smart car, or health care) based on 5G communication technology and IoT-related technology.

2 is a block diagram of a wearable electronic device according to various embodiments of the present disclosure; Referring to FIG. 2 , the wearable electronic device 200 may include a sound input device 210 , a sound output device 220 , a communication circuit 230 , and a processor 240 . According to various embodiments, the sound input device 210 may detect a sound generated in the vicinity of the wearable electronic device 200 and generate data corresponding to the sound. According to various embodiments, the sound input device 210 may transmit the generated data to the processor 240 . For example, the sound input device 210 may include at least one microphone.

According to various embodiments, the sound output device 220 may output a sound based on an electrical signal received from the processor 240 . According to various embodiments, the sound input device 210 may include a speaker.

According to various embodiments, the communication circuit 230 may support wireless communication between the wearable electronic device 200 and other electronic devices other than the wearable electronic device 200 . According to various embodiments, the type of wireless communication supported by the communication circuit 230 is not limited. For example, the communication circuit 230 may support Bluetooth and Bluetooth low energy (BLE) communication.

According to various embodiments, the processor 240 obtains data from other components of the wearable electronic device 200 , performs various data processing or operations based on the obtained data, and controls operations of other components. It may transmit a signal to other components. According to various embodiments, the wearable electronic device 200 may include a plurality of processors 240 .

3 illustrates operations performed by an electronic device according to various embodiments. In operation 310 , at least one processor (eg, the processor 120 ) of the electronic device (eg, the electronic device 101 ) performs a wearable device through a communication circuit (eg, the communication module 190 ). Sound data regarding the surrounding environment may be obtained from the electronic device (eg, the wearable electronic device 200 ). According to various embodiments, sound data may be acquired through the sound input device 210 of the wearable electronic device 200 and transmitted to the electronic device 101 through the communication circuit 230 .

In operation 320 , the at least one processor 120 of the electronic device 101 may acquire sensor data through at least one sensor (eg, the sensor module 176 ). According to various embodiments, the at least one sensor 176 may include a GPS sensor, and the sensor data may indicate location information of the electronic device 101 . According to various embodiments, the at least one sensor 176 may include a vibration sensor. According to various embodiments, the at least one sensor 176 may include a microphone. According to various embodiments, the at least one sensor 176 may include a sensor that detects a motion state of the electronic device 101 , such as an acceleration sensor and/or a gyro sensor.

In operation 330 , the at least one processor 120 of the electronic device 101 may determine whether the surrounding environment matches one of a plurality of preset situations based on the sound data and the sensor data. According to various embodiments of the present disclosure, the plurality of preset situations may include a situation in which the user of the electronic device 101 speaks, a situation in which another person calls the user of the electronic device 101 indoors, and an activity of the user of the electronic device 101 . It may include at least one of a current situation, an indoor event situation, a situation where there is a safety issue indoors, or a situation in which the user of the electronic device 101 is using a means of transportation.

According to various embodiments, the at least one processor 120 may confirm that the surrounding environment matches the situation in which the user speaks, based on the vibration sensor data and the sound data acquired through the vibration sensor. According to various embodiments, when it is confirmed that vibration has occurred when the utterance indicated by the sound data occurs, the at least one processor 120 may confirm that the surrounding environment matches the situation in which the user utters the utterance.

According to various embodiments, when it is confirmed that the user is singing according to the music output from the wearable electronic device 200 based on the sound data, the at least one processor 120 may change the surrounding environment according to a situation in which the user speaks. You can see that they don't match. According to various embodiments, the at least one processor 120 performs a STT (Speaking-To-Text) function based on sound data, and when the STT accuracy is less than a specific level, the user is output from the wearable electronic device 200 You can confirm that you are singing according to the music.

According to various embodiments of the present disclosure, the at least one processor 120 matches a situation in which another person is calling the user of the electronic device 101 in an indoor environment based on the GPS sensor data and sound data acquired through the GPS sensor. can confirm that According to various embodiments, the at least one processor 120 indicates that the GPS sensor data indicates that the electronic device 101 is indoors, and that the sound data indicates one of predetermined keywords corresponding to the user of the electronic device 101 . If it is confirmed, it may be confirmed that the surrounding environment matches a situation in which another person is calling the user of the electronic device 101 indoors. According to various embodiments, the predetermined keyword may be set as an input to the electronic device 101 . For example, the predetermined keyword may include at least one of a name, a nickname, or a job title of the user of the electronic device 101 .

According to various embodiments, the at least one processor 120 may confirm that the surrounding environment matches the situation in which the user of the electronic device 101 is active, based on the vibration sensor data and sound data acquired through the vibration sensor. have. According to various embodiments, the at least one processor 120 may store a program for classifying various activities based on the vibration sensor data, and apply the program to the vibration sensor data to determine whether the vibration sensor data corresponds to a specific activity. can check whether According to various embodiments, when the vibration sensor data corresponds to some activities such as running and walking among various activities that can be distinguished by a program, the surrounding environment is determined by the user of the electronic device 101 by further referring to the GPS sensor data. You can check that it matches the active situation. For example, when the vibration sensor data corresponds to running, the at least one processor 120 determines whether the moving speed of the electronic device 101 is within a general running speed range based on the GPS sensor data, and the electronic device 101 ) is within the general running speed range, it may be confirmed that the surrounding environment matches the situation in which the user of the electronic device 101 is active.

According to various embodiments, when the vibration sensor data corresponds to a specific activity and the repetition cycle of the vibration sensor data and the repetition cycle of the sound data are similar, the at least one processor 120 determines that the user of the electronic device 101 is active. You can check that it matches the situation. For example, when the user is running, the vibration sensor data may be repeated at a constant cycle as the user periodically steps on his/her foot, and the sound data may also be repeated at a regular cycle as the user's foot periodically touches the ground. . In addition to running, when performing an exercise in which a constant motion is repeated, such as jumping rope, walking, or swimming, vibration sensor data and sound data may be constantly repeated, and at least one processor 120 may As the repetition period of the sound data is similar, it may be confirmed that the user of the electronic device 101 is in a specific activity.

According to various embodiments, the at least one processor 120 may determine that the vibration sensor data corresponds to a specific activity, and the moving speed of the electronic device 101 confirmed by the GPS sensor data is within a general speed range that may be of the specific activity. and, when the repetition period of the vibration sensor data and the repetition period of the sound data are similar, it may be confirmed that the user of the electronic device 101 is active.

According to various embodiments, the at least one processor 120 may confirm that the surrounding environment matches the indoor event situation based on the GPS sensor data and the sound data. According to various embodiments, when the GPS sensor data indicates a predetermined location and the sound data indicates one of the preset sounds, the at least one processor 120 may confirm that the surrounding environment matches the indoor event situation. . According to various embodiments, the predetermined location may include at least one of a location specified by a user input of the electronic device 101 or a location frequently visited by the user of the electronic device 101 . According to various embodiments, the preset sounds may include at least one of a doorbell sound, a door knocking sound, a human screaming sound, a dog barking sound, a cat cry, a glass breaking sound, a gunshot, or a baby crying sound. can According to various embodiments, the indoor event situation may be defined corresponding to each preset sound. For example, an indoor event situation might be a doorbell rang, someone knocks on the door, a person screams, a dog barks, a cat cries, a glass is broken, a gun fires, or a baby. It may include at least one of a crying situation.

According to various embodiments, the at least one processor 120 may confirm that the surrounding environment matches a situation in which there is a safety issue indoors, based on the GPS sensor data and the sound data. According to various embodiments, when the GPS sensor data indicates that the electronic device 101 is located indoors, and the sound data indicates one of preset sounds, the at least one processor 120 may determine that the surrounding environment is safe indoors. You can check that it matches the situation with the issue. According to various embodiments, the preset sounds may be sounds related to safety issues that may occur indoors. For example, the preset sounds may include at least one of a fire alarm sound and a fire engine sound. According to various embodiments, a situation in which there is a safety issue in the room may be defined corresponding to each preset sound. For example, the situation in which there is a safety issue in the room may include at least one of a situation in which a fire alarm sounds or a situation in which a sound of a fire engine is heard.

According to various embodiments, the at least one processor 120 may determine that the surrounding environment matches the situation in which the user is using the transportation means, based on the GPS sensor data, the vibration sensor data, and the sound data. According to various embodiments, the at least one processor 120 may determine that the vibration sensor data indicates a vibration pattern corresponding to the transportation means, and the movement speed of the electronic device 101 indicated by the GPS sensor data is within the general movement speed range of the transportation means. And, when the sound data represents a characteristic sound generated by the means of transportation, it can be confirmed that the surrounding environment matches the situation in which the user is using the means of transportation. For example, the means of transportation may include buses and subways.

In operation 340 , the at least one processor 120 of the electronic device 101 may check the state of the display (eg, the display module 160 ) and whether the wearable electronic device 200 is outputting a sound. According to various embodiments, the state of the display 160 may include a state in which the display 160 is turned on and a state in which the display 160 is turned off.

According to various embodiments, the at least one processor 120 receives information about the sound being output from the wearable electronic device 200 through the communication circuit 190 to determine whether the wearable electronic device 200 is outputting a sound can be checked. In operation 340, the meaning of checking whether the wearable electronic device 200 is outputting a sound means whether the wearable electronic device 200 is outputting a sound to be recognized by the user, such as a sound corresponding to music or a video. It could mean to confirm. For example, when the wearable electronic device 200 is outputting a sound for canceling ambient noise in order to perform a noise canceling function, the at least one processor 120 is not outputting the sound from the wearable electronic device 200 can confirm that it is not.

In operation 350 , the at least one processor 120 determines whether the wearable electronic device 200 is outputting a sound and the state of the display 160 confirmed in operation 340 and the matching situation confirmed in operation 330 . At least one of transmitting a signal for controlling the electronic device 200 to the wearable electronic device 200 through the communication circuit 190 or displaying a notification on the display 160 may be performed.

According to various embodiments, the at least one processor 120 checks the first state when it is confirmed that the display 160 is turned on regardless of whether the wearable electronic device 200 is outputting a sound, and the display 160 . When is off and the wearable electronic device 200 is outputting sound, the second state is checked, and when the display 160 is off and the wearable electronic device 200 is not outputting sound, the third state can be checked. .

According to various embodiments, when it is determined in operation 330 that the surrounding environment matches the situation in which the user utters, the at least one processor 120 performs the steps according to the first state, the second state, and the third state in operation 350 . , a control signal for causing the wearable electronic device 200 to perform the operation shown in Table 1 may be transmitted to the wearable electronic device 200 .

first state second state third state noise canceling function Disabled Disabled Disabled Ambient sound listening function activate activate activate sound output stop stop -

According to various embodiments, when the user performs an input for resuming the stopped sound output after performing operation 350, the at least one processor 120 returns to a state prior to transmission of the control signal. can control

According to various embodiments, when it is determined in operation 330 that the surrounding environment matches a situation in which another person is calling the user of the electronic device 101 in the room, in operation 330 , the at least one processor 120 sets the first state, According to the second state and the third state, the operations shown in Table 2 may be performed.

first state second state third state control signal - Alert sound output TTS audio output message display "Someone is calling you" message type toast message heads-up message any type

As shown in Table 2, in the first state, the at least one processor 120 may display a toast message indicating that someone is calling the user on the display 160 . In the second state, the at least one processor 120 displays a heads-up message indicating that someone is calling the user on the display 160 and causes the wearable electronic device 200 to output a notification sound. A control signal to be executed may be transmitted to the wearable electronic device 200 . In the third state, the at least one processor 120 displays a message indicating that someone is calling the user on the display 160 in any type, and causes the wearable electronic device 200 to indicate that someone is calling the user. A control signal for outputting TTS voice may be transmitted to the wearable electronic device 200 .

According to various embodiments, if it is determined in operation 330 that the surrounding environment matches a situation in which the user is active, the at least one processor 120 is irrespective of the first state, the second state, and the third state in operation 350 . Thus, a control signal for causing the wearable electronic device 200 to deactivate the noise canceling function and activate the ambient sound listening function may be transmitted to the wearable electronic device 200 . According to various embodiments, the at least one processor 120 may display, on the display 160 , a message indicating that the noise canceling function is deactivated and the ambient sound listening function is activated because the user is active in the first state. According to various embodiments, when it is confirmed that the user has finished the activity after performing operation 350, the at least one processor 120 may control the wearable electronic device 200 to return to a state prior to transmission of the control signal. .

According to various embodiments, when it is determined in operation 330 that the surrounding environment matches the indoor event situation, in operation 350 , the at least one processor 120 performs a table according to the first state, the second state, and the third state. 3 may be performed.

first state second state third state control signal - Alert sound output TTS audio output message display Messages corresponding to indoor event situations message type toast message head up message any type

As shown in Table 3, in the first state, the at least one processor 120 displays a message corresponding to an indoor event situation, for example, a doorbell ringing situation on the display 160 in the form of a toast message. can In the second state, the at least one processor 120 displays a message corresponding to an indoor event situation on the display 160 in the form of a head-up message, and controls the wearable electronic device 200 to output a notification sound A signal may be transmitted to the wearable electronic device 200 . In the third state, the at least one processor 120 displays a message corresponding to the indoor event situation on the display 160 in an arbitrary type, and causes the wearable electronic device 200 to respond to the message corresponding to the indoor event situation. A control signal for outputting a corresponding TTS sound may be transmitted to the wearable electronic device 200. According to various embodiments, when it is determined in operation 330 that the surrounding environment matches a situation in which there is a safety issue in the room, The at least one processor 120 may perform the operations shown in Table 4 according to the first state, the second state, and the third state in operation 350 .

first state second state third state control signal Disable noise canceling function
Activate the ambient sound function message display Messages in response to safety issues - -

As shown in Table 4, the at least one processor 120 causes the wearable electronic device 200 to deactivate the noise canceling function and perform the ambient sound listening function regardless of the first state, the second state, and the third state. A control signal for activation may be transmitted to the wearable electronic device 200 . According to various embodiments, the at least one processor 120 may display a message corresponding to a safety issue, for example, a situation in which a fire alarm sounds, on the display 160 in the first state.

According to various embodiments, if it is determined in operation 330 that the surrounding environment matches a situation in which the user is using a means of transportation, the at least one processor 120 performs the first state, the second state, and the third state in operation 350 . Depending on the state, the operations shown in Table 5 may be performed.

first state second state third state control signal Disable noise canceling function
Activate the ambient sound function message message display Show message + beep sound TTS message

As shown in Table 5, the at least one processor 120 causes the wearable electronic device 200 to deactivate the noise canceling function and perform the ambient sound listening function regardless of the first state, the second state, and the third state. A control signal for activation may be transmitted to the wearable electronic device 200 .

As shown in Table 5, according to various embodiments, the at least one processor 120 may display a message asking the user's destination on the display 160 in the first state. According to various embodiments, the at least one processor 120 controls the display 160 in the on state in the second state, displays a message asking the user's destination on the display 160, and the wearable electronic device 200 ) to output a notification sound prompting the user to look at the display 160 may be transmitted to the wearable electronic device 200 . According to various embodiments, the at least one processor 120 outputs a control signal that causes the wearable electronic device 200 to output a TTS voice corresponding to a message asking for a user's destination in the third state. can be sent to

According to various embodiments, the at least one processor 120 receives the user's destination in the form of voice or text, and when the location of the electronic device 101 indicated by the GPS sensor data approaches the user's destination, The wearable electronic device 200 may be controlled to display a message indicating that it will arrive soon on the display 160 or to output a TTS voice corresponding to the message.

4 illustrates operations performed by an electronic device according to various embodiments. In operation 410 , at least one processor (eg, the processor 120 ) of the electronic device (eg, the electronic device 101 ) performs a wearable device through a communication circuit (eg, the communication module 190 ). Sound data regarding the surrounding environment may be obtained from the electronic device (eg, the wearable electronic device 200 ). For operation 410 , details regarding operation 310 of FIG. 3 may be applied.

In operation 420 , the at least one processor 120 of the electronic device 101 may determine whether the surrounding environment matches one of a plurality of preset situations based on the sound data. According to various embodiments, the plurality of preset situations may include at least one of a situation in which another person is calling the user of the electronic device 101 , a situation in which a loud noise is generated, or a situation in which there is a safety issue.

According to various embodiments, when it is determined that the sound data represents one of the predetermined keywords corresponding to the user of the electronic device 101 , the at least one processor 120 determines that the surrounding environment is the user of the electronic device 101 . You can check that it matches the situation in which you are calling .

According to various embodiments, when the sound data indicates a sound having a threshold level or higher, the at least one processor 120 may confirm that the surrounding environment matches a situation in which a large noise is generated. According to various embodiments, the at least one processor 120 may confirm that the surrounding environment matches a situation in which a large noise is generated in response to confirming that the sound data represents a sound having a threshold level or more and represents a preset pattern. have. According to various embodiments, the preset pattern may be a pattern learned in advance in response to a situation in which a large noise is likely to occur, such as noise around a construction site or airport noise.

According to various embodiments, when the sound data represents one of preset sounds, the at least one processor 120 may confirm that the surrounding environment matches a situation in which there is a safety issue. According to various embodiments, the preset sounds may be sounds related to safety issues. For example, the preset sounds may include at least one of a fire alarm sound, a fire engine sound, a police car sound, an ambulance sound, or a sound generated when a large vehicle such as a truck is reversing.

In operation 430 , the at least one processor 120 of the electronic device 101 may check the state of the display (eg, the display module 160 ) and whether the wearable electronic device 200 is outputting a sound. Details regarding operation 340 of FIG. 3 may be equally applied to operation 430 .

In operation 440 , the at least one processor 120 determines whether the wearable electronic device 200 is outputting a sound and the state of the display 160 confirmed in operation 430 and the matching situation confirmed in operation 420 . At least one of transmitting a signal for controlling the electronic device 200 to the wearable electronic device 200 through the communication circuit 190 or displaying a notification on the display 160 may be performed.

According to various embodiments, the at least one processor 120 checks the first state when it is confirmed that the display 160 is turned on regardless of whether the wearable electronic device 200 is outputting a sound, and the display 160 . When is off and the wearable electronic device 200 is outputting sound, the second state is checked, and when the display 160 is off and the wearable electronic device 200 is not outputting sound, the third state can be checked. .

According to various embodiments, when it is determined in operation 420 that the surrounding environment matches a situation in which another person is calling the user of the electronic device 101 , in operation 440 , the at least one processor 120 determines the first state and the second state. According to the state and the third state, the operations described in Table 2 may be performed. With respect to the operations shown in Table 2, the details described above with reference to FIG. 3 may be equally applied to operation 440 .

According to various embodiments, when it is determined in operation 420 that the surrounding environment corresponds to a situation in which a large noise is generated, the at least one processor 120 enters the first state, the second state, and the third state in operation 440 . Accordingly, a control signal for causing the wearable electronic device 200 to perform the operations shown in Table 6 may be transmitted to the wearable electronic device 200 . After performing operation 440, the at least one processor 120 controls the wearable electronic device 200 to return to the state before the control signal transmission when the state in which the sound data indicates the sound less than the threshold level continues for a predetermined time or longer. can be controlled

first state second state third state noise canceling function activate activate activate Ambient sound listening function Disabled Disabled Disabled sound output volume increase decrease in volume -

According to various embodiments, if it is determined in operation 420 that the surrounding environment matches a situation in which there is a safety issue, in operation 440 , the at least one processor 120 is configured according to the first state, the second state, and the third state. The operations described in Table 4 may be performed. With respect to the operations shown in Table 4, the details described above with reference to FIG. 3 may be equally applied to operation 440 .

5 illustrates operations performed by an electronic device according to various embodiments. In operation 510 , at least one processor (eg, the processor 120 ) of the electronic device (eg, the electronic device 101 ) performs a wearable device through a communication circuit (eg, the communication module 190 ). Sound data regarding the surrounding environment may be obtained from the electronic device (eg, the wearable electronic device 200 ). Details regarding operation 310 of FIG. 3 may be applied to operation 510 .

In operation 520 , the at least one processor 120 of the electronic device 101 may determine whether the sound data is data related to a sound reproduced in an electronic device other than the electronic device 101 . According to various embodiments, the at least one processor 120 may convert sound data into a log-mel signal, and extract a sound characteristic vector and an ambient noise characteristic vector based on the converted log-mel signal. The at least one processor 120 determines whether the sound data is data related to a sound reproduced by another electronic device or data related to an actual noise source based on the log-mel signal, the sound characteristic vector, and the ambient noise characteristic vector. can be checked According to various embodiments, the at least one processor 120 analyzes a frequency band of the sound data, and determines that the sound data is data related to a sound reproduced by another electronic device based on the absence of a component of a specific frequency band. can

In operation 530 , if the sound data is data related to a sound reproduced by an electronic device other than the electronic device 101 , the at least one processor 120 performs operations 510 through 510 until sound data related to an actual noise source is identified. Operation 530 may be repeated. In other words, in determining the surrounding environment, the at least one processor 120 may not utilize sound data related to a sound reproduced by another electronic device.

In operation 530 , when the sound data is not data related to a sound reproduced in an electronic device other than the electronic device 101 , the at least one processor 120 may perform operation 540 . In operation 540 , the at least one processor 120 may acquire sensor data through at least one sensor (eg, the sensor module 176 ). Details regarding operation 320 of FIG. 3 may be equally applied to operation 540 .

In operation 550 , the at least one processor 120 may determine whether the surrounding environment matches one of a plurality of preset situations based on the sound data and the sensor data. Details regarding operation 330 of FIG. 3 may be equally applied to operation 550 .

In operation 560 , the at least one processor 120 may check the state of the display (eg, the display module 160 ) and whether the wearable electronic device 200 is outputting a sound. Details regarding operation 340 of FIG. 3 may be equally applied to operation 560 .

In operation 570 , the at least one processor 120 determines whether the wearable electronic device 200 is outputting a sound and the state of the display 160 confirmed in operation 560 and the matching situation confirmed in operation 550 . At least one of transmitting a signal for controlling the electronic device 200 to the wearable electronic device 200 through the communication circuit 190 or displaying a notification on the display 160 may be performed. Details regarding operation 350 of FIG. 3 may be equally applied to operation 570 .

In FIG. 5 , the at least one processor 120 confirms whether the sound data is sound data related to a sound reproduced in another electronic device. However, according to various embodiments, the sound data is a sound reproduced in another electronic device. Whether or not it is sound data related to may be checked by the processor 240 of the wearable electronic device 200 . In this case, the wearable electronic device 200 may transmit information indicating whether the sound data is sound data related to a sound reproduced by another electronic device to the electronic device 101 . According to various embodiments, the wearable electronic device 200 does not transmit sound data related to a sound reproduced by another electronic device to the electronic device 101 , but only transmits sound data related to an actual noise source to the electronic device 101 . can be transmitted

When determining the surrounding environment, the electronic device 101 according to various embodiments may ignore sound data related to a sound reproduced by another electronic device and consider only sound data related to an actual noise source. Since noise generated by content being reproduced in another electronic device does not represent the surrounding environment of the electronic device 101 , the electronic device 101 ignores sound data related to sounds reproduced in the other electronic device to improve the surrounding environment. can be judged accurately.

6A to 6C illustrate exemplary screens displayed on an electronic device according to various embodiments. FIG. 6A illustrates a setting interface 600a displayed on the display 160 of the electronic device 101 for setting whether or not to be recognized by the electronic device 101 for each scenario of the surrounding environment. Referring to FIG. 6A , the setting interface is an interface 610a for controlling on/off of the context recognition function, and an interface for controlling whether to activate or deactivate indoor sound recognition when the context recognition function is turned on It may include a 620a and an interface 630a that can control whether to activate or deactivate outdoor sound recognition.

6B illustrates an outdoor sound recognition setting interface 600b that can set whether or not to be recognized by the electronic device 101 for each specific scenario that may occur outdoors with respect to outdoor sound recognition. Referring to FIG. 6B , the outdoor sound recognition setting interface 600b may include an interface 610b that can control whether to activate or deactivate outdoor sound recognition. According to various embodiments, the outdoor sound recognition setting interface 600b activates the interface 621b that can control whether to activate or deactivate the function to recognize the fire alarm and to activate the function to recognize the ambulance siren or not. It may include an interface 622b that can control whether to deactivate. According to various embodiments, the outdoor sound recognition setting interface 600b includes an interface 631b that can control whether to activate or deactivate a function for recognizing a situation in which the user boards the subway, and a situation in which a bicycle bell rings. An interface 632b capable of controlling whether to activate or deactivate the recognition function may be included.

6C may include an interface 600c for setting recognition keywords for recognizing a situation in which the user boards the subway in relation to the function of recognizing the situation in which the user boards the subway. The interface 600c may display previously set recognition keywords 611c, 612c, and 613c, and may include an interface 620c for adding a new keyword. According to various embodiments of the present disclosure, the at least one processor 120 of the electronic device 101 determines that the set recognition keywords 611c , 612c , and 613c are detected based on the sound data as a situation in which the user is boarding the subway. It can be done as at least part of the conditions for confirmation.

7 is a block diagram of a wearable electronic device according to various embodiments of the present disclosure; Referring to FIG. 7 , the wearable electronic device 700 may include a sound input device 710 , a sound output device 720 , a communication circuit 730 , a processor 740 , and a sensor 750 . The sound input device 710 , the sound output device 720 , the communication circuit 730 , and the processor 740 are the sound input device 210 , the sound output device 220 , and the communication circuit described above with reference to FIG. 2 . Details regarding 230 and processor 240 may apply. According to various embodiments, the sensor 750 may include a gyro sensor 751 , an acceleration sensor 752 , a GPS sensor 753 , and a vibration sensor 754 .

8 illustrates operations performed by a wearable electronic device according to various embodiments of the present disclosure. In operation 810 , at least one processor (eg, processor 740 ) of the wearable electronic device (eg, wearable electronic device 700 ) receives sound data regarding the surrounding environment through the sound input device 710 . can be obtained

In operation 820 , the at least one processor 740 may acquire sensor data through at least one sensor (eg, the sensor 750 ).

In operation 830 , the at least one processor 740 may determine whether the surrounding environment matches one of a plurality of preset situations based on the sound data and the sensor data. Details regarding operation 330 of FIG. 3 may be applied to operation 830 . While operation 330 of FIG. 3 is an operation performed by an electronic device (eg, the electronic device 101) interacting with the wearable electronic device 700, operation 830 is performed by the wearable electronic device 700 There is a difference in action.

In operation 840 , the at least one processor 740 determines the state of the display (eg, the display module 160 ) of the external electronic device (eg, the electronic device 101 ) and the sound of the wearable electronic device 700 . It can be checked whether sound is being output through the output device 720 . The at least one processor 740 may check the state of the display 160 by receiving information about the state of the display 160 from the external electronic device 101 through the communication circuit 730 . According to various embodiments, the state of the display 160 may include a state in which the display 160 is turned on and a state in which the display 160 is turned off.

As in operation 340 , checking whether a sound is being output through the sound output device 720 of the wearable electronic device 700 means outputting a sound to be recognized by the user, such as a sound corresponding to music or a video This may mean checking whether or not it is in progress.

In operation 850 , the at least one processor 740 controls the sound output device 720 , or transmits a signal to the external electronic device to display a notification on the display 160 of the external electronic device 101 . at least one of The details of operation 350 of FIG. 3 may be applied to operation 850 , but displaying on the display 160 in operation 350 means that the external electronic device 101 displays on the display 160 in operation 850 . may be replaced with transmitting a signal to the external electronic device 101, and in operation 350 , transmitting the control signal to the wearable electronic device 200 so that the wearable electronic device 200 performs a specific operation includes operation 850 may be replaced with controlling the sound output device 720 so that at least one processor 740 performs a specific operation.

9 illustrates operations performed by a wearable electronic device according to various embodiments of the present disclosure. In operation 910 , at least one processor (eg, the processor 740 ) of the wearable electronic device (eg, the wearable electronic device 700 ) receives sound data regarding the surrounding environment through the sound input device 710 . can be obtained

In operation 920 , the at least one processor 740 may determine whether the surrounding environment matches one of a plurality of preset situations based on the sound data. Details regarding operation 420 of FIG. 4 may be applied to operation 920 . Operation 420 of FIG. 4 is an operation performed by an electronic device (eg, the electronic device 101) interacting with the wearable electronic device 700 , while operation 920 is performed by the wearable electronic device 700 . There is a difference in the action

In operation 930 , the at least one processor 740 determines the state of the display (eg, the display module 160 ) of the external electronic device (eg, the electronic device 101 ) and the sound of the wearable electronic device 700 . It can be checked whether sound is being output through the output device 720 . Details regarding operation 840 of FIG. 8 may be applied to operation 930 .

In operation 940 , the at least one processor 740 controls the sound output device 720 , or transmits a signal to the external electronic device to display a notification on the display 160 of the external electronic device 101 . at least one of The details of operation 440 of FIG. 4 may be applied to operation 940 , but displaying on the display 160 in operation 440 means that the external electronic device 101 displays on the display 160 in operation 940 . may be replaced with transmitting a signal to the external electronic device 101, and in operation 440 , transmitting the control signal to the wearable electronic device 200 so that the wearable electronic device 200 performs a specific operation is performed in operation 940 may be replaced with controlling the sound output device 720 so that at least one processor 740 performs a specific operation.

10 illustrates operations performed by a wearable electronic device according to various embodiments of the present disclosure. In operation 1010 , at least one processor (eg, processor 740 ) of the wearable electronic device (eg, wearable electronic device 700 ) receives sound data regarding the surrounding environment through the sound input device 710 . can be obtained

In operation 1020 , the at least one processor 740 may determine whether the sound data is data related to a sound reproduced in an electronic device other than the wearable electronic device 700 . Details regarding operation 520 of FIG. 5 may be applied to operation 1020 . There is a difference in that operation 520 is performed by at least one processor 120 of the electronic device 101 , while operation 1020 is performed by at least one processor 740 of the wearable electronic device 700 .

In operation 1030 , when the sound data is data related to a sound reproduced in an electronic device other than the wearable electronic device 700 , the at least one processor 740 performs operation 1010 until sound data related to an actual noise source is identified. Operations 1030 to 1030 may be repeated. In other words, in determining the surrounding environment, the at least one processor 740 may not utilize sound data related to sounds reproduced by other electronic devices.

In operation 1030 , when the sound data is not data related to a sound reproduced in an electronic device other than the wearable electronic device 700 , the at least one processor 740 may perform operation 1040 . In operation 540 , the at least one processor 740 may acquire sensor data through at least one sensor (eg, the sensor 750 ).

In operation 1050 , the at least one processor 740 may determine whether the surrounding environment matches one of a plurality of preset situations based on the sound data and the sensor data. Details regarding operation 830 of FIG. 8 may be applied to operation 1050 .

In operation 1060 , the at least one processor 740 determines the state of the display (eg, the display module 160 ) of the external electronic device (eg, the electronic device 101 ) and the sound of the wearable electronic device 700 . It can be checked whether sound is being output through the output device 720 . Details regarding operation 840 of FIG. 8 may be applied to operation 1060 .

In operation 1070 , the at least one processor 740 controls the sound output device 720 , or displays a notification on the display 160 of the external electronic device 101 . At least one of transmitting a signal to the electronic device may be performed. Details regarding operation 850 of FIG. 8 may be applied to operation 1070 .

In determining the surrounding environment, the wearable electronic device 700 according to various embodiments ignores sound data related to sounds reproduced by other electronic devices other than the wearable electronic device 700 and considers only sound data related to an actual noise source. can Since noise generated by content being reproduced in another electronic device does not represent the surrounding environment of the wearable electronic device 700 , the wearable electronic device 700 ignores sound data related to sounds reproduced in the other electronic device. environment can be accurately assessed.

11 is a front view and a rear view of a first wireless audio device that may be included in a wearable electronic device according to an exemplary embodiment.

A structure of the first wireless audio device 1100 that may be included in the wearable electronic device 200 will be described with reference to FIG. 11 . Although redundant description is omitted for convenience of description, the wearable electronic device 200 may further include a second wireless audio device (not shown) having the same or similar structure to the first wireless audio device 1100 .

Reference numeral 1101 shows a front view of the first wireless audio device 1100 . The first wireless audio device 1100 may include a housing 1110 . The housing 1110 may form at least a part of the exterior of the first wireless audio device 1100 . It may include a button 1113 and a plurality of microphones 1181a and 1181b disposed on the first surface of the housing 1110 (eg, the surface facing the outside when worn). The button 1113 may be set to receive a user input (eg, a touch input or a push input). The first microphone 1181a and the second microphone 1181b may be included in the audio receiving circuit 1181 of FIG. 3 . The first microphone 1181a and the second microphone 1181b may be arranged to sense a sound in a direction facing the outside of the user when the first wireless audio device 1100 is worn. The first microphone 1181a and the second microphone 1181b may be referred to as external microphones. The first microphone 1181a and the second microphone 1181b may sense a sound outside the housing 1110 . For example, the first microphone 1181a and the second microphone 1181b may detect a sound generated in the vicinity of the first wireless audio device 1100 . The sound of the surrounding environment sensed by the first wireless audio device 1100 may be output by the speaker 1170 . In an embodiment, the first microphone 1181a and the second microphone 1181b may be sound-collecting microphones for a noise canceling function (eg, active noise cancellation (ANC)) of the first wireless audio device 1100 . In addition, the first microphone 1181a and the second microphone 1181b may be sound pickup microphones for a function of listening to an ambient sound (eg, a transparency function or an ambient aware function) of the first wireless audio device 1100 . For example, the first microphone 1181a and the second microphone 1181b may include various types of microphones including an electronic condenser microphone (ECM) and a micro electro mechanical system (MEMS) microphone. A wing tip 1111 may be coupled to the circumference of the housing 1110 . The wing tip 1111 may be formed of an elastic material. The wing tip 1111 may be detached from the housing 1110 or attached to the housing 1110 . The wingtip 1111 may improve wearability of the first wireless audio device 1100 .

Reference numeral 1102 shows a rear view of the first wireless audio device 1100 . A first electrode 1114, a second electrode 1115, a proximity sensor 1150, a third microphone 1181c, and A speaker 1170 may be included. The speaker 1170 may be included in the audio output circuit 1171 of FIG. 3 . The speaker 1170 may convert an electrical signal into a sound signal. The speaker 1170 may output sound to the outside of the first wireless audio device 1100 . For example, the speaker 1170 may convert an electrical signal into a sound that the user can audibly recognize and output. At least a portion of the speaker 1170 may be disposed inside the housing 1110 . The speaker 1170 may be coupled to the ear tip 1112 through one end of the housing 1110 . The ear tip 1112 may be formed in a cylindrical shape with a hollow formed therein. For example, when the ear tip 1112 is coupled to the housing 1110, the sound (audio) output from the speaker 1170 can be transmitted to an external object (eg, a user) through the hollow of the ear tip 1112 . have.

According to an embodiment, the first wireless audio device 1100 may include a sensor 1151a (eg, an acceleration sensor, a bone conduction sensor, and/or a gyro sensor) disposed on the second surface of the housing 1110 . . The position and shape of the sensor 1151a illustrated in FIG. 11 are exemplary, and embodiments of the present document are not limited thereto. For example, the sensor 1151a may be disposed inside the housing 1110 and may not be exposed to the outside. The sensor 1151a, when worn, may be located at a position capable of contacting the wearer's ear or at a portion of the housing 1110 contacting the wearer's ear.

The ear tip 1112 may be formed of an elastic material (or a flexible material). The ear tip 1112 may assist the first wireless audio device 1100 to be inserted in close contact with the user's ear. For example, the ear tip 1112 may be formed of a silicon material. At least one region of the ear tip 1112 may be deformed according to the shape of the external object (eg, the shape of the ear kernel). According to various embodiments of the present disclosure, the ear tip 1112 may be formed by a combination of at least two of silicone, foam, and plastic material. For example, a region in which the ear tip 1112 is inserted into and abuts against a user's ear may be formed of a silicone material, and a region into which the housing 1110 is inserted may be formed of a plastic material. The ear tip 1112 may be detached from the housing 1110 or attached to the housing 1110 . The first electrode 1114 and the second electrode 1115 may be connected to an external power source (eg, a case) and receive an electrical signal from the external power source. The proximity sensor 1150 may be used to detect a wearing state of the user. The proximity sensor 1150 may be disposed inside the housing 1110 . At least a portion of the proximity sensor 1150 may be disposed to be exposed to the exterior of the first wireless audio device 1100 . The first wireless audio device 1100 may determine whether the first wireless audio device 1100 is worn by the user based on data measured by the proximity sensor 1150 . For example, the proximity sensor 1150 may include an IR sensor. The IR sensor may detect whether the housing 1110 is in contact with the user's body, and the first wireless audio device 1100 determines whether the first wireless audio device 1100 is worn based on the detection of the IR sensor. can do. The proximity sensor 1150 is not limited to the IR sensor, and may be implemented using various types of sensors (eg, an acceleration sensor or a gyro sensor). The third microphone 1181c may be disposed to detect a sound in a direction toward the user when the first wireless audio device 1100 is worn. The third microphone 1181c may be referred to as an internal microphone.

12 shows a block diagram of a wireless audio device according to an embodiment.

According to various embodiments, the wireless audio device 1200 of FIG. 12 may be a type of the wearable electronic device 200 . Configurations of the wireless audio device 1200 of FIG. 12 may include exemplary software modules. For example, the above configurations may be implemented by the first wireless audio device 1100 or the second wireless audio device. At least some of the components may be omitted. At least some of the components may be implemented as one software module. The division of the components is logical, and any program, thread, application, or code that performs the same function may correspond to the components.

The pre-processing module 1210 may perform pre-processing on the received audio (or audio signal) using an audio receiving circuit (eg, the sound input device 210 of FIG. 2 ). . For example, the preprocessing module 1210 may cancel an echo of an audio signal obtained by using an acoustic echo canceller (AEC) 1211 . The preprocessing module 1210 may reduce noise of the acquired audio signal using noise suppression (NS) 1212 . The preprocessing module 1210 may reduce a signal of a specified band of the acquired audio signal using a high pass filter (HPF) 1213 . The preprocessing module 1210 may change a sampling rate of the audio input signal by using the converter 1214 . For example, the converter 1214 may be configured to perform downsampling or upsampling on an audio input signal. The preprocessing module 1210 may selectively apply at least one of the AEC 1211 , the NS 1212 , the HPF 1213 , and the converter 1214 to the audio signal.

The dialog mode module 1220 may determine initiation and termination of the dialog mode. For example, the conversation mode module 1220 may detect whether a wearer (eg, a user) of the wireless audio device 1200 speaks by using a first voice activity detection (VAD) 1221 . The conversation mode module 1220 may use the second VAD 1222 to detect whether the wearer's utterance and an outsider's utterance are uttered. The conversation mode module 1220 may identify and/or specify the utterance section of the wearer through the first VAD 1221 . The conversation mode module 1220 may identify and/or specify an outsider's utterance section through the first VAD 1221 and the second VAD 1222 . For example, the conversation mode module 1220 identifies the utterance section of an outsider by excluding the section in which the utterance of the wearer is identified through the first VAD 1221 from the section where the utterance is identified through the second VAD 1222 . and/or may be specified. The conversation mode module 1220 may determine whether to execute or terminate the sound agent by using the first VAD 1221 , the second VAD 1222 , and the conversation mode function 1223 .

According to an embodiment, the conversation mode module 1220 may detect whether the user's utterance and whether an outsider's utterance is uttered by using the first VAD 1221 and the second VAD 1222 . In one example, the interactive mode module 1220 uses the audio signal pre-processed by the pre-processing module 1210 or the audio signal not processed by the pre-processing module 1210 to the first VAD 1221 or the second VAD 1222 . ) can be performed at least one of. The wireless audio device 1200 may detect the movement of the wireless audio device 1200 using the sensor 1151a (eg, a motion sensor, an acceleration sensor, and/or a gyro sensor). For example, when an audio signal (eg, a sound signal) having a specified size or larger is detected in a specified band (eg, a human voice range), the wireless audio device 1200 may detect the sound signal from the audio signal. When a designated movement is sensed simultaneously or substantially simultaneously while a sound signal is detected, the wireless audio device 1200 may detect a user's utterance (eg, a wearer's utterance) based on the sound signal. For example, the designated movement may be a movement sensed by the wireless audio device 1200 due to the utterance of the wearer of the wireless audio device 1200 . For example, movement due to the wearer's utterance may be transmitted to a motion sensor, an acceleration sensor, and/or a gyro sensor in the form of movement or vibration. The motion caused by the wearer's utterance may be input to the motion sensor, the acceleration sensor, and/or the gyro sensor in a form similar to the input of the bone conduction microphone. The wireless audio device 1200 may acquire information on the start time and end time of the wearer's utterance based on the specified motion and sound signal. While a sound signal is sensed, simultaneously or substantially simultaneously, if a designated movement is not detected, the wireless audio device 1200 may utter an outsider's utterance based on the sound signal (eg, a utterance of a person other than the wearer (eg, an outsider or a counterpart)). ) can be detected. The wireless audio device 1200 may acquire information on the start time and end time of an outsider's utterance based on a designated motion and sound signal. The conversation mode module 1220 may store information on start and end times of the user's or an outsider's utterance in a memory, and determine the start or end of the conversation mode based on the information stored in the memory.

For example, the first VAD 1221 and the second VAD 1222 may be a serial process. When a sound signal is detected using the second VAD 1222 , the wireless audio device 1200 detects a motion using a motion sensor (eg, an acceleration sensor and/or a gyro sensor) so that the sound signal corresponds to the user's utterance. It can be identified whether

For example, the first VAD 1221 and the second VAD 1222 may be parallel processes. For example, the first VAD 1221 may be configured to detect a user's utterance independently of the second VAD 1222 . The second VAD 1222 may be set to detect a sound signal regardless of whether a user speaks.

For example, the wireless audio device 1200 may use different microphones to detect the user's utterance and the utterance of an outsider. The wireless audio device 1200 may use an external microphone (eg, the first microphone 1181a and the second microphone 1181b of FIG. 11 ) to detect an outsider's utterance. The wireless audio device 1200 may use an internal microphone (eg, the third microphone 1181c of FIG. 11 ) to detect a user's utterance. When using the internal microphone, the electronic device 1200 may determine whether the wearer speaks based on a sound signal and motion information based on the internal microphone. The wireless audio device 1200 may determine whether the wearer is uttering based on a sound signal input through a sensor input in order to detect the user's utterance. The signal flowing into the sensor input may include at least one of an acceleration sensor and a gyro sensor input.

According to an embodiment, the conversation mode module 1220 may determine the start of the conversation mode by using the first VAD 1221 and/or the second VAD 1222 . In the conversation mode OFF state, the conversation mode module 1220 may determine whether to start the conversation mode. For example, the conversation mode module 1220 may determine the start of the conversation mode when the user's utterance is maintained for a specified time period. As another example, the conversation mode module 1220 may determine the start of the conversation mode when the other party's utterance is maintained for a specified time period after the user's utterance is terminated.

According to an embodiment, the conversation mode module 1220 may determine whether to maintain or terminate the conversation mode by using the first VAD 1221 and/or the second VAD 1222 . In the conversation mode ON state, the conversation mode module 1220 may determine whether to maintain or terminate the conversation mode. For example, during the conversation mode, the conversation mode module 1220 may determine the end of the conversation mode if no sound signal is detected for a specified time period. During the conversation mode, the conversation mode module 1220 may determine to maintain the conversation mode when a sound signal is detected within a specified time period from the end of the previous sound signal.

According to an embodiment, the conversation mode module 1220 may determine the start and/or end of the conversation mode based on the conversation mode function 1223 . The conversation mode function 1223 may detect the start and/or end of the conversation mode based on a user input. For example, the user input may include a user's sound command, a user's touch input, or a user's button input.

According to an embodiment, the conversation mode module 1220 may determine the length of the specified time period based on the ambient sound. For example, the conversation mode module 1220 may determine the length of the designated time period based on at least one of a background noise sensitivity, an SNR value, and a noise type of a sound acquired using an external microphone. In a noisy environment, the conversation mode module 1220 may increase the length of the designated time interval.

According to an embodiment, the conversation mode module 1220 may determine the start and/or end of the conversation mode based on a user's sound command. In one example, the sound agent module 1230 may detect a sound command instructing the user to initiate the conversation mode, and transmit information instructing the start of the conversation mode to the conversation mode function 1223 in response to the detection of the sound command. have. The sound command instructing initiation of the conversation mode may include a wake-up utterance (eg, high Bixby) and a sound command for waking up of the sound agent. For example, the sound command may have a form such as “Hi Bixby, start conversation mode”. As another example, the sound command instructing initiation of the conversation mode may have a form such as “start the conversation mode” that does not include a wake-up utterance. When the conversation mode function 1223 receives information instructing the start of the conversation mode from the sound agent module 1230 , the conversation mode module 1220 may determine the start of the conversation mode. In one example, the sound agent module 1230 detects a sound command instructing the user to end the conversation mode, and transmits information instructing the end of the conversation mode to the conversation mode function 1223 in response to the detection of the sound command. can For example, the sound command instructing end of the conversation mode may include a wake-up utterance and a sound command for waking up of the sound agent. The sound command may have a form such as “Hi Bixby, end the conversation mode”. For example, the sound command instructing the end of the conversation mode may have a form such as “End the conversation mode” that does not include a wake-up utterance. When the conversation mode function 1223 receives information instructing the end of the conversation mode from the sound agent module 1230 , the conversation mode module 1220 may determine the end of the conversation mode.

According to an embodiment, the conversation mode module 1220 may determine the start and/or end of the conversation mode based on the user's touch input. For example, the electronic device 101 may provide a UI for controlling the conversation mode of the wireless audio device 1200 . Through the UI, the electronic device 101 may receive a user input for setting activation or deactivation of the conversation mode. When a user input instructing to activate the conversation mode is received, the electronic device 101 may transmit a signal instructing the start of the conversation mode to the wireless audio device 1200 . When the conversation mode function 1223 obtains information indicating the start of the conversation mode from the signal, the conversation mode module 1220 may determine the start of the conversation mode. When a user input instructing deactivation of the conversation mode is received through the UI, the electronic device 101 may transmit a signal instructing the end of the conversation mode to the wireless audio device 1200 . When the conversation mode function 1223 obtains information indicating the end of the conversation mode from the signal, the conversation mode module 1220 may determine the end of the conversation mode.

According to an embodiment, when the conversation mode module 1230 determines the start or end of the conversation mode, the wireless audio device 1200 transmits a signal indicating that the start or end of the conversation mode is determined to the electronic device 101 can do. The electronic device 101 may provide information indicating that the start or end of the conversation mode has been determined obtained from the signal through the UI for controlling the conversation mode of the wireless audio device 1200 .

According to an embodiment, the conversation mode module 1220 may determine the start and/or end of the conversation mode based on the user's button input. For example, the wireless audio device 1200 may include at least one button (eg, the button 1113 of FIG. 11 ). The conversation mode function 1223 may be set to detect a designated input for a button (eg, a double tap or a long press). When an input indicating initiation of the conversation mode is received through the button, the conversation mode module 1220 may determine the start of the conversation mode. When an input instructing the end of the conversation mode is received through the button, the conversation mode module 1220 may determine the end of the conversation mode.

According to one embodiment, the conversation mode function 1223 may be configured to interact with the sound agent module 1230 . For example, the conversation mode function 1223 may obtain information indicating whether the utterance is for a sound agent call from the sound agent module 1230 . For example, the wearer's utterance maintained for a specified time or longer may be detected by the first VAD 1221 . In this case, the conversation mode module 1220 may use the conversation mode function 1223 to identify whether the wearer's utterance is for the call of the sound agent. When the conversation mode function 1223 confirms using the sound agent module 1230 that the sound agent call is performed by the corresponding utterance, the conversation mode module 1220 may ignore the utterance. For example, even if the corresponding utterance continues for a specified time or longer, the conversation mode module 1220 may not determine to start the conversation mode only with the corresponding utterance. For example, the sound agent module 1230 may identify a sound command instructing initiation of a conversation mode from the corresponding utterance. In this case, the sound agent module 1230 may transmit a signal instructing the start of the conversation mode to the conversation mode module 1220 , and the conversation mode module 1220 may determine the start of the conversation mode. That is, in this case, the conversation mode module 1220 may determine the start of the conversation mode based on the instruction of the sound agent module 1230 rather than the length of the utterance itself.

According to an embodiment, the conversation mode module 1220 may determine the end of the conversation mode based on the operation time of the conversation mode. For example, when a predetermined time elapses after the conversation mode is turned on, the conversation mode module 1220 may determine the end of the conversation mode.

According to an embodiment, the sound agent module 1230 may include a wakeup speech recognition module 1231 and a sound agent control module 1232 . In one example, the sound agent module 1230 may further include a sound command recognition module 1233 . The wake-up utterance recognition module 1231 may obtain an audio signal by using the audio receiving circuit 210 and recognize a wake-up utterance (eg, high Bixby) from the audio signal. When a specified sound command is recognized, the wakeup speech recognition module 1231 may control the sound agent using the sound agent control module 1232 . For example, the sound agent control module 1232 transmits a sound signal received to the electronic device (eg, the electronic device 101 ), and receives a task or command corresponding to the sound signal from the electronic device 101 . can do. For example, when a sound signal instructs to adjust the volume, the electronic device 101 may transmit a signal instructing to adjust the volume to the wireless audio device 1200 . The sound command recognition module 1233 may obtain an audio signal using the audio receiving circuit 210 and recognize a specified sound command from the audio signal. In one example, the designated sound utterance may include a sound command (eg, start the conversation mode, end the conversation mode) for controlling the conversation mode. The sound command recognition module 1233 may perform a function corresponding to the specified sound command when the specified sound command is recognized without recognizing the wake-up utterance. For example, the sound command recognition module 1233 may transmit a signal instructing the end of the conversation mode to the electronic device 101 when recognizing the utterance of a specified command such as “end of the conversation mode”. For example, the sound command recognition module 1233 may perform a function corresponding to a specified sound command without interaction with the sound agent. The electronic device 101 may perform sound control of the wireless audio device 1200, which will be described later, in response to a signal instructing the end of the conversation mode.

According to an embodiment, the conversation mode module 1220 may transmit a decision on the conversation mode (eg, the end of the conversation mode or the start of the conversation mode) to the conversation mode control module 1250 . The conversation mode control module 1250 may control the function of the wireless audio device 1200 according to the activation and/or deactivation of the conversation mode. For example, the conversation mode control module 1250 may control the output sound of the wireless audio device 1200 using the sound control module 1240 according to activation and/or deactivation of the conversation mode.

For example, the sound control module 1240 may include an active noise canceling (ANC) module 1241 and an ambient sound control module 1242 . The ANC module 1241 may be configured to acquire an ambient sound and perform noise cancellation based on the ambient sound. For example, the ANC module 1241 may acquire an ambient sound using an external microphone and perform noise cancellation using the acquired ambient sound. The ambient sound control module 1242 may be configured to provide ambient sounds to the wearer. For example, the ambient sound control module 1242 may be set to provide ambient sound by acquiring ambient sound using an external microphone and outputting the obtained ambient sound using a speaker of the wireless audio device 1200. .

According to an embodiment, when the conversation mode starts, the conversation mode control module 1250 may control the output sound of the wireless audio device 1200 using the sound control module 1240 . For example, the conversation mode control module 1250 may deactivate ANC and activate ambient sounds in response to the start of the conversation mode. For another example, when music is being output from the wireless audio device 1200 , the conversation mode control module 1250 may reduce the volume level of the music being output by a certain percentage or more in response to the start of the conversation mode, or set it to maximum mute. can The user of the wireless audio device 1200 can clearly hear ambient sounds according to the initiation of the conversation mode.

According to an embodiment, when the conversation mode ends, the conversation mode control module 1250 may control the output sound of the wireless audio device 1200 using the sound control module 1240 . For example, the conversation mode control module 1250 may restore the ANC setting and/or the ambient sound settings to the settings prior to initiation of the conversation mode in response to the end of the conversation mode, and inactivate the ambient sound. For example, before starting the conversation mode, the conversation mode control module 1250 may store the ANC setting and/or the ambient sound setting in the memory. When the conversation mode ends, the conversation mode control module 1250 may activate or deactivate the ANC and/or the ambient sound according to the ANC setting and/or the ambient sound setting stored in the memory.

As another example, the conversation mode control module 1250 may restore the output sound of the wireless audio device 1200 to a setting prior to initiation of the conversation mode in response to the end of the conversation mode. For example, when music is being output from the wireless audio device 1200 before the conversation mode starts, the conversation mode control module 1250 may store the music output sound setting in the memory. When the conversation mode ends, the conversation mode control module 1250 may restore the music output sound to the music output sound setting stored in the memory. The conversation mode control module 1250 may decrease or mute the media output volume to a specified value according to a setting in the conversation mode. In the conversation mode, the wireless audio device 1200 may output a notification of the sound agent (eg, a response to a user's utterance) independently of the volume in the conversation mode. For example, in the conversation mode, the wireless audio device 1200 may output a notification (eg, a TTS-based response) of the sound agent as a specified volume value.

According to an embodiment, the conversation mode control module 1250 may control the output sound using the sound control module 1240 during the operation of the conversation mode. For example, the conversation mode control module 1250 may control the intensity of the ANC and/or ambient sound. The conversation mode control module 1250 may control the gain value of the ambient sound to amplify the intensity of the ambient sound. The conversation mode control module 1250 may amplify only a section in which a sound exists or a frequency band corresponding to the sound in the surrounding sound. In the conversation mode, the conversation mode control module 1250 may decrease the strength of the ANC. The conversation mode control module 1250 may control an output volume of an audio signal.

Tables 7 and 8 below show examples of sound control of the conversation mode control module 1250 according to the start (eg, ON) and end (eg, OFF) of the conversation mode.

sound control previous state Conversation Mode ON Conversation Mode OFF ANC ON OFF ON ambient sound OFF ON OFF

Referring to Table 7, the wearer of the wireless audio device 1200 may be listening to music using the wireless audio device 1200 . For example, the wireless audio device 1200 may output music while performing ANC. For example, the wireless audio device 1200 may output music at a first volume. According to the start of the conversation mode, the conversation mode control module 1250 may activate an ambient sound and deactivate the ANC. In this case, the conversation mode control module 1250 may reduce the volume of the music being output to a specified value or less, or may decrease it by a specified ratio. For example, the conversation mode control module 1250 may decrease the volume of the music being output to a second value in the conversation mode. Upon termination of the conversation mode, the conversation mode control module 1250 may restore settings related to the output sound. For example, the conversation mode control module 1250 may activate ANC and deactivate ambient sounds. Also, the conversation mode control module 1250 may increase the volume of the music being output to the first value.

sound control previous state Conversation Mode ON Conversation Mode OFF ANC OFF OFF OFF ambient sound OFF ON OFF

Referring to Table 8, the wearer of the wireless audio device 1200 may be listening to music using the wireless audio device 1200 . For example, the wireless audio device 1200 may output music without applying ANC. For example, the wireless audio device 1200 may output music at a first volume. According to the start of the conversation mode, the conversation mode control module 1250 may activate an ambient sound and maintain the ANC in an inactive state. In this case, the conversation mode control module 1250 may reduce the volume of the music being output to a specified value or less, or may decrease it by a specified ratio. For example, the conversation mode control module 1250 may reduce the volume of the music being output to a second value in the conversation mode. Upon termination of the conversation mode, the conversation mode control module 1250 may restore settings related to the output sound. For example, the conversation mode control module 1250 may maintain the ANC in a deactivated state and deactivate ambient sounds. Also, the conversation mode control module 1250 may increase the volume of the music being output to the first value.

In the examples of Tables 7 and 8, it has been described that the wireless audio device 1200 deactivates the ambient sound when the conversation mode is not set, but embodiments of the present document are not limited thereto. For example, even when the conversation mode is not set, the wireless audio device 1200 may activate ambient sounds according to a user's setting.

According to an embodiment, the environment classification module 1260 may obtain an audio signal using an audio receiving circuit and classify the environment based on the audio signal. For example, the environment classification module 1260 may obtain at least one of a background noise, a signal to noise ratio (SNR), and a type of noise from the audio signal. The environment classification module 1260 may detect the environment based on the intensity of the background noise, the SNR, or the type of noise. For example, the environment classification module 1260 may identify the environment of the wireless audio device 1200 by comparing the environment information stored in the memory with at least one of the background noise intensity, SNR, and noise type. For example, the environment classification module 1260 may control an output sound based on the identified environment.

For example, in a state in which the conversation mode is activated, the environment classification module 1260 may control an output sound based on the identified environment. The environment classification module 1260 may control the ambient sound based on the intensity and/or SNR of the background noise. For example, the environment classification module 1260 may determine the overall output of the ambient sound, the amplification of a sound band among the ambient sounds, or the amplification of a specified sound (eg, an alarm or a siren) among the ambient sounds. For example, the environment classification module 1260 may determine the strength of the ANC. For example, the environment classification module 1260 may adjust parameters (eg, coefficients) of a filter for ANC. In the above-described example, the environment classification module 1260 identifies the environment using the audio signal, but embodiments of the present document are not limited thereto. For example, the environment classification module 1260 may identify the environment using Wi-Fi information and/or GPS information.

According to an embodiment, the environment classification module 1260 may control the conversation mode based on the identified environment. For example, the environment classification module 1260 may activate the conversation mode based on the identified environment. If the environment classification module 1260 determines that the user is in an environment in which ambient sounds are to be heard, the conversation mode control module 1250 may be used to activate the conversation mode, and may provide ambient sounds to the user according to the conversation mode. For example, when the user is in a dangerous environment (eg, an environment in which a siren sound is sensed), the environment classification module 1260 may activate a conversation mode.

According to an embodiment, the electronic device 101 may display a user interface instructing the end or start of the conversation mode on the display 360 . The electronic device 101 may provide the user interface in a manner synchronized with the conversation mode of the wireless audio device 1200 . The electronic device 101 may display the user interface when the electronic device 101 determines the end or start of the conversation mode or receives a signal instructing the end or start of the conversation mode from the wireless audio device 1200 . can For example, when the conversation mode starts, the electronic device 101 may display a first user interface including information indicating that the conversation mode has been set. The first user interface may include an interface for controlling output sound settings in the conversation mode. For example, when the conversation mode ends, the electronic device 101 may display a second user interface including information indicating that the conversation mode has ended. The electronic device 101 may display the first user interface and the second user interface on an execution screen of an application (eg, a wearable application) for controlling the wireless audio device 1200 .

According to an embodiment, the conversation mode module 1220 may determine the start and end of the conversation mode further based on whether or not the user is wearing it. For example, when the wireless audio device 1200 is worn by a user, the conversation mode module 1220 may initiate a conversation mode based on a user's (eg, a wearer) utterance or a user input. When the wireless audio device 1200 is not worn by the user, the conversation mode module 1220 may not start the conversation mode even if the user's utterance is detected.

For example, each of the first wireless audio device 1100 and the second wireless audio device (not shown) may include the components of the wireless audio device 1200 shown in FIG. 12 . Each of the first wireless audio device 1100 and the second wireless audio device may be configured to determine whether to initiate a conversation mode. According to an embodiment, when the first wireless audio device 1100 or the second wireless audio device determines to start the conversation mode, the first wireless audio device 1100 and the second wireless audio device are configured to operate in the conversation mode can be For example, the first wireless audio device 1100 or the second wireless audio device having decided to initiate the conversation mode is set to transmit a signal instructing the initiation of the conversation mode to another wireless audio device and/or the electronic device 101 can be According to an embodiment, when both the first wireless audio device 1100 and the second wireless audio device decide to start the conversation mode, the first wireless audio device 1100 and the second wireless audio device operate in the conversation mode. can be set. For example, the first wireless audio device 1100 or the second wireless audio device that has decided to initiate the conversation mode checks whether another wireless audio device has decided to initiate the conversation mode, and both wireless audio devices initiate the conversation mode is determined, the first wireless audio device 1100 and the second wireless audio device may operate in a conversation mode. As another example, the first wireless audio device 1100 or the second wireless audio device that has decided to start the chat mode may transmit a signal instructing the start of the chat mode to the electronic device 101 . The electronic device 101 receives a signal instructing to initiate a conversation mode from both the first wireless audio device 1100 and the second wireless audio device within a specified time, the first wireless audio device 1100 and the second wireless audio device It may transmit a signal that causes the device to operate in an interactive mode.

According to various embodiments, the electronic device (eg, the electronic device 101 ) includes a communication circuit (eg, the communication module 190 ), a display (eg, the display module 160 ), and at least one of a sensor (eg, a sensor module 176), and at least one processor (eg, a processor 120), wherein the at least one processor 120 includes Through this, sound data regarding the surrounding environment is obtained from the wearable electronic device 700 , and sensor data is obtained through the at least one sensor 176 , and based on the sound data and the sensor data, the surrounding environment is determining whether the match matches one of a plurality of preset contexts, and in response to determining that the surrounding environment matches one of a plurality of preset contexts, the matching context, the state of the display 160, and the Transmitting a signal for controlling the wearable electronic device 700 to the wearable electronic device 700 through the communication circuit 190 based on whether the wearable electronic device 700 is outputting a sound, or At least one of displaying a notification message on the display 160 may be set.

According to various embodiments, the at least one processor 120 checks whether the sound data is data related to a sound reproduced in another electronic device, and the sound data is data related to a sound reproduced in another electronic device. based on it is determined that not , it may be configured to check whether the surrounding environment matches one of a plurality of preset situations.

According to various embodiments, the at least one sensor 176 includes a vibration sensor, and the at least one processor 120, based on the vibration sensor data and the sound data obtained through the vibration sensor, In response to confirming that the surrounding environment matches the situation in which the user of the electronic device 101 speaks, and confirming that the surrounding environment matches the situation in which the user of the electronic device 101 speaks: the display 160 ) is on, transmitting a signal for controlling the wearable electronic device 700 to deactivate the noise canceling function, activate the ambient sound listening function, and stop sound output to the wearable electronic device 700, When the display 160 is turned off and the wearable electronic device 700 is outputting sound, the wearable electronic device 700 disables the noise canceling function, activates the ambient sound listening function, and stops the sound output. ) to the wearable electronic device 700, and when the display 160 is turned off and the wearable electronic device 700 is not outputting a sound, the noise canceling function is deactivated, It may be configured to transmit a signal for controlling the wearable electronic device 700 to the wearable electronic device 700 to activate an ambient sound listening function.

According to various embodiments, the at least one processor 120 is, in response to confirming that the sound data represents a sound corresponding to the preset text, the surrounding environment is the preset among the plurality of situations in which the electronic In response to confirming that the context in which the user of the device 101 is being called is consistent and the surrounding environment is consistent with the context in which another person is calling the user of the electronic device 101: the display 160 turns on If there is, a notification message is displayed on the display 160 , and when the display 160 is turned off and the wearable electronic device 700 is outputting a sound, a notification message is displayed on the display 160 . and transmits a signal for controlling the wearable electronic device 700 to output a notification sound to the wearable electronic device 700, the display 160 is turned off, and a sound is generated from the wearable electronic device 700 When not outputting, a signal for controlling the wearable electronic device 700 to output a sound corresponding to the notification message may be transmitted to the wearable electronic device 700 .

According to various embodiments, it is confirmed that the surrounding environment matches a situation in which another person is calling the user of the electronic device 101 , the display 160 is turned off, and a sound is output from the wearable electronic device 700 . In this case, the notification message displayed on the display 160 may be a heads-up message.

According to various embodiments, when it is confirmed that the surrounding environment matches a situation in which another person is calling the user of the electronic device 101 and the display 160 is turned on, the display 160 is displayed. The notification message may be a toast message.

According to various embodiments, the at least one processor 120, in response to confirming that the sound data represents a sound having a threshold level or more and represents a preset first pattern, a situation in which a large noise is generated in the surrounding environment , and in response to confirming that the surrounding environment matches a situation in which a loud noise is generated: When the display 160 is turned on and the wearable electronic device 700 is outputting a sound, noise canceling The wearable electronic device 700 transmits a signal for controlling the wearable electronic device 700 to activate a function, deactivate the ambient sound listening function, and increase the volume of the sound output from the wearable electronic device 700 . to activate the noise canceling function and deactivate the ambient sound listening function when the display 160 is turned on and the wearable electronic device 700 is not outputting sound. When a signal for controlling Deactivate a function and transmit a signal for controlling the wearable electronic device 700 to increase the volume of the sound output from the wearable electronic device 700 to the wearable electronic device 700, and the display 160 ) is turned off and the wearable electronic device 700 is not outputting a sound, a signal for controlling the wearable electronic device 700 to activate the noise canceling function and deactivate the ambient sound listening function is transmitted to the wearable electronic device 700 . It may be configured to transmit to the electronic device 700 .

According to various embodiments, the at least one sensor 176 includes a GPS sensor, and the at least one processor 120 indicates a location where GPS sensor data acquired through the GPS sensor is preset, and the sound When the data indicates one of preset sounds, confirming that the surrounding environment matches an indoor event situation among the plurality of preset situations, and in response to confirming that the surrounding environment matches an indoor event situation: the display When 160 is turned on, a notification message corresponding to one of the preset sounds is displayed on the display 160 , the display 160 is turned off, and the wearable electronic device 700 emits a sound. When outputting, a notification message corresponding to one of the preset sounds is displayed on the display 160, and a signal for controlling the wearable electronic device 700 to output a notification sound is transmitted to the wearable electronic device ( 700), output a sound corresponding to a notification message corresponding to one of the preset sounds when the display 160 is turned off and the wearable electronic device 700 is not outputting a sound It may be configured to transmit a signal for controlling the wearable electronic device 700 to the wearable electronic device 700 .

According to various embodiments, the predetermined location may include at least one of a location specified by a user input of the electronic device 101 or a location frequently visited by the user of the electronic device 101 .

According to various embodiments, the wearable electronic device (eg, the wearable electronic device 700 ) includes a communication circuit (eg, the communication circuit 730 ) and a sound input device (eg, the sound input device 710 ). )), a sound output device (eg, sound output device 720), at least one sensor (eg, sensor 750), and at least one processor (eg, processor 740). The at least one processor 740 establishes a communication connection with the external electronic device 101 through the communication circuit 730 and provides sound data related to the surrounding environment through the sound input device 710 . obtains, acquires sensor data through the at least one sensor 750, and checks whether the surrounding environment matches one of a plurality of preset situations based on the sound data and the sensor data, In response to confirming that the surrounding environment matches one of a plurality of preset situations, the matching situation, the status of the display 160 of the external electronic device 101 , and a sound from the sound output device 720 . At least one of controlling the sound output device 720 or transmitting a signal to the external electronic device 101 so as to display a notification on the display 160 is performed based on whether the sound output device 720 is output. can be set to

According to various embodiments, the at least one processor 740 checks whether the sound data is data related to a sound reproduced by another electronic device, and the sound data is data related to a sound reproduced in another electronic device. based on it is determined that the surrounding environment does not correspond to one of a plurality of preset situations.

According to various embodiments, the at least one sensor 750 includes a vibration sensor 754 , and the at least one processor 740 includes the vibration sensor data and the sound acquired through the vibration sensor 754 . In response to confirming that the surrounding environment matches a situation in which the user of the electronic device speaks, based on data, and confirming that the surrounding environment matches a situation in which the user of the electronic device speaks: the external electronic device When the display 160 of 101 is turned on, the sound output device 720 is controlled to deactivate the noise canceling function, activate the ambient sound listening function, and stop outputting the sound, and the external electronic device 101 ) is turned off and the wearable electronic device 700 is outputting sound, the sound output device ( 720), when the display 160 of the external electronic device 101 is turned off and the wearable electronic device 700 is not outputting a sound, the noise canceling function is deactivated and the ambient sound listening function is enabled. It may be configured to control the sound output device 720 to activate.

According to various embodiments of the present disclosure, a method performed in an electronic device (eg, the electronic device 101 ) includes obtaining sound data regarding a surrounding environment from a wearable electronic device (eg, the wearable electronic device 700 ). operation, acquiring sensor data through at least one sensor (eg, the sensor module 176) of the electronic device 101, based on the sound data and the sensor data, the surrounding environment is preset In response to an operation of ascertaining whether it matches one of a plurality of situations, and determining that the surrounding environment matches one of a plurality of preset situations, the matching situation, the display (eg, a display module ( 160)) and transmitting a signal for controlling the wearable electronic device 700 to the wearable electronic device 700 based on whether the wearable electronic device 700 is outputting a sound, or It may include an operation of performing at least one of displaying a notification message on the display 160 .

According to various embodiments, the method further includes checking whether the sound data is data related to a sound reproduced by another electronic device, and whether the surrounding environment matches one of a plurality of preset situations The confirming operation may be performed based on confirming that the sound data is not data related to a sound reproduced by another electronic device.

According to various embodiments, the operation of acquiring the sensor data through the at least one sensor 176 includes the operation of acquiring the vibration sensor data through a vibration sensor, and the surrounding environment is set in one of a plurality of preset situations; The operation of determining whether the match includes an operation of confirming that the surrounding environment matches a situation in which a user of the electronic device 101 speaks, based on the vibration sensor data and the sound data, and the surrounding environment When it is confirmed that the situation in which the user of the electronic device 101 is uttering, a signal for controlling the wearable electronic device 700 is transmitted to the wearable electronic device 700 , or the display 160 ), the operation of displaying at least one of displaying a notification message on When the operation of transmitting a signal for controlling the electronic device 700 to the wearable electronic device 700, the display 160 is turned off, and the wearable electronic device 700 is outputting a sound, the noise canceling function is performed transmitting, to the wearable electronic device 700, a signal for controlling the wearable electronic device 700 to deactivate, activate an ambient sound listening function, and stop outputting the sound to the wearable electronic device 700, and the display 160 is turned off and when the wearable electronic device 700 is not outputting a sound, a signal for controlling the wearable electronic device 700 to deactivate the noise canceling function and activate the ambient sound listening function is transmitted to the wearable electronic device ( 700) may include the operation of transmitting.

According to various embodiments, the determining whether the surrounding environment matches one of a plurality of preset situations may include, in response to it being confirmed that the sound data represents a sound corresponding to a preset text, the surrounding environment and confirming that a situation in which another person is calling the user of the electronic device 101 among the plurality of preset situations matches, and the surrounding environment is a situation in which another person is calling the user of the electronic device 101 When it is confirmed that they match, at least one of transmitting a signal for controlling the wearable electronic device 700 to the wearable electronic device 700 or displaying a notification message on the display 160 is performed The operations to be performed include: when the display 160 is turned on, displaying a notification message on the display 160 , when the display 160 is off and the wearable electronic device 700 is outputting a sound , an operation of displaying a notification message on the display 160 and transmitting a signal for controlling the wearable electronic device 700 to output a notification sound to the wearable electronic device 700, and the display 160 is off and the wearable electronic device 700 is not outputting a sound, a signal for controlling the wearable electronic device 700 to output a sound corresponding to a notification message is transmitted to the wearable electronic device 700 It may include an operation of transmitting.

According to various embodiments, it is confirmed that the surrounding environment matches a situation in which another person is calling the user of the electronic device 101 , the display 160 is turned off, and a sound is output from the wearable electronic device 700 . In this case, the notification message displayed on the display 160 may be a heads-up message.

According to various embodiments, when it is confirmed that the surrounding environment matches a situation in which another person is calling the user of the electronic device 101 and the display 160 is turned on, the display 160 is displayed. The notification message may be a toast message.

According to various embodiments, the determining whether the surrounding environment matches one of a plurality of preset situations may include, in response to it being confirmed that the sound data represents a sound having a threshold level or higher and represents a preset first pattern. , a signal for controlling the wearable electronic device 700 when it is confirmed that the surrounding environment matches a situation in which a large noise is generated, and it is confirmed that the surrounding environment matches a situation in which a large noise is generated The operation of performing at least one of transmitting to the wearable electronic device 700 and displaying a notification message on the display 160 is: when the display 160 is turned on, the wearable electronic device ( When the sound is being output by the wearable electronic device 700 , the wearable electronic device 700 is controlled to activate the noise canceling function, deactivate the ambient sound listening function, and increase the volume of the sound being output from the wearable electronic device 700 . When the display 160 is turned on and the wearable electronic device 700 is not outputting a sound, the noise canceling function is activated, An operation of transmitting a signal for controlling the wearable electronic device 700 to the wearable electronic device 700 to deactivate a listening function, the display 160 is turned off, and the wearable electronic device 700 outputs a sound a signal for controlling the wearable electronic device 700 to activate the noise canceling function, deactivate the ambient sound listening function, and increase the volume of the sound output from the wearable electronic device 700 When the operation of transmitting to the electronic device 700 and the display 160 are turned off and the wearable electronic device 700 is not outputting sound, the noise canceling function is activated and the ambient sound listening function is deactivated. the above The operation may include transmitting a signal for controlling the wearable electronic device 700 to the wearable electronic device 700 .

According to various embodiments, the operation of acquiring the sensor data through the at least one sensor 176 includes the operation of acquiring the GPS sensor data through the GPS sensor, and the surrounding environment is set in one of a plurality of preset situations; The operation of determining whether the match is performed may include, when the GPS sensor data indicates a predetermined location and the sound data indicates one of preset sounds, the surrounding environment matches an indoor event situation among the plurality of preset situations transmitting a signal for controlling the wearable electronic device 700 to the wearable electronic device 700 when it is confirmed that the surrounding environment matches the indoor event situation, or the display The performing of at least one of displaying a notification message on the display 160 is: when the display 160 is turned on, a notification message corresponding to one of the preset sounds is displayed on the display 160 When the display 160 is turned off and the wearable electronic device 700 is outputting a sound, a notification message corresponding to one of the preset sounds is displayed on the display 160, transmitting a signal for controlling the wearable electronic device 700 to output a notification sound to the wearable electronic device 700; When not outputting, transmitting a signal for controlling the wearable electronic device 700 to output a sound corresponding to a notification message corresponding to one of the preset sounds to the wearable electronic device 700 may include

The electronic device according to various embodiments disclosed in this document may have various types of devices. The electronic device may include, for example, a portable communication device (eg, a smart phone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a home appliance device. The electronic device according to the embodiment of the present document is not limited to the above-described devices.

The various embodiments of this document and terms used therein are not intended to limit the technical features described in this document to specific embodiments, but it should be understood to include various modifications, equivalents, or substitutions of the embodiments. In connection with the description of the drawings, like reference numerals may be used for similar or related components. The singular form of the noun corresponding to the item may include one or more of the item, unless the relevant context clearly dictates otherwise. As used herein, "A or B", "at least one of A and B", "at least one of A or B", "A, B or C", "at least one of A, B and C", and "A , B, or C" each may include any one of the items listed together in the corresponding one of the phrases, or all possible combinations thereof. Terms such as “first”, “second”, or “first” or “second” may simply be used to distinguish an element from other such elements, and may refer elements to other aspects (e.g., importance or order) is not limited. It is said that one (eg, first) component is "coupled" or "connected" to another (eg, second) component, with or without the terms "functionally" or "communicatively". When referenced, it means that one component can be connected to the other component directly (eg by wire), wirelessly, or through a third component.

The term “module” used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and is interchangeable with terms such as, for example, logic, logic block, component, or circuit. can be used as A module may be an integrally formed part or a minimum unit or a part of the part that performs one or more functions. For example, according to an embodiment, the module may be implemented in the form of an application-specific integrated circuit (ASIC).

Various embodiments of the present document include one or more instructions stored in a storage medium (eg, the internal memory 136 or the external memory 138) readable by a machine (eg, the electronic device 101). may be implemented as software (eg, the program 140) including For example, the processor (eg, the processor 120 ) of the device (eg, the electronic device 101 ) may call at least one of one or more instructions stored from a storage medium and execute it. This makes it possible for the device to be operated to perform at least one function according to the called at least one command. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' only means that the storage medium is a tangible device and does not contain a signal (eg, electromagnetic wave), and this term is used in cases where data is semi-permanently stored in the storage medium and It does not distinguish between temporary storage cases.

According to one embodiment, the method according to various embodiments disclosed in this document may be provided as included in a computer program product. Computer program products may be traded between sellers and buyers as commodities. The computer program product is distributed in the form of a machine-readable storage medium (eg compact disc read only memory (CD-ROM)), or through an application store (eg Play Store™) or on two user devices ( It can be distributed (eg downloaded or uploaded) directly or online between smartphones (eg: smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created in a machine-readable storage medium such as a memory of a server of a manufacturer, a server of an application store, or a relay server.

According to various embodiments, each component (eg, a module or a program) of the above-described components may include a singular or a plurality of entities, and some of the plurality of entities may be separately disposed in other components. have. According to various embodiments, one or more components or operations among the above-described corresponding components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (eg, a module or a program) may be integrated into one component. In this case, the integrated component may perform one or more functions of each component of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. . According to various embodiments, operations performed by a module, program, or other component are executed sequentially, in parallel, repeatedly, or heuristically, or one or more of the operations are executed in a different order, or omitted. or one or more other operations may be added.

Claims (20)

In an electronic device,
communication circuit,
display,
at least one sensor, and
at least one processor;
the at least one processor,
Acquire sound data about the surrounding environment from the wearable electronic device through the communication circuit,
Obtaining sensor data through the at least one sensor,
based on the sound data and the sensor data, check whether the surrounding environment matches one of a plurality of preset situations,
In response to confirming that the surrounding environment matches one of a plurality of preset situations, based on the matching situation, the state of the display, and whether the wearable electronic device is outputting a sound, the wearable electronic device is configured The electronic device is configured to perform at least one of transmitting a control signal to the wearable electronic device through the communication circuit and displaying a notification message on the display. According to claim 1,
the at least one processor,
Checking whether the sound data is data related to a sound reproduced by another electronic device,
and check whether the surrounding environment matches one of a plurality of preset situations based on it being confirmed that the sound data is not data related to a sound reproduced by another electronic device. According to claim 1,
the at least one sensor comprises a vibration sensor,
the at least one processor,
Based on the vibration sensor data and the sound data acquired through the vibration sensor, it is confirmed that the surrounding environment matches a situation in which the user of the electronic device utters,
In response to determining that the surrounding environment matches a situation in which the user of the electronic device is uttering:
When the display is turned on, transmitting a signal for controlling the wearable electronic device to deactivate the noise canceling function, activate the ambient sound listening function, and stop outputting sound to the wearable electronic device,
When the display is turned off and the wearable electronic device is outputting a sound, a signal for controlling the wearable electronic device to deactivate a noise canceling function, activate an ambient sound listening function, and stop outputting the sound is transmitted to the wearable electronic device transmit to an electronic device;
When the display is turned off and the wearable electronic device is not outputting a sound, transmit a signal for controlling the wearable electronic device to deactivate the noise canceling function and activate the ambient sound listening function to the wearable electronic device Consisting of, an electronic device. According to claim 1,
the at least one processor,
In response to confirming that the sound data represents a sound corresponding to the preset text, confirming that the surrounding environment matches a situation in which another person is calling a user of the electronic device from among the plurality of preset situations,
In response to determining that the surrounding environment is consistent with a situation in which another person is calling the user of the electronic device:
When the display is turned on, display a notification message on the display,
When the display is turned off and the wearable electronic device is outputting a sound, displaying a notification message on the display and transmitting a signal for controlling the wearable electronic device to output a notification sound to the wearable electronic device,
When the display is turned off and the wearable electronic device is not outputting a sound, the electronic device is configured to transmit a signal for controlling the wearable electronic device to output a sound corresponding to a notification message to the wearable electronic device . 5. The method of claim 4,
When it is confirmed that the surrounding environment matches a situation in which another person is calling the user of the electronic device, the display is turned off, and the wearable electronic device is outputting a sound, the notification message displayed on the display is a head-up (heads-up) message, an electronic device. 5. The method of claim 4,
If it is confirmed that the surrounding environment matches a situation in which another person is calling the user of the electronic device, and the display is turned on, the notification message displayed on the display is a toast message. According to claim 1,
the at least one processor,
In response to confirming that the sound data represents a sound above a threshold level and represents a preset first pattern, confirming that the surrounding environment is consistent with a situation in which a loud noise is generated,
In response to confirming that the surrounding environment is consistent with a loud noise situation:
When the display is turned on and the wearable electronic device is outputting a sound, the wearable electronic device activates a noise canceling function, disables an ambient sound listening function, and increases the volume of a sound output by the wearable electronic device Transmitting a signal for controlling the wearable electronic device,
When the display is turned on and the wearable electronic device is not outputting a sound, a signal for controlling the wearable electronic device to activate a noise canceling function and deactivate an ambient sound listening function is transmitted to the wearable electronic device, ,
When the display is turned off and the wearable electronic device is outputting a sound, the wearable electronic device activates a noise canceling function, disables an ambient sound listening function, and increases the volume of a sound output by the wearable electronic device Transmitting a signal for controlling the wearable electronic device,
When the display is turned off and the wearable electronic device is not outputting a sound, transmit a signal for controlling the wearable electronic device to activate a noise canceling function and deactivate an ambient sound listening function to the wearable electronic device Consisting of, an electronic device. According to claim 1,
The at least one sensor comprises a GPS sensor,
the at least one processor,
When the GPS sensor data obtained through the GPS sensor indicates a predetermined location and the sound data indicates one of the preset sounds, confirm that the surrounding environment matches the indoor event situation among the plurality of preset situations, and ,
In response to determining that the surrounding environment is consistent with an indoor event situation:
When the display is turned on, a notification message corresponding to one of the preset sounds is displayed on the display,
When the display is turned off and the wearable electronic device is outputting a sound, displaying a notification message corresponding to one of the preset sounds on the display and controlling the wearable electronic device to output a notification sound sending a signal to the wearable electronic device;
When the display is turned off and the wearable electronic device is not outputting a sound, a signal for controlling the wearable electronic device to output a sound corresponding to a notification message corresponding to one of the preset sounds is transmitted to the wearable electronic device. An electronic device configured to transmit to an electronic device. 9. The method of claim 8,
The electronic device, wherein the predetermined location includes at least one of a location specified by a user input of the electronic device and a location frequently visited by the user of the electronic device. A method performed in an electronic device, comprising:
acquiring sound data about the surrounding environment from the wearable electronic device;
acquiring sensor data through at least one sensor of the electronic device;
based on the sound data and the sensor data, determining whether the surrounding environment matches one of a plurality of preset situations; and
In response to determining that the surrounding environment matches one of a plurality of preset situations, based on the matching situation, a state of a display of the electronic device, and whether the wearable electronic device is outputting a sound, the wearable A method comprising: transmitting a signal for controlling an electronic device to the wearable electronic device; and performing at least one of displaying a notification message on the display. 11. The method of claim 10,
The method further includes the operation of confirming whether the sound data is data related to a sound reproduced in another electronic device,
The operation of determining whether the surrounding environment matches one of a plurality of preset situations is performed based on confirming that the sound data is not data related to a sound reproduced in another electronic device. 11. The method of claim 10,
The operation of acquiring sensor data through the at least one sensor includes the operation of acquiring vibration sensor data through a vibration sensor,
The operation of determining whether the surrounding environment matches one of a plurality of preset situations may include, based on the vibration sensor data and the sound data, confirming that the surrounding environment matches a situation uttered by the user of the electronic device including actions to
Transmitting a signal for controlling the wearable electronic device to the wearable electronic device or displaying a notification message on the display when it is confirmed that the surrounding environment matches a situation in which the user of the electronic device utters The action of performing at least one of:
transmitting, to the wearable electronic device, a signal for controlling the wearable electronic device to deactivate a noise canceling function, activate an ambient sound listening function, and stop outputting sound when the display is turned on;
When the display is turned off and the wearable electronic device is outputting a sound, a signal for controlling the wearable electronic device to deactivate a noise canceling function, activate an ambient sound listening function, and stop outputting the sound is transmitted to the wearable electronic device sending to the electronic device, and
When the display is turned off and the wearable electronic device is not outputting a sound, transmitting a signal for controlling the wearable electronic device to deactivate the noise canceling function and activate the ambient sound listening function to the wearable electronic device A method comprising an action. According to claim 1,
The operation of determining whether the surrounding environment matches one of the plurality of preset situations may include, in response to it being confirmed that the sound data represents a sound corresponding to the preset text, that the surrounding environment matches the one of the plurality of preset situations. and confirming that it matches the situation in which another person is calling the user of the electronic device,
When it is confirmed that the surrounding environment matches a situation in which another person is calling the user of the electronic device, transmitting a signal for controlling the wearable electronic device to the wearable electronic device, or displaying a notification message on the display An action that does at least one of:
displaying a notification message on the display when the display is turned on;
When the display is turned off and the wearable electronic device is outputting a sound, displaying a notification message on the display and transmitting a signal for controlling the wearable electronic device to output a notification sound to the wearable electronic device , and
When the display is turned off and the wearable electronic device is not outputting a sound, transmitting a signal for controlling the wearable electronic device to output a sound corresponding to a notification message to the wearable electronic device, Way. 14. The method of claim 13,
When it is confirmed that the surrounding environment matches a situation in which another person is calling the user of the electronic device, the display is turned off, and the wearable electronic device is outputting a sound, the notification message displayed on the display is a head-up The (heads-up) message, the method. 14. The method of claim 13,
If it is confirmed that the surrounding environment matches a situation in which another person is calling the user of the electronic device, and the display is turned on, the notification message displayed on the display is a toast message. According to claim 1,
The operation of determining whether the surrounding environment matches one of a plurality of preset situations may include, in response to it being confirmed that the sound data represents a sound having a threshold level or higher and represents a preset first pattern, the surrounding environment is large including actions to ensure that the noise is consistent with the situation in which it occurs;
At least one of transmitting a signal for controlling the wearable electronic device to the wearable electronic device or displaying a notification message on the display when it is confirmed that the surrounding environment matches a situation in which a loud noise is generated The action to perform is:
When the display is turned on and the wearable electronic device is outputting a sound, the wearable electronic device activates a noise canceling function, disables an ambient sound listening function, and increases the volume of a sound output by the wearable electronic device transmitting a signal for controlling the wearable electronic device,
When the display is turned on and the wearable electronic device is not outputting sound, a signal for controlling the wearable electronic device to activate the noise canceling function and deactivate the ambient sound listening function to the wearable electronic device movement,
When the display is turned off and the wearable electronic device is outputting a sound, the wearable electronic device activates a noise canceling function, disables an ambient sound listening function, and increases the volume of a sound output by the wearable electronic device transmitting a signal for controlling the wearable electronic device to the wearable electronic device, and
When the display is turned off and the wearable electronic device is not outputting a sound, transmitting a signal for controlling the wearable electronic device to activate a noise canceling function and deactivate an ambient sound listening function to the wearable electronic device A method comprising an action. According to claim 1,
The operation of acquiring sensor data through the at least one sensor includes the operation of acquiring GPS sensor data through a GPS sensor,
The operation of determining whether the surrounding environment coincides with one of a plurality of preset situations may include, if the GPS sensor data indicates a preset location, and the sound data indicates one of preset sounds, the surrounding environment Including the operation of confirming that it matches the indoor event situation among the plurality of preset situations,
When it is confirmed that the surrounding environment matches the indoor event situation, at least one of transmitting a signal for controlling the wearable electronic device to the wearable electronic device or displaying a notification message on the display The action is:
displaying a notification message corresponding to one of the preset sounds on the display when the display is turned on;
When the display is turned off and the wearable electronic device is outputting a sound, displaying a notification message corresponding to one of the preset sounds on the display and controlling the wearable electronic device to output a notification sound transmitting a signal to the wearable electronic device; and
When the display is turned off and the wearable electronic device is not outputting a sound, a signal for controlling the wearable electronic device to output a sound corresponding to a notification message corresponding to one of the preset sounds is transmitted to the wearable electronic device. A method comprising transmitting to an electronic device. A wearable electronic device comprising:
communication circuit,
sound input device,
sound output device,
at least one sensor, and
at least one processor;
the at least one processor,
Forming a communication connection with an external electronic device through the communication circuit,
Acquire sound data about the surrounding environment through the sound input device,
Obtaining sensor data through the at least one sensor,
based on the sound data and the sensor data, check whether the surrounding environment matches one of a plurality of preset situations,
In response to determining that the surrounding environment matches one of a plurality of preset situations, based on the matching situation, a state of a display of the external electronic device, and whether the sound output device is outputting a sound, the The wearable electronic device is configured to perform at least one of controlling a sound output device or transmitting a signal to the external electronic device to display a notification on the display. 19. The method of claim 18,
the at least one processor,
Checking whether the sound data is data related to a sound reproduced by another electronic device,
and check whether the surrounding environment matches one of a plurality of preset situations based on it being confirmed that the sound data is not data related to a sound reproduced by another electronic device. 19. The method of claim 18,
the at least one sensor comprises a vibration sensor,
the at least one processor,
Based on the vibration sensor data and the sound data acquired through the vibration sensor, it is confirmed that the surrounding environment matches a situation in which the user of the electronic device utters,
In response to determining that the surrounding environment matches a situation in which the user of the electronic device is uttering:
when the display of the external electronic device is turned on, disable the noise canceling function, activate the ambient sound listening function, and control the sound output device to stop outputting sound,
When the display of the external electronic device is turned off and the wearable electronic device is outputting sound, the sound output device is controlled to deactivate the noise canceling function, activate the ambient sound listening function, and stop outputting the sound,
When the display of the external electronic device is turned off and the wearable electronic device is not outputting a sound, the wearable electronic device is configured to control the sound output device to deactivate a noise canceling function and activate an ambient sound listening function .

Images