KR102496225B1 - Method for video encoding and electronic device supporting the same - Google Patents

Abstract

The present invention relates to a video encoding method and an electronic device supporting the same, and in the video encoding method according to various embodiments of the present invention, an operation of receiving a signal related to video call reception from a first electronic device through a communication unit, a user Receiving a user input accepting the video call request through an interface, transmitting a signal accepting the video call request to the first electronic device through the communication unit, state information of the electronic device and the communication unit Receiving a video stream encoded based on state information from the first electronic device through the communication unit, displaying the received video stream and playing the received audio, and when a user's voice is received through a microphone, An operation of transmitting the received voice to the first electronic device through the communication unit may be included. Other embodiments may also be possible.

Description

Video encoding method and electronic device supporting the same {METHOD FOR VIDEO ENCODING AND ELECTRONIC DEVICE SUPPORTING THE SAME}

Various embodiments of the present disclosure relate to a video encoding method and an electronic device supporting the same.

With the development of hardware technology of electronic devices and the development of mobile communication technology, real-time video calls have become possible between electronic devices, and furthermore, real-time transmission of high-resolution image data has become possible.

When a real-time video call is connected between electronic devices, a user may be photographed through a camera of the electronic device, and the photographed video data may be encoded and transmitted to the other electronic device through a network. Such a real-time video call can be performed between two parties or multiple parties, and the electronic device can transmit a compressed stream by synchronizing video data with audio data to the other party's electronic device.

When an electronic device performs a video call with a counterpart's electronic device, the electronic device may encode and transmit a captured image to the counterpart's electronic device.

However, electronic devices generally encode image data for playback between fixed devices having high resolution and high battery level, and it may be difficult to reproduce the encoded image in a wearable device having low resolution and limited battery capacity. there is. In other words, the encoding operation is only suitable for reproducing image data between electronic devices having a high resolution and a high battery level, and a wearable device having a low resolution, limited transmission speed, and limited battery capacity receives the encoded image data. Below, not only does it take a long time due to the limited transmission speed, but also battery shortage may occur due to excessive power consumption.

An electronic device according to various embodiments of the present disclosure includes a communication unit communicating with a first electronic device, a touch screen displaying a user interface for performing a video call, one or more processors electrically connected to the communication unit and the touch screen, and and a memory electrically connected to the one or more processors, wherein, when executed, the one or more processors receive a signal related to video call reception from the first electronic device through the communication unit, and the user interface When a user input accepting the video call request is received through, a signal accepting the video call request is transmitted to the first electronic device through the communication unit, and state information of the electronic device and state information of the communication unit are transmitted. Receives a video stream encoded based on the basis from the first electronic device through the communication unit, displays the received encoded video stream, reproduces received audio, and transmits voice received through a microphone through the communication unit. 1 may include instructions for transmitting to an electronic device.

In the video encoding method according to various embodiments of the present disclosure, an operation of receiving a signal related to reception of a video call from a first electronic device through a communication unit, and an operation of receiving a user input accepting the video call request through a user interface. , Transmitting a signal accepting the video call request to the first electronic device through the communication unit, and transmitting a video stream encoded based on state information of the electronic device and status information of the communication unit from the first electronic device. receiving through the communication unit, displaying the received video stream and playing the received audio, and transmitting the received voice to the first electronic device through the communication unit when a user's voice is received through a microphone Actions may be included.

A video encoding method and an electronic device supporting the same according to various embodiments of the present invention provide video data suitable for performing a video call in a wearable device connected to the electronic device receiving the video call, so that the user can use the wearable device worn on the body. You can feel the convenience of being able to conduct video calls through the device.

A video encoding method according to various embodiments of the present invention and an electronic device supporting the same analyze the state of a wearable device in real time and provide encoded video data accordingly, thereby smoothly performing a video call even at a limited battery level and limited transmission speed. can do.

1 is a block diagram illustrating a network environment according to various embodiments of the present invention.
2 is a block diagram of an electronic device according to various embodiments of the present disclosure.
3 is a block diagram of program modules according to various embodiments of the present invention.
4 is a diagram illustrating connections between a first electronic device and a third electronic device according to various embodiments of the present disclosure.
5 is a diagram illustrating a configuration of an electronic device according to various embodiments of the present disclosure.
6 is a diagram illustrating a signal flow between a first electronic device and a third electronic device for explaining an image encoding method in the first electronic device according to various embodiments of the present disclosure.
7 is a diagram illustrating a signal flow between a first electronic device and a third electronic device for explaining an image encoding method in the first electronic device according to various embodiments of the present disclosure.
8 is a diagram illustrating a signal flow between a first electronic device and a third electronic device for explaining an image encoding method in the first electronic device according to various embodiments of the present disclosure.
9 is a diagram illustrating a signal flow between a first electronic device and a third electronic device for explaining an image encoding method in a second electronic device according to various embodiments of the present disclosure.
10 is a diagram for explaining an image encoding method performed in a first electronic device according to various embodiments of the present disclosure.
11 is a diagram for explaining an image encoding method performed in a first electronic device according to various embodiments of the present disclosure.
12 is a diagram for explaining a method of reproducing an encoded image in a third electronic device according to various embodiments of the present disclosure.
13 is a diagram for explaining an image encoding method performed in a second electronic device according to various embodiments of the present disclosure.
14A is a diagram for explaining an image encoding method performed in a second electronic device according to various embodiments of the present disclosure.
14B is a diagram for explaining a post process according to various embodiments of the present disclosure.
15 is a diagram for explaining an image encoding method performed in a second electronic device according to various embodiments of the present disclosure.
16 is a diagram for explaining a method of reproducing image data in a third electronic device according to various embodiments of the present disclosure.
17A and 17B are diagrams for explaining an image encoding method performed in a second electronic device according to various embodiments of the present disclosure.
18 is a diagram for explaining an image encoding method performed in a second electronic device according to various embodiments of the present disclosure.
19A and 19B are diagrams illustrating user interfaces related to receiving and sending a video call according to various embodiments of the present disclosure.
20A to 20D are diagrams illustrating a user interface for receiving a video call according to various embodiments of the present disclosure.
20E is a diagram for explaining a method of performing a video call in a third electronic device according to various embodiments of the present disclosure.
21A and 21B are views illustrating a user interface according to a pressure level of a user input according to various embodiments of the present disclosure.
21C is a diagram for explaining a function according to a pressure level of a user input according to various embodiments of the present disclosure.
21D is a diagram for explaining a function according to a pressure level of a user input according to various embodiments of the present disclosure.
21E is a diagram for explaining a function according to a direction in which a user input is sensed according to various embodiments of the present disclosure.
22A is a diagram illustrating a user interface including an indicator indicating quality of a video call according to various embodiments of the present disclosure.
22B is a diagram illustrating a method of adjusting quality of a video call according to various embodiments of the present disclosure.
23A and 23B are views illustrating a user interface for explaining an operation of switching a video call to a fourth electronic device according to various embodiments of the present disclosure.
23C is a flowchart illustrating an operation of switching a video call to a fourth electronic device according to various embodiments of the present disclosure.
24 is a diagram for explaining an operation of switching a video call to a fourth electronic device according to various embodiments of the present disclosure.
25 is a diagram illustrating a signal flow between a first electronic device, a second electronic device, and a fourth electronic device to describe a method of switching a video call to a fourth electronic device according to various embodiments of the present disclosure.
26 illustrates a signal flow between a first electronic device, a second electronic device, and a fourth electronic device for explaining a method of switching a video call to a fourth electronic device using an account management server according to various embodiments of the present disclosure. It is a drawing showing
27 is a diagram illustrating a user interface for explaining a function of converting a video call by searching for nearby electronic devices in a third electronic device according to various embodiments of the present disclosure.
28 is a diagram illustrating connections between first electronic devices and fourth electronic devices according to various embodiments of the present disclosure.
29 is a diagram illustrating a multiple connection method according to various embodiments of the present invention.
30 is a diagram for explaining an image data storage method according to various embodiments of the present disclosure.
31 is a diagram illustrating software according to various embodiments of the present invention.

Hereinafter, various embodiments of this document will be described with reference to the accompanying drawings. Examples and terms used therein are not intended to limit the technology described in this document to specific embodiments, and should be understood to include various modifications, equivalents, and/or substitutes of the embodiments. In connection with the description of the drawings, like reference numerals may be used for like elements. Singular expressions may include plural expressions unless the context clearly dictates otherwise. In this document, expressions such as "A or B" or "at least one of A and/or B" may include all possible combinations of the items listed together. Expressions such as "first," "second," "first," or "second," may modify the corresponding components regardless of order or importance, and are used to distinguish one component from another. It is used only and does not limit the corresponding components. When a (e.g., first) element is referred to as being "(functionally or communicatively) coupled to" or "connected to" another (e.g., second) element, that element refers to the other (e.g., second) element. It may be directly connected to the component or connected through another component (eg, a third component).

In this document, "configured (or configured to)" means "suitable for," "having the ability to," "changed to," depending on the situation, for example, hardware or software. ," can be used interchangeably with "made to," "capable of," or "designed to." In some contexts, the expression "device configured to" can mean that the device is "capable of" in conjunction with other devices or components. For example, the phrase "a processor configured (or configured) to perform A, B, and C" may include a dedicated processor (eg, embedded processor) to perform the operation, or by executing one or more software programs stored in a memory device. , may mean a general-purpose processor (eg, CPU or application processor) capable of performing corresponding operations.

Electronic devices according to various embodiments of the present document include, for example, a smart phone, a tablet PC, a mobile phone, a video phone, an e-book reader, a desktop PC, a laptop PC, a netbook computer, a workstation, a server, a PDA, and a PMP. It may include at least one of a portable multimedia player, an MP3 player, a medical device, a camera, or a wearable device. A wearable device may be in the form of an accessory (e.g. watch, ring, bracelet, anklet, necklace, eyeglasses, contact lens, or head-mounted-device (HMD)), integrated into textiles or clothing (e.g. electronic garment); In some embodiments, the electronic device may include, for example, a television, a digital video disk (DVD) player, Audio, refrigerator, air conditioner, vacuum cleaner, oven, microwave, washing machine, air purifier, set top box, home automation control panel, security control panel, media box (e.g. Samsung HomeSyncTM, Apple TVTM, or Google TVTM), game console (eg, XboxTM, PlayStationTM), an electronic dictionary, an electronic key, a camcorder, or an electronic photo frame.

In another embodiment, the electronic device may include various types of medical devices (e.g., various portable medical measuring devices (such as blood glucose meter, heart rate monitor, blood pressure monitor, or body temperature monitor), magnetic resonance angiography (MRA), magnetic resonance imaging (MRI), CT (computed tomography), imager, or ultrasonicator, etc.), navigation device, global navigation satellite system (GNSS), EDR (event data recorder), FDR (flight data recorder), automobile infotainment device, marine electronic equipment (e.g. navigation devices for ships, gyrocompasses, etc.), avionics, security devices, head units for vehicles, industrial or home robots, drones, ATMs in financial institutions, point of sale (POS) in stores of sales), or IoT devices (eg, light bulbs, various sensors, sprinkler devices, fire alarms, thermostats, street lights, toasters, exercise equipment, hot water tanks, heaters, boilers, etc.). According to some embodiments, the electronic device may be a piece of furniture, a building/structure or a vehicle, an electronic board, an electronic signature receiving device, a projector, or various measuring devices (eg, water, electricity, gas, radio wave measuring device, etc.). In various embodiments, the electronic device may be flexible or a combination of two or more of the various devices described above. An electronic device according to an embodiment of the present document is not limited to the aforementioned devices. In this document, the term user may refer to a person using an electronic device or a device using an electronic device (eg, an artificial intelligence electronic device).

1 is a block diagram illustrating a network environment according to various embodiments of the present invention.

Referring to FIG. 1 , an electronic device 101 in a network environment 100 in various embodiments is described. The electronic device 101 may include a bus 110, a processor 120, a memory 130, an input/output interface 150, a display 160, and a communication interface 170. In some embodiments, the electronic device 101 may omit at least one of the components or may additionally include other components. Bus 110 may include circuitry that connects components 110-170 to each other and carries communications (eg, control messages or data) between components. The processor 120 may include one or more of a central processing unit, an application processor, or a communication processor (CP). The processor 120 may, for example, execute calculations or data processing related to control and/or communication of at least one other component of the electronic device 101 .

The memory 130 may include volatile and/or non-volatile memory. The memory 130 may store, for example, commands or data related to at least one other component of the electronic device 101 . According to one embodiment, memory 130 may store software and/or programs 140 . The program 140 may include, for example, a kernel 141 , middleware 143 , an application programming interface (API) 145 , and/or an application program (or “application”) 147 . At least part of the kernel 141, middleware 143, or API 145 may be referred to as an operating system. Kernel 141 is responsible for, for example, system resources (eg, bus 110, processor 120, or memory 130, etc.) may be controlled or managed. In addition, the kernel 141 may provide an interface capable of controlling or managing system resources by accessing individual components of the electronic device 101 from the middleware 143, API 145, or application program 147.

The middleware 143 may perform an intermediary role so that, for example, the API 145 or the application program 147 communicates with the kernel 141 to exchange data. Also, the middleware 143 may process one or more job requests received from the application program 147 according to priority. For example, the middleware 143 assigns a priority for using system resources (eg, a bus 110, a processor 120, or a memory 130, etc.) of the electronic device 101 to at least one of the application programs 147, and processes the one or more task requests. can be dealt with API 145 is an interface for the application 147 to control functions provided by the kernel 141 or middleware 143, and includes, for example, at least one interface or function for file control, window control, image processing, or character control (e.g., command) may be included. The input/output interface 150 transmits, for example, a command or data input from a user or other external device to other element(s) of the electronic device 101, or a command received from another element(s) of the electronic device 101. Alternatively, data can be output to a user or other external device.

The display 160 may include, for example, a liquid crystal display (LCD), a light emitting diode (LED) display, an organic light emitting diode (OLED) display, a microelectromechanical system (MEMS) display, or an electronic paper display. can The display 160 may display various types of content (eg, text, image, video, icon, and/or symbol) to the user. The display 160 may include a touch screen, and may receive, for example, a touch, gesture, proximity, or hovering input using an electronic pen or a part of the user's body. The communication interface 170 may establish communication between the electronic device 101 and an external device (eg, the first external electronic device 102 , the second external electronic device 104 , or the server 106 ). For example, the communication interface 170 may be connected to the network 162 through wireless or wired communication to communicate with an external device (eg, the second external electronic device 104 or the server 106).

Wireless communication includes, for example, LTE, LTE Advance (LTE-A), code division multiple access (CDMA), wideband CDMA (WCDMA), universal mobile telecommunications system (UMTS), wireless broadband (WiBro), or global system for mobile communications) and the like. According to one embodiment, wireless communication, for example, WiFi (wireless fidelity), Bluetooth, Bluetooth Low Energy (BLE), Zigbee (zigbee), near field communication (NFC), magnetic secure transmission (magnetic secure transmission), radio It may include at least one of a frequency (RF) and a body area network (BAN). According to one embodiment, wireless communication may include GNSS. GNSS may be, for example, a global positioning system (GPS), a global navigation satellite system (Glonass), a Beidou Navigation Satellite System (hereinafter “Beidou”) or Galileo, the European global satellite-based navigation system. Hereinafter, in this document, “GPS” may be used interchangeably with “GNSS”. Wired communication may include, for example, at least one of universal serial bus (USB), high definition multimedia interface (HDMI), recommended standard 232 (RS-232), power line communication, or plain old telephone service (POTS). there is. Network 162 may include at least one of a telecommunications network, for example, a computer network (eg, LAN or WAN), the Internet, or a telephone network.

Each of the first and second external electronic devices 102 and 104 may be the same as or different from the electronic device 101 . According to various embodiments, all or part of operations executed in the electronic device 101 may be executed in one or more electronic devices (eg, the electronic device 102, 104, or the server 106. According to one embodiment, the electronic device When the electronic device 101 needs to perform a certain function or service automatically or upon request, the electronic device 101, instead of or in addition to executing the function or service by itself, performs at least some of the functions related to it with another device (e.g., an electronic device). 102, 104, or server 106). Another electronic device (eg, the electronic device 102, 104, or server 106) may execute the requested function or additional function and deliver the result to the electronic device 101. The electronic device 101 may provide the requested function or service by directly or additionally processing the received result, for which, for example, cloud computing, distributed computing, or client-server computing technology may be used.

2 is a block diagram of an electronic device according to various embodiments of the present disclosure.

The electronic device 201 may include all or part of the electronic device 101 shown in FIG. 1 , for example. Electronic device 201 includes one or more processors (eg AP) 210, communication module 220, subscriber identity module 224, memory 230, sensor module 240, input device 250, display 260, interface 270, audio module 280, camera module 291, power management module 295, battery 296, indicator 297, and motor 298. The processor 210 may control a plurality of hardware or software components connected to the processor 210 by driving, for example, an operating system or an application program, and may perform various data processing and calculations. The processor 210 may be implemented as, for example, a system on chip (SoC). According to one embodiment, the processor 210 may further include a graphic processing unit (GPU) and/or an image signal processor. The processor 210 may include at least some of the components shown in FIG. 2 (eg, the cellular module 221). The processor 210 may load and process a command or data received from at least one of other components (eg, non-volatile memory) into a volatile memory, and store resultant data in the non-volatile memory.

It may have the same or similar configuration as the communication module 220 (eg, the communication interface 170). The communication module 220 may include, for example, a cellular module 221, a WiFi module 223, a Bluetooth module 225, a GNSS module 227, an NFC module 228, and an RF module 229. The cellular module 221 may provide, for example, a voice call, a video call, a text service, or an Internet service through a communication network. According to an embodiment, the cellular module 221 may perform identification and authentication of the electronic device 201 within a communication network using the subscriber identity module (eg, SIM card) 224 . According to one embodiment, the cellular module 221 may perform at least some of the functions that the processor 210 can provide. According to one embodiment, the cellular module 221 may include a communication processor (CP). According to some embodiments, at least some (eg, two or more) of the cellular module 221, the WiFi module 223, the Bluetooth module 225, the GNSS module 227, or the NFC module 228 may be included in one integrated chip (IC) or IC package. . The RF module 229 may transmit and receive communication signals (eg, RF signals), for example. The RF module 229 may include, for example, a transceiver, a power amp module (PAM), a frequency filter, a low noise amplifier (LNA), or an antenna. According to another embodiment, at least one of the cellular module 221, the WiFi module 223, the Bluetooth module 225, the GNSS module 227, or the NFC module 228 may transmit and receive RF signals through a separate RF module. The subscriber identification module 224 may include, for example, a card or embedded SIM including a subscriber identification module, and may include unique identification information (eg, integrated circuit card identifier (ICCID)) or subscriber information (eg, international IMSI (international information)). mobile subscriber identity)).

The memory 230 (eg, the memory 130) may include, for example, an internal memory 232 or an external memory 234. The built-in memory 232 includes, for example, volatile memory (eg, DRAM, SRAM, or SDRAM, etc.), non-volatile memory (eg, OTPROM (one time programmable ROM), PROM, EPROM, EEPROM, mask ROM, flash ROM, flash ROM, etc.) It may include at least one of a memory, a hard drive, or a solid state drive (SSD) The external memory 234 may include a flash drive, for example, a compact flash (CF), secure digital (SD), micro- It may include SD, Mini-SD, extreme digital (xD), multi-media card (MMC), memory stick, etc. The external memory 234 may be functionally or physically connected to the electronic device 201 through various interfaces.

The sensor module 240 may, for example, measure a physical quantity or detect an operating state of the electronic device 201 and convert the measured or sensed information into an electrical signal. The sensor module 240 includes, for example, a gesture sensor 240A, a gyro sensor 240B, an air pressure sensor 240C, a magnetic sensor 240D, an acceleration sensor 240E, a grip sensor 240F, a proximity sensor 240G, and a color sensor 240H (eg, RGB (red, green, blue) sensor), a bio sensor 240I, a temperature/humidity sensor 240J, an illuminance sensor 240K, or an ultra violet (UV) sensor 240M. Additionally or alternatively, the sensor module 240 may include, for example, an e-nose sensor, an electromyography (EMG) sensor, an electroencephalogram (EEG) sensor, an electrocardiogram (ECG) sensor, an IR ( infrared) sensor, an iris sensor, and/or a fingerprint sensor. The sensor module 240 may further include a control circuit for controlling one or more sensors included therein. In some embodiments, the electronic device 201 may further include a processor configured to control the sensor module 240 as part of the processor 210 or separately to control the sensor module 240 while the processor 210 is in a sleep state. .

The input device 250 may include, for example, a touch panel 252 , a (digital) pen sensor 254 , a key 256 , or an ultrasonic input device 258 . The touch panel 252 may use at least one of, for example, a capacitive type, a pressure-sensitive type, an infrared type, or an ultrasonic type. Also, the touch panel 252 may further include a control circuit. The touch panel 252 may further include a tactile layer to provide a tactile response to the user. The (digital) pen sensor 254 may be, for example, a part of the touch panel or may include a separate recognition sheet. Key 256 may include, for example, a physical button, an optical key, or a keypad. The ultrasonic input device 258 may detect ultrasonic waves generated from an input tool through a microphone (eg, the microphone 288) and check data corresponding to the detected ultrasonic waves.

The display 260 (eg, the display 160) may include a panel 262, a hologram device 264, a projector 266, and/or a control circuit for controlling them. Panel 262 may be implemented to be flexible, transparent, or wearable, for example. The panel 262 may include the touch panel 252 and one or more modules. According to one embodiment, the panel 262 may include a pressure sensor (or force sensor) capable of measuring the strength of a user's touch. The pressure sensor may be implemented integrally with the touch panel 252 or may be implemented as one or more sensors separate from the touch panel 252 . The hologram device 264 may display a 3D image in the air using interference of light. The projector 266 may display an image by projecting light onto a screen. The screen may be located inside or outside the electronic device 201, for example. The interface 270 may include, for example, an HDMI 272, a USB 274, an optical interface 276, or a D-subminiature (D-subminiature) 278. Interface 270 may be included in, for example, the communication interface 170 shown in FIG. 1 . Additionally or alternatively, the interface 270 may include, for example, a mobile high-definition link (MHL) interface, an SD card/multi-media card (MMC) interface, or an infrared data association (IrDA) standard interface.

The audio module 280 may, for example, convert a sound and an electrical signal in both directions. At least some components of the audio module 280 may be included in the input/output interface 145 shown in FIG. 1 , for example. The audio module 280 may process sound information input or output through, for example, a speaker 282, a receiver 284, an earphone 286, or a microphone 288. The camera module 291 is, for example, a device capable of capturing still images and moving images, and according to one embodiment, one or more image sensors (eg, a front sensor or a rear sensor), a lens, an image signal processor (ISP), or May contain a flash (e.g. LED or xenon lamp). The power management module 295 may manage power of the electronic device 201 , for example. According to one embodiment, the power management module 295 may include a power management integrated circuit (PMIC), a charger IC, or a battery or fuel gauge. A PMIC may have a wired and/or wireless charging method. The wireless charging method includes, for example, a magnetic resonance method, a magnetic induction method, or an electromagnetic wave method, and may further include an additional circuit for wireless charging, for example, a coil loop, a resonance circuit, or a rectifier. there is. The battery gauge may measure, for example, remaining capacity of the battery 296, voltage, current, or temperature during charging. Battery 296 may include, for example, a rechargeable battery and/or a solar cell.

The indicator 297 may indicate a specific state of the electronic device 201 or a part thereof (eg, the processor 210), for example, a booting state, a message state, or a charging state. The motor 298 may convert electrical signals into mechanical vibrations and generate vibrations or haptic effects. The electronic device 201 is, for example, a mobile TV support device (eg, GPU) capable of processing media data according to standards such as digital multimedia broadcasting (DMB), digital video broadcasting (DVB), or mediaFloTM. can include Each of the components described in this document may be composed of one or more components, and the name of the corresponding component may vary depending on the type of electronic device. In various embodiments, the electronic device (eg, the electronic device 201) is composed of a single entity by omitting some components, further including additional components, or combining some of the components, but prior to combining The functions of the corresponding components may be equally performed.

3 is a block diagram of program modules according to various embodiments of the present invention. According to one embodiment, the program module 310 (eg, the program 140) may include an operating system (eg, the electronic device 101) that controls resources related to the electronic device (eg, the electronic device 101) and/or various applications (eg, the application program 147) running on the operating system. can include The operating system may include, for example, Android™, iOS™, Windows™, Symbian™, Tizen™, or Bada™. Referring to FIG. 3 , a program module 310 includes a kernel 320 (eg, kernel 141), middleware 330 (eg, middleware 143), API 360 (eg, API 145), and/or an application 370 (eg, application program 147). can include At least a part of the program module 310 is preloaded on the electronic device or can be downloaded from an external electronic device (eg, the electronic device 102, 104, server 106, etc.).

The kernel 320 may include, for example, a system resource manager 321 and/or a device driver 323 . The system resource manager 321 may control, allocate, or recover system resources. According to one embodiment, the system resource manager 321 may include a process management unit, a memory management unit, or a file system management unit. The device driver 323 may include, for example, a display driver, a camera driver, a Bluetooth driver, a shared memory driver, a USB driver, a keypad driver, a WiFi driver, an audio driver, or an inter-process communication (IPC) driver. The middleware 330 may, for example, provide functions commonly required by the application 370 or provide various functions to the application 370 through the API 360 so that the application 370 can use limited system resources inside the electronic device. According to one embodiment, the middleware 330 includes a runtime library 335, an application manager 341, a window manager 342, a multimedia manager 343, a resource manager 344, a power manager 345, a database manager 346, a package manager 347, a connectivity manager 348, and a notification manager 349. , a location manager 350, a graphic manager 351, or a security manager 352.

The runtime library 335 may include, for example, a library module used by a compiler to add new functions through a programming language while the application 370 is being executed. The runtime library 335 may perform I/O management, memory management, or arithmetic function processing. The application manager 341 may manage the life cycle of the application 370, for example. The window manager 342 may manage GUI resources used in the screen. The multimedia manager 343 may identify a format required for reproducing media files, and encode or decode the media file using a codec suitable for the format. The resource manager 344 may manage the source code of the application 370 or memory space. The power manager 345 may manage, for example, battery capacity or power, and provide power information necessary for the operation of the electronic device. According to one embodiment, the power manager 345 may interoperate with a basic input/output system (BIOS). The database manager 346 may create, search, or change a database to be used in the application 370, for example. The package manager 347 may manage installation or update of applications distributed in the form of package files.

The connectivity manager 348 may manage wireless connections, for example. The notification manager 349 may provide a user with an event such as an arrival message, an appointment, or proximity notification. The location manager 350 may manage, for example, location information of an electronic device. The graphic manager 351 may manage, for example, graphic effects to be provided to users or user interfaces related thereto. Security manager 352 may provide system security or user authentication, for example. According to one embodiment, the middleware 330 may include a telephony manager for managing voice or video call functions of an electronic device or a middleware module capable of forming a combination of functions of the aforementioned components. According to one embodiment, the middleware 330 may provide modules specialized for each type of operating system. The middleware 330 may dynamically delete some existing components or add new components. The API 360 is, for example, a set of API programming functions, and may be provided in different configurations depending on the operating system. For example, in the case of Android or iOS, one API set can be provided for each platform, and in the case of Tizen, two or more API sets can be provided for each platform.

Applications 370 include, for example, home 371, dialer 372, SMS/MMS 373, IM (instant message) 374, browser 375, camera 376, alarm 377, contacts 378, voice dial 379, email 380, calendar 381, media player 382, album 383, watch 384, health care (eg, measuring exercise or blood sugar, etc.), or environmental information (eg, atmospheric pressure, humidity, or temperature information) providing applications. According to one embodiment, the application 370 may include an information exchange application capable of supporting information exchange between an electronic device and an external electronic device. The information exchange application may include, for example, a notification relay application for delivering specific information to an external electronic device or a device management application for managing an external electronic device. For example, a notification delivery application may transfer notification information generated by another application of an electronic device to an external electronic device or may receive notification information from an external electronic device and provide the notification information to a user. The device management application is, for example, a function of an external electronic device communicating with the electronic device (eg, turn-on/turn-off of the external electronic device itself (or some component parts) or display brightness (or resolution)). adjustment), or an application operating in an external electronic device may be installed, deleted, or updated. According to an embodiment, the application 370 may include an application designated according to the properties of an external electronic device (eg, a health management application of a mobile medical device). According to one embodiment, the application 370 may include an application received from an external electronic device. At least a portion of the program module 310 may be implemented (eg, executed) in software, firmware, hardware (eg, the processor 210), or a combination of at least two of them, and may include modules, programs, routines, It may contain an instruction set or process.

The term "module" used in this document includes a unit composed of hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. A “module” may be an integrally constructed component or a minimal unit or part thereof that performs one or more functions. A "module" may be implemented mechanically or electronically, for example, a known or future developed application-specific integrated circuit (ASIC) chip, field-programmable gate arrays (FPGAs), or A programmable logic device may be included. At least some of devices (eg, modules or functions) or methods (eg, operations) according to various embodiments are implemented as instructions stored in a computer-readable storage medium (eg, memory 130) in the form of program modules. It can be. When the command is executed by a processor (eg, the processor 120), the processor may perform a function corresponding to the command. Computer-readable recording media include hard disks, floppy disks, magnetic media (e.g. magnetic tape), optical recording media (e.g. CD-ROM, DVD, magneto-optical media (e.g. floptical disks), built-in memory, etc.) A command may include code generated by a compiler or code executable by an interpreter A module or program module according to various embodiments may include at least one or more of the above-described components or , some may be omitted, or may further include other elements.According to various embodiments, operations performed by modules, program modules, or other elements may be executed sequentially, in parallel, iteratively, or heuristically, or at least Some actions may be performed in a different order, omitted, or other actions may be added.

4 is a diagram illustrating connections between a first electronic device and a third electronic device according to various embodiments of the present disclosure.

Referring to FIG. 4 , in various embodiments of the present disclosure, a first electronic device 401 is a transmitting electronic device that transmits a video call request signal, and a second electronic device 403 is a transmitting electronic device that receives a video call request signal from the first electronic device 401. It is assumed that the receiving electronic device, the third electronic device 405, is an electronic device that is connected to the second electronic device 403, receives a video call request signal from the second electronic device 403, and performs a video call with the first electronic device 401. Explain. For example, the third electronic device 405 may be a wearable device having a display size of, for example, 4 inches or less.

In various embodiments of the present disclosure, the first electronic device 401 and the second electronic device 403 may be connected through a first network 411, for example, 3G or LTE, and the second electronic device 403 and the third electronic device 405 The second network 413 may be connected through a low power network such as Bluetooth or wifi.

In various embodiments of the present disclosure, the display size of the third electronic device 405 may be smaller than the display sizes of the first electronic device 401 and the second electronic device 403 . For example, the display size of the third electronic device 405 may be 4 inches, and the display sizes of the first electronic device 401 and the second electronic device 403 may be 7 inches. As the display size of the third electronic device 405 is small, the resolution supported by the third electronic device 405 may be lower than the resolution supported by the first electronic device 401 and the second electronic device 403 . As described above, the third electronic device 405 operates by receiving a control signal from the second electronic device 403 connected through a low power network such as a second network 413 such as Bluetooth or wifi, or operates through a sensor unit of the third electronic device 405. It can operate through signals sensed through.

In various embodiments of the present invention, video data for a third electronic device 405 connected to the first electronic device 401 and the second electronic device 403 through the second network 413 to perform a video call is the first electronic device 401 or the second electronic device 403 . 2 It may be processed (eg, encoded) through the electronic device 403 .

In various embodiments of the present disclosure, an operation of processing video data for performing a video call by the first electronic device 401 will be described in detail with reference to FIGS. 6 to 8 to be described later.

In various embodiments of the present disclosure, an operation of processing video data for performing a video call by the second electronic device 403 will be described in detail with reference to FIG. 9 .

5 is a diagram illustrating a configuration of an electronic device according to various embodiments of the present disclosure.

Referring to FIG. 5 , an electronic device according to various embodiments of the present disclosure may include a communication unit 510, a memory 520, a touch screen 530, a sensor unit 540, a camera 550, an audio processing unit 560, and a processor 570.

In various embodiments of the present invention, the electronic device includes a transmitting electronic device requesting a video call, for example, the first electronic device 401 of FIG. 4 , a receiving electronic device receiving the video call, for example, a second electronic device 403 of FIG. 4 , An electronic device that accepts the video call request and performs the video call may operate as, for example, the third electronic device 405 of FIG. 4 .

In various embodiments of the present invention, the communication unit 510 (eg, the communication interface 170 of FIG. 1 and the communication module 220 of FIG. 2 ) may include a first network and a second network. The first network may connect communication between a transmitting electronic device requesting the video call, e.g., the first electronic device 401 of FIG. 4 and a receiving electronic device receiving the video call, e.g., the second electronic device 403 of FIG. 4 . The second network includes a receiving electronic device that receives the video call, e.g., the second electronic device 403 of FIG. 4 and an electronic device that accepts the video call request and performs the video call, e.g., the third electronic device 405 of FIG. of communication can be connected. A video call request signal and an acceptance signal for the video call request signal may be transmitted and received between electronic devices through the first network and the second network. Video data for performing a video call between electronic devices and information of each electronic device may be transmitted and received through the first network and the second network.

In various embodiments of the present invention, the memory 520 (eg, the memory 130 of FIG. 1 and the memory 230 of FIG. 2 ) includes information about at least one electronic device capable of performing a video call (eg, the information of the at least one electronic device). It can store the type, display size, shape (eg round, rectangular, or square), battery capacity, screen resolution, or CPU power. The memory 520 may map and store a user interface or function displayed according to a user input, a holding time of the user input, a hovering input level, a pressure level of the user input, or a direction in which the user input is sensed. there is.

In various embodiments of the present invention, the touch screen 530 may be integrally configured including a display unit 531 (eg, the display 160 of FIG. 1 and the display 260 of FIG. 2 ) and a touch panel 533 (eg, the input device 250 of FIG. 2 ). there is.

In various embodiments of the present disclosure, the display unit 531 may display a user interface for controlling functions related to video calls. The display unit 531 may display video data for performing the video call. When a user input is detected under the control of the processor 570, the display unit 531 may display a user interface mapped to the holding time of the detected user input, the pressure level of the detected user input, or the direction in which the user input is sensed. there is. Alternatively, when detecting a hovering input under the control of the processor 570, the display unit 531 may analyze the level of the sensed hovering input and display a user interface mapped to the analyzed hovering input level.

In various embodiments of the present invention, the sensor unit 540 (eg, the sensor module 240 of FIG. 2 ) measures a physical quantity or detects an operating state of an electronic device, and converts the measured or sensed information into an electrical signal. The sensor unit 540 may include, for example, at least one of a gesture sensor, a gyro sensor, an acceleration sensor, and a geomagnetic sensor. In various embodiments of the present disclosure, a movement angle of the head of the electronic device may be measured using the earth magnetic sensor. In addition, the movement of the electronic device may be detected using the acceleration sensor and the gyro sensor. The sensor unit 540 may transmit sensor information according to the measured movement angle and movement of the electronic device to the processor 570 . The sensor unit 540 may collect external situation information about the electronic device including at least one of weather information, brightness information, and sound information.

In various embodiments of the present invention, when an electronic device operates as the first electronic device 401 (eg, a transmitting electronic device requesting a video call), a camera 550 (eg, the camera module 291 of FIG. 2 ) captures an image of a subject. And, under the control of the processor 570, image data of the photographed subject may be transmitted to the second electronic device 403 through the communication unit 510.

In various embodiments of the present invention, when the electronic device operates as the third electronic device 405 (eg, an electronic device that accepts a video call request and performs a video call), the camera 550 captures an image of a subject, and the processor 570 Image data of the photographed subject may be transmitted to the second electronic device 403 through the communication unit 510 under the control of . The camera 550 may recognize at least one of the user's face, face, or eyes from the image data of the photographed subject. In various embodiments of the present invention, the audio processing unit 560 (eg, the audio module 280 of FIG. 2 ) A speaker (SPK, speaker) 561 for outputting audio signals transmitted and received during a call (singing tone or receiving tone), an audio signal included in a received message, and an audio signal according to playback of an audio file stored in the memory 433, etc. It may be connected to at least one microphone (MIC, microphone) 563 for collecting voice or other audio signals.

In various embodiments of the present invention, when receiving a voice for controlling a function related to a video call through the microphone 563, the audio processing unit 560 may transmit the voice signal to the processor 570. The audio processor 560 may output audio extracted under the control of the processor 570 from image data received from the first electronic device 401 through the speaker 561 . The audio processing unit 560 may output an audio signal related to notification of video call reception and video call conversion through the speaker 561 .

In various embodiments of the present invention, when an electronic device operates as the first electronic device 401 (eg, a transmitting electronic device requesting a video call), the processor 570 (eg, the processor 120 in FIG. 1 or the processor 210 in FIG. 2 ) A video call is connected to the second electronic device 403 (eg, a receiving electronic device that receives a video call) or a third electronic device 405 (eg, an electronic device that accepts a video call request and performs the video call) through the communication unit 510 can request When receiving an acceptance signal for the video call connection request from the second electronic device 403 or the third electronic device 405, the processor 570 receives state information of the second electronic device 403, state information of the third electronic device 405, and network connection. information can be obtained.

In various embodiments of the present invention, the processor 570 may determine an encoding scheme for image data (eg, a video stream) photographed by the camera 550 based on the obtained information. The processor 570 may encode the image data based on the determined encoding scheme and transmit the encoded image data to the second electronic device 403 or the third electronic device 405 . Alternatively, the processor 570 may generate a plurality of images based on the image data, extract an audio signal, and transmit the extracted audio signal to the second electronic device 403 or the third electronic device 405 . The processor 570 checks at least one of maximum resolution, maximum quality, or maximum fps supported by the second electronic device 403 or the third electronic device 405, and based on the state information of the second network, the maximum resolution, maximum quality, Alternatively, at least one of the maximum fps may be adjusted, and image data may be encoded based on at least one of the adjusted resolution, quality, or fps. In various embodiments of the present disclosure, when receiving video data for the third electronic device 405 from the second electronic device 403, the processor 570 determines an encoding scheme based on the video data for the third electronic device 405, Image data may be encoded based on the determined encoding scheme. In various embodiments of the present invention, when receiving a quality adjustment value (eg, a picture quality change value, a resolution change value, or an fps change value) of image data from the second electronic device 403, the processor 570 outputs the received quality adjustment value and The image data may be encoded and transmitted to the second electronic device 403 based on a transmission rate according to a network condition.

In various embodiments of the present invention, the operation of determining the above-described encoding scheme and the operation of encoding image data based on the determined encoding scheme may be performed in the same way in the first electronic device 401 and the second electronic device 403. .

In various embodiments of the present disclosure, when the electronic device operates as the second electronic device 403 (eg, a receiving electronic device receiving a video call), the processor 570 transmits the first electronic device 401 (eg, video call) through the communication unit 431. When a video call request signal is received from the requesting sending electronic device), the video call request signal may be transmitted to the third electronic device 405 (eg, an electronic device that accepts the video call request and performs the video call). When receiving an acceptance signal for the video call connection request from the third electronic device 405, the processor 570 may transmit the acceptance signal to the first electronic device 401. The processor 570 may transmit and receive video data and control signals for performing a video call between the first electronic device 401 and the third electronic device 305 .

In various embodiments of the present disclosure, when the electronic device operates as the third electronic device 405 (eg, an electronic device that accepts a video call request and performs a video call), the processor 570 connects the second electronic device 403 (through the communication unit 510). For example, a video call request signal may be received from a receiving electronic device receiving a video call. When a user input accepting the video call request is detected, the processor 570 may transmit an acceptance signal for the video call request to the second electronic device 403 . The processor 570 may receive and reproduce encoded video data from the second electronic device 403 based on the state information of the third electronic device 405 and the second network information. Alternatively, in various embodiments of the present disclosure, the processor 570 may receive and reproduce a plurality of images and audio signals generated based on the state information of the third electronic device 405 and the second network information from the second electronic device 403 and reproduce them. there is.

In various embodiments of the present disclosure, the processor 570 may capture an image of a subject through the camera 550 and transmit image data of the captured subject to the second electronic device 403 . Image data transmitted to the second electronic device 403 may be image data encoded based on state information about the third electronic device 405 and second network state information. In various embodiments of the present invention, the processor 570 may display the encoded video data and display an indicator capable of adjusting the quality of the video data. The processor 570 may detect an input for adjusting the quality of image data through the indicator. The processor 570 may determine an adjusted quality value (eg, a picture quality change value, a resolution change value, or an fps change value) in response to an input for adjusting the sensed quality, and transmit the adjusted quality value to the second electronic device 403 . The processor 570 may receive and reproduce video data encoded based on the adjusted quality value from the second electronic device 403 . The processor 570 may detect movement of the third electronic device 405 through the sensor unit 540 . The processor 570 may collect sensor information about the movement of the third electronic device 405 and reproduce image data and/or audio signals based on the collected sensor information. For example, the processor 570 may reproduce an audio signal and store the image data in the memory 520 when the movement of the electronic device is within a preset range based on the sensor information. The processor 570 may reproduce image data stored in the memory 520 when the movement of the electronic device is not within a preset range. When a signal for switching the video call is detected while a video call is being performed, the processor 570 may search for and display at least one electronic device capable of switching the video call. The processor 570 may transmit a video call switching signal to the selected electronic device when detecting an input for selecting an electronic device to switch the video call from among the at least one displayed electronic device for a preset time. The processor 570 analyzes state information and second network information about the at least one displayed electronic device when an input for selecting an electronic device to transfer the video call from among the at least one displayed electronic device is not sensed for a preset time. priorities are given, and an electronic device having a higher priority is automatically selected as an electronic device to convert a video call to.

In various embodiments of the present invention, an electronic device includes a communication unit communicating with a first electronic device, a touch screen displaying a user interface for performing a video call, one or more processors electrically connected to the communication unit and the touch screen, and one or more processors electrically connected to the communication unit and the touch screen. and a memory electrically connected to one or more processors, wherein the one or more processors, when executed, receive a signal related to video call reception from the first electronic device through the communication unit, and through the user interface When a user input accepting the video call request is received, a signal accepting the video call request is transmitted to the first electronic device through the communication unit, and based on state information of the electronic device and state information of the communication unit Receives an encoded video stream from the first electronic device through the communication unit, displays the received encoded video stream and reproduces received audio, and transmits voice received through a microphone through the communication unit to the first electronic device. It may contain instructions for sending to the device.

According to various embodiments, the instructions may include at least one of information about the type of the electronic device, the size of the touch screen, processing power, maximum resolution, battery status, memory status, or presence or absence of a camera, by the one or more processors. An encoded video stream may be received from the first electronic device through the communication unit based on state information of the communication unit including state information of the electronic device making a video call and a video call quality state for performing a video call.

According to various embodiments, when the one or more processors detect that the state of the electronic device or the state of the communication unit is changed, at least one of resolution, quality, or frame rate is different from the received video stream. A quality video stream may be received from the first electronic device through the communication unit.

According to various embodiments, the electronic device further includes a camera, and the instructions include encoding, by the one or more processors, a video stream captured through the camera based on state information of the electronic device and state information of the communication unit. and transmit it to the first electronic device through the communication unit.

According to various embodiments, the instructions include, by the one or more processors, displaying the encoded video stream, displaying an indicator for adjusting the quality of the video stream, and adjusting the quality of the video stream in the indicator. When an input is detected, an encoding scheme corresponding to the quality of the adjusted video stream is determined and transmitted to the first electronic device, and a video stream encoded based on the encoding scheme is received and displayed from the first electronic device. can

According to various embodiments, the instructions may include setting a region of interest (ROI) in the displayed video stream and transmitting the instructions to the first electronic device, changing focus, cropping, and quality of the ROI; Alternatively, a video stream encoded based on at least one of exposure adjustment may be received from the first electronic device and displayed.

According to various embodiments, the electronic device further includes a sensor unit, and the instructions include, when the one or more processors sense the motion of the electronic device through the sensor unit, first sensor information about the motion of the electronic device. Collects and analyzes the data, reproduces the received audio and stores the received video stream in the memory when the movement of the electronic device is within a predetermined range based on the analyzed sensor information, and stores the received video stream in the memory, When the motion of the electronic device is detected, second sensor information about the motion of the electronic device is collected and analyzed, and based on the analyzed sensor information, if the motion of the electronic device is not within a preset range, the memory is stored. You can play the saved video stream.

According to various embodiments, the instructions may determine whether at least one of the user's face, face, or pupil is recognized by the one or more processors through the camera, and if not recognized, play the received audio; The received video stream may be stored in the memory, and the video stream stored in the memory may be reproduced when at least one of the user's face, face, or eyes is recognized through the camera.

According to various embodiments, the instructions may include at least one capable of switching the video call when the one or more processors detects a signal for switching the video call while displaying the encoded video stream and playing received audio. One electronic device is searched for and displayed, and when an input for selecting an electronic device to which the video call is to be switched for a predetermined time is detected from among the at least one displayed electronic device, a video call conversion signal may be transmitted to the selected electronic device. .

According to various embodiments of the present disclosure, the instructions include, when the one or more processors do not detect an input for selecting an electronic device to switch the video call from among the one or more displayed electronic devices for a predetermined time, the at least one displayed electronic device is not detected. Priority is given by analyzing the state information of the electronic device and the state information of the communication unit, the electronic device having the highest priority is automatically selected as the electronic device to which the video call is switched, and the automatically selected electronic device is selected. A video call conversion signal may be transmitted to

According to various embodiments, the instructions may analyze state information of the electronic device and state information of the communication unit when the one or more processors receive a video call reception signal from the first electronic device through the communication unit. and display the user interface including a function related to the video call that can be performed based on the analyzed state information of the electronic device and state information of the communication unit.

According to various embodiments, the instructions include, by the one or more processors, performing a video call by receiving a high-definition video stream based on the analyzed state information of the electronic device and the state information of the communication unit, and receiving a low-definition video stream. to display the user interface including at least one function related to the video call among performing a video call, performing a voice call, performing a video call with the second electronic device, or performing a voice call with the second electronic device.

According to various embodiments, the instructions may be performed when the one or more processors receive a video call request from a third electronic device and detect a user input accepting the received video call request from the third electronic device. The encoded video stream received from the first electronic device is controlled to be displayed on the first area of the touch screen, and a video stream for a video call with the third electronic device is received from the third electronic device to display the touch screen. It can be displayed in the second area of .

According to various embodiments, the instructions may include, when a user input is detected while the one or more processors display the received video stream and reproduce the received audio, the pressure level for the detected user input or the user input The sensed direction may be checked, and a function corresponding to the pressure level or the direction in which the user input is sensed may be performed.

According to various embodiments, the instructions may include, when the one or more processors detect an input for storing the received video stream, collect external context information through the sensor unit, and collect the external context information and the input A request for storing a video stream for the detected time point is transmitted to the first electronic device, and a video stream for the time point at which the input was detected encoded based on the external situation information is received from the first electronic device. and can be stored in the memory.

According to various embodiments, the instructions include at least one of weather information, brightness information, and sound information for a time when the input for storing the video stream from the first electronic device is sensed by the one or more processors. A video stream encoded based on the external context information may be received and stored in the memory.

According to various embodiments, the instructions may search for at least one electronic device capable of performing the video call when the one or more processors receive a signal related to receiving a video call from the first electronic device through the communication unit. , When an input for controlling the touch screen to display the one or more searched electronic devices and selecting an electronic device for performing the video call is detected from the one or more displayed electronic devices, a video image with the selected electronic device is detected. A signal for accepting a call may be transmitted to the first electronic device through the communication unit.

According to various embodiments, the instructions may cause the one or more processors to display the at least one electronic device as an icon, and to select the at least one electronic device based on state information of the communication unit and state information of the electronic device. Attribute information of an icon including at least one of the size, color, or shape of the icon, or a location where the icon is disposed on the touch screen may be determined.

According to various embodiments, the communication unit may include at least one of Bluetooth and Wi-Fi.

According to various embodiments, the electronic device may include a wearable device.

In various embodiments of the present invention, an electronic device includes a communication unit communicating with a first electronic device, a touch screen displaying a user interface for performing a video call, one or more processors electrically connected to the communication unit and the touch screen, and one or more processors electrically connected to the communication unit and the touch screen. and a memory electrically connected to one or more processors, wherein the one or more processors, when executed, receive a signal related to video call reception from the first electronic device through the communication unit, and transmit the video call through a user interface. When a user input accepting a call request is received, a signal accepting the video call request is transmitted to the first electronic device through the communication unit, and a plurality of plurality of electronic devices generated based on state information of the electronic device and state information of the communication unit Receives images and audio signals from the first electronic device through the communication unit, displays the received plurality of images and reproduces the received audio signal, and transmits a voice received through a microphone through the communication unit. It may include instructions for transmitting to the first electronic device.

In various embodiments of the present invention, an electronic device includes a communication unit including a first communication unit communicating with a first electronic device and a second communication unit communicating with a second electronic device, a touch screen displaying a user interface for performing a video call, It includes one or more processors electrically connected to the communication unit and the touch screen, and a memory electrically connected to the one or more processors, wherein the memory, when executed, controls the one or more processors from the first electronic device or server. When a video call request signal is received through the first communication unit, the received video call request signal is displayed, a notification is generated based on the received video call request signal, and the generated notification is sent through the second communication unit. transmitting to the second electronic device and displaying the video call request signal in response to the received acceptance signal when receiving a signal accepting the video call request from the second electronic device through the second communication unit and encodes the video stream received from the first electronic device based on the state information of the second electronic device and the state information of the second communication unit, and transmits the encoded video stream to the first electronic device through the second communication unit. It may include instructions for transmitting to 2 electronic devices and transferring an audio signal received from the second electronic device through the second communication unit to the first electronic device or the server through the first communication unit.

According to various embodiments, when the one or more processors detect that the state of the electronic device or the state of the second communication unit is changed, at least one of the encoded video stream and resolution, quality, or frame rate A video stream received from the first electronic device may be encoded differently, and the encoded video stream may be transmitted to the second electronic device through the second communication unit.

According to various embodiments, the instructions include determining, by the one or more processors, at least one of maximum resolution, maximum quality, or maximum fps for a video stream supported by the second electronic device, and a state of the second communication unit At least one of the maximum resolution, maximum quality, and maximum fps may be adjusted based on information, and a video stream received from the first electronic device may be encoded based on at least one of the adjusted resolution, quality, or fps. there is.

According to various embodiments, the instructions may, when the one or more processors receive a video stream from the second electronic device, determine an encoding scheme based on the received video stream, and determine the encoding scheme based on the determined encoding scheme. A video stream received from the first electronic device may be encoded.

According to various embodiments of the present disclosure, when the one or more processors receive the ROI set from the second electronic device, encoding through at least one of focus, cropping, image quality change, or exposure adjustment based on the ROI, A video stream encoded based on the ROI may be transmitted to the second electronic device through the second communication unit.

According to various embodiments, the instructions may extract a corresponding video stream when the one or more processors receive a signal requesting storage of a video stream from the second electronic device and external context information of the second electronic device. A video stream encoded and stored based on external context information of the second electronic device or encoded based on external context information of the second electronic device may be transmitted to the second electronic device through the second communication unit. .

According to various embodiments, the instructions may include registering the second electronic device in a server by the one or more processors, and based on state information of the second electronic device registered in the server and state information of the second communication unit. A video stream received from the first electronic device may be encoded as

According to various embodiments, the instructions include, when the one or more processors receive a video call request signal and information on the third electronic device from the third electronic device, transmit video call connection information to the third electronic device, , Information of the third electronic device may be transmitted to the first electronic device through the first communication unit.

In various embodiments of the present invention, an electronic device includes a communication unit including a first communication unit communicating with a first electronic device and a second communication unit communicating with a second electronic device, a touch screen displaying a user interface for performing a video call, and the A communication unit and one or more processors electrically connected to the touch screen, and a memory electrically connected to the one or more processors, wherein the memory, when executed, causes the one or more processors to receive the information from the first electronic device or server. When a video call request signal is received through the first communication unit, a notification is generated based on the received video call request signal, and the generated notification is transmitted to the second electronic device through the second communication unit. 2 When a signal accepting the video call request is received from the electronic device through the second communication unit, an operation of displaying the video call request signal in response to the received acceptance signal is stopped and received from the first electronic device generating a plurality of images based on a video stream, extracting an audio signal from the video stream, and transmitting the generated plurality of images and the extracted audio signal to the second electronic device through the second communication unit; It may include instructions for transferring an audio signal received from the second electronic device through the second communication unit to the first electronic device or the server through the first communication unit.

According to various embodiments, the instructions may include obtaining, by the one or more processors, state information of the second electronic device and state information of the second communication unit, and obtaining the obtained state information of the second electronic device and the second electronic device. Based on the state information of the communication unit, at least one image among the plurality of images may be transmitted to the second electronic device.

According to various embodiments, the instructions include generating, by the one or more processors, the plurality of images including at least one still image, and an I-frame (intra-frame) in a video stream received from the first electronic device. ) is extracted, and the similarity between the previously extracted I-frame and the extracted I-frame is determined by comparing the previously extracted I-frame with the extracted I-frame, and the similarity is set to a preset reference value. If it exceeds, a still image may be generated based on the extracted I-frame.

According to various embodiments, the instructions may include, by the one or more processors, a change amount of a direction vector, a key frame change amount, a dominant color change amount, or a base color change amount with respect to the video stream received from the first electronic device. The at least one still image may be generated based on at least one of the set time intervals.

According to various embodiments, when the second communication unit is Bluetooth, the instructions include: the one or more processors transmitting the plurality of images to the second electronic device through a serial port profile (SPP) of the Bluetooth; The audio signal may be transmitted to the second electronic device through a hands free profile (HFP) of the Bluetooth.

In various embodiments of the present invention, an electronic device includes a first communication unit communicating with a first electronic device, a touch screen displaying a user interface for performing a video call, one or more processors electrically connected to the touch screen of the first communication unit, and and a memory electrically connected to the one or more processors, wherein, when the memory is executed, the one or more processors transmit a video call request signal to the first electronic device through the first communication unit, A signal accepting the video call request from an electronic device, state information of a second electronic device for performing the video call, and state information of a second communication unit are received through the first communication unit, and the received second electronic device It may include instructions for encoding a video stream based on the state information of and the state information of the second communication unit, transmitting the video stream to the first electronic device through the first communication unit, and receiving a voice signal from the first electronic device. there is.

In addition, although not shown in FIG. 5, the first to third electronic devices include components having additional functions, such as a Global Positioning System (GPS) module for receiving location information and a broadcast receiving module for receiving broadcasts. Optionally, more may be included.

6 is a diagram illustrating a signal flow between a first electronic device and a third electronic device for explaining an image encoding method in the first electronic device according to various embodiments of the present disclosure.

In various embodiments of the present disclosure, the first electronic device 601 (eg, the electronic device 401 of FIG. 4 ) connects the second electronic device 603 (eg, the second electronic device 403 of FIG. 4 ) to a second network (eg, the electronic device 401 of FIG. 4 ). Video data for performing a video call with the third electronic device 605 (eg, the third electronic device 405 of FIG. 4 ) connected through the second network 413 is stored as an image suitable for the state of the third electronic device 605 or the second network state. data can be converted.

In various embodiments of the present disclosure, the first electronic device 601 may be at least partially or entirely similar to or the same as the aforementioned first electronic device 401 of FIG. 4 . In addition, the second electronic device 603 may be at least partially or entirely similar to or identical to the aforementioned second electronic device 403 of FIG. 4 . In addition, the third electronic device 605 may be at least partially or entirely similar to or identical to the aforementioned third electronic device 405 of FIG. 4 .

Referring to FIG. 6 , a first electronic device 601 may request 611 a video call connection to a second electronic device 603 connected to a first network (eg, the first network 411 of FIG. 4 ). The second electronic device 603 may transmit 613 a notification notifying that a video call connection request has been received from the first electronic device 601 to the third electronic device 605 connected to the second network. The third electronic device 605 may accept the video call connection request and transmit 615 the acceptance signal to the second electronic device 603 . The second electronic device 603 may forward 617 the video call connection acceptance signal received from the third electronic device 605 to the first electronic device 601 .

In various embodiments of the present disclosure, the first electronic device 601 provides status information of the second electronic device 603 from the second electronic device 603 and information about a third electronic device 605 connected to the second electronic device 603 through a second network. Status information and network connection information can be obtained. The state information of the second electronic device 603 includes type information of the second electronic device 603, display size, processing power, maximum resolution supported by the second electronic device 603, battery state (eg, battery level), memory state, Or it may include at least one of the presence or absence of a camera. The information of the third electronic device 605 may include type information, display size, processing power, maximum resolution supported by the third electronic device 605, battery status (eg, battery level), memory status, or camera of the third electronic device 605 . It may include at least one of the presence or absence of having. The network connection information may include state information of the second network or state information of the first network.

In various embodiments of the present disclosure, the first electronic device 601 may determine 619 an encoding scheme based on the acquired information. The first electronic device 601 may encode 621 video data for performing a video call based on the determined encoding scheme and transmit 623 the video data to the second electronic device 603 .

In various embodiments of the present invention, the encoding scheme may include a transcoding scheme, a scalable encoding scheme, or a mirroring scheme, and image data may be transmitted through at least one of the above schemes. can be encoded. The second electronic device 603 may transmit 625 the encoded image data received from the first electronic device 601 to the third electronic device 605 . The third electronic device 605 may reproduce 627 the image data received from 603 as the second electronic device.

An operation of encoding video data through the transcoding method in various embodiments of the present invention will be described in detail with reference to FIGS. 10 to 16 described later.

An operation of encoding image data through the scalable encoding method in various embodiments of the present invention will be described in detail with reference to FIGS. 17A and 17B.

An operation of encoding image data through the mirroring method in various embodiments of the present invention will be described in detail with reference to FIG. 18 to be described later.

In various embodiments of the present disclosure, the third electronic device 605 may receive 629 voice and transmit 631 the voice signal to the second electronic device 603 . The second electronic device 603 may transmit 633 the voice signal received from the third electronic device 605 to the first electronic device 601 .

In various embodiments of the present disclosure, when the third electronic device 605 has a camera, it can encode an image received through the camera and transmit it along with a voice signal. For example, as described above with reference to FIG. 4 , the resolution supported by the third electronic device 605 may be lower than the resolution supported by the first electronic device 601 and the second electronic device 603 . As the resolution supported by the third electronic device 605 is lower than that of the first electronic device 601 and the second electronic device 603, the third electronic device 605 can reproduce image data received through a camera as a low-quality image even with low power consumption. data can be encoded. The third electronic device 605 may transmit the encoded low-quality video data and audio signal to the second electronic device 603 together. However, it is not limited thereto, and the third electronic device 605 may encode video data received through the camera into an image or an image sequence, and convert the image and the audio signal or the image sequence and audio signal into the second electronic device 605 . It can be transmitted to the electronic device 603.

7 is a diagram illustrating a signal flow between a first electronic device and a third electronic device for explaining an image encoding method in the first electronic device according to various embodiments of the present disclosure.

In various embodiments of the present disclosure, the first electronic device 701 (eg, the first electronic device 401 in FIG. 4 or the first electronic device 601 in FIG. 6 ) is a second electronic device 703 (eg, the second electronic device in FIG. 4 ). 403 or a third electronic device 705 connected to the second electronic device 603 of FIG. 6 through a second network (eg, the second network 413) (eg, the third electronic device 405 of FIG. 4 or the third electronic device 405 of FIG. 6 ) A third electronic device capable of performing a video call with the electronic device 605) and based on information of the third electronic device 705 receiving video data of the first electronic device 701 for performing a video call from a server 711 It can be converted into video data suitable for 705.

In various embodiments of the present disclosure, the server 711 may manage an electronic device capable of performing a video call based on a user account. For example, the server 711 may monitor an electronic device registered based on the user account, and update state information of the electronic device in real time through the monitoring. In other words, the server 711 can monitor the electronic device and check whether a video call can be performed through status information of the electronic device. The state information may include information on whether the electronic device is currently in use, on/off state, or away state. The first electronic device 701 selects an electronic device to conduct a video call based on state information of the electronic device updated through the server 711 (eg, selects an electronic device to conduct a video call from a phone book), and selects the electronic device to perform a video call. A video call request signal may be transmitted to the electronic device.

Referring to FIG. 7 , the third electronic device 705 may request 721 registration to the second electronic device 703 . For example, the third electronic device 705 may request the second electronic device 703 to register the third electronic device 705 with the server 711 as an electronic device capable of performing a video call. Upon receiving the registration request 721 from the third electronic device 705, the second electronic device 703 may issue an update request 723 to the server 711. For example, the second electronic device 703 may request the server 711 to register information on the third electronic device 705 or to update information on the third electronic device 705 . The server 711 may request 725 an update to the first electronic device 701 based on the updated information of the third electronic device 705 .

In various embodiments of the present disclosure, the first electronic device 701 may request a video call to a specific electronic device based on information registered in the server 711, for example, information of an electronic device capable of performing a video call. For example, the first electronic device 701 is an electronic device capable of performing a video call based on the information of the second electronic device 703 or the information of the third electronic device 705 received from the server 711 and updated, for example, the third electronic device. 705 can be checked. In addition, the server 711 receives type information of the second electronic device 703 and the third electronic device 705 from the second electronic device 703, display size, processing power, maximum resolution supported by the third electronic device 705, battery status ( For example, at least one of battery level), memory status, presence or absence of a camera, status information of the first network (eg, the first network 411 of FIG. 4 ), or status information of the second network may be acquired and stored.

The first electronic device 701 determines the third electronic device 705 as an electronic device capable of performing the video call, and requests 727 a video call connection to the second electronic device 703 connected to the first network. The second electronic device 703 may transmit 729 a notification notifying that a video call connection request has been received from the first electronic device 701 to the third electronic device 705 connected to the second network. The third electronic device 705 may accept the video call connection request and transfer 731 the acceptance signal to the second electronic device 703 . The second electronic device 703 may transmit 733 the video call connection acceptance signal received from the third electronic device 705 to the first electronic device 701 .

The first electronic device 701 may determine 735 an encoding scheme based on information of the second electronic device 703 stored in the server 711, information of the third electronic device 705 connected to the second electronic device 703, and network connection information. . The first electronic device 701 may encode 737 video data for performing a video call based on the determined encoding scheme and transmit 739 it to the second electronic device 703 . The second electronic device 703 may transfer 741 the encoded image data received from the first electronic device 701 to the third electronic device 705 . The third electronic device 705 may reproduce 743 the encoded image data received from 703 as the second electronic device.

In various embodiments of the present disclosure, the third electronic device 705 may receive 745 voice and transmit 747 the voice signal to the second electronic device 703 . The second electronic device 703 may transmit 749 the voice signal received from the third electronic device 705 to the first electronic device 701 .

8 is a diagram illustrating a signal flow between a first electronic device and a third electronic device for explaining an image encoding method in the first electronic device according to various embodiments of the present disclosure.

Referring to FIG. 8 , a first electronic device 801 (eg, the first electronic device 401 of FIG. 4 , the first electronic device 601 of FIG. 6 , or the first electronic device 701 of FIG. 7 ) connects to a first network (eg, FIG. A video call connection is made to the second electronic device 803 (eg, the second electronic device 403 in FIG. 4 , the second electronic device 603 in FIG. 6 , or the second electronic device 703 in FIG. 7 ) connected to the first network 411 in FIG. 4 ). You can request 811. The second electronic device 803 transmits a notification notifying that a video call connection request has been received from the first electronic device 801 to the third electronic device 805 connected through a second network (eg, the second network 413 of FIG. 4 ) 813 can do. The third electronic device 805 may accept the video call connection request and transmit 815 the acceptance signal to the second electronic device 803 .

In various embodiments of the present disclosure, the third electronic device 805 may transmit 815 image data of the third electronic device 805 to the second electronic device 803 as well as the acceptance signal described above.

For example, in various embodiments of the present disclosure, the third electronic device 805 may include a camera, and transmit image data acquired through the camera to the second electronic device 803 . For example, the third electronic device 805 may encode the obtained image data based on state information of the third electronic device 805 and second network state information. The third electronic device 805 may transmit the encoded image data to the second electronic device 803 .

In various embodiments of the present disclosure, the second electronic device 803 may transfer 817 the acceptance signal received from the third electronic device 805 and image data of the third electronic device 805 to the first electronic device 801 . Alternatively, the third electronic device 805 may generate and transmit only a voice signal to the second electronic device 803 .

The first electronic device 801 may check 819 the quality of video data of the third electronic device 805 received from the second electronic device 803 . For example, the first electronic device 801 can check the encoding scheme, quality, or size of the image of the received image data of the third electronic device 805 .

The first electronic device 801 may determine an encoding scheme of video data to be transmitted based on the encoding scheme, quality, or size of the video data received from the identified third electronic device 805 . The present invention is not limited thereto, and in various embodiments of the present disclosure, the first electronic device 801 receives display size and processing of the third electronic device 805 received through the second electronic device 803 in addition to image data of the third electronic device 805. Encoding image data with power, maximum resolution supported by the third electronic device 805, battery status (eg, battery level), memory status, presence or absence of a camera, first network status information, or second network status information may be used to determine 821 an encoding scheme for

The first electronic device 801 may encode 823 image data based on the determined encoding scheme and transmit 825 the encoded image data to the second electronic device 803 . The second electronic device 803 may transfer 827 the encoded image data received from the first electronic device 801 to the third electronic device 805 . The third electronic device 805 may reproduce 829 the image data received from the 803 as the second electronic device. Upon receiving 831 the voice, the third electronic device 805 may transmit 833 the voice signal to the second electronic device 803 . The second electronic device 803 may transmit 835 the voice signal received from the third electronic device 805 to the first electronic device 801 .

9 is a diagram illustrating a signal flow between a first electronic device and a third electronic device for explaining an image encoding method in a second electronic device according to various embodiments of the present disclosure.

In various embodiments of the present invention, the second electronic device 903 (eg, the second electronic device 403 in FIG. 4 , the second electronic device 603 in FIG. 6 , the second electronic device 703 in FIG. 7 , or the second electronic device in FIG. 8 ) 803) is the first electronic device 901 (eg, the first electronic device 401 of FIG. 4, the first electronic device 601 of FIG. 6, the first electronic device 701 of FIG. 7, or the first electronic device 801 of FIG. 8) and the first electronic device 801 of FIG. A third electronic device 905 connected to the second electronic device 903 through a second network (eg, the second network 413 in FIG. 4 ) (eg, the third electronic device 405 in FIG. 4 , the third electronic device 605 in FIG. 6 , FIG. Video data for performing a video call with the third electronic device 705 of 7 or the third electronic device 805 of FIG. 8 may be converted into video data suitable for the third electronic device 905 .

Referring to FIG. 9 , in various embodiments of the present disclosure, a first electronic device 901 may request 911 a video call connection to a second electronic device 903 connected to a first network (eg, the first network 411 of FIG. 4 ). . The second electronic device 903 may transmit 913 a notification notifying that a video call connection request has been received from the first electronic device 901 to the third electronic device 905 connected to the second network. The third electronic device 905 may accept the video call connection request and transmit 915 the acceptance signal to the second electronic device 903 . The second electronic device 903 may forward 917 the video call connection approval signal received from the third electronic device 905 to the first electronic device 901 .

In various embodiments of the present disclosure, the first electronic device 901 may transmit 919 video data for a video call captured through a camera to the second electronic device 903 . The second electronic device 903 converts the image data received from the first electronic device 901 to state information of the third electronic device 905 (eg, type, screen size, processing power, maximum resolution, and battery status of the third electronic device 905). , memory state, presence/absence of a camera, etc.), state information of the first network, and state information of the second network, the encoding scheme may be determined 921 . The second electronic device 903 may encode 923 video data for performing a video call based on the determined encoding scheme and transmit 925 it to the third electronic device 905 . The third electronic device 905 may reproduce 927 the image data received from 903 as the second electronic device. When receiving 939 the voice, the third electronic device 905 may transmit 931 the voice signal to the second electronic device 903 . The second electronic device 903 may transmit 933 the voice signal received from the third electronic device 905 to the first electronic device 901 .

10 is a diagram for explaining an image encoding method performed in a first electronic device according to various embodiments of the present disclosure.

As described above, in various embodiments of the present invention, as described above, the first electronic device (eg, the first electronic device 401 in FIG. 4 , the first electronic device 601 in FIG. 6 , the first electronic device 701 in FIG. 7 , 8, or the first electronic device 901 in FIG. 9) or the second electronic device (eg, the second electronic device 403 in FIG. 4, the second electronic device 603 in FIG. 6, the second electronic device in FIG. 7) The device 703, the second electronic device 803 of FIG. 8, or the second electronic device 903 of FIG. 9 converts image data of the first electronic device to a third electronic device (eg, the second electronic device 903 of FIG. 4) using a transcoding method. image data suitable for the third electronic device 405, the third electronic device 605 of FIG. 6, the third electronic device 705 of FIG. 7, the third electronic device 805 of FIG. 8, or the third electronic device 905 of FIG. there is.

10 described later in various embodiments of the present invention will be described assuming that video data is encoded in the first electronic device. However, it is not limited thereto, and the operation of encoding the image data may be equally performed in the second electronic device.

In various embodiments of the present invention, in the case of Full HD (1920 * 1080) or HD (1280 * 720) resolution and frame image data of 30 fps (frame per second), it is difficult to reproduce them in the first electronic device or the second electronic device. It is suitable, and in the case of QVGA (320 * 204), CIF (352 * 288), or QCIF (176 * 240) resolution and frame image data of 30 fps or 15 fps, it can be played back on a third-party electronic device without significant degradation in image quality. can In other words, Full HD (1920*1080), or There is no need to reproduce HD (1280*720) resolution and 30fps (frame per second) frame image data as it is. In addition, when the second electronic device and the third electronic device are connected to a second network (eg, the second network 413 of FIG. 4 ), for example, a low power network such as Bluetooth or wifi, the second electronic device connects to the third electronic device. Not only is it difficult to secure a transmission speed for transmitting a high-capacity streaming file to a device, but also battery life performance problems may occur due to excessive power consumption when playing high-definition streaming in a third electronic device.

As described above, according to an embodiment, the third electronic device transmits an image having a resolution of QVGA (320 * 204), CIF (352 * 288), or QCIF (176 * 240) and a frame of 30 fps or 15 fps. On the small screen of the third electronic device, it can be reproduced efficiently without significant deterioration of picture quality. In addition, as the screen for reproducing the image data of the third electronic device is small, it is difficult for the user to find deterioration in image quality due to the highly compressed image. Accordingly, various embodiments of the present invention can obtain a great advantage in a poor network environment or a network with a small bandwidth for low power connection because the required bandwidth for streaming decreases as the file size of the video stream becomes relatively small.

In various embodiments of the present invention, a transcoding scheme parameter for converting image data of a first electronic device into image data suitable for a third electronic device using a transcoding method is an image quality (compression rate) , fps (frames per second), resolution (resize), focus (focus), or crop (crop).

In various embodiments of the present disclosure, the first electronic device may provide information on the second electronic device or a third electronic device connected to the second electronic device through a second network (eg, type of third electronic device, display size, camera availability). Presence, processing power, supported codec, current battery level, memory usage, number of applications in use, or application type), a first network connecting the first electronic device and the second electronic device (eg, the first electronic device in FIG. 4 ). 1 Determine transcoding scheme parameters for encoding based on information (state/signal quality) of the network 411 and information (state/signal quality) of a second network connecting the second electronic device and the third electronic device can

Referring to FIG. 10 , in various embodiments of the present disclosure, in operation 1001, the first electronic device may check the maximum resolution, maximum quality, or maximum fps of image data required by the third electronic device.

In operation 1003, the first electronic device may determine an available bit rate based on the network state. The network status includes first network information (status/signal quality) connecting the first electronic device and the second electronic device and second network information (status/signal quality) connecting the second electronic device and the third electronic device. can include

In various embodiments of the present disclosure, the first electronic device may determine resolution, fps, quality, or I-frame refresh corresponding to the determined bit rate. In detail, the first electronic device may determine the resolution of the image data corresponding to the bit rate in operation 1005, and resize the image data based on the determined resolution in operation 1007. The first electronic device may determine the fps of the video data corresponding to the bit rate in operation 1009 and change the fps of the video data based on the fps in operation 1011. In operation 1013, the first electronic device determines the quality (compression rate) of the video data corresponding to the bit rate, and in operation 1015, the compression rate of the video data can be changed based on the determined quality (compression rate). In operation 1017, the first electronic device may determine I-frame refresh of image data corresponding to the bit rate.

In various embodiments of the present disclosure, in operation 1019, the first electronic device may encode image data based on the changed resolution, fps (frames per second), quality, or I-frame refresh. The first electronic device may transmit the video data encoded in operation 1021 to the second electronic device through the first network.

11 is a diagram for explaining an image encoding method performed in a first electronic device according to various embodiments of the present disclosure.

Referring to FIG. 11 , in various embodiments of the present disclosure, a first electronic device (eg, the first electronic device 401 of FIG. 4 , the first electronic device 601 of FIG. 6 , the first electronic device 701 of FIG. 7 , the first electronic device 701 of FIG. 8 ) The first electronic device 801 (or the first electronic device 901 of FIG. 9) includes a network interface 1101, a video call client 1103, a network manager 1105, and a video call manager. ) 1107, image processor 1109, camera sensor 1111, captured image processing unit 1113, camera driver 1115, encoder driver 1117, encoder ) 1119, and a memory 1121.

Referring to FIG. 11 , in various embodiments of the present disclosure, 1131 indicates the flow of a control signal, and 1133 indicates a signal from a first electronic device to a second electronic device 1123 (eg, the second electronic device 403 in FIG. 4 or the second electronic device 403 in FIG. 6 ). 2 shows a video call signal flow to the electronic device 603, the second electronic device 703 of FIG. 7, the second electronic device 803 of FIG. 8, or the second electronic device 903 of FIG.

In various embodiments of the present invention, the network manager 1101 performs basic network connection management such as connection establishment, connection monitoring, connection adjustments, or connection tear down. action can be performed. In addition, the network manager 1101 may generate packets of data to be transmitted according to the network environment and de-packet the received packets to restore a desired signal. The network manager 1101 may measure network quality using feedback data such as packet loss information and round trip delay, and the third electronic device (eg, the third electronic device 405 in FIG. 4, the third electronic device 605 in FIG. 6, the third electronic device 705 in FIG. 7, the third electronic device 805 in FIG. 8, or the third electronic device 905 in FIG. 9) The quality of the network can be measured through a training sequence received at regular intervals. The measured network quality may be delivered to a video call manager 1107.

In various embodiments of the present invention, a video call client 1103 is an application for performing a video call. When a user input for a video call is detected, the video call client 1103 may transfer the user input signal to a video call manager 1107 .

The video call manager 1107 is a module that controls video calls and can control a network manager 1105 and an encoder driver 1117 for video call services. The video call manager 1107 may control functions for performing a video call based on a user input received from the video call client 1103 . The video call manager 1105 may perform a video call by loading information of a third electronic device stored in a memory 1121 . For example, the video call manager 1105 may control the quality of video data necessary for a video call based on some of the information of the third electronic device stored in the memory 1121 .

The video call manager 1107 obtains the first information obtained from the network manager 1105, memory 1121, and video call client 1103 in order to perform a video call with a third electronic device. 3 Based on the properties of the electronic device, a maximum bit rate may be calculated. The video call manager 1107 checks the network status of the second network (eg, the second network 413 of FIG. 4 ) from the network manager 1105 and determines the bit rate based on this. can be calculated The video call manager 1107 determines the resolution, fps, or quality corresponding to the calculated bit rate, and the encoder 1119 outputs video based on the determined resolution, fps, or quality. An encoding scheme for encoding data can be determined.

In various embodiments of the present invention, the video call manager 1107 may control a vertical blanking interval (VBI) signal of a camera sensor 1111. The video call manager 1107 controls the vertical blanking interval (VBI) of the camera sensor 1111 to frame drop video data input through the camera sensor 1111. The frame rate can be adjusted through For example, assuming that the frame rate of video data input through the camera sensor 1111 is 30 fps, the video call manager 1107 performs a frame drop of the 30 fps frame rate. rate) can be adjusted to 15 fps.

In various embodiments of the present invention, the network manager 1105 may control the buffer to adjust the frame rate of video data encoded by frame drop. The network manager 1105 may adjust the frame rate of the transmitted video data through frame drop of the encoded video data. For example, when receiving encoded video data having a frame rate of 30 fps, the network manager 1105 reduces the frame rate of the received video data to 15 fps through frame drop. can be adjusted with

In various embodiments of the present disclosure, a camera sensor 1111 may acquire image data and transmit it to a captured image processing unit 1113. The captured image processing unit 1113 may transmit image data received from the camera sensor 1111 to an image processor 1109 through a camera driver 1115 .

In various embodiments of the present invention, a camera sensor 1111 acquires image data at regular time intervals, and the image data obtained at regular time intervals is transferred to an image processor 1109 through a camera driver 1115. can be sent to

In various embodiments of the present invention, an image processor 1109 may adjust image quality and size of image data. The image processor 1109 reduces the image data to correspond to the display ratio of the third electronic device, adjusts the image quality by changing the quantization level of the image, or crops the screen. Can be resized. For example, the image processor 1109 determines a region of interest (ROI) by recognizing a face or an object in image data, and crops the image data to fit the properties of the third electronic device based on the ROI. can do.

According to another embodiment, the image processor 1109 may differentiate an encoding scheme between an ROI part and a non-ROI part based on the ROI. Accordingly, various embodiments of the present invention may provide a region of interest to the user (the ROI) as a higher quality image and a portion of less interest (the non-ROI) as a relatively low quality image. The image processor 1109 may transmit the image data whose quality and size have been adjusted to an encoder 1119 through an encoder driver 1117 .

In various embodiments of the present invention, the encoder 1119 encodes the video data whose picture quality and size are adjusted based on an encoding scheme, and encodes the second video data through a network manager 1105 and a network interface 1101. It can be transmitted to an electronic device.

In various embodiments of the present disclosure, a memory 1121 means a volatile or non-volatile storage location, and may store information of one or more electronic devices, for example, a wearable device. The memory 1121 may store information such as a display size of one or more electronic devices, a battery capacity (level), or a type of electronic device. The stored information of one or more electronic devices may be used in a video call manager 1107 to control a video call. The information of the electronic device may be obtained in a discovery operation performed by the first electronic device or the second electronic device to find a connectable device or may be obtained through a pre-stored connection information history. Alternatively, the information of the electronic device may be information pre-registered at the time of manufacturing the product.

12 is a diagram for explaining a method of reproducing an encoded image in a third electronic device according to various embodiments of the present disclosure.

In various embodiments of the present disclosure, the third electronic device 405 may reproduce encoded image data received from the second electronic device 403 .

Referring to FIG. 12 , in various embodiments of the present disclosure, a third electronic device (eg, the third electronic device 405 in FIG. 4 , the third electronic device 605 in FIG. 6 , the third electronic device 705 in FIG. 7 , the third electronic device 705 in FIG. 8 ) The third electronic device 805 or the third electronic device 905 of FIG. 9 includes a network interface 1201 (eg, the network interface 1101 of FIG. 11 ) and a network manager 1203 (eg, the network interface 1101 of FIG. 11 ). 11 network manager 1105), decoder driver 1205, decoder 1207, video call client 1209 (eg, video call client 1103 of FIG. 11) , and a video call manager 1211 (eg, the video call manager 1107 of FIG. 11).

In various embodiments of the present invention, 1221 indicates a control signal flow, and 1223 indicates a second electronic device (e.g., the second electronic device 403 in FIG. 4, the second electronic device 603 in FIG. 6, and the second electronic device in FIG. 7). 703 shows a signal flow of receiving and reproducing encoded data from the second electronic device 803 of FIG. 8 , the second electronic device 903 of FIG. 9 , or the second electronic device 1123 of FIG. 11 .

In various embodiments of the present disclosure, a third electronic device may receive image data optimized for the third electronic device from the second electronic device through a network manager 1203 . The received video data may be transferred to a video call client 1209 through a decoder driver 1205 and a decoder 1207 and reproduced. The third electronic device may transmit status information related to network quality of the second electronic device or information of the third electronic device through a network manager 1203 . In various embodiments of the present disclosure, the third electronic device may transmit status information related to network quality of other electronic devices around the third electronic device and information of the other electronic devices to the second electronic device.

In various embodiments of the present disclosure, the third electronic device may determine a transcoding scheme parameter. The third electronic device converts the determined parameter to the first electronic device (eg, the first electronic device 401 of FIG. 4 , the first electronic device 601 of FIG. 6 , the first electronic device 701 of FIG. 7 , the first electronic device of FIG. 8 ). It is transmitted to the first electronic device 801 or the first electronic device 901 of FIG. 9 or the second electronic device and may request encoding of image data. The first electronic device or the second electronic device encodes video data based on the parameter in response to an encoding request received from the third electronic device, and converts the encoded video data to the third electronic device. can be sent to

13 is a diagram for explaining an image encoding method performed in a second electronic device according to various embodiments of the present disclosure.

Referring to FIG. 13 , in various embodiments of the present disclosure, a second electronic device (eg, the second electronic device 403 of FIG. 4 , the second electronic device 603 of FIG. 6 , the second electronic device 703 of FIG. 7 , the second electronic device 703 of FIG. 8 ) The second electronic device 803, the second electronic device 903 of FIG. 9, the second electronic device 1123 of FIG. 11, or the second electronic device 1215 of FIG. 12 is a network interface 1301 (eg, the network interface of FIG. 11) (network interface) 1101 or network interface 1201 of FIG. 12), network manager 1303 (eg, network manager 1105 of FIG. 11 or network manager of FIG. 12) 1203), transcoder driver 1305, encoder/decoder 1307, video call manager 1309 (eg, video call manager 1107 of FIG. 11 or FIG. 12) video call manager 1211), video call client 1311 (eg, video call client 1103 of FIG. 11 or video call client 1209 of FIG. 12 ) ), and a memory 1313 (eg, memory 1121 of FIG. 11 ).

In various embodiments of the present invention, 1321 denotes a control signal flow, 1323 denotes a first electronic device 1315 (eg, the first electronic device 401 in FIG. 4 , the first electronic device 601 in FIG. 6 , the first electronic device in FIG. 7 ) 701, the first electronic device 801 in FIG. 8 or the first electronic device 901 in FIG. 9) represents a signal flow from the second electronic device to the second electronic device, and 1325 indicates a signal flow from the second electronic device to the third electronic device 1317 (eg, FIG. 4 ). , the third electronic device 405 in FIG. 6 , the third electronic device 705 in FIG. 7 , the third electronic device 805 in FIG. 8 , or the third electronic device 905 in FIG. 9 ).

In various embodiments of the present invention, a network interface 1301, a network manager 1303, a video call manager 1309, a video call client 1311, and a memory 1311 are respectively The network interface 1101, network manager 1105, video call client 1103, video call manager 1107, and memory 1121 described above in FIG. A detailed description will be omitted.

In various embodiments of the present invention, a video call client 1311 is an application for performing a video call. The video call client 1311 may detect an input for selecting an electronic device to perform a video call and adjusting the quality of video data.

In various embodiments of the present invention, the network manager 1303 performs basic network connection management such as connection establishment, connection monitoring, connection adjustments, or connection tear down. action can be performed. In addition, the network manager 1303 may measure network quality using feedback data such as packet loss information and round trip delay, and the third Network quality may be measured through a training sequence received at regular intervals from the electronic device 1317 . The measured network quality may be transmitted to a video call manager 1311.

In various embodiments of the present invention, the transcoder driver 1305 uses encoding criteria (eg, attribute information of the second electronic device and the third electronic device 1317 in the video call manager 1311, An image is generated using an encoding parameter determined by the state information of the first network (eg, the first network 411 of FIG. 4 ) or the state information of the second network (eg, the second network 413 of FIG. 4 ). The resolution, fps, or image quality of the data may be determined, and based on this, image data suitable for the third electronic device 1317 may be converted.

In various embodiments of the present disclosure, the memory 1313 may store information such as a display size of one or more electronic devices, a battery capacity, or a type of electronic device.

The video call manager 1309 is a module that controls a video call and can control a network manager 1303 and an encoder/decoder driver 1307 for video call service. The video call manager 1311 may control functions for performing a video call based on a user input received from the video call client 1311 . The video call manager 1309 may load information of the third electronic device 1317 eg, a wearable device stored in the memory 1313 to perform a video call and control the quality of video data.

The video call manager 1309 is a property of the third electronic device 1317 for performing a video call acquired from a network manager 1303, a memory 1313, and a video call client 1311. It is possible to calculate the requested transmission rate (maximum bit rate) based on. The video call manager 1309 evaluates the quality state of the second network connecting the third electronic device 1317 and the second electronic device from the network manager 1303 to support the bit rate. ) can be calculated. The video call manager 1309 determines the resolution, fps, or quality corresponding to the calculated bit rate, and encodes video data based on the determined resolution, fps, or quality. scheme can be determined.

In various embodiments of the present invention, the video call manager 1309 may adjust the quality and size of video data through an image processor module in a transcoder driver 1305. The image processor may adjust the image quality by reducing the screen of the image data at a predetermined ratio or by changing a quantization level. Alternatively, the image processor crops and resizes the screen of image data, recognizes a face or object, sets a region of interest (ROI), and sets the set ROI as the center of the third electronic device. The property of 1317, for example, can be cropped according to the display size.

For example, assuming that the touch screen of the first electronic device 1315 has a shape longer than the width and the touch screen of the third electronic device 1317 has the same square shape, the image processor of the first electronic device 1315 ( The image processor may set an ROI (eg, face) in image data having a height longer than width, crop the image data around the set ROI, and transmit the image data to the third electronic device 1317 . If the ROI cannot be found or the ROI cannot be tracked in the image data, the second electronic device crops the image data around the center and transmits the image data to the third electronic device 1317 or the third electronic device 1317 without cropping. can be sent to

In various embodiments of the present invention, the ROI may be set based on a tool such as a rectangle or circle, a closed curve, a touch input, or the touch input time.

In various embodiments of the present disclosure, the second electronic device may encode the set ROI with differentiated quality. For example, when the user's face is set as an ROI, the image processor of the second electronic device generates high-quality image data by adjusting the quantization level of the face and the peripheral portion, and the portion other than the ROI has low-quality image data. of image data can be generated.

In various embodiments of the present disclosure, the second electronic device may receive the ROI selected by the third electronic device 1317 .

In various embodiments of the present invention, at least one ROI may be set and may be changed during a video call.

14A is a diagram for explaining an image encoding method performed in a second electronic device according to various embodiments of the present disclosure.

In various embodiments of the present invention, the second electronic device (eg, the second electronic device 403 in FIG. 4 , the second electronic device 603 in FIG. 6 , the second electronic device 703 in FIG. 7 , the second electronic device 803 in FIG. 8 , The second electronic device 903 of FIG. 9 , the second electronic device 1123 of FIG. 11 , or the second electronic device 1215 of FIG. 12 ) is a first electronic device 1401 (eg, the first electronic device 401 of FIG. Image data received from the first electronic device 601, the first electronic device 701 of FIG. 7, the first electronic device 801 of FIG. 8, or the first electronic device 901 of FIG. signal and non-continuous still images are converted into a third electronic device 1413 (e.g., the third electronic device 405 in FIG. 4, the third electronic device 605 in FIG. 6, the third electronic device 705 in FIG. 7, the third electronic device in FIG. 8) It can be transmitted to the electronic device 805 or the third electronic device 905 of FIG. 9 ). For example, the third electronic device 1413 may continuously output an audio signal received from the second electronic device and output a video signal asynchronously with the audio signal. For example, when the state of the second network (eg, the second network 413 of FIG. 4 ) connecting the second electronic device and the third electronic device 1413 is very poor or the battery level of the third electronic device 1413 is very low. , The third electronic device 1413 may continuously output an audio signal and output a video signal asynchronously with the audio signal to maintain a video call service using minimal resources.

Referring to FIG. 14A , in various embodiments of the present disclosure, 1421 represents a control signal flow, 1423 represents a video call signal flow from a first electronic device 1401 to a second electronic device, and 1425 represents a flow of a video call signal from the second electronic device 1401 to the second electronic device. A signal flow for transmitting encoded image data to the third electronic device 1413 is shown.

In various embodiments of the present invention, after an encoding operation, a decoder 1407 may additionally perform an image data processing operation by a post process 1406 . For example, in various embodiments of the present invention, the post process 1406 may generate a still image based on image data decoded by the decoder 1407 . The decoder 1407 can decode the video data received from the decoder driver 1405 by separating it into an audio signal and a video signal. The decoder 1407 may transfer the decoded voice signal to the third electronic device 1413 through a network manager 1403 through a separate channel. The decoder 1407 may transfer the decoded video signal to the post process 1406 . The post process 1406 converts the video signal into a still image, and converts the converted still image into a decoder driver 1405 and a network manager 1403 (e.g., the network manager of FIG. 11). ) 1105, the network manager 1203 of FIG. 12, or the network manager 1303 of FIG. 13) to the third electronic device 1413.

The post process 1406 will be described in detail with reference to FIG. 14B to be described later.

14B is a diagram for explaining a post process according to various embodiments of the present disclosure.

Referring to FIG. 14B , in various embodiments of the present invention, a post process 1406 may include an intra-frame (I-frame) extraction unit 1415, a comparison unit 1417, and an image generation unit 1419.

In various embodiments of the present invention, 1421 represents the flow of a control signal, and 1423 represents a first electronic device (e.g., the first electronic device 401 in FIG. 4, the first electronic device 601 in FIG. 6, and the first electronic device in FIG. 7). 701, the first electronic device 801 of FIG. 8, the first electronic device 901 of FIG. 9, or the first electronic device 1401 of FIG. 14A) to the second electronic device (eg, the second electronic device 403 of FIG. The second electronic device 603, the second electronic device 703 in FIG. 7, the second electronic device 803 in FIG. 8, the second electronic device 903 in FIG. 9, the second electronic device 1123 in FIG. 11, or the second electronic device in FIG. 12 1215), and 1425 indicates a video call signal flow from the second electronic device to the third electronic device (eg, the third electronic device 405 in FIG. 4, the third electronic device 605 in FIG. 6, and the third electronic device 705 in FIG. 7). , a signal flow for transmitting encoded image data to the third electronic device 805 of FIG. 8 , the third electronic device 905 of FIG. 9 , or the third electronic device 1413 of FIG. 14A ).

In various embodiments of the present invention, each of the I-frame extraction unit 1415, the comparison unit 1417, and the image generation unit 1419 may transmit and receive control signals to and from the video call manager 1409. The I-frame extraction unit 1415 may receive video data decoded by the decoder 1407, extract an I-frame from the received video data, and transmit the extracted I-frame to the comparison unit 1417.

The comparator 1417 may compare similarities between a previous I-frame, eg, a first I-frame, and a current I-frame, eg, a second I-frame. The comparator 1417 may determine similarity based on a reference value received from a video call manager 1409, and through this, adjust the number of images to be generated based on the video data. For example, the comparator 1417 may transfer the second I-frame to the image generator 1419 when it is determined that the degree of similarity exceeds the reference value. The image generator 1419 may generate a still image based on the second I-frame received from the comparator 1417. The image generator 1419 may transmit the generated still image to a third electronic device through a network manager 1403 .

15 is a diagram for explaining an image encoding method performed in a second electronic device according to various embodiments of the present disclosure.

Referring to FIG. 15 , in various embodiments of the present disclosure, 1551 indicates a flow of a control signal, and 1553 indicates a first electronic device (eg, the first electronic device 401 in FIG. 4 or the first electronic device 601 in FIG. 6 ). 7, the first electronic device 801 in FIG. 8, the first electronic device 901 in FIG. 9, or the first electronic device 1401 in FIG. 14A) in the second electronic device (eg, the second electronic device in FIG. 4). device 403, second electronic device 603 in FIG. 6, second electronic device 703 in FIG. 7, second electronic device 803 in FIG. 8, second electronic device 903 in FIG. 9, second electronic device 1123 in FIG. 11, or FIG. 12 indicates a video call signal flow from the second electronic device 1215, 1555 from the second electronic device to the third electronic device 1505 (eg, the third electronic device 405 in FIG. 4, the third electronic device 605 in FIG. 6, FIG. 7, the third electronic device 805 in FIG. 8, the third electronic device 905 in FIG. 9, or the third electronic device 1413 in FIG. 14A) shows a signal flow for transmitting encoded image data.

In various embodiments of the present invention, in a still image generating method, the second electronic device decodes the image 1521 input through the first network 1523 (eg, the first network 411 of FIG. 4 ) from the first electronic device. ) 1507 (eg, the decoder 1407 of FIG. 14a or 14b). A post process 1506 (eg, the post process 1406 of FIG. 14A or 14B ) may receive decoded image data from the decoder 1507 . The received image data is extracted through an I-frame extractor 1515 (eg, the I-frame extractor 1415 of FIG. 14B) of the post processing 1506, and the comparison unit 1517 (eg, the comparison unit 1415 of FIG. 14B) section 1417). The comparator 1517 compares the previous I-frame, e.g., the first I-frame, and the current I-frame, e.g., the second I-frame, based on the criteria 1535 received from the video call manager. The degree of similarity can be determined by comparison. For example, if the comparison unit 1517 determines that the similarity exceeds the reference value 1535 (eg, when the displacement of the image is large), the second I-frame is converted into an image generator 1519 (eg, the image generator 1419 of FIG. 14B). can be forwarded to

In various embodiments of the present disclosure, the comparator 1517 of the second electronic device transmits information (e.g., Information on the type of the third electronic device 1505, display size, processing power, maximum resolution, current battery level (capacity), memory usage, presence or absence of a camera, window size; Resolution difference according to full-screen video call), supported codec, number of applications in use, application type, etc.), network connecting the second electronic device and the third electronic device 1505, for example, of the second network 1531 A still image acquisition period may be adjusted based on state information (eg, signal quality). For example, if the state of the second network 1531 is good for video call and the battery state is sufficient, the second electronic device may set the still image acquisition cycle to once per second, and the state of the second network 1531 If is good for video call and the battery condition is insufficient, the still image acquisition cycle can be set to once every 3 seconds, the state of the second network 1531 is poor for video call, and the battery condition is poor. If this is not enough, the still image acquisition period can be set to once every 5 seconds. Numerical values representing the still image acquisition cycle set based on the above-described state of the second network 1531 and the state of the battery are only examples and are not limited thereto.

In various embodiments of the present invention, the image generator 1519 may generate a still image based on the second I-frame received from the comparator 1517 .

In various embodiments of the present invention, the I-frame may be stored in a buffer before being transmitted to the image generator 1519. Each I-frame may be assigned a recognition code using numbers, symbols, or characters and stored in the buffer. The recognition code may be assigned based on time information, sequence information, account information of a call participant, or information of a terminal providing a service (eg, terminal name or mac address). When the second electronic device receives a request for capturing or storing displayed image data from the third electronic device 1505, the image data corresponding to the capture or saving request is stored in the buffer using the identification code of the given I-frame. -You can search for high-resolution images before encoding frames and save them as high-resolution images. The high resolution image may be stored in the second electronic device or the third electronic device 1505 . The buffer may be a circular buffer.

The image generator 1519 may transmit the generated still image to the third electronic device 1505 through a network manager, for example, a BT modem 1533.

Although a method of generating a still image by comparing I-frames has been described in various embodiments of the present invention, the present invention is not limited thereto, and the direction (motion) vector change amount, key frame change amount, dominant color of the image A still image can be created at each (dominant color) variation, specific time interval, or random time interval.

Various embodiments of the present invention may use a content change of actual image data as a trigger point for generating an image. For example, when the image generator 1519 of the second electronic device detects a change in a specific area of image data, for example, generation or change of subtitles, or a change in a character's facial expression or a change in the number of characters, it is recognized as a trigger point and stopped. You can create video. The image generator 1519 of the second electronic device may generate a still image when a specific image appears or when a specific region of the image is changed. For example, the image generating unit 1519 of the second electronic device may acquire a still image whenever a score information screen displayed on a certain area of a TV screen appears in a baseball relay broadcast. Alternatively, when recognizing that there is a change in score information through the comparator 1517 of the second electronic device, the image generator 1519 may acquire a still image.

In various embodiments of the present disclosure, the image generator 1519 of the second electronic device may determine at least one of the quality or size of the still image based on the information of the third electronic device 1505 and the information of the second network 1531 . For example, when the display of the third electronic device 1505 is small, the image quality may be reduced by changing the size of the image to a window suitable for the display window or adjusting the quantization level regardless of the size of the input image.

In various embodiments of the present disclosure, the second electronic device may transmit the generated still image 1541 to the third electronic device 1505 through a serial port profile (SPP) 1543 of the BT modem 1533, and the audio signal 1545 may be transmitted to the HFP of the BT modem 1533. It can be delivered to the third electronic device 1505 through (hands free profile) 1547 in real time.

In various embodiments of the present invention, the SPP 1543 and the HFP 1547 may be connected to different electronic devices. For example, the still image 1541 transmitted through the SPP 1543 can be connected to and reproduced by the third electronic device 1505, and the audio signal 1545 transmitted through the HFP 1547 can be transmitted to a hearing device such as a Bluetooth headset or an external speaker. It can be connected and played.

As described above, various embodiments of the present invention have been described assuming that video and audio are delivered to the third electronic device 1505 in different ways, but the present invention is not limited thereto, and video and audio can be encoded and delivered as a single file. . For example, a voice signal may be encrypted with an image file or an image file may be encrypted with a voice signal and transmitted to the third electronic device 1505 through Bluetooth SPP.

According to various embodiments of the present invention, video data and audio signals necessary for performing the above-described video call may be selectively reproduced according to circumstances. For example, video data and audio data may be reproduced or only audio data may be reproduced based on state information of the electronic device and network state information. A configuration for selectively reproducing video data and audio data based on the aforementioned network state information will be described in detail below.

In various embodiments of the present invention, a first network (eg, the first network 411 in FIG. 4 or the first network 1523 in FIG. 15 ) or a second network (eg, the second network 413 in FIG. 4 or the first network 1523 in FIG. 15 ) The state of the second network 1531 is good for performing a video call and the third electronic device (eg, the third electronic device 405 in FIG. 4 , the third electronic device 605 in FIG. 6 , the third electronic device 705 in FIG. 7 , If the battery of the third electronic device 805 of FIG. 8, the third electronic device 905 of FIG. 9, the third electronic device 1413 of FIG. 14A, or the third electronic device 1505 of FIG. 15 is sufficient, the second electronic device (eg, The second electronic device 403 of FIG. 4, the second electronic device 603 of FIG. 6, the second electronic device 703 of FIG. 7, the second electronic device 803 of FIG. 8, the second electronic device 903 of FIG. 9, the second electronic device of FIG. 11 The electronic device 1123 or the second electronic device 1215 of FIG. 12 may encode video data for performing a video call using a transcoding method by adjusting a transcoding scheme parameter suitable for a network condition. there is. In other words, the second electronic device may adjust the resolution, fps, or picture quality to generate image data capable of a video call even at a low transmission speed, and transmit the generated image data to the third electronic device.

In various embodiments of the present disclosure, when the state of the first network or the second network deteriorates, the second electronic device transmits an audio signal to the third electronic device in real time and transmits a still image rather than image data at regular intervals (eg, transmission period determined based on network conditions). Accordingly, the user can perform a limited video call.

In various embodiments of the present disclosure, when the condition of the first network or the second network deteriorates further, the second electronic device may perform a voice call by transmitting only an audio signal to the third electronic device without transmitting video data.

In various embodiments of the present disclosure, when the first network or the second network is in the worst state or the distance between the second electronic device and the third electronic device exceeds a reference value, the second electronic device transmits an image through the third electronic device. It is determined that the call service cannot be provided, and the second electronic device or another electronic device may be controlled to transfer or terminate the video call.

In another embodiment, the second electronic device may detect movement of the third electronic device while performing a video call through the third electronic device. The third electronic device may predict the network quality with the second electronic device based on the movement direction and location change and provide notification. For example, if the third electronic device detects that the network quality deteriorates as the distance from the second electronic device gradually increases, it may output information about deterioration of video call quality through a notification such as vibration or a message.

16 is a diagram for explaining a method of reproducing image data in a third electronic device according to various embodiments of the present disclosure.

In various embodiments of the present invention, the third electronic device (eg, the third electronic device 405 in FIG. 4 , the third electronic device 605 in FIG. 6 , the third electronic device 705 in FIG. 7 , the third electronic device 805 in FIG. 8 , The third electronic device 905 of FIG. 9 , the third electronic device 1413 of FIG. 14A , or the third electronic device 1505 of FIG. 15) controls the video call service according to the movement when the movement of the third electronic device is detected. can

Referring to FIG. 16 , the third electronic device may perform a video call in operation 1601 . In operation 1603, the third electronic device may determine whether motion of the third electronic device is detected.

When the motion of the third electronic device is detected, the third electronic device collects sensor information about the motion of the third electronic device through the sensor unit in operation 1605, and based on the sensor information collected in operation 1607, image data and Audio signals can be selectively reproduced.

For example, the third electronic device may measure a movement angle of the third electronic device in the direction of the head using a geomagnetic sensor. Also, the third electronic device may detect movement of the third electronic device using an acceleration sensor and a gyro sensor. Sensor information according to movement of the third electronic device may be collected through the earth magnetic sensor, acceleration sensor, or gyro sensor.

In various embodiments of the present invention, if the third electronic device is assumed to be a watch, the third electronic device is a motion of a hand wearing the third electronic device (eg, a head direction of the third electronic device through a geomagnetic sensor). It is possible to control the reproduction of image data according to the measurement of the motion angle of . For example, when the user is in a state where the user's hand is lowered (eg, when the movement of the third electronic device in the direction of the head is changed from 0 degree to 90 degree), the third electronic device is in a state in which the user is not viewing the video of the video call. It can be decided to make a call by voice only. When the user's hand is raised (eg, when the head direction of the third electronic device is changed from 90 degrees to 0 degrees), the third electronic device determines that the user is viewing a video of a video call and transmits video and audio. You can decide to make a call. In other words, in various embodiments of the present disclosure, the third electronic device analyzes the collected sensor information, reproduces audio and stores image data in a memory when the movement of the electronic device is within a preset range, and stores the electronic device in a memory. When the motion of the device is not within a preset range, the video stream stored in the memory may be reproduced.

In various embodiments of the present disclosure, when the third electronic device includes an image input device such as a camera, gaze information of the user is obtained through a sensor to determine whether or not to gaze at the screen, and reproduction of image data can be controlled based on this. Alternatively, the third electronic device may determine whether the user gazes at the screen through at least one of face recognition, face recognition, and pupil recognition of the user.

In various embodiments of the present disclosure, when the third electronic device determines to perform a call only by voice, the third electronic device may store the corresponding image data in a temporary buffer without updating the image data, and may receive only the voice.

When recognizing at least one of face recognition, face recognition, or pupil recognition through the camera, the third electronic device can reproduce the image stored in the buffer at a high-speed slide. Accordingly, the user can check the updated image without missing it.

In various embodiments of the present invention, the reproduction speed of the slide may be determined by the number of images to be reproduced. For example, if the slide reproduction time is determined as a specific time, different reproduction speeds may be set for a case where 30 slides are to be played and a play speed when 10 slides are to be played. Alternatively, the reproduction request time may be differently set according to the number of images to be reproduced. Accordingly, various embodiments of the present invention can prevent fast or slow reproduction of an image in reproducing an image.

In various embodiments of the present invention, the displayed image data may be compressed and stored. For example, if at least one of the user's face recognition, face recognition, or pupil recognition is not recognized, the third electronic device may determine that the user is not gazing at the screen and store image data when the user is not gazing at the screen.

In various embodiments of the present disclosure, when at least one of face recognition, face recognition, or pupil recognition is recognized through the camera, the third electronic device may reproduce the compressed and stored image data.

In various embodiments of the present invention, when at least one of face recognition, face recognition, or pupil recognition is not recognized through a camera, the third electronic device extracts a key frame from the image data to generate and store an image. can When at least one of face recognition, face recognition, or pupil recognition is recognized through the camera, the third electronic device can reproduce the stored images in a slide format.

17A and 17B are diagrams for explaining an image encoding method performed in a second electronic device according to various embodiments of the present disclosure.

In various embodiments of the present invention, the second electronic device 1703 (eg, the second electronic device 403 in FIG. 4 , the second electronic device 603 in FIG. 6 , the second electronic device 703 in FIG. 7 , the second electronic device 803 in FIG. 8 ) , the second electronic device 903 of FIG. 9, the second electronic device 1123 of FIG. 11, or the second electronic device 1215 of FIG. 12 uses a scalable encoding (spatial and time domain) method to convert image data. can be encoded. The scalable encoding means encoding an input video stream by dividing it into two or more hierarchical structured video streams. The scalable encoding may compress a video sequence into several layers, for example, a base layer and one or more enhanced layers. The base layer is an independently decodable bit stream and may include information for restoring an image of minimum quality, and the enhanced layer is an additional bit stream for improving the base layer bit stream and is decoded in addition to the base layer. can Accordingly, the scalable encoding can encode several video layers into one bit stream. In other words, the layered stream can be reproduced only by the base layer, and when the base layer and other enhanced layers are combined and reproduced, high-quality images can be hierarchically obtained according to the number of enhanced layers used.

In various embodiments of the present disclosure, the second electronic device 1703 is the first electronic device 1701 (eg, the first electronic device 401 in FIG. 4 , the first electronic device 601 in FIG. 6 , the first electronic device 701 in FIG. 7 , and the first electronic device 701 in FIG. 8 ). All layers of a scalable encoded signal may be received from the first electronic device 801 of FIG. 9 , the first electronic device 901 of FIG. 9 , or the first electronic device 1401 of FIG. 14A . The second electronic device 1703 transmits only the base layer from all the received layers to the third electronic device 1705 (eg, the third electronic device 405 in FIG. 4 , the third electronic device 605 in FIG. 6 , and the third electronic device 705 in FIG. 7 ). , the third electronic device 805 of FIG. 8, the third electronic device 905 of FIG. 9, the third electronic device 1413 of FIG. 14A, or the third electronic device 1505 of FIG. 15). Alternatively, the second electronic device 1703 may use basic information of the third electronic device 1705, current state information, or a second network between the second electronic device 1703 and the third electronic device 1705 (eg, the second network 413 of FIG. 4 ). , or the second network 1531 of FIG. 15) determines an enhanced layer to be transmitted in addition to the base layer based on state information, and transmits the base layer and the determined enhanced layer to the third electronic device 1705.

In various embodiments of the present disclosure, when the first electronic device 1701 encodes data using a scalable encoding method and transmits it to the second electronic device 1703, the second electronic device 1703 transmits basic information of the third electronic device 1705 (eg, the third electronic device 1703). The type of device 1705, display size, shape (eg, round, rectangular, or square), battery capacity, display resolution, CPU power) of the third electronic device 1705, current state information, or the second electronic device 1703 and the third electronic device 1705. An enhanced layer to be transmitted to the third electronic device 1705 may be determined based on state information of the second network between the electronic devices 1705 (eg, the second network 413 in FIG. 4 and the second network 1531 in FIG. 15 ). In other words, the enhanced layer to be transmitted to the third electronic device 1705 may be determined. In various embodiments of the present invention, the second electronic device 1703 considers the state of the third electronic device 1705 that wants to connect a video call to the base layer received from the first electronic device 1701, or the base layer and at least one enhanced layer. may be combined and transmitted to the third electronic device 1705.

In various embodiments of the present disclosure, the information of the second electronic device 1703 may be registered in a user account interworking server. Alternatively, the first electronic device 1701 may obtain information from the second electronic device 1703 during an outgoing operation. For example, the user account linkage server may manage the user's electronic device and information (eg, status information) on the electronic device based on the user account (eg, ID). Accordingly, the user account interworking server may provide information about the electronic device (eg, the first electronic device) to perform a video call to the electronic device desiring to perform a video call service.

For example, as shown in FIG. 17A, a third electronic device 1705, for example, a watch having a 2-inch screen and a carkit having a 7-inch screen are connected to the second electronic device 1703 via Bluetooth as shown in FIG. 17B. It will be described on the assumption that this information is updated in the user account interlocking server as the user's device usage information.

In various embodiments of the present disclosure, the first electronic device 1701 may determine a method of encoding image data, for example, a scalable encoding method, based on device information stored in the user account linkage server.

For example, in various embodiments of the present invention, a user may use an electronic device, a watch and a car kit (or a car entertainment system, or a car control system) connected to the electronic device as an electronic device in use. settings, and the watch and car kit can be registered in the user account interworking server. The user account interworking server may monitor registered electronic devices, for example, a watch and a car kit, detect when the user changes state information of the vehicle (eg, when the vehicle is in use) during monitoring, and detect the change in the vehicle state information. status information can be updated.

In various embodiments of the present invention, the enhanced layer transmitted from the first electronic device 1701 to the third electronic device 1705 connected to the second electronic device 1703 through the second network varies according to time and spatial requirements. can be configured. The first electronic device 1701 may generate a base layer and an enhanced layer in consideration of temporal resolution (7.5 fps, 15 fps, or 30 fps) and spatial resolution (QCIF, CIF, or QVGA).

For example, since watch and car kit information is stored in a user account interworking server, the first electronic device 1701 can perform a video call with at least one electronic device of the watch or car kit. The first electronic device 1701 may encode video data for the watch and car kit in consideration of this. For example, the first electronic device 1701 is based on the base layer 1711, with enhanced layer #1 1713, e.g. low quality encoding for watch, enhanced layer #2 1715, e.g. high quality encoding for watch, enhanced layer #3 1717 For example, low-quality encoding for carkit and high-quality encoding for carkit may be performed using the enhanced layer #4 1719 and transmitted to the second electronic device 1703 .

For example, as shown in FIG. 17A , when a video call is received using a third electronic device 1705, for example, a watch, the second electronic device 1703 provides state information of the second network, memory usage of the watch, or battery level. According to the current information of the watch, only the base layer 1711 may be transmitted to the watch, or the base layer 1711 and enhanced layer #1 1713 corresponding to low-quality encoding for the watch may be transmitted to the third electronic device 1705.

In various embodiments of the present invention, as shown in FIG. 17B , when a video call is received using a third electronic device 1705, for example, a car kit, the second electronic device 1703 provides state information of the second network, memory usage of the car kit, According to the current information of the carkit, such as CPU power or process occupancy in the operating system, only the base layer 1711 is transmitted to the carkit, or the base layer 1711 and the enhanced layer #3 1717 corresponding to low-quality encoding for the carkit are transmitted to the carkit. 3 to the electronic device 1705.

18 is a diagram for explaining an image encoding method performed in a second electronic device according to various embodiments of the present disclosure.

Referring to FIG. 18 , in various embodiments of the present invention, a second electronic device (eg, the second electronic device 403 of FIG. 4 , the second electronic device 603 of FIG. 6 , the second electronic device 703 of FIG. 7 , the second electronic device 703 of FIG. 8 ) The second electronic device 803, the second electronic device 903 in FIG. 9, the second electronic device 1123 in FIG. 11, the second electronic device 1215 in FIG. 12, and the second electronic device 1703 in FIG. 17A or 17B) are mirroring/resized- Video data can be encoded using a mirroring method.

In various embodiments of the present invention, the second electronic device is a first electronic device (eg, the first electronic device 401 in FIG. 4 , the first electronic device 601 in FIG. 6 , the first electronic device 701 in FIG. 7 , the first electronic device 701 in FIG. 8 ). 1 Provides resized mirroring image data based on image data received from the electronic device 801, the first electronic device 901 of FIG. 9, the first electronic device 1401 of FIG. 14A, or the first electronic device 1701 of FIG. 17A or 17B). 3 electronic devices (eg, the third electronic device 405 in FIG. 4 , the third electronic device 605 in FIG. 6 , the third electronic device 705 in FIG. 7 , the third electronic device 805 in FIG. 8 , the third electronic device 905 in FIG. 9 ) , the third electronic device 1413 of FIG. 14A, the third electronic device 1505 of FIG. 15, or the third electronic device 1705 of FIG. 17A or 17B). The second electronic device connects the second electronic device with information on the third electronic device (eg, type of third electronic device, display size, CPU power, screen type, battery level, or memory usage) The resize size of image data may be determined based on state information of the second network (eg, the second network 413 of FIG. 4 or the second network 1531 of FIG. 15 ).

In various embodiments of the present disclosure, the second electronic device moves the focus of the mirroring image data or crops the mirroring image data by using information about the display shape (eg, circular, square, or rectangular) and contents of the third electronic device. (crop) and transmit it to the third electronic device. The second electronic device may focus or crop the mirroring image data around the center, or focus or crop the mirrored image data using a face recognition or object recognition method, and transmit the mirrored image data to the third electronic device.

For example, as shown in FIG. 18 , the third electronic device may be an electronic device 1803 having a square display shape and an electronic device 1805 having a circular display shape. When image data of the electronic device 1801 whose display shape is a rectangle longer than the width is displayed on an electronic device 1803 having a square display shape or an electronic device 1805 having a circular display shape, the second electronic device is displayed on the electronic device 1801. In the displayed image data, only the face area may be recognized and cropped into the display shape, for example, a square shape 1803 or a circular shape 1805. The second electronic device may transmit only the cropped face region to the third electronic device. The third electronic device may receive and display the image data of the face area received from the second electronic device.

19A and 19B are diagrams illustrating user interfaces related to receiving and sending a video call according to various embodiments of the present disclosure.

Referring to FIGS. 19A and 19B , the second electronic device (eg, the second electronic device 403 of FIG. 4 , the second electronic device 603 of FIG. 6 , the second electronic device 703 of FIG. 7 , the second electronic device of FIG. 8 ) 803, the second electronic device 903 of FIG. 9, the second electronic device 1123 of FIG. 11, the second electronic device 1215 of FIG. 12, the second electronic device 1703 of FIG. 17A or 17B) is a first electronic device (eg, FIG. 4, the first electronic device 601 in FIG. 6, the first electronic device 701 in FIG. 7, the first electronic device 801 in FIG. 8, the first electronic device 901 in FIG. 9, the first electronic device in FIG. 14A When a video call request is received from the device 1401 or the first electronic device 1701 of FIG. 17A or 17B , the second electronic device sends a second network (eg, the second network 413 of FIG. 4 or the second network of FIG. 15 ). 1531) (eg, the third electronic device 405 in FIG. 4, the third electronic device 605 in FIG. 6, the third electronic device 705 in FIG. 7, the third electronic device 805 in FIG. 8, and the third electronic device 805 in FIG. 9 A notification may be transmitted to the third electronic device 905 of FIG. 14A , the third electronic device 1505 of FIG. 15 , or the third electronic device 1705 of FIG. 17A or 17B . Upon receiving the notification, the third electronic device uses information such as state information of the second network connected to the second electronic device, battery levels of the second electronic device and the third electronic device, memory state, process schedule state, etc. It is possible to determine whether a video call is possible and display it on the display.

For example, if the state of the second network is good for performing a video call and the battery state is sufficient, the third electronic device performs video call with the second electronic device 1901, as shown in FIG. 19A, to the third electronic device. User interfaces such as performing a video call 1903, performing a voice call with a third electronic device 1905, and rejecting reception 1907 may be displayed.

In various embodiments of the present invention, when receiving video data for a video call, the third electronic device displays video call quality information such as resolution information or fps information of the video data together, so that the user for receiving low-quality or high-quality video interface can be displayed.

In various embodiments of the present disclosure, when a video call is made to an external electronic device, for example, the first electronic device, the third electronic device provides state information of the second network with the second electronic device, the first network (eg, FIG. 4 ). Whether or not a video call is available is determined using information such as status information of the first network 411 or the first network 1523 of FIG. 15 , battery levels of the first to third electronic devices, or schedule status of the memory state processor. determined and displayed in the user interface. For example, when the condition of the second network is poor to perform a video call or the battery is insufficient, the third electronic device transmits audio and video to an external electronic device, for example, the first electronic device as shown in FIG. 19B. Only user interfaces such as making a call 1913 and making a video call with the second electronic device 1911 may be displayed.

In various embodiments of the present disclosure, the third electronic device may operate as a hub or a controller for video call connection to other electronic devices. Accordingly, the first electronic device may perform a video call with a device registered to the first electronic device to the third electronic device or an unregistered but usable device through the third electronic device.

20A to 20D are diagrams illustrating a user interface for receiving a video call according to various embodiments of the present disclosure.

20A to 20D, the third electronic device (eg, the third electronic device 405 of FIG. 4, the third electronic device 605 of FIG. 6, the third electronic device 705 of FIG. 7, the third electronic device of FIG. 8) 805, the third electronic device 905 of FIG. 9, the third electronic device 1413 of FIG. 14A, the third electronic device 1505 of FIG. 15, the first electronic device 1701 of FIG. 17A or 17B) is the third electronic device or the first electronic device 3 An electronic device connected to the electronic device, a second electronic device (eg, the second electronic device 403 in FIG. 4 , the second electronic device 603 in FIG. 6 , the second electronic device 703 in FIG. 7 , the second electronic device 803 in FIG. 8 ) , the second electronic device 903 of FIG. 9, the second electronic device 1123 of FIG. 11, the second electronic device 1215 of FIG. 12, or the second electronic device 1703 of FIG. 17A or 17B) connected or registered, the 2 Connectable electronic devices that are not registered as electronic devices or third-party electronic devices may be displayed. When receiving an input for selecting one electronic device from among the displayed electronic devices capable of conducting a video call, the third electronic device may transmit a video call signal to the selected electronic device.

In various embodiments of the present disclosure, the size of an icon representing an electronic device capable of performing a video call may indicate a video call quality state. The video call quality state may be determined by a network state of each electronic device and a state of each electronic device, and the size of the icon may be proportional to the video call quality state. For example, the larger the size of the icon, the higher the video call quality, and the smaller the icon, the lower the call quality.

In various embodiments of the present invention, the description has been made on the assumption that the size of the icon is displayed differently according to the quality of the video call, but the present invention is not limited thereto, and the shape, color (eg, contrast/black and white) of the icon, and surroundings of the icon are not limited thereto. An electronic device suitable for a video call may be displayed, or a different icon depending on the quality of a video call (eg, a torn icon if the quality is poor) may be displayed. In other words, according to various embodiments of the present disclosure, a video call quality state may be indicated by combining at least one of the size, shape, or color of the aforementioned icons.

As shown in FIGS. 20A to 20D , the third electronic device may display icons representing electronic devices capable of conducting a video call. For example, the third electronic device includes an icon 2011 representing a TV, an icon 2013 representing a smartphone, an icon 2015 representing a car kit, an icon 2017 representing a tablet PC, an icon 2019 representing a headset, an icon 2021 representing a video call rejection, and an icon representing a watch. Can display icon 2023.

For example, the size of an icon 2011 representing a TV may be different from other electronic devices such as a smartphone icon 2013, a car kit icon 2015, a tablet PC icon 2017, a headset icon 2019, a video call rejection icon 2021, and a watch. It can be seen that the icon representing the is larger than the size of 2023. In other words, when the icon 2011 representing a TV is selected, the first electronic device (eg, the first electronic device 401 in FIG. 4 , the first electronic device 601 in FIG. 6 , the first electronic device 701 in FIG. 7 , The first electronic device 801, the first electronic device 901 of FIG. 9, the first electronic device 1401 of FIG. 14A, and the first electronic device 1701 of FIG. 17A or 17B) can perform a video call with the TV in high quality. It can be seen that the size of the icon 2015 representing the car kit is relatively small compared to the size of the icon 2111 representing the TV. In other words, when the icon 2015 representing the car kit is selected, the communication state may be of lower quality than the communication state with the TV, and the first electronic device receives the car kit and, for example, voice in real time and receives the video at regular intervals to obtain the video image. A call may be made or a call may be made only by voice.

As described above, various embodiments of the present invention have been described on the assumption that the size of an icon representing a specific electronic device is adjusted based on the quality of a video call, but is not limited thereto, and the size of an icon representing a specific electronic device is Based on user history, user usage pattern, or context awareness information, the position where the icon is placed or adjusted may be changed. For example, the icon representing the specific electronic device may be displayed in a sorted form according to size. Also, the number of electronic device icons to be displayed on the third electronic device, the type or properties of the electronic device to be displayed may be set by the user.

In various embodiments of the present invention, when a video call is received and an electronic device for a video call is changed while the video call is being performed, an input for changing the electronic device for the video call is performed by the user while the video call is being performed. When detected, the third electronic device may display a user interface including a switchable electronic device to perform a video call.

In various embodiments of the present invention, the third electronic device analyzes the network state of each electronic device or the state of each electronic device among electronic devices capable of performing a displayed video call to determine priorities, and determines the priorities. Based on this, the size of the icon can be differentiated and displayed.

In various embodiments of the present invention, the third electronic device may monitor a network state of each electronic device or a state change (eg, battery level change) of each electronic device among electronic devices capable of performing a video call in real time or at regular time intervals. After analysis, the size of the icon representing each electronic device or the position where the icon is displayed may be adjusted and displayed.

In various embodiments of the present invention, the network status of each electronic device or status information of each electronic device is obtained based on a predetermined time interval, an average value of a predetermined time, or a reference threshold, so that an icon representing each electronic device is displayed. You can update the size and/or placement position. Accordingly, according to various embodiments of the present disclosure, it is possible to prevent a sudden change in size of an icon representing each electronic device.

In various embodiments of the present disclosure, an icon representing each electronic device may be displayed along with additional information. For example, the additional information may include the owner (account registrant) of the electronic device, the setting name of the electronic device, or authority information.

In various embodiments of the present disclosure, upon receiving an input for selecting a specific electronic device, the third electronic device may transmit a video call connection signal to the selected specific electronic device. For example, as shown in FIG. 20A, select the icon 2023 representing the watch 2025, drag the icon 2023 representing the watch as shown in FIG. 20B to the caller display window or the video call window 2001 2041, as shown in FIG. When an operation of dragging 2051 from the caller display window or the video call window 2001 to the icon 2023 representing the watch is detected, the third electronic device may transmit a video call connection signal to the watch.

In various embodiments of the present disclosure, when the third electronic device is a watch and the watch does not have a separate crown, a video call can be performed by selecting an icon representing the electronic device as described above. Alternatively, when the third electronic device is a watch and the watch has a separate crown (not shown), the watch may control the crown to select an icon representing an electronic device to perform a video call. Alternatively, when the third electronic device is a circular watch, an icon representing an electronic device to perform a video call may be selected by rotating a wheel 2061 as shown in FIG. 20D .

20E is a diagram for explaining a method of performing a video call in a third electronic device according to various embodiments of the present disclosure.

Referring to FIG. 20E , the third electronic device may receive a video call in operation 2071.

In various embodiments of the present disclosure, upon receiving a video call, the third electronic device may additionally perform an operation of determining whether the third electronic device is worn.

In various embodiments of the present disclosure, the third electronic device may determine whether the user is wearing the third electronic device through a sensor unit, such as a geomagnetic sensor, a gravity sensor, a heart rate sensor, or a temperature sensor. Alternatively, the third electronic device may determine whether or not the user is wearing the third electronic device through whether or not the watch strap is coupled through a hall sensor.

In various embodiments of the present disclosure, when it is determined that the third electronic device is worn, the third electronic device may transmit a sensor signal regarding the wearing of the third electronic device to the second electronic device. Based on the received sensor signal, the second electronic device may transmit a control signal to display the notification only to the third electronic device without displaying the notification on the second electronic device. Accordingly, in various embodiments of the present invention, a notification indicating that a video call has been received may not be displayed on the display of the second electronic device, and the video call is received only on the display of the third electronic device that directly performs the video call. You can display a notification notifying you that it has been done.

In various embodiments of the present disclosure, the third electronic device may display different alarms according to the location of the second electronic device. For example, when the second electronic device is located in the living room, the third electronic device may output a first notification sound. When the second electronic device is located in a bedroom, the third electronic device may output a second notification sound. When the second electronic device is located in the kitchen, the third electronic device may output vibration. When the second electronic device is located at the front door, the third electronic device may notify the user of the reception of the video call request by turning on/off the flash. In various embodiments of the present invention, the location of the second electronic device may be obtained through, for example, indoor navigation using wifi or BLE, or a communication network such as GPS, LTE, or 3G.

In various embodiments of the present disclosure, upon receiving a video call request, the third electronic device may display additional information including the location of the second electronic device along with a notification indicating that the video call request has been received. For example, the third electronic device may display the location of the second electronic device acquired by using map information stored in a memory, and the name of a location registered by the user or the name and properties of a location stored in a server together. can be displayed

In various embodiments of the present disclosure, upon receiving a video call request, the third electronic device may display an arrow indicating the location of the second electronic device together with a notification indicating that the video call request has been received. For example, the third electronic device may display map degree, direction information, distance information between the third electronic device and the second electronic device, and a reachable time to the second electronic device based on the distance information. The distance information may be displayed in the form of numbers, colors, or icons.

In various embodiments of the present disclosure, in operation 2073, the third electronic device may display at least one electronic device capable of performing a video call. The third electronic device may be the third electronic device or an electronic device connected to the third electronic device, an electronic device connected to or registered with the second electronic device, or not registered with the second electronic device or the third electronic device but connectable. Electronic devices can be displayed.

In various embodiments of the present invention, the at least one electronic device may be displayed as an icon, and the size of the icon may be determined by a video call quality state (eg, a state of the electronic device (eg, battery level) or a network state), a user's It may be determined based on user history, user usage patterns, or context awareness information. For example, the third electronic device may set the icon size of an electronic device having good video call quality to be larger than a preset size. In addition, the third electronic device may analyze the user's previous usage history or the user's usage pattern and set the icon size of an electronic device that frequently performs video calls to be larger than a preset size. On the other hand, the third electronic device may set the icon size of an electronic device that does not frequently perform a video call to be smaller than a preset size by analyzing the user's previous use history or user's use pattern, or the quality of the video call is poor.

In various embodiments of the present disclosure, when an input for selecting an electronic device to perform a video call is detected from among the one or more displayed electronic devices in operation 2075, the third electronic device transmits a video call signal to the selected electronic device in operation 2077. can For example, as described above with reference to FIGS. 20A to 20C , the third electronic device performs a touch input on an electronic device to perform a video call, drags an icon representing an electronic device to perform a video call to a caller display window or a video call window, An electronic device to conduct a video call may be selected through an operation of dragging the display window or the video call window to the icon representing the electronic device to conduct the video call.

In various embodiments of the present disclosure, when the third electronic device does not have a separate crown, a video call may be performed by selecting an icon representing the electronic device as described above. Alternatively, when the third electronic device has a separate crown (not shown), an icon of an electronic device to perform a video call may be selected using the crown. Alternatively, an icon of an electronic device to perform a video call may be selected through an operation of rotating a wheel of the third electronic device.

21A and 21B are views illustrating a user interface according to a pressure level of a user input according to various embodiments of the present disclosure.

Referring to FIGS. 21A and 21B , a third electronic device (eg, the third electronic device 405 of FIG. 4 , the third electronic device 605 of FIG. 6 , the third electronic device 705 of FIG. 7 , the third electronic device of FIG. 8 ) 805, the third electronic device 905 of FIG. 9, the third electronic device 1413 of FIG. 14A, the third electronic device 1505 of FIG. 15, the third electronic device 1705 of FIG. 17A or 17B) receives an input for switching a video call. If detected, at least one electronic device capable of switching a video call, for example, icon 2101 representing a TV, icon 2103 representing a tablet PC, icon 2105 representing a headset, icon 2107 representing a smartphone, icon 2109 representing a car kit, and video call termination An icon 2111 indicating may be arranged according to the shape of the third electronic device.

In various embodiments of the present disclosure, different functions may be performed according to a pressure level of a touch input for selecting an icon representing the specific electronic device. For example, when a touch input with a pressure level of 1 is detected, the third electronic device can switch to a video call with a specific electronic device in which a touch input with a pressure level of 1 is detected. When a touch input with a pressure level of 2 is detected, the third electronic device may transmit a notification to a specific electronic device in which a touch input with a pressure level of 2 is detected. Through this notification, the user can intuitively check the specific electronic device to which the video call will be switched.

For example, when a touch input 2113 is detected at a pressure level of 2 in an icon 2101 representing a TV among the displayed icons, the third electronic device may transmit a notification to the TV corresponding to the icon 2101 representing the TV. The TV controls the screen of the TV to flicker based on a notification received from the third electronic device, displays a specific image pattern on at least a portion of the TV screen, and provides user-related information, such as user account information or third party information. Information of the electronic device may be displayed on at least a portion of the TV screen. In addition, when a touch input 2113 is detected as a pressure level of 2 in the icon 2101 indicating a TV, the third electronic device may change the user interface to “device change” and display the changed user interface as shown in FIGS. 21A and 21B .

In various embodiments of the present invention, the screen displayed on the TV and the screen displayed on the third electronic device are temporarily matched to each other, so that the user can intuitively check the electronic device to which the video call will be transmitted before connecting the video call.

In various embodiments of the present invention, the third electronic device temporarily substitutes an icon representing an electronic device to convert the video call on a screen displaying icons representing an electronic device capable of converting a video call, or displays caller information. An icon indicating an electronic device to which the video call is to be converted may be replaced and displayed at the location.

In various embodiments of the present disclosure, when a touch input with a pressure level of 3 is detected, the third electronic device may display detailed property information of the electronic device for which the touch input with a pressure level of 3 is detected. For example, the third electronic device may display detailed information related to properties of the electronic device where the touch input is detected and network connection properties.

21C is a diagram for explaining a function according to a pressure level of a user input according to various embodiments of the present disclosure.

Referring to FIG. 21C , when the third electronic device detects an input for switching the video call in operation 2131, it can display at least one electronic device capable of switching the video call in operation 2133.

In operation 2135, the third electronic device may detect an input for selecting an electronic device to switch a video call from among the at least one displayed electronic device, and may check a pressure level for an input for selecting the detected electronic device. . In operation 2137, the third electronic device may perform a function related to the selected electronic device corresponding to the pressure level.

For example, when a touch input with a pressure level of 1 is detected, the third electronic device may perform an operation of switching a video call to the electronic device in which the touch input with a pressure level of 1 is detected. When a touch input with a pressure level of 2 is detected, the third electronic device may transmit a notification so that a notification is output to the electronic device in which the touch input with the pressure level of 2 is detected. When a touch input with a pressure level of 3 is detected, the third electronic device 405 provides attribute information (eg, information of the electronic device in which the touch input is detected or network information) of the electronic device in which the touch input of the pressure level 3 is detected. can be displayed

21D is a diagram for explaining a function according to a pressure level of a user input according to various embodiments of the present disclosure.

In various embodiments of the present disclosure, the third electronic device may perform different functions according to a detected user input and a pressure level of the detected user input.

In various embodiments of the present disclosure, the third electronic device recognizes an object of image data based on a location where a user input is sensed, and selects resolution, focus, or exposure according to a pressure level applied based on the object. At least one can be adjusted.

In various embodiments of the present disclosure, the third electronic device may include a user input, a time for which the user input is maintained, an area of the user input (eg, an area according to the user's finger being used), a hovering input level (eg, an input tool Different user interfaces may be displayed based on at least one of (for example, a user's finger or stylus) or a distance interval between touch screens.

Referring to FIG. 21D , the third electronic device may perform a video call in operation 2141. The third electronic device may detect a user input in operation 2143. In operation 2145, the third electronic device may check the pressure level of the sensed user input. In operation 2147, the third electronic device may perform a function corresponding to the pressure level.

For example, the third electronic device may display a video call menu when a user input is detected on at least a partial area of the display during a video call. The third electronic device may display a user interface for changing the resolution of image data when detecting a user input of pressure level 1. When the third electronic device detects a user input of pressure level 2, it may display a user interface for switching the video call. When the third electronic device detects a user input of pressure level 3, it can display a user interface for changing the quality of image data.

In various embodiments of the present disclosure, when the third electronic device detects a user input of pressure level 1 in at least a portion of a displayed image, the third electronic device converts the image data at a specific time interval or key frame while maintaining the user input of pressure level 1. It can be captured and stored as a still image at (key frame) intervals. When the third electronic device detects the user input of pressure level 2, it can store image data as a video while the user input of pressure level 2 is maintained.

In various embodiments of the present disclosure, the third electronic device may display at least a portion of an image or video stored by the user's input of the pressure level 2 on at least a portion of the display device for a predetermined time. For example, the third electronic device may display the stored image in response to the user input of the pressure level 2 as a snapshot in a predetermined size on at least a portion of the display device displaying image data, for example, on the lower portion. Alternatively, the third electronic device may display an icon indicating that a video is being recorded on at least a portion of the display device in response to the user input of the pressure level 2. In various embodiments of the present invention, the user can intuitively check that an image or video is being saved (or recorded) through an icon indicating that the snapshot or recording is being performed.

In various embodiments of the present invention, when the third electronic device detects a user input of pressure level 1 in a partial region of a displayed image, a high-resolution image corresponding to the image at the time when the user input of pressure level 1 is sensed is displayed as a first image. An electronic device (eg, the first electronic device 401 of FIG. 4 , the first electronic device 601 of FIG. 6 , the first electronic device 701 of FIG. 7 , the first electronic device 801 of FIG. 8 , the first electronic device 901 of FIG. 9 , The first electronic device 1401 of FIG. 14A, the first electronic device 1701 of FIG. 17A or 17B) or the second electronic device (eg, the second electronic device 403 of FIG. 4, the second electronic device 603 of FIG. 6, or the second electronic device 603 of FIG. 7 The second electronic device 703, the second electronic device 803 of FIG. 8, the second electronic device 903 of FIG. 9, the second electronic device 1123 of FIG. 11, the second electronic device 1215 of FIG. 12, the second electronic device of FIG. 17A or 17B A request may be made to the electronic device 1703). A third electronic device (eg, the third electronic device 405 in FIG. 4 , the third electronic device 605 in FIG. 6 , the third electronic device 705 in FIG. 7 , the third electronic device 805 in FIG. 8 , the third electronic device in FIG. 9 ) 905, the third electronic device 1413 of FIG. 14A, the third electronic device 1505 of FIG. 15, and the second electronic device 1703 of FIG. 17A or 17B) detect a user input of pressure level 2 in a part of the displayed image. The first electronic device or the second electronic device may be requested to adjust the focus of the camera around the area where the user input of the pressure level 2 is sensed. When the third electronic device detects a user input of pressure level 3 in a partial area of the displayed image, the third electronic device adjusts the exposure of the camera based on the area where the user input of pressure level 3 is detected to the first electronic device or the second electronic device. can request

In various embodiments of the present invention, it is assumed that different functions are performed according to the pressure level of the user input, but the present invention is not limited thereto, and different functions may be performed according to the direction in which the user input is sensed. In this regard, it will be described in detail with reference to FIG. 21E.

21E is a diagram for explaining a function according to a direction in which a user input is sensed according to various embodiments of the present disclosure.

In various embodiments of the present disclosure, the third electronic device may perform different functions according to a direction in which a user input is sensed.

Referring to FIG. 21E , the third electronic device may perform a video call in operation 2151. The third electronic device may detect a user input in operation 2153. In various embodiments of the present disclosure, the user input may include at least one of a swipe input and a drag input. The third electronic device may check the direction in which the user input is sensed in operation 2155, and perform a function corresponding to the direction in which the user input is sensed in operation 2157.

According to various embodiments of the present disclosure, the user input in FIG. 21E is assumed to be a swipe. The third electronic device may display a menu related to the video call when a user input, for example, a swipe is detected on at least a partial area of the display during a video call. The third electronic device may analyze the direction in which the swipe is sensed. The direction in which the swipe is detected may include at least one of a swipe from left to right, a swipe from right to left, a swipe from top to bottom, or a swipe from bottom to top. can

For example, when detecting a swipe from the left to the right, the third electronic device may display a user interface for changing the resolution of image data. When detecting a swipe from right to left, the third electronic device may display a user interface for converting a video call. When the third electronic device detects a swipe from the top to the bottom, it can display a user interface for changing the image quality of the image data. The present invention is not limited thereto, and the resolution of the image data, the focus of the image data, or the exposure of the image data may be adjusted according to the direction in which the swipe is sensed.

As described above, in various embodiments of the present invention, different user interfaces are displayed based on at least one of a user input, a holding time of the user input, a hovering level, a pressure level of the user input, or a direction in which the user input is sensed. or perform different functions. In addition, the user input, the holding time of the user input, the level of the hovering input (eg, the distance between the input tool (eg, the user's finger or stylus) and the touch screen), the pressure level of the user input, or the user input is detected. A user interface displayed based on at least one of the directions may be mapped and stored in a memory.

22A is a diagram illustrating a user interface including an indicator indicating quality of a video call according to various embodiments of the present disclosure.

Referring to FIG. 22A , in various embodiments of the present disclosure, a third electronic device (eg, the third electronic device 405 in FIG. 4 , the third electronic device 605 in FIG. 6 , the third electronic device 705 in FIG. 7 , and the third electronic device 705 in FIG. 8 ) The third electronic device 805, the third electronic device 905 of FIG. 9, the third electronic device 1413 of FIG. 14A, the third electronic device 1505 of FIG. 15, and the second electronic device 1703 of FIG. 17A or 17B) are shown in 2201. As described above, while performing a video call, video data 2209 and an indicator 2211 indicating the quality of the video call may be displayed. Also, as shown in 2203, the third electronic device may display video data 2209, an indicator 2211 indicating the quality of the video call, and a button 2213 for lowering the resolution of the video data.

In various embodiments of the present invention, as shown at 2205, the third electronic device indicates the video call quality determined based on video data 2209, the network state, and information of the third electronic device, and the quality of the video call can be adjusted. A user interface including indicator 2215 may be displayed. For example, the indicator 2215 may include the maximum usable video call quality 2221 determined based on the network status and the information of the third electronic device and the currently used video call quality 2223 . The user can intuitively adjust the video call quality through the slide control 2223 in the indicator 2215 .

For example, indicators 2215 through 2221 indicate the maximum playable video call quality, and 2223 indicate the quality of a currently playing video call. The third electronic device may adjust the video call quality when the indicator 2215 detects an input 2225 for adjusting the video call quality (eg, dragging in a left or right direction after selecting the video call quality area). there is. The third electronic device may determine an encoding scheme corresponding to the adjusted video call quality. The third electronic device converts the determined encoding scheme to the first electronic device (eg, the first electronic device 401 of FIG. 4 , the first electronic device 601 of FIG. 6 , the first electronic device 701 of FIG. 7 , the first electronic device of FIG. 8 ). 801, the first electronic device 901 of FIG. 9, the first electronic device 1401 of FIG. 14A, or the first electronic device 1701 of FIG. 17A or 17B). The first electronic device or the second electronic device may encode image data based on the encoding scheme and transmit the encoded image data to a third electronic device. The third electronic device may reproduce the received encoded image data.

In various embodiments of the present invention, as shown at 2207, if there is a shadow in the left area of the indicator 2231 based on the reference value 2230, the video call quality may mean low quality, and if there is a shadow from the indicator 2233 to the reference value 2230, the video The call quality may mean medium quality, and if a right area of the indicator 2235 is shaded based on the reference value 2230, the video call quality may mean high quality.

Various embodiments of the present invention have been described on the assumption that the video call quality is adjusted through the indicator as described above, but the present invention is not limited thereto, and rotation of a crown (not shown) separately provided in the third electronic device, for example, a watch Alternatively, the quality of the video call may be adjusted through a rotation of the wheel, voice input, gesture input, or a combination thereof.

For example, according to various embodiments of the present disclosure, when a user wants to minimize battery consumption or perform other tasks through multitasking, the quality of a video call may be adjusted to a minimum level by a user. Accordingly, since the video call quality can be adjusted according to the user's situation, satisfaction in the video call service can be high.

In various embodiments of the present disclosure, when receiving and displaying video data for a video call, the third electronic device may also display video call quality information such as resolution information or fps information for the video data. Also, when an input for adjusting the indicator is detected, the third electronic device may display video call quality information corresponding to the adjusted input. In various embodiments of the present invention, as the video call quality information is also displayed, the user can intuitively check the currently displayed video data information or the video data information according to the adjustment of the indicator.

22B is a diagram illustrating a method of adjusting quality of a video call according to various embodiments of the present disclosure.

Referring to FIG. 22B , while displaying video data necessary for a video call in operation 2231, the third electronic device may determine whether an input for adjusting the quality of video data is detected in operation 2233.

In various embodiments of the present disclosure, the third electronic device may display video data for a video call, and may display an indicator for adjusting the quality of the video data on a partial area of the display unit. The indicator may include a maximum usable image data quality determined based on a state of the third electronic device or a network state and a current image data quality. When an input for adjusting the quality of image data is detected through the indicator, the third electronic device may perform operation 2235 to be described later.

On the other hand, if an input for adjusting the quality of image data is not sensed through the indicator, the third electronic device may perform an operation 2231 of displaying the image data.

In various embodiments of the present disclosure, the third electronic device may determine an encoding scheme based on the adjusted video call quality in operation 2235 and transmit the determined encoding scheme to the second electronic device in operation 2237.

In various embodiments of the present disclosure, when an operation of encoding image data is performed by a second electronic device, the second electronic device encodes the image data based on the encoding scheme received from the third electronic device and converts the video data into the third electronic device. can be transmitted to the device. Alternatively, when the operation of encoding image data is performed by the first electronic device, the second electronic device may transmit the encoding scheme received from the third electronic device to the first electronic device. The second electronic device may receive image data encoded based on the encoding scheme from the first electronic device and transmit it to the third electronic device.

In operation 2239, the third electronic device may receive and display image data encoded based on the encoding scheme from the second electronic device.

23A and 23B are diagrams illustrating a user interface for explaining video call conversion operations according to various embodiments of the present disclosure.

In various embodiments of the present invention, the third electronic device (eg, the third electronic device 405 in FIG. 4 , the third electronic device 605 in FIG. 6 , the third electronic device 705 in FIG. 7 , the third electronic device 805 in FIG. 8 , The third electronic device 905 of FIG. 9 , the third electronic device 1413 of FIG. 14A , the third electronic device 1505 of FIG. 15 , and the third electronic device 1705 of FIG. 17A or 17B ) may move or externally while performing a video call. A state change of the second network (eg, the second network 413 of FIG. 4 or the second network 1531 of FIG. 15 ) may be detected by the interference signal generation. Alternatively, while performing a video call, the third electronic device may change battery status of the third electronic device, lack of memory due to multitasking, lack of processing power, or the first electronic device (eg, the first electronic device 401 of FIG. 4 ). The first electronic device 601 of FIG. 6 , the first electronic device 701 of FIG. 7 , the first electronic device 801 of FIG. 8 , the first electronic device 901 of FIG. 9 , the first electronic device 1401 of FIG. 14A , or the first electronic device 17A or 17B the first electronic device 1701 of FIG. 4 and the second electronic device (eg, the second electronic device 403 of FIG. 4 , the second electronic device 603 of FIG. 6 , the second electronic device 703 of FIG. 7 , and the second electronic device 803 of FIG. 8 ). , a first network (e.g., FIG. While performing a video call due to quality deterioration of the first network 411 of 4 or the first network 1523 of FIG. 15 , battery level change of the first electronic device or the second electronic device, insufficient memory, or insufficient processing power, etc. Changes in call quality can be detected.

In various embodiments of the present disclosure, upon detecting a change in quality of a video call, the third electronic device may request the first electronic device or the second electronic device performing encoding to change an encoding scheme. The third electronic device may receive and reproduce image data encoded based on the changed encoding scheme in response to an encoding scheme change request from the first electronic device or the second electronic device.

In various embodiments of the present invention, when a video call cannot be performed, the third electronic device may change to a voice call instead of a video call. A video call may be switched to another electronic device connected to the second electronic device. When the video call is converted to a voice call, or when the video call is switched to another electronic device connected to the second electronic device, the third electronic device converts the video call to a voice call and converts the video call to another electronic device. A video call conversion notification may be provided to the user through a user interface (eg, a pop-up message), voice, or vibration.

In various embodiments of the present disclosure, the third electronic device may obtain information on electronic devices that are currently connectable from the second electronic device, and may acquire information on a user's usage pattern, status of each electronic device, or network connection status information. Electronic devices capable of converting video calls may be displayed according to priority.

In various embodiments of the present invention, the third electronic device includes an electronic device capable of switching the connection so that the user can select an electronic device to switch the video call to when the environment for providing the video call of the third electronic device is not good. A pop-up message can be displayed. For example, as shown in FIG. 23A , a video call conversion pop-up message 2311 may be displayed, and an electronic device connected to a second electronic device capable of video call conversion, such as a smartphone 2313 or a car kit 2315, may be displayed. When an input for selecting one of the electronic devices displayed in the pop-up message, for example, the smartphone 2313 or the car kit 2315, is detected, the third electronic device may transmit a video call conversion signal to the selected electronic device.

In various embodiments of the present invention, the video call conversion operation may be operated through the third electronic device or the second electronic device performing the video call. For example, as shown in FIG. 23B , when a video call quality change signal is detected while the third electronic device is performing a video call, the second electronic device may display a pop-up message 2331 indicating the video call conversion. When an input for selecting one of the electronic devices displayed in the pop-up message 2331, for example, the watch 2333, the car kit 2335, and the phone 2337, is detected, a video call conversion signal may be transmitted to the selected electronic device.

In various embodiments of the present disclosure, if an input for selecting an electronic device for switching a video call is not detected, the third electronic device may control such that an optimal electronic device is automatically selected among electronic devices capable of switching a video call. there is. The optimum electronic device may be selected based on current user use or previous connection information using state information (eg, battery, process power, or memory) of each electronic device and network connection information.

23C is a flowchart illustrating an operation of switching a video call to a fourth electronic device according to various embodiments of the present disclosure.

Referring to FIG. 23C , in operation 2341, the third electronic device may determine whether a video call quality change signal is detected. For example, while performing a video call, the third electronic device may detect a signal in which the state of the second network is changed due to location movement or external interference signal generation. Alternatively, the third electronic device may include a change in battery state of the third electronic device, insufficient memory, insufficient processing power, deterioration in quality of a first network connecting the first electronic device and the second electronic device, and a first electronic device or a second electronic device. A signal in which the quality of a video call is changed due to a change in battery level of an electronic device, insufficient memory, or insufficient processing power may be detected.

When the signal for changing the video call quality is detected, the third electronic device may display at least one electronic device capable of converting the video call in operation 2343. For example, as described above with reference to FIG. 23A , a pop-up message including at least one electronic device capable of converting a video call may be displayed.

In operation 2345, the third electronic device may determine whether or not an input for selecting an electronic device to which a video call is to be switched is detected for a predetermined time period. If an input for selecting an electronic device to switch the video call is detected during the preset time, the third electronic device may transmit a signal for switching the video call to the selected electronic device in operation 2349. For example, when an input for selecting an electronic device to convert a video call is detected in the pop-up message, the third electronic device may transmit a video call signal to the selected electronic device. Accordingly, the first electronic device and the selected electronic device may perform a video call.

If an input for selecting an electronic device to switch the video call is not detected during the preset time, the third electronic device automatically selects an electronic device to switch the video call according to a preset condition in operation 2347, and in operation 2349 A signal for converting a video call may be transmitted to the electronic device. In various embodiments of the present disclosure, the preset condition may be a state of at least one electronic device capable of converting a video call and network state information. The third electronic device receives a priority determined based on the status of at least one electronic device or network status information among electronic devices capable of converting a video call from the second electronic device, and the video call starts with the electronic device having the higher priority. It can be controlled to switch automatically.

As described above, the operation of switching the video call to the selected electronic device has been described as determining whether or not an input for selecting an electronic device to which the video call is to be switched is detected for a predetermined time, but is not limited thereto, and the video call is not limited thereto. A pop-up window indicating whether to automatically select an electronic device to convert a call or to select by user input may be displayed. When it is determined through the displayed pop-up window to select an electronic device to switch the video call to by user input, the third electronic device may perform operations 2345 and 2349 of FIG. 23C . Alternatively, when it is determined to automatically select an electronic device to convert a video call through the displayed pop-up window, the third electronic device may perform operations 2347 and 2349 of FIG. 23C.

24 is a diagram for explaining an operation of switching a video call to a fourth electronic device according to various embodiments of the present disclosure.

Referring to FIG. 24 , in various embodiments of the present invention, a video call currently being serviced may be switched to a standalone device having an independent network access function, for example, a fourth electronic device 2411 .

In various embodiments of the present invention, the first electronic device 2401 (eg, the first electronic device 401 in FIG. 4 , the first electronic device 601 in FIG. 6 , the first electronic device 701 in FIG. 7 , the first electronic device 801 in FIG. 8 ) , the first electronic device 901 of FIG. 9, the first electronic device 1401 of FIG. 14A, the first electronic device 1701 of FIG. 17A or 17B) and the second electronic device 2405 (eg, the second electronic device 403 of FIG. 4, FIG. 6, the second electronic device 703 in FIG. 7, the second electronic device 803 in FIG. 8, the second electronic device 903 in FIG. 9, the second electronic device 1123 in FIG. 11, the second electronic device in FIG. 12 The device 1215 or the second electronic device 1703 of FIG. 17A or 17B may be connected through a first network 2403 (eg, the first network 411 of FIG. 4 or the first network 1523 of FIG. 15 ), and the second electronic device 2405 and the third electronic device 2409 (eg, the third electronic device 405 in FIG. 4 , the third electronic device 605 in FIG. 6 , the third electronic device 705 in FIG. 7 , the third electronic device 805 in FIG. 8 , the third electronic device 805 in FIG. 9 ) 3 The electronic device 905, the third electronic device 1413 in FIG. 14A, the third electronic device 1505 in FIG. 15, or the third electronic device 1705 in FIG. 17A or 17B) is the second network 2407 (eg, the second network 413 in FIG. 4 ). , or the second network 1531 of FIG. 15). In addition, the fourth electronic device 2411 may be independently connected to an external network through a third network 2413, for example, wifi, or may be connected to an external network through a wired network.

The fourth electronic device 2411 may be a device connected to the second electronic device 2405 or registered to the second electronic device 2405 , or a new device not connected to or registered with the second electronic device 2405 . The fourth electronic device 2411 may have different users and owners of the second electronic device 2405, and may be, for example, a common device such as a TV or monitor installed in a public place.

The second electronic device 2405 or the third electronic device 2409 connected to the second electronic device 2405 through the second network 2407 may request to switch a video call currently being serviced to the fourth electronic device 2411 . The fourth electronic device 2411 may request the second electronic device 2405 or the third electronic device 2409 connected to the second electronic device 2405 through the second network 2407 to switch the video call. The fourth electronic device 2411 and the second electronic device 2405 or the third electronic device 2409 transmit data for switching the video call, for example, IP information, MAC information, security authentication information, VPN configuration information, or electronic device identification information. Can send and receive 2417, 2419. For example, the fourth electronic device 2411 may request conversion of a video call to the second electronic device 2405 or the third electronic device 2409 through NFC tagging, RFID, BLE, or BT broadcasting.

In various embodiments of the present disclosure, the third electronic device 2409 providing a video call service or the second electronic device 2405 connected to the third electronic device 2409 through the second network 2407 includes an image pattern and a QR code for converting a video call. , or encryption data. The fourth electronic device 2411 may request a video call connection by obtaining video call information through an image sensor of the displayed image pattern, QR code, or encryption data.

For example, when the fourth electronic device 2411 detects an input of selecting a “Connect video call” menu to request a video call conversion, the fourth electronic device 2411 activates a camera to transmit a predetermined pattern of a flash or LED or LCD screen brightness signal to a second electronic device. It can be transmitted to the electronic device 2405 or the third electronic device 2409. The second electronic device 2405 or the third electronic device 2409 may temporarily display a specific image pattern for video call switching on the display unit based on the signal of the predetermined pattern received from the fourth electronic device 2411 . The fourth electronic device 2411 may read the specific image pattern through an image sensor and transmit/receive data for video call conversion based on this.

In various embodiments of the present disclosure, the second electronic device 2405 or the third electronic device 2409 makes a video call with the fourth electronic device 2411 before connecting to the fourth electronic device 2411 or before transmitting data for converting a video call. An operation for approving an operation may be additionally performed.

In various embodiments of the present disclosure, the fourth electronic device 2411 inserts the information of the fourth electronic device 2411 into a specific pattern of ultrasonic waves or sound waves to make a video call to the second electronic device 2405 or the third electronic device 2409 through a speaker and a microphone. or send/receive data for converting a video call.

For example, the fourth electronic device 2411 inserts information of the fourth electronic device 2411, for example, the type, MAC information, and connection information of the fourth electronic device 2411 into a specific pattern of ultrasonic waves or sound waves or a BLE signal, so that the second electronic device 2405 or It can be transmitted to the third electronic device 2409. The second electronic device 2405 or the third electronic device 2409 determines data for a video call to be transmitted to the fourth electronic device 2411 based on the information of the fourth electronic device 2411 received from the fourth electronic device 2411, and displays the data. can be displayed The second electronic device 2405 or the third electronic device 2409 can switch the video call to the fourth electronic device 2411 when detecting an input for selecting the determined data.

In various embodiments of the present invention, the third electronic device 2409 may search for electronic devices capable of connecting to a video call in the vicinity in a video call connection standby state, and select electronic devices capable of a video call connection from the electronic devices displayed through the search. You can choose.

In various embodiments of the present disclosure, the third electronic device 2409 may temporarily register the fourth electronic device 2411 in the account server in an operation of searching for and registering the nearby connectable electronic devices.

25 is a diagram illustrating a signal flow between a first electronic device, a second electronic device, and a fourth electronic device to describe a method of switching a video call to a fourth electronic device according to various embodiments of the present disclosure.

Referring to FIG. 25 , in various embodiments of the present invention, a fourth electronic device 2507 (eg, the fourth electronic device 2411 in FIG. 24 ) is a second electronic device 2503 (eg, the second electronic device 403 in FIG. 4 , the fourth electronic device 2411 in FIG. 6 ). second electronic device 603 of FIG. 7, second electronic device 703 of FIG. 7, second electronic device 803 of FIG. 8, second electronic device 903 of FIG. 9, second electronic device 1123 of FIG. 11, second electronic device of FIG. 12 1215, a video call switching request may be requested to the second electronic device 1703 of FIGS. 17A and 17B or the second electronic device 2405 of FIG. 24, and information of the fourth electronic device 2507 may be transmitted 2511. When the second electronic device 2503 receives a video call transfer request signal and information about the fourth electronic device 2507 from the fourth electronic device 2507, the second electronic device 2503 receives a signal for authenticating the fourth electronic device 2507 and an image of the fourth electronic device 2507. A signal 2513 accepting the call transfer request may be transmitted to the fourth electronic device 2507 . In addition, the second electronic device 2503 is the first electronic device 2501 (eg, the first electronic device 401 of FIG. 4 , the first electronic device 601 of FIG. 6 , the first electronic device 701 of FIG. 7 , the first electronic device of FIG. 8 ) Video call connection information with the device 801, the first electronic device 901 of FIG. 9, the first electronic device 1401 of FIG. 14A, the first electronic device 1701 of FIGS. 17A and 17B, or the first electronic device 2401 of FIG. 24) Transmission 2515 may be performed to the fourth electronic device 2507 . The second electronic device 2503 may transmit 2517 the information of the fourth electronic device 2507 to the first electronic device 2501 . The first electronic device 2501 may determine 2519 an encoding scheme based on the information of the fourth electronic device 2507 received from the second electronic device 2503 and encode 2521 image data based on the determination. The first electronic device 2501 may transmit encoded video data to the fourth electronic device 2507 and convert 2523 a video call to the fourth electronic device 2507 . The fourth electronic device 2507 may reproduce 2525 encoded image data received from the first electronic device 2501 . The fourth electronic device 2507 may receive 2527 voice and transmit 2529 the voice signal to the second electronic device 2503 . The second electronic device 2503 may transmit 2531 the voice signal received from the fourth electronic device 2507 to the first electronic device 2501 . Accordingly, the first electronic device 2501 and the fourth electronic device 2507 may perform a video call.

In the foregoing embodiment, it has been described assuming that an operation of switching a video call to the fourth electronic device 2507 is controlled by the second electronic device 2503, but is not limited thereto, and although not shown in the drawings, the third electronic device ( For example, the third electronic device 405 in FIG. 4, the third electronic device 605 in FIG. 6, the third electronic device 705 in FIG. 7, the third electronic device 805 in FIG. 8, the third electronic device 905 in FIG. 9, and the third electronic device 905 in FIG. 14A. The video call is switched to the fourth electronic device 2507 by the third electronic device 1413, the third electronic device 1505 in FIG. 15, the third electronic device 1705 in FIGS. 17A and 17B, or the third electronic device 2409 in FIG. 24). operation can be controlled.

In various embodiments of the present disclosure, when a video call is switched to the fourth electronic device 2507, the video call with the first electronic device 2501 may be relayed and connected by the second electronic device 2503. Alternatively, when the video call is switched to the fourth electronic device 2507, the video call with the first electronic device 2501 is not relayed by the second electronic device 2503, and an independent network (eg, the third electronic device 2501 in FIG. A video call may be performed by being directly connected through the network 2413).

In another embodiment of the present invention, the second electronic device 2503 may receive a video call request signal from an independent device, for example, the fourth electronic device 2507 while performing a video call with the first electronic device 2501 . In various embodiments of the present disclosure, the second electronic device 2503 maintains a video call with the first electronic device 2501 and creates a new video call connection with the fourth electronic device 2507 that has transmitted the video call request signal, thereby creating a three-way video call. call can be made. The new fourth electronic device 2507 may obtain video call connection information with the first electronic device 2501 from the second electronic device 2503 . Based on the acquired video call connection information of the first electronic device 2501, the fourth electronic device 2507 can additionally participate in a video call with the second electronic device 2503 and perform a three-way call with the first electronic device 2501. In other words, the first electronic device 2501, the second electronic device 2503, and the fourth electronic device 2507 may perform a three-way video call, and the second electronic device 2503 and the fourth electronic device 2507 may each perform the first electronic device 2503. A 1:1 video call may be performed with the electronic device 2501 . Various embodiments of the present invention enable a video call with the first electronic device 2501 without providing information of the second electronic device 2503 to other nearby devices, thereby increasing the user's usability of the video call service.

In various embodiments of the present disclosure, after the operation of switching the video call to the fourth electronic device 2507, the second electronic device 2503 maintains the authority to control the video call of the fourth electronic device 2507 or the video call is switched. The video call may be controlled by the fourth electronic device 2507.

26 illustrates a signal flow between a first electronic device, a second electronic device, and a fourth electronic device for explaining a method of switching a video call to a fourth electronic device using an account management server according to various embodiments of the present disclosure. It is a drawing showing

Referring to FIG. 26 , in various embodiments of the present invention, a third electronic device (eg, the third electronic device 405 in FIG. 4 , the third electronic device 605 in FIG. 6 , the third electronic device 605 in FIG. 7 ) is used by using the account management server 2600 Electronic device 705, third electronic device 805 in FIG. 8, third electronic device 905 in FIG. 9, third electronic device 1413 in FIG. 14A, third electronic device 1505 in FIG. 15, third electronic device in FIGS. 17A and 17B A video call may be switched from 1705 or the third electronic device 2409 of FIG. 24 to the fourth electronic device 2607 (eg, the fourth electronic device 2411 of FIG. 24 or the fourth electronic device 2507 of FIG. 25 ). The fourth electronic device 2607 includes the second electronic device 2603 (eg, the second electronic device 403 in FIG. 4 , the second electronic device 603 in FIG. 6 , the second electronic device 703 in FIG. 7 , the second electronic device 803 in FIG. 8 , The second electronic device 903 of FIG. 9, the second electronic device 1123 of FIG. 11, the second electronic device 1215 of FIG. 12, the second electronic device 1703 of FIGS. 17A and 17B, the second electronic device 2405 of FIG. 24, or the FIG. A video call transfer request may be requested to the second electronic device 2503 of 25 , and information of the fourth electronic device 2607 may be transmitted 2611 . Upon receiving a video call switch request signal and information on the fourth electronic device 2607 from the fourth electronic device 2607, the second electronic device 2603 can authenticate 2613 the fourth electronic device 2607. In addition, the second electronic device 2603 may transmit a signal 2615 accepting the video call switching request of the fourth electronic device 2607 to the fourth electronic device 2607 .

Although operation 2613 has been described assuming that the second electronic device 2603 authenticates the fourth electronic device 2607, the present invention is not limited thereto. The video call is switched by selecting another electronic device, for example, the fourth electronic device 2607 from the third electronic device connected through the second network 413 of FIG. 4, the second network 1531 of FIG. 15, or the second network 2407 of FIG. 24). In this case, an operation of separately authenticating the fourth electronic device 2607 may not be performed.

The second electronic device 2603 is the first electronic device 2601 (eg, the first electronic device 401 of FIG. 4 , the first electronic device 601 of FIG. 6 , the first electronic device 701 of FIG. 7 , the first electronic device 801 of FIG. 8 , The first electronic device 901 of FIG. 9, the first electronic device 1401 of FIG. 14A, the first electronic device 1701 of FIGS. 17A and 17B, the first electronic device 2401 of FIG. 24, or the first electronic device 2501 of FIG. 25) Video call connection information of may be transmitted 2617 to the fourth electronic device 2607 .

In various embodiments of the present disclosure, the second electronic device 2603 may temporarily register 2619 information on the fourth electronic device 2607 with the server 2600 .

In various embodiments of the present invention, the server 2600 may include a user account server, and the user account server may register and manage a user account and a user's electronic device. For example, the user account server may manage the user's electronic device and information (eg, status information) on the electronic device based on the user account. As described above, the user account server may temporarily register an electronic device to perform a video call with the temporarily registered electronic device, in addition to the method of managing based on the user account. For example, the user account server may have a smart phone, a tablet PC, and a watch registered thereon based on the user account, and may additionally temporarily register an electronic device other than the user account, such as a TV. As the TV is temporarily registered in the user account server, an electronic device that wants to make a video call can make a video call with the TV instead of the user account.

In various embodiments of the present disclosure, the first electronic device 2601 is an electronic device for performing a video call and may receive the fourth electronic device 2607 from the server 2600 and update 2621 . The second electronic device 2603 may transmit 2623 information of the fourth electronic device 2607 to the first electronic device 2601 .

In various embodiments of the present disclosure, the fourth electronic device 2607 may request 2625 a video call connection to the first electronic device 2601 based on video call connection information with the first electronic device 2601 . The first electronic device 2601 may determine 2627 an encoding scheme based on the information of the fourth electronic device 2607 received from the second electronic device 2603 and encode 2629 image data based on the determination. The first electronic device 2601 may transmit 2631 the encoded video data to the fourth electronic device 2607, and the video call may be transferred to the fourth electronic device 2607 2523. The fourth electronic device 2607 may reproduce 2633 encoded video data received from the first electronic device 2601 and perform a video call with the first electronic device 2601 . When receiving 2635 the voice, the fourth electronic device 2607 may transmit 2637 the voice signal to the second electronic device 2603 . The second electronic device 2603 may transmit 2639 the voice signal received from the fourth electronic device 2607 to the first electronic device 2601 . Accordingly, the first electronic device 2601 and the fourth electronic device 2607 may perform a video call.

When the video call ends, information on the fourth electronic device 2607 temporarily registered in the account management server 2600 may be deleted.

In the foregoing embodiment, it has been described assuming that an operation of switching a video call to the fourth electronic device 2607 is controlled by the second electronic device 2603, but is not limited thereto, and although not shown in the drawings, the third electronic device An operation of switching a video call to the fourth electronic device 2607 can be controlled by

In various embodiments of the present disclosure, when a video call is switched to the fourth electronic device 2607, the video call with the first electronic device 2601 may be relayed and connected by the second electronic device 2603. Alternatively, when the video call is switched to the fourth electronic device 2607, the video call with the first electronic device 2601 is not relayed by the second electronic device 2603, and an independent network (eg, the third electronic device 2601 in FIG. A video call may be performed by being directly connected through the network 2413).

27 is a diagram illustrating a user interface for explaining a function of converting a video call by searching for nearby electronic devices in a third electronic device according to various embodiments of the present disclosure.

Referring to FIG. 27 , in various embodiments of the present disclosure, a third electronic device (eg, the third electronic device 405 of FIG. 4 , the third electronic device 605 of FIG. 6 , the third electronic device 705 of FIG. 7 , the third electronic device 705 of FIG. 8 ) The third electronic device 805, the third electronic device 905 in FIG. 9, the third electronic device 1413 in FIG. 14A, the third electronic device 1505 in FIG. 15, the third electronic device 1705 in FIGS. 17A and 17B, or the third electronic device 1705 in FIG. 24 3 The electronic device 2409) may search for an electronic device capable of converting a video call. For example, in various embodiments of the present disclosure, the third electronic device may search for an electronic device capable of switching the video call through at least one of BLE, wifi, and zigbee. In response to the operation of searching for the electronic device, the third electronic device may display electronic devices connected to the same network and capable of switching a video call. For example, the third electronic device may search not only home appliances and electronic devices connected to the same network, but also electronic devices such as health care, remote meter reading, smart home, or smart car, and share information of the searched electronic devices. can do.

In various embodiments of the present disclosure, the electronic device capable of switching the video call may include an electronic device registered in a user account or not registered. If the third electronic device is an electronic device registered to the user account, it may display an icon (eg, a profile picture) indicating the electronic device along with account information. In the case of an electronic device not registered in the user account, the third electronic device may display only an icon indicating a connectable electronic device without account information. In various embodiments of the present invention, the account information may be included in the search signal or obtained from a user account server based on the search signal.

In various embodiments of the present disclosure, upon receiving an input for selecting a specific electronic device from among the discovered electronic devices capable of converting video calls, the third electronic device may set the first electronic device (eg, the first electronic device 401 of FIG. 4 ). , the first electronic device 2401 of FIG. 24, the first electronic device 2501 of FIG. 25, or the first electronic device 2601 of FIG. 26) and the selected specific electronic device may be connected to perform a video call.

For example, in response to a search operation for an electronic device capable of switching a video call, the third electronic device displays a pop-up window 2701 including Jesse 2703, Kathryn 2705, and a TV icon 2707 capable of switching a video call, as shown in FIG. 27 . can do. In various embodiments of the present disclosure, when an electronic device registered in a user account is searched for, for example, 2703 and 2705, account information of Jesse 2703 and Kathryn 2705 may be displayed along with an icon representing the electronic device. Alternatively, when an electronic device not registered in a user account is searched for, for example, only an icon 2707 representing a TV, such as 2707, may be displayed. When an input for selecting one of the Jesse 2703, Kathryn 2705, and TV icon 2707 is detected, a video call may be switched to the electronic device corresponding to the selected icon.

In another embodiment of the present invention, switching a video call may be performed through infrared communication. For example, according to various embodiments of the present disclosure, an electronic device to be switched to a video call may be selected using a remote control using a TV remote control light receiving unit and an infrared emitting unit of the electronic device. For example, the above-described operation of exchanging information between a signal for switching a video call and an electronic device for performing a video call may be performed through the infrared communication. Accordingly, in various embodiments of the present invention, a user can intuitively perform an operation of switching a video call to an electronic device for performing a video call.

28 is a diagram illustrating connections between first electronic devices and fourth electronic devices according to various embodiments of the present disclosure.

In various embodiments of the present invention, a video call may be performed through a chain structure of two or more electronic devices or extended connection devices.

Referring to FIG. 28 , in various embodiments of the present disclosure, a first electronic device 2801 (eg, the first electronic device 401 of FIG. 4 , the first electronic device 601 of FIG. 6 , the first electronic device 701 of FIG. 7 , and the first electronic device 701 of FIG. 8 ) The first electronic device 801 of FIG. 9, the first electronic device 901 of FIG. 9, the first electronic device 1401 of FIG. 14A, the first electronic device 1701 of FIGS. 17A and 17B, the first electronic device 2401 of FIG. 24, the first electronic device 2401 of FIG. 25 1 electronic device 2501 or the first electronic device 2601 in FIG. 26 is connected via a first network 2821 (eg, the first network 411 in FIG. 4 , the first network 1523 in FIG. 15 , or the first network 2403 in FIG. 24 ). The second electronic device 2803 (eg, the second electronic device 403 in FIG. 4 , the second electronic device 603 in FIG. 6 , the second electronic device 703 in FIG. 7 , the second electronic device 803 in FIG. 8 , the second electronic device in FIG. 9 ) Device 903, second electronic device 1123 in FIG. 11, second electronic device 1215 in FIG. 12, second electronic device 1703 in FIGS. 17A and 17B, second electronic device 2405 in FIG. 24, second electronic device 2503 in FIG. , or the second electronic device 2603 of FIG. 26). The second electronic device 2803 is connected to a third electronic device 2805 (eg, the second network 2823 in FIG. 4 , the second network 1531 in FIG. 15 , or the second network 2407 in FIG. 24 ). 4, the third electronic device 605 in FIG. 6, the third electronic device 705 in FIG. 7, the third electronic device 805 in FIG. 8, the third electronic device 905 in FIG. 9, the third electronic device in FIG. 14A Device 1413, the third electronic device 1505 of FIG. 15, the third electronic device 1705 of FIGS. 17A and 17B, or the third electronic device 2409 of FIG. 24) may be connected. The third electronic device 2805 may be connected to a home network server 2807 through a third network 2825 . The fourth electronic device 2809 may be connected to the home network server 2807 through a fourth network 2827 .

Also, in various embodiments of the present invention, a plurality of electronic devices may be connected to the home network server 2807. It is possible to access an Internet of Things (IoT) network through the home network server 2807 . For example, n electronic devices 2811 as well as a fourth electronic device 2809 may be connected to the home network server 2807 through n networks 2829 . Through this, the home network server 2807 can register electronic devices existing in a wide area and monitor status information of the electronic devices. Accordingly, the user can check the state information of the electronic device registered in the home network server 2807, and can quickly check whether a video call is possible based on the state information.

In various embodiments of the present disclosure, a video call between the third electronic device 2805 and the first electronic device 2801 may be connected to the fourth electronic device 2809 through the home network server 2807 .

For example, when three or more electronic devices are connected in a chain form, an electronic device connected in the middle of the chain, for example, the home network server 2807, may relay information about the network and connected electronic devices.

In various embodiments of the present invention, the home network server 2807 may operate as an encoder. For example, the home network server 2807 may transmit image data to the fourth electronic device 2809 by controlling the encoders of the first electronic device 2801, the second electronic device 2803, the third electronic device 2805, and the home network server 2807. A first network 2821 connecting the first electronic device 2801 and the second electronic device 2803, a second network 2823 connecting the second electronic device 2803 and the third electronic device 2805, and the third electronic device 2805 and the home network server If the state of the third network 2825 connecting the 2807 is good, the first electronic device 2801 or the second electronic device 2803 may transmit video data for a video call to the home network server 2807 without encoding the video data.

In various embodiments of the present invention, the home network server 2807 may encode the received video data for a video call by encoding, for example, a transcoding method. For example, if the home network server 2807 uses a fixed power supply, encoding image data using a battery of the first electronic device 2801 or the second electronic device 2803 may be more efficient.

In various embodiments of the present invention, an operation of encoding image data using a scalable encoding method can also maximize a scalable encoding effect while transferring some of the layers received by each electronic device to the next connected electronic device. For example, the first electronic device 2801 may transmit the base layer and the enhanced layer #1 to the second electronic device 2803 according to circumstances. The second electronic device 2803 may transmit the received base layer and enhanced layer #1 to the third electronic device 2805. In addition, when the third electronic device 2805 detects switching of a video call to the fourth electronic device 2809, it can transmit the base layer and enhanced layer #1 received from the second electronic device 2803 to the home network server 2807. The home network server 2807 may transmit only the base layer to the fourth electronic device 2809, instead of transmitting both the base layer and the enhanced layer #1 received from the third electronic device 2805.

In various embodiments of the present disclosure, a video call switching operation may be performed based on the third electronic device 2805 or a video call switching operation based on the home network server 2807 .

29 is a diagram illustrating a multiple connection method according to various embodiments of the present invention.

In various embodiments of the present invention, the second electronic device (eg, the second electronic device 403 in FIG. 4 , the second electronic device 603 in FIG. 6 , the second electronic device 703 in FIG. 7 , the second electronic device 803 in FIG. 8 , The second electronic device 903 of FIG. 9, the second electronic device 1123 of FIG. 11, the second electronic device 1215 of FIG. 12, the second electronic device 1703 of FIGS. 17A and 17B, the second electronic device 2405 of FIG. 24, and the second electronic device 25 The second electronic device 2503 of FIG. 26, the second electronic device 2603 of FIG. 28, or the second electronic device 2803 of FIG. 28) or a third electronic device connected to the second electronic device (eg, the third electronic device 405 of FIG. 4, The third electronic device 605 of FIG. 6, the third electronic device 705 of FIG. 7, the third electronic device 805 of FIG. 8, the third electronic device 905 of FIG. 9, the third electronic device 1413 of FIG. 14A, and the third electronic device of FIG. 15 The electronic device 1505, the third electronic device 1705 of FIGS. 17A and 17B, the third electronic device 2409 of FIG. 24, or the third electronic device 2805 of FIG. 28) may independently perform multiple video calls. In various embodiments of the present disclosure, when a multi-video call is performed through the second electronic device or a third electronic device connected to the second electronic device, the first video call with the second electronic device or the third electronic device is performed. It is possible to perform a normal video call (eg, video and audio signal reception / transmission), and for a second video call with a new electronic device, STT (speech to text) or TTS (text to speech) A video call can be partially performed using the function.

Referring to FIG. 29 , the third electronic device may perform a first video call in operation 2901. For example, a first video call may be in progress through a third electronic device, for example, a tablet PC.

The third electronic device may receive a second video call request from the new electronic device in operation 2903 . For example, while performing a first video call through the tablet PC, a second video call request may be received from a new electronic device.

The third electronic device may accept the second video call in operation 2905 . For example, the third electronic device may perform the second video call with the new electronic device by selecting an electronic device other than the tablet PC, such as a TV or a car kit, as an electronic device to perform the second video call.

In operation 2907, the third electronic device may display first video data for a first video call and second video data for a second video call. For example, a first video call may be performed in a normal manner (eg, video and audio signal reception/transmission), and a second video call may process voice information through an STT/TTS function while only a screen is provided. For example, the voice information may be displayed as a caption, and when the third electronic device detects an input of text, the third electronic device may transmit the input text to the other electronic device as a voice. The text may be input through a separate device, for example, a Bluetooth keyboard.

In various embodiments of the present disclosure, an activated video call and a video call in a standby state (eg, inactive) may be distinguished using a control device such as a third electronic device, such as a watch. For example, the third electronic device may receive/transmit audio in an activated video call. When receiving a voice signal from the other electronic device in a video call in a standby state, the third electronic device may output that a voice signal has been received from the other electronic device using a notification or vibration, flicker, or beep sound. there is.

In various embodiments of the present disclosure, in operation 2909, the third electronic device may perform the first video call and the second video call. For example, the screen of the third electronic device may be divided into, for example, two areas to display first video data for a first video call in a first area and display data for a second video call in a second area. 2 Video data can be displayed. The two divided areas may be adjusted by a user. As described above, the first video call may be a normal video call, and the second video call may be performed simultaneously using the STT/TTS function. Also, the first video call and the second video call may be partially performed simultaneously by distinguishing between active/standby video calls. In various embodiments of the present invention, a notification based on voice information may be displayed in a separate window and output through a third electronic device, for example, a watch.

30 is a diagram for explaining an image data storage method according to various embodiments of the present disclosure.

In various embodiments of the present invention, a second electronic device that receives a video call (eg, the second electronic device 403 in FIG. 4 , the second electronic device 603 in FIG. 6 , the second electronic device 703 in FIG. 7 , the second electronic device 703 in FIG. 8 ) 2 electronic device 803, the second electronic device 903 in FIG. 9, the second electronic device 1123 in FIG. 11, the second electronic device 1215 in FIG. 12, the second electronic device 1703 in FIGS. 17A and 17B, the second electronic device in FIG. 24 Device 2405, the second electronic device 2503 in FIG. 25, the second electronic device 2603 in FIG. 26, or the second electronic device 2803 in FIG. 28) or a third electronic device connected to the second electronic device (e.g., the second electronic device in FIG. 3 electronic device 405, the third electronic device 605 in FIG. 6, the third electronic device 705 in FIG. 7, the third electronic device 805 in FIG. 8, the third electronic device 905 in FIG. 9, the third electronic device 1413 in FIG. 14A, The third electronic device 1505 of FIG. 15, the third electronic device 1705 of FIGS. 17A and 17B, the third electronic device 2409 of FIG. 24, or the third electronic device 2805 of FIG. can be saved For example, a screen or audio may be stored in the video data, and the screen and audio may be stored together.

Referring to FIG. 30 , while performing a video call in operation 3001, the third electronic device may detect an input for storing video data in operation 3003. When an input for storing the image data is detected, the input for storing the image data may be transmitted to the second electronic device. The second electronic device may request transmission of video data for storing an image from the first electronic device that transmits a video call based on the input for storing the received video data. For example, when the second electronic device receives an input for capturing or storing a specific screen while playing video data for a video call, the first electronic device that generates the video data for the video call is received from the third electronic device. For example, transmission of image data for a specific screen may be requested. Upon receiving a request to transmit image data for a specific screen from the second electronic device, the first electronic device can extract and transmit image data corresponding to the specific screen to the second electronic device. The first electronic device may transmit a high-quality image, not a reprocessed image, to the second electronic device in consideration of network state information. The high-quality image may be transmitted through a control signal channel or by forming a new channel.

In various embodiments of the present disclosure, the third electronic device may store image data in operation 3005. For example, in various embodiments of the present disclosure, the third electronic device may store image data for the specific screen based on external context information of the second electronic device or the third electronic device. For example, when the external situation information of the first electronic device is bright and noisy and the external situation information of the second electronic device is dark and quiet, the third electronic device transmits image data for the specific screen to the second electronic device. It can be set and stored as low illumination intensity and low volume corresponding to external situation information. In other words, the image or voice may be captured or stored in consideration of surrounding information of the second electronic device or the third electronic device.

In various embodiments of the present invention, when the third electronic device transmits an input for storing the detected image of the screen being played back to the second electronic device, the third electronic device receives and stores image data before image processing from the second electronic device. can Through this, the user can acquire higher quality image data. In this case, the image may be transferred to and stored in the third electronic device or stored in the second electronic device. For example, when an external TV device is temporarily selected as an electronic device for performing a video call service, the third electronic device is a second electronic device or a third electronic device other than the TV providing the video call service. It can be controlled to save the image. In other words, the electronic device in which image data is to be stored may be changed according to properties of the electronic device, such as whether an account is registered or not, whether or not it is a shared device, and the like.

In various embodiments of the present invention, the image may be stored in a cloud storage. The cloud storage may include a cloud storage of an account of the owner of the electronic device or a cloud server of an account receiving a video call service. The cloud storage may be selected by the user or may be determined according to properties of the electronic device, eg, whether an account is registered or not, whether or not it is a public device, and the like.

In various embodiments of the present invention, when video data, for example, video and audio is stored, the third electronic device inserts, for example, video call time and place, or information on the electronic device as a meta tag, It can be displayed in the form of a watermark on the image.

In various embodiments of the present invention, when storing video data in a low capacity or storing only audio, the third electronic device may extract key frames and store image slides. For example, the third electronic device may extract key frames based on 10 minutes of image data and store them in an image slide method. Accordingly, various embodiments of the present invention may store image data having a reduced storage capacity. Also, the stored video may be stored in a form synchronized with audio.

In the video data stored in various embodiments of the present invention, a specific image can be inserted or removed in the middle through an editing device, and audio and video of the existing video data can be synchronized regardless of the inserted/removed image.

In various embodiments of the present invention, a video encoding method of an electronic device includes receiving a signal related to video call reception from a first electronic device through a communication unit, and receiving a user input accepting the video call request through a user interface. An operation of transmitting a signal accepting the video call request to the first electronic device through the communication unit, and transmitting a video stream encoded based on state information of the electronic device and state information of the communication unit to the first electronic device. receiving from the communication unit through the communication unit, displaying the received video stream and playing the received audio, and when a user's voice is received through a microphone, the received voice is transmitted through the communication unit to the first electronic device. It may include an operation to transmit to.

According to various embodiments, the operation of transmitting the received voice to the first electronic device may include encoding a video stream captured by a camera based on state information of the electronic device and state information of the communication unit; and and transmitting the encoded video stream to the first electronic device through the communication unit.

According to various embodiments, the method may further include displaying the received video stream and displaying an indicator for adjusting a quality of the video stream, wherein the displaying of the indicator may include displaying the video stream in the displayed indicator. When an input for adjusting the quality is detected, determining an encoding scheme corresponding to the quality of the adjusted video stream and transmitting the determined encoding scheme to the first electronic device, and video encoded based on the encoding scheme from the first electronic device. An operation of receiving and displaying a stream may be further included.

According to various embodiments, when an input for setting an ROI is detected in the received video stream, an operation of transmitting the set ROI to the first electronic device, and focusing based on the ROI from the first electronic device; The method may further include receiving and displaying a video stream encoded by at least one of cropping, quality change, and exposure adjustment.

An image encoding method according to various embodiments includes an operation of detecting a motion of the electronic device, an operation of collecting and analyzing first sensor information about the motion of the electronic device, and an operation of detecting a motion of the electronic device based on the analyzed sensor information. When the movement is included in the preset range, the operation of playing the received audio and storing the received video stream, and when the movement of the electronic device is detected, collecting second sensor information about the movement of the electronic device The method may further include an operation of analyzing, and an operation of reproducing the stored video stream when the movement of the electronic device does not fall within a preset range based on the analyzed sensor information.

A video encoding method according to various embodiments includes an operation of detecting a signal for switching the video call, an operation of searching for and displaying at least one electronic device capable of switching the video call, and selecting a selected electronic device from among the displayed at least one electronic device. When an input for selecting an electronic device to switch the video call for a set time is detected, transmitting a video call switch signal to the selected electronic device, and performing the video call for a preset time among the at least one displayed electronic device. When an input for selecting an electronic device to be switched is not sensed, a priority is given by analyzing the state information of the at least one displayed electronic device and the state information of the communication unit, and the electronic device having the highest priority is imaged. The method may further include automatically selecting an electronic device to transfer the call and transmitting a video call switching signal to the automatically selected electronic device.

According to various embodiments, the receiving of the video call reception signal through the communication unit may include analyzing state information of the electronic device and state information of the communication unit, and analyzing state information of the electronic device and the communication unit. An operation of displaying the user interface including a function related to the video call, which can be performed based on state information, may be included.

A video encoding method according to various embodiments includes receiving a video call request from a third electronic device, detecting a user input accepting the received video call request from the third electronic device, and the first electronic device. The method may further include displaying a video stream received from the device on a first area of the touch screen and displaying a video stream for a video call with the third electronic device on a second area of the touch screen.

An image encoding method according to various embodiments includes an operation of detecting a user input, an operation of determining a pressure level for the detected user input or a direction in which the user input is sensed, and an operation of detecting the pressure level or the user input. An operation of performing a function corresponding to the direction may be further included.

An image encoding method according to various embodiments includes an operation of detecting an input for storing the received video stream, collecting external context information of the electronic device, and detecting the collected external reimbursement information and the input at a time point. The method may further include transmitting a request for storing a video stream to the first electronic device, and receiving and storing a video stream encoded based on the external context information from the first electronic device.

According to various embodiments, the external situation information may include at least one of weather information, brightness information, and sound information about a time point when an input for saving the video stream is sensed.

According to various embodiments, the receiving of the video call reception signal through the communication unit includes searching for at least one electronic device capable of conducting the video call, and displaying the at least one electronic device that has been searched for. can include

According to various embodiments, the operation of receiving the user input may be an operation of selecting an electronic device for performing the video call from the displayed at least one electronic device.

According to various embodiments, the displaying of the at least one electronic device may include displaying the at least one electronic device as an icon, analyzing state information of the communication unit and state information of the electronic device, and Based on the analyzed state information of the communication unit and state information of the electronic device, attribute information about an icon including at least one of size, brightness, color, or shape of an icon representing the at least one electronic device or the icon It may include an operation of determining the location where the object is placed.

In various embodiments of the present invention, a video encoding method may include: receiving a signal related to video call reception from a first electronic device through a communication unit; receiving a user input accepting the video call request through a user interface; Transmitting a signal accepting the video call request to the first electronic device through the communication unit, and transmitting a plurality of images and audio signals generated based on state information of the electronic device and status information of the communication unit to the communication unit. operation of receiving the received image, displaying the received images and reproducing the audio signal, and transmitting the received voice to the first electronic device through the communication unit when the user's voice is received through the microphone. action may be included.

In various embodiments of the present invention, a video encoding method may include an operation of receiving a video call request signal from a first electronic device or server through a first communication unit, an operation of displaying the received video call request signal, and an operation of displaying the received video call request signal. An operation of generating a notification based on a request signal, an operation of transmitting the generated notification to a second electronic device through a second communication unit, and a signal accepting the video call request from the second electronic device through the second communication unit. stopping the operation of displaying the video call request signal in response to the received acceptance signal, and transmitting the video stream received from the first electronic device in response to the received acceptance signal to the second electronic device. An operation of encoding based on state information of and the state information of the second communication unit, an operation of transmitting the encoded video stream to the second electronic device through the second communication unit, and an operation of transmitting the encoded video stream from the second electronic device to the second communication unit. It may include an operation of transmitting the audio signal received through the first electronic device or the server through the first communication unit.

According to various embodiments, the encoding operation may include an operation of checking at least one of maximum resolution, maximum quality, or maximum fps for a video stream supported by the second electronic device, based on a state of the second communication unit It may include adjusting at least one of the maximum resolution, maximum quality, or maximum fps, and encoding the video stream based on at least one of the adjusted resolution, quality, or fps.

According to various embodiments, the encoding may include receiving a video stream from the second electronic device, determining an encoding scheme based on the received video stream, and performing the encoding scheme based on the determined encoding scheme. 1 An operation of encoding a video stream received from an electronic device may be included.

According to various embodiments, the encoding operation may include receiving an ROI set from the second electronic device and encoding the ROI through at least one of focus, cropping, image quality change, and exposure adjustment. can

According to various embodiments, the encoding may include registering the second electronic device in a server, and based on state information about the second electronic device registered in the server and state information of the second communication unit. An operation of encoding a video stream received from the first electronic device may be included.

An image encoding method according to various embodiments includes an operation of receiving a video call request signal from a third electronic device, an operation of receiving information of the third electronic device, and an operation of transmitting video call connection information to the third electronic device. , and transmitting the information of the third electronic device to the first electronic device through the first communication unit.

A video encoding method according to various embodiments includes receiving a video call request signal from a first electronic device or server through a first communication unit, generating a notification based on the received video call request signal, and generating a notification based on the received video call request signal. Sending a notification to a second electronic device through a second communication unit, receiving a signal accepting the video call request from the second electronic device through the second communication unit, and responding to the received acceptance signal to the second electronic device. Stopping an operation of displaying a video call request signal, generating a plurality of images based on a video stream received from the first electronic device in response to the received acceptance signal, and extracting an audio signal from the video stream , transmitting the generated plurality of images and the extracted audio signal to the second electronic device through the second communication unit, and transmitting the audio signal received from the second electronic device through the second communication unit to the second electronic device. 1 It may include an operation of transmitting to the electronic device or the server through the first communication unit.

According to various embodiments, the operation of transmitting the generated plurality of images and the extracted audio signal to the second electronic device through the second communication unit includes state information of the second electronic device and the second communication unit. Obtaining state information, and transmitting at least one image from among the plurality of images to the second electronic device based on the obtained state information of the second electronic device and the state information of the second communication unit. can include

According to various embodiments, the plurality of images include at least one still image, and the operation of generating the plurality of images includes an operation of extracting an I-frame from a video stream received from the first electronic device; Comparing the extracted I-frame with the extracted I-frame to determine the similarity between the previously extracted I-frame and the extracted I-frame, if the similarity exceeds a preset reference value, the extraction and generating the at least one still image based on the I-frame.

According to various embodiments, the operation of transmitting the generated plurality of images to the second electronic device through the second communication unit may include transmitting the plurality of images to the second electronic device through a serial port profile (SPP) of the second communication unit. 2 An operation of transmitting the extracted audio signal to the second electronic device, and the operation of transmitting the extracted audio signal to the second electronic device through a second communication unit includes transmitting the audio signal to the second electronic device through a hands free profile (HFP) of the second communication unit. 2 It may be an operation to transmit to an electronic device.

A video encoding method according to various embodiments includes transmitting a video call request signal to a first electronic device through a first communication unit, and performing the video call with a signal accepting the video call request from the first electronic device. Receiving state information of a second electronic device and state information of a second communication unit for a purpose through the first communication unit, and encoding a video stream based on the received state information of the second electronic device and state information of the second communication unit. and transmitting to the first electronic device, and receiving a voice signal from the first electronic device.

31 is a diagram illustrating software according to various embodiments of the present invention.

Referring to FIG. 31 , in various embodiments of the present invention, a method for operating a video call and a method for providing a user interface may be provided using an application programming interface (API). In this case, not only applications made by electronic device manufacturers but also 3rd party applications can perform video calls and provide corresponding user interfaces through these APIs. For example, the API 3105 may receive a parameter return value 3103 from the API-Calling components 3101 when the API-Calling components 3101 call 3109 parameters for performing a video call. The API may transmit the parameter return value 3103 to the API-Implementing components. The API-Implementing components may implement a user interface for performing a video call using the parameter return value 3103.

In the above, preferred embodiments of the video encoding method according to an embodiment of the present invention and a device supporting the same have been described through this specification and drawings, and although specific terms have been used, this merely explains the technical content of the present invention. and is used in a general sense to aid understanding of the present invention, but the present invention is not limited to the above-described embodiments. That is, it is obvious to those skilled in the art that various embodiments based on the technical idea of the present invention are possible.

510: communication department
520: memory
530: touch screen
540: sensor unit
550: camera
560: audio processing unit
570: processor

Claims (61)

In a wearable electronic device,
a wireless communication circuit that communicates with the first electronic device;
display;
speaker;
microphone;
one or more processors electrically connected to the wireless communication circuitry, the display, the speaker, and the microphone; and
a memory electrically connected to the one or more processors;
The memory, when executed, the one or more processors,
Conducting a video call with a second electronic device through the first electronic device;
During the video call, a plurality of still images are received from the first electronic device and displayed on the display, and the plurality of still images are video data streams received by the first electronic device from the second electronic device. Contains a still image extracted from,
While performing the video call, a first audio data stream corresponding to the video call is received from the first electronic device and outputted through the speaker, and the first audio data stream is transmitted by the first electronic device to the first audio data stream. 2 includes an encoded audio data stream generated based on audio data received from the electronic device, and
During the video call, a second audio data stream corresponding to the video call is transmitted to the second electronic device through the first electronic device, and the second audio data stream is audio received through the microphone. Contains encoded audio stream data generated based on the data;
the first voice data stream and the second voice data stream are continuous streaming data; and
The plurality of still images are non-continuous still images. According to claim 1,
A display of the wearable electronic device is smaller than a display of the first electronic device;
Battery performance of the wearable electronic device is lower than battery performance of the first electronic device, and
The wearable electronic device may have processing performance lower than that of the first electronic device. According to claim 1,
Each of the plurality of still images corresponds to an intra-frame of the video data stream received by the first electronic device from the second electronic device. According to claim 2,
The wearable electronic device wherein the wireless communication circuit includes a Bluetooth communication circuit and/or a Wi-Fi communication circuit. According to claim 4,
The plurality of still images are received through a first Bluetooth profile, and
The first audio data stream and the second audio data stream are received through a second Bluetooth profile. According to claim 5,
The first Bluetooth profile includes a serial port profile (SPP), and
The second Bluetooth profile includes a hands free profile (HFP). In electronic devices,
a first wireless communication circuit that communicates with the wearable electronic device;
a second wireless communication circuit that communicates with a second electronic device;
display;
speaker;
microphone;
one or more processors electrically connected to the first wireless communication circuit, the second wireless communication circuit, the display, the speaker, and the microphone; and
a memory electrically connected to the one or more processors;
The memory, when executed, the one or more processors,
detect the occurrence of an event related to initiation or acceptance of a video call;
When the occurrence of the event is detected by the wearable electronic device connected by communication:
When a video data stream corresponding to the video call is received from the second electronic device, the video data stream is configured based on attribute information of the wearable electronic device and attribute information of the first wireless communication circuit and the second wireless communication circuit. encoding to generate a first audio data stream and a plurality of still images, the plurality of still images being non-contiguous still images;
transmitting the first audio data stream and the plurality of still images to the wearable electronic device so that the wearable electronic device performs the video call;
receiving a second audio data stream corresponding to the video call from the wearable electronic device, the second audio data stream being encoded audio stream data generated based on audio data received by a microphone of the wearable electronic device; contains, and
transmitting the second audio data stream to the second electronic device so that the second electronic device performs the video call, the first audio data stream and the second audio data stream being continuous streaming data; and
When the occurrence of the event is detected by the electronic device, the video data stream received from the second electronic device is converted into property information of the electronic device and the second wireless communication circuit so that the electronic device performs the video call. An electronic device that encodes based on attribute information of and displays it on the display. According to claim 7,
The attribute information of the wearable electronic device includes at least one of a type of the wearable electronic device, a display size, a maximum resolution, a battery state, and a memory capacity;
The attribute information of the first and second wireless communication circuits includes at least one of a network state and signal quality, and
The attribute information of the electronic device includes at least one of a type of the electronic device, a size of a display, a maximum resolution, a battery state, and a memory capacity. In the image encoding method of a wearable electronic device,
performing a video call with a second electronic device through the first electronic device;
receiving, from the first electronic device, a plurality of still images including still images extracted from a video data stream received by the first electronic device from the second electronic device while performing the video call;
While performing the video call, a first audio data stream including an encoded audio data stream generated by the first electronic device based on audio data received from the second electronic device is received from the first electronic device. receiving action; and
Transmitting a second audio data stream including encoded audio stream data generated based on audio data received through a microphone to the second electronic device through the first electronic device while performing the video call. including,
the first voice data stream and the second voice data stream are continuous streaming data; and
The method of claim 1 , wherein the plurality of still images are non-contiguous still images. In the video encoding method of the electronic device,
detecting occurrence of an event related to initiation or acceptance of a video call;
When the occurrence of the event is detected by a wearable electronic device connected to the electronic device:
When a video data stream corresponding to the video call is received from the second electronic device, encoding the video data stream based on attribute information of the wearable electronic device and attribute information of the first wireless communication circuit and the second wireless communication circuit generating a first audio data stream and a plurality of still images;
transmitting the first audio data stream and the plurality of still images to the wearable electronic device so that the wearable electronic device performs the video call;
receiving, from the wearable electronic device, a second audio data stream including encoded audio stream data generated based on audio data received by a microphone of the wearable electronic device; and
transmitting the second audio data stream to the second electronic device so that the second electronic device performs the video call; and
When the occurrence of the event is detected by the electronic device, the electronic device performs the video call based on attribute information of the electronic device and attribute information of the second wireless communication circuit from the second electronic device. Receiving an encoded video data stream and displaying it on a display;
The plurality of still images are non-contiguous still images, and
wherein the first audio data stream and the second audio data stream comprise continuous streaming data. delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete delete

Images