KR20220061786A - Electronic device including variable display and method of operation therof - Google Patents

Abstract

In accordance with one embodiment of the present disclosure, an electronic device comprises: a housing including a first side, a second side, a third side, and a fourth side; a variable display combined with the housing, wherein a position of a side area surrounding the first side can be changed; a sensor sensing a position change of the side area; and a processor electrically connected with the variable display and the sensor. The processor can be configured to: acquire data about a state change of the variable display through the sensor, calculate variations of the position of the side area based on the data about the state change, and change display coordinates of a soft key based on the calculated variations. Therefore, the present invention is capable of adaptively displaying a soft key considering at least one state.

Description

ELECTRONIC DEVICE INCLUDING VARIABLE DISPLAY AND METHOD OF OPERATION THEROF

Embodiments disclosed in this document relate to an electronic device including a variable display having a variable shape such as a slide type, a rollable type, and a foldable type, and an operating method thereof.

In recent years, various types of electronic devices are being developed in order to secure a more extended display area without impeding portability. For example, the variable display electronic device is a slide type electronic device in which at least one side of the housing is slid and the state of the display is changed, and the display wound inside or outside the housing is unfolded while the electronic device is outside. It may correspond to any one of a rollable type electronic device in which the display area exposed by .

The variable display electronic device has the advantage of being able to change the shape of the display as needed, but since at least one side of the display is bent to cover the outer surface of the housing of the electronic device, a hardware button mounted on the side housing is generally mounted. You may not have enough space to do it. Accordingly, the soft key can be displayed on the display screen, but as the shape of the display is changed in the variable display, it may be necessary to change the position where the soft key is displayed.

An electronic device including a variable display, the electronic device for adaptively displaying a soft key on the variable display by detecting a change in a display shape and updating display coordinates of the soft key to correspond to the changed display shape, and an operation thereof We want to provide a way.

Also, in an electronic device including a variable display, the orientation of the electronic device (horizontal direction, vertical direction). An object of the present invention is to provide an electronic device that adaptively displays a soft key in consideration of at least one of a holding state, a low power mode, and an AOD mode of the electronic device, and an operating method thereof.

An electronic device according to an embodiment of the present disclosure includes a housing including a first side, a second side, a third side, and a fourth side, and is coupled to the housing, and the position of the side area surrounding the first side is changed A possible variable display, a sensor for detecting a change in the position of the side area, and a processor electrically connected to the variable display and the sensor, wherein the processor obtains data on a change in state of the variable display through the sensor, , calculate a change amount of the position of the side area based on the data on the state change, and change the display coordinates of the soft key based on the calculated change amount.

In a method of displaying a soft key on a variable display of an electronic device according to an embodiment of the present disclosure, data on a state change of the variable display is obtained through a sensor of the electronic device, and the state change a change amount of the position of the side area of the variable display is calculated based on the data for Based on the display coordinates of the soft key can be changed.

The soft key adaptively to the variable display by detecting that the shape of the face, housing and/or display is changed and updating the display coordinates of the soft key to correspond to the changed shape of the housing and/or display according to various embodiments of the present disclosure It is possible to provide an electronic device for displaying and an operating method thereof.

Orientation (landscape direction, portrait direction) of an electronic device according to various embodiments of the present disclosure. Provided are an electronic device that adaptively displays a soft key in consideration of at least one of a holding state, a low power mode, and an AOD mode of the electronic device, and an operating method thereof.

In addition, various effects directly or indirectly identified through this document may be provided.

1 is a block diagram of an electronic device 101 in a network environment 100 according to various embodiments of the present disclosure.
2 is a diagram 201 and 202 illustrating an electronic device according to an exemplary embodiment.
3 is a flowchart 300 illustrating an operation of an electronic device according to an exemplary embodiment.
4 is a diagram 400 illustrating components of an electronic device according to an exemplary embodiment.
5 is a flowchart 500 illustrating an operation of an electronic device according to an exemplary embodiment.
6 is a diagram 610 and 620 of an electronic device according to an exemplary embodiment.
7 is a flowchart 700 illustrating an operation of an electronic device according to an exemplary embodiment.
8 is a diagram 810 and 820 illustrating an electronic device according to an exemplary embodiment.
9 is a flowchart 900 illustrating an operation of an electronic device according to an exemplary embodiment.
10 is a diagram 1010 and 1020 illustrating an electronic device according to an exemplary embodiment.
11 is a flowchart 1100 illustrating an operation of an electronic device according to an exemplary embodiment.
12 is a diagram 1210 and 1220 of an electronic device according to an exemplary embodiment.
13 is a flowchart 1300 illustrating an operation of an electronic device according to an exemplary embodiment.
14 is a diagram 1410 and 1420 of an electronic device according to an exemplary embodiment.
15 is a diagram 1510 and 1520 illustrating an electronic device according to an exemplary embodiment.
16 is a flowchart 1600 illustrating an operation of an electronic device according to an exemplary embodiment.
17 is a diagram 1710 and 1720 of an electronic device according to an exemplary embodiment.
18 is a diagram 1810 and 1580 illustrating an electronic device according to an exemplary embodiment.
In connection with the description of the drawings, the same or similar reference numerals may be used for the same or similar components.

Hereinafter, various embodiments of the present invention will be described with reference to the accompanying drawings. However, this is not intended to limit the present invention to specific embodiments, and it should be understood that various modifications, equivalents, and/or alternatives of the embodiments of the present invention are included.

1 is a block diagram of an electronic device 101 in a network environment 100 according to various embodiments. Referring to FIG. 1 , in a network environment 100 , an electronic device 101 communicates with an electronic device 102 through a first network 198 (eg, a short-range wireless communication network) or a second network 199 . It may communicate with the electronic device 104 or the server 108 through (eg, a long-distance wireless communication network). According to an embodiment, the electronic device 101 may communicate with the electronic device 104 through the server 108 . According to an embodiment, the electronic device 101 includes a processor 120 , a memory 130 , an input module 150 , a sound output module 155 , a display module 160 , an audio module 170 , and a sensor module ( 176), interface 177, connection terminal 178, haptic module 179, camera module 180, power management module 188, battery 189, communication module 190, subscriber identification module 196 , or an antenna module 197 may be included. In some embodiments, at least one of these components (eg, the connection terminal 178 ) may be omitted or one or more other components may be added to the electronic device 101 . In some embodiments, some of these components (eg, sensor module 176 , camera module 180 , or antenna module 197 ) are integrated into one component (eg, display module 160 ). can be

The processor 120, for example, executes software (eg, a program 140) to execute at least one other component (eg, a hardware or software component) of the electronic device 101 connected to the processor 120 . It can control and perform various data processing or operations. According to one embodiment, as at least part of data processing or operation, the processor 120 converts commands or data received from other components (eg, the sensor module 176 or the communication module 190 ) to the volatile memory 132 . may be stored in the volatile memory 132 , and may process commands or data stored in the volatile memory 132 , and store the result data in the non-volatile memory 134 . According to an embodiment, the processor 120 is the main processor 121 (eg, a central processing unit or an application processor) or a secondary processor 123 (eg, a graphic processing unit, a neural network processing unit) a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor). For example, when the electronic device 101 includes the main processor 121 and the sub-processor 123 , the sub-processor 123 may use less power than the main processor 121 or may be set to be specialized for a specified function. can The auxiliary processor 123 may be implemented separately from or as a part of the main processor 121 .

The auxiliary processor 123 is, for example, on behalf of the main processor 121 while the main processor 121 is in an inactive (eg, sleep) state, or the main processor 121 is active (eg, executing an application). ), together with the main processor 121, at least one of the components of the electronic device 101 (eg, the display module 160, the sensor module 176, or the communication module 190) It is possible to control at least some of the related functions or states. According to an embodiment, the co-processor 123 (eg, an image signal processor or a communication processor) may be implemented as part of another functionally related component (eg, the camera module 180 or the communication module 190). there is. According to an embodiment, the auxiliary processor 123 (eg, a neural network processing device) may include a hardware structure specialized for processing an artificial intelligence model. Artificial intelligence models can be created through machine learning. Such learning may be performed, for example, in the electronic device 101 itself on which artificial intelligence is performed, or may be performed through a separate server (eg, the server 108). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but in the above example not limited The artificial intelligence model may include a plurality of artificial neural network layers. Artificial neural networks include deep neural networks (DNNs), convolutional neural networks (CNNs), recurrent neural networks (RNNs), restricted boltzmann machines (RBMs), deep belief networks (DBNs), bidirectional recurrent deep neural networks (BRDNNs), It may be one of deep Q-networks or a combination of two or more of the above, but is not limited to the above example. The artificial intelligence model may include, in addition to, or alternatively, a software structure in addition to the hardware structure.

The memory 130 may store various data used by at least one component of the electronic device 101 (eg, the processor 120 or the sensor module 176 ). The data may include, for example, input data or output data for software (eg, the program 140 ) and instructions related thereto. The memory 130 may include a volatile memory 132 or a non-volatile memory 134 .

The program 140 may be stored as software in the memory 130 , and may include, for example, an operating system 142 , middleware 144 , or an application 146 .

The input module 150 may receive a command or data to be used in a component (eg, the processor 120 ) of the electronic device 101 from the outside (eg, a user) of the electronic device 101 . The input module 150 may include, for example, a microphone, a mouse, a keyboard, a key (eg, a button), or a digital pen (eg, a stylus pen).

The sound output module 155 may output a sound signal to the outside of the electronic device 101 . The sound output module 155 may include, for example, a speaker or a receiver. The speaker can be used for general purposes such as multimedia playback or recording playback. The receiver may be used to receive an incoming call. According to one embodiment, the receiver may be implemented separately from or as part of the speaker.

The display module 160 may visually provide information to the outside (eg, a user) of the electronic device 101 . The display module 160 may include, for example, a control circuit for controlling a display, a hologram device, or a projector and a corresponding device. According to an embodiment, the display module 160 may include a touch sensor configured to sense a touch or a pressure sensor configured to measure the intensity of a force generated by the touch.

The audio module 170 may convert a sound into an electric signal or, conversely, convert an electric signal into a sound. According to an embodiment, the audio module 170 acquires a sound through the input module 150 , or an external electronic device (eg, a sound output module 155 ) connected directly or wirelessly with the electronic device 101 . A sound may be output through the electronic device 102 (eg, a speaker or headphones).

The sensor module 176 detects an operating state (eg, power or temperature) of the electronic device 101 or an external environmental state (eg, user state), and generates an electrical signal or data value corresponding to the sensed state. can do. According to an embodiment, the sensor module 176 may include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, It may include a temperature sensor, a humidity sensor, or an illuminance sensor.

The interface 177 may support one or more designated protocols that may be used by the electronic device 101 to directly or wirelessly connect with an external electronic device (eg, the electronic device 102 ). According to an embodiment, the interface 177 may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

The connection terminal 178 may include a connector through which the electronic device 101 can be physically connected to an external electronic device (eg, the electronic device 102 ). According to an embodiment, the connection terminal 178 may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (eg, a headphone connector).

The haptic module 179 may convert an electrical signal into a mechanical stimulus (eg, vibration or movement) or an electrical stimulus that the user can perceive through tactile or kinesthetic sense. According to an embodiment, the haptic module 179 may include, for example, a motor, a piezoelectric element, or an electrical stimulation device.

The camera module 180 may capture still images and moving images. According to an embodiment, the camera module 180 may include one or more lenses, image sensors, image signal processors, or flashes.

The power management module 188 may manage power supplied to the electronic device 101 . According to an embodiment, the power management module 188 may be implemented as, for example, at least a part of a power management integrated circuit (PMIC).

The battery 189 may supply power to at least one component of the electronic device 101 . According to one embodiment, battery 189 may include, for example, a non-rechargeable primary cell, a rechargeable secondary cell, or a fuel cell.

The communication module 190 is a direct (eg, wired) communication channel or a wireless communication channel between the electronic device 101 and an external electronic device (eg, the electronic device 102, the electronic device 104, or the server 108). It can support establishment and communication performance through the established communication channel. The communication module 190 may include one or more communication processors that operate independently of the processor 120 (eg, an application processor) and support direct (eg, wired) communication or wireless communication. According to one embodiment, the communication module 190 is a wireless communication module 192 (eg, a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module 194 (eg, : It may include a LAN (local area network) communication module, or a power line communication module). A corresponding communication module among these communication modules is a first network 198 (eg, a short-range communication network such as Bluetooth, wireless fidelity (WiFi) direct, or infrared data association (IrDA)) or a second network 199 (eg, legacy It may communicate with the external electronic device 104 through a cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (eg, a telecommunication network such as a LAN or a WAN). These various types of communication modules may be integrated into one component (eg, a single chip) or may be implemented as a plurality of components (eg, multiple chips) separate from each other. The wireless communication module 192 uses the subscriber information (eg, International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module 196 within a communication network such as the first network 198 or the second network 199 . The electronic device 101 may be identified or authenticated.

The wireless communication module 192 may support a 5G network after a 4G network and a next-generation communication technology, for example, a new radio access technology (NR). NR access technology includes high-speed transmission of high-capacity data (eMBB (enhanced mobile broadband)), minimization of terminal power and access to multiple terminals (mMTC (massive machine type communications)), or high reliability and low latency (URLLC (ultra-reliable and low-latency) -latency communications)). The wireless communication module 192 may support a high frequency band (eg, mmWave band) to achieve a high data rate, for example. The wireless communication module 192 includes various technologies for securing performance in a high-frequency band, for example, beamforming, massive multiple-input and multiple-output (MIMO), all-dimensional multiplexing. It may support technologies such as full dimensional MIMO (FD-MIMO), an array antenna, analog beam-forming, or a large scale antenna. The wireless communication module 192 may support various requirements specified in the electronic device 101 , an external electronic device (eg, the electronic device 104 ), or a network system (eg, the second network 199 ). According to an embodiment, the wireless communication module 192 may include a peak data rate (eg, 20 Gbps or more) for realizing eMBB, loss coverage (eg, 164 dB or less) for realizing mMTC, or U-plane latency for realizing URLLC ( Example: downlink (DL) and uplink (UL) each 0.5 ms or less, or round trip 1 ms or less).

The antenna module 197 may transmit or receive a signal or power to the outside (eg, an external electronic device). According to an embodiment, the antenna module 197 may include an antenna including a conductor formed on a substrate (eg, a PCB) or a radiator formed of a conductive pattern. According to an embodiment, the antenna module 197 may include a plurality of antennas (eg, an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network such as the first network 198 or the second network 199 is connected from the plurality of antennas by, for example, the communication module 190 . can be selected. A signal or power may be transmitted or received between the communication module 190 and an external electronic device through the selected at least one antenna. According to some embodiments, other components (eg, a radio frequency integrated circuit (RFIC)) other than the radiator may be additionally formed as a part of the antenna module 197 .

According to various embodiments, the antenna module 197 may form a mmWave antenna module. According to one embodiment, the mmWave antenna module comprises a printed circuit board, an RFIC disposed on or adjacent to a first side (eg, bottom side) of the printed circuit board and capable of supporting a designated high frequency band (eg, mmWave band); and a plurality of antennas (eg, an array antenna) disposed on or adjacent to a second side (eg, top or side) of the printed circuit board and capable of transmitting or receiving signals of the designated high frequency band. can do.

At least some of the components are connected to each other through a communication method between peripheral devices (eg, a bus, general purpose input and output (GPIO), serial peripheral interface (SPI), or mobile industry processor interface (MIPI)) and a signal ( eg commands or data) can be exchanged with each other.

According to an embodiment, the command or data may be transmitted or received between the electronic device 101 and the external electronic device 104 through the server 108 connected to the second network 199 . Each of the external electronic devices 102 or 104 may be the same as or different from the electronic device 101 . According to an embodiment, all or a part of operations executed in the electronic device 101 may be executed in one or more external electronic devices 102 , 104 , or 108 . For example, when the electronic device 101 is to perform a function or service automatically or in response to a request from a user or other device, the electronic device 101 may perform the function or service itself instead of executing the function or service itself. Alternatively or additionally, one or more external electronic devices may be requested to perform at least a part of the function or the service. One or more external electronic devices that have received the request may execute at least a part of the requested function or service, or an additional function or service related to the request, and transmit a result of the execution to the electronic device 101 . The electronic device 101 may process the result as it is or additionally and provide it as at least a part of a response to the request. For this, for example, cloud computing, distributed computing, mobile edge computing (MEC), or client-server computing technology may be used. The electronic device 101 may provide an ultra-low latency service using, for example, distributed computing or mobile edge computing. In another embodiment, the external electronic device 104 may include an Internet of things (IoT) device. Server 108 may be an intelligent server using machine learning and/or neural networks. According to an embodiment, the external electronic device 104 or the server 108 may be included in the second network 199 . The electronic device 101 may be applied to an intelligent service (eg, smart home, smart city, smart car, or health care) based on 5G communication technology and IoT-related technology.

The electronic device according to various embodiments disclosed in this document may have various types of devices. The electronic device may include, for example, a portable communication device (eg, a smart phone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a home appliance device. The electronic device according to the embodiment of the present document is not limited to the above-described devices.

The various embodiments of this document and terms used therein are not intended to limit the technical features described in this document to specific embodiments, but it should be understood to include various modifications, equivalents, or substitutions of the embodiments. In connection with the description of the drawings, like reference numerals may be used for similar or related components. The singular form of the noun corresponding to the item may include one or more of the item, unless the relevant context clearly dictates otherwise. As used herein, "A or B", "at least one of A and B", "at least one of A or B", "A, B or C", "at least one of A, B and C", and "A , B, or C" each may include any one of the items listed together in the corresponding one of the phrases, or all possible combinations thereof. Terms such as "first", "second", or "first" or "second" may be used simply to distinguish the element from other elements in question, and may refer to elements in other aspects (e.g., importance or order) is not limited. It is said that one (eg, first) component is "coupled" or "connected" to another (eg, second) component, with or without the terms "functionally" or "communicatively". When referenced, it means that one component can be connected to the other component directly (eg by wire), wirelessly, or through a third component.

The term “module” used in various embodiments of this document may include a unit implemented in hardware, software, or firmware, and is interchangeable with terms such as, for example, logic, logic block, component, or circuit. can be used as A module may be an integrally formed part or a minimum unit or a part of the part that performs one or more functions. For example, according to an embodiment, the module may be implemented in the form of an application-specific integrated circuit (ASIC).

According to various embodiments of the present document, one or more instructions stored in a storage medium (eg, internal memory 136 or external memory 138) readable by a machine (eg, electronic device 101) may be implemented as software (eg, the program 140) including For example, a processor (eg, processor 120 ) of a device (eg, electronic device 101 ) may call at least one command among one or more commands stored from a storage medium and execute it. This makes it possible for the device to be operated to perform at least one function according to the called at least one command. The one or more instructions may include code generated by a compiler or code executable by an interpreter. The device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, 'non-transitory' only means that the storage medium is a tangible device and does not include a signal (eg, electromagnetic wave), and this term is used in cases where data is semi-permanently stored in the storage medium and It does not distinguish between temporary storage cases.

According to one embodiment, the method according to various embodiments disclosed in this document may be provided as included in a computer program product. Computer program products may be traded between sellers and buyers as commodities. The computer program product is distributed in the form of a machine-readable storage medium (eg compact disc read only memory (CD-ROM)), or via an application store (eg Play Store™) or on two user devices ( It can be distributed (eg downloaded or uploaded) directly between smartphones (eg: smartphones) and online. In the case of online distribution, at least a part of the computer program product may be temporarily stored or temporarily created in a machine-readable storage medium such as a memory of a server of a manufacturer, a server of an application store, or a relay server.

According to various embodiments, each component (eg, module or program) of the above-described components may include a singular or a plurality of entities, and some of the plurality of entities may be separately disposed in other components. there is. According to various embodiments, one or more components or operations among the above-described corresponding components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of components (eg, a module or a program) may be integrated into one component. In this case, the integrated component may perform one or more functions of each component of the plurality of components identically or similarly to those performed by the corresponding component among the plurality of components prior to the integration. . According to various embodiments, operations performed by a module, program, or other component are executed sequentially, in parallel, repeatedly, or heuristically, or one or more of the operations are executed in a different order, or omitted. or one or more other operations may be added.

Hereinafter, an electronic device according to an exemplary embodiment will be described with reference to FIGS. 2, 3 and 4 .

2 is a diagram 201 and 202 illustrating an electronic device according to an exemplary embodiment. 3 is a flowchart 300 illustrating an operation of an electronic device according to an exemplary embodiment. 4 is a diagram 400 illustrating components of an electronic device according to an exemplary embodiment.

Referring to FIG. 2 , an electronic device 101 (eg, the electronic device 101 of FIG. 1 ) includes a housing 210 constituting a body of the electronic device 101 and a display 160 coupled to the housing 210 . (eg, the display module 160 of FIG. 1 ).

The display 160 has a left side 211 (eg, -x direction), an upper side 212 (eg, +y direction), and a right side 213 (eg, +x) of the electronic device 101 with respect to the front side of the electronic device 101 . direction), and four side surfaces of the lower side surface 214 (eg, -y direction), and at least one side surface may be exposed outside without being covered by the housing. Taking the first case 201 of FIG. 2 as an example, the left side 211 , the upper side 212 , and the lower side 214 of the display 160 are covered by the housing 210 , but the right side 213 is It may be positioned on the outer surface of the housing 210 and bent so as to cover the housing 210 . A first side view 270 of FIG. 2 is a view showing the electronic device 101 viewed from the +x axis to the -x axis direction, and in the first side view 270 , the display 160 includes the housing 210 . may be covering Hereinafter, in various embodiments of the present disclosure, a portion of the display 160 corresponding to one side bent to cover the outer surface of the housing 210 may be referred to as a side area 220 of the display 160 .

Hereinafter, in various embodiments of the present disclosure, the display 160 may be a variable display. According to an embodiment, the display 160 of the electronic device 101 may have a width of the first distance AA from the minimum size. According to an exemplary embodiment, the width in which the display 160 is extended based on the minimum size state of the display 160 may be defined as the second distance BB. The third distance CC may be defined as the sum of the first distance AA and the second distance BB, and the variable display may mean a display in which the third distance CC is changeable. In addition, the above-described side area 220 of the display 160 may be located at a third distance CC along the +x axis from the left end of the display 160 , and in the variable display, the position of the side area 220 is It may mean a changeable display.

The variable display 160 is a slide type in which at least one side of the housing 210 of the electronic device 101 is slid while the side area 220 is changed, and is wound inside or outside the housing 210 . A rollable type in which the display area exposed to the outside of the electronic device 101 expands and the side area 220 also moves as the display 160 is unfolded, or the display ( The display area of 160 ) may be expanded/reduced, and the side area 220 may correspond to any one of a foldable type for children. The electronic device 101 of FIG. 2 is a slide-type electronic device 101 , and the second side view 280 and the third side view 290 of FIG. 2 show the electronic device 101 along the -y axis. It is a view showing a view in the +y-axis direction, and as one side of the housing 210 slides in the second side view 280 and expands like the third side view 290 , the electronic device of the display 160 The front portion of 101 may also be expanded.

Referring to the first case 201 of FIG. 2 , the side area 220 may move in the +x-axis direction on the entire screen of the display 160 while the display area of the display 160 expands in the +x-axis direction. . Referring to the second case 202 of FIG. 2 , while the display area of the display 160 is reduced in the -x-axis direction, the side area 220 may move in the -x-axis direction on the entire screen of the display 160 . .

The soft key 230 may be a button displayed on the display 160, and according to settings, a power key, a volume key, a home key, a back key, a recently used app key, a voice assistant key, a shortcut key, It may correspond to at least one of an app shortcut key, a contact shortcut key, a capture key, and a shortcut bar key, but is not limited thereto.

The soft key 230 includes the left side 211 , the upper side 212 , the right side 213 , and the lower side 214 of the display 160 for the user's touch convenience without interfering with the displayed contents of the display 160 . ) may be displayed in an area corresponding to at least one of them or may be displayed in a position adjacent thereto. In various embodiments of the present disclosure, the soft key 230 is displayed on the display 160, in particular, in the repositionable side area 220 or displayed in the first area 240 adjacent to the side area 220 . can be assumed to be

According to an embodiment, the soft key 230 may be displayed based on reference coordinates in which the third distance CC is the x-coordinate and the fourth distance DD is the y-coordinate. The fourth distance DD may be a distance set by a user or may be a distance that is changed according to circumstances, as in an embodiment to be described with reference to FIGS. 7 and 8 . A method for the electronic device 101 to determine the third distance CC will be described later in detail with reference to FIGS. 3 and 4 .

The processor 120 of the electronic device 101 may determine the position of the pixel (pixel) in the x-axis direction by multiplying the third distance CC, which is the x-axis coordinate of the reference coordinate, by the pixel density (display density or pixel density). Also, the processor 120 of the electronic device 101 may determine the position of the pixel (pixel) in the y-axis direction by multiplying the pixel density by the fourth distance DD, which is the y-axis coordinate of the reference coordinate.

Alternatively, the processor 120 of the electronic device 101 may determine the pixel (pixel) position in the x-axis direction by adding the pixel position corresponding to the first distance AA to a value obtained by multiplying the pixel density by the second distance BB. .

In FIG. 2 , the soft key 230 is illustrated as being displayed in the side area 220 , but the present invention is not limited thereto, and may be displayed in the first area 240 adjacent to the side area 220 . A method of determining reference coordinates for displaying the soft key 230 when the soft key 230 is displayed in the side area 230 even when the soft key 230 is displayed in the first area 240 The same can be applied. However, the x-axis coordinate of the reference coordinate when the soft key 230 is displayed in the first region 240 is (the first region 240 is moved by a predetermined value in the -x-axis direction rather than the side light region 220 ). position), it may be determined to be less than the x-axis coordinate of the reference coordinate when the soft key 230 is displayed in the side region 230 by a predetermined value.

Referring to the first case 201 of FIG. 2 , when the side area 220 moves in the +x-axis direction on the entire screen of the display 160 while the display area of the display 160 expands in the +x-axis direction. , the x-axis coordinates of the reference coordinates for displaying the soft key 230 corresponding to the side region 220 may also be moved to the +x-axis.

Referring to the second case 202 of FIG. 2 , when the side area 220 moves in the -x-axis direction on the entire screen of the display 160 while the display area of the display 160 is reduced in the -x-axis direction , the x-axis coordinates of the reference coordinates for displaying the soft key 230 corresponding to the side region 220 may also be moved to the -x-axis.

Hereinafter, an operation of the electronic device 101 following identification of a display change event will be described with reference to FIG. 3 .

Referring to FIG. 3 , in operation 301 , the electronic device (eg, the processor 120 of the electronic device 101 of FIG. 2 ) may identify a change event of the display (eg, the display 160 of FIG. 2 ). . According to an embodiment, the change event of the display refers to the occurrence of change (expand, reduction, or change) of a housing in which the display is mounted, and/or change (expand, reduction, or change) of the display area of the display through change of the housing. could mean occurrence. Hereinafter, in the present specification, identification of a change in a display may include not only identifying a change in the display area of the display itself, but also identifying through a change in a housing in which the display is mounted. According to an embodiment, the display change event may mean that the third distance CC changes.

In operation 303 , the processor 120 of the electronic device may determine a side area of the display (eg, the side area 220 of FIG. 2 ) based on the obtained measurement value. The description of the side area of the display may be the same as the description with reference to FIG. 2 above. According to an embodiment, the determination of the side area may mean determination of the x-axis coordinate (= the third distance CC) of the reference coordinate for displaying the soft key.

The processor 120 of the electronic device may first obtain a measurement value from a sensor (eg, at least one sensor included in the sensor module 176 of FIG. 4 ) in order to determine the side area. According to an embodiment, the processor 120 of the electronic device 101 may obtain a measurement value for a display change event through a sensor (eg, at least one sensor included in the sensor module 176 of FIG. 4 ). there is. According to an embodiment, the measured value for the change event of the display may be a value directly measured by the third distance CC or a value measured by the second distance BB to calculate the third distance CC. A method by which the sensor of the electronic device obtains a measurement value for a display change event will be described in detail later with reference to FIG. 4 .

The processor 120 of the electronic device may determine the side area based on the acquired measurement value.

In operation 305, the processor 120 of the electronic device may determine whether the calculated side area is changed from the existing one.

When it is determined that the calculated side area is changed from the existing one, the processor 120 of the electronic device may change the display position of the soft key to correspond to the changed side area in operation 307 .

The processor of the electronic device may determine the third distance CC based on the acquired measurement value and change the reference coordinate of the soft key by using the third distance CC as the x-axis coordinate. The processor of the electronic device may display the soft key based on reference coordinates in which the third distance CC is the x-coordinate and the fourth distance DD is the y-coordinate.

Alternatively, according to an embodiment, the processor of the electronic device may change the x-coordinate of the reference coordinate of the soft key as much as the side area is changed. For example, if the side area is moved by a in the +x direction, the x-axis coordinate of the soft key display coordinate can be changed by +a. Also, the electronic device may change at least one of a display size and/or a display direction of the soft key to correspond to the changed side area.

The processor of the electronic device may determine the position of the pixel (pixel) in the x-axis direction by multiplying the third distance CC that is the x-axis coordinate of the reference coordinate by the pixel density (display density or pixel density). Also, the processor of the electronic device may determine the position of the pixel (pixel) in the y-axis direction by multiplying the pixel density by the fourth distance DD, which is the y-axis coordinate of the reference coordinate. Alternatively, the processor of the electronic device may determine the position of the pixel (pixel) in the x-axis direction by adding the pixel position corresponding to the first distance AA to a value obtained by multiplying the second distance BB by the pixel density. The processor of the electronic device may display the soft key based on the determined pixel position.

Hereinafter, a method of determining the position of the side area 220 of the display 160 of the electronic device 101 will be described with reference to FIGS. 2 and 4 . The electronic device 101 of FIG. 2 may include the sensor module 176 of FIG. 4 (eg, the sensor module 176 of FIG. 1 ) and middleware 144 (eg, the middleware 144 of FIG. 1 ). can In the present specification, determining the position of the side area 220 of the display 160 may mean determining the x-axis coordinate (= third distance CC) of the reference coordinate of the soft key.

The sensor module 176 is a time-of-flight (ToF) sensor 431 , an illuminance sensor 432 , a hall sensor 433 , a proximity sensor 434 , a touch screen panel 435 , a rotation encoder ( It may include at least one of an encoder 436 and a bending sensor 437 . At least one of the sensors included in the sensor module 176 of FIG. 4 may be omitted.

The middleware 144 may include at least one of a display state manager 441 , a display manager 442 , a soft key manager 443 , a window manager 444 , and an input manager 445 . At least one of the modules included in the middleware 144 of FIG. 4 may be omitted.

According to an embodiment, the electronic device 101 may measure the third distance CC using the ToF sensor 431 . For example, the ToF sensor 431 is positioned inside a housing corresponding to the left side 211 of the display 160 of the electronic device 101 , and the ToF sensor 431 is located in the side area 220 of the display 160 . A strong light can be shot towards the The ToF sensor 431 may measure the third distance CC by detecting light reflected back from the side area 220 (or the housing corresponding to the side area). The ToF sensor 431 may transmit the measured value to the display state manager 441 .

According to an embodiment, the electronic device 101 measures the third distance CC itself of the display 160 or the second distance BB for calculating the third distance CC using the illuminance sensor 432 . can do. For example, the illuminance sensor 432 may be located inside a housing corresponding to the side area 220 of the display 160 of the electronic device 101 . Also for example, the display 160 may include a plurality of micro-configurations that gradually increase or decrease in size along the upper side 212 of the display 160 . Accordingly, when the position of the side region 220 is changed, the size of the micro-hole corresponding to the side region 220 is also changed, and the amount of light incident to the illuminance sensor 432 through the micro-pore may also vary. The illuminance sensor 432 may transmit a measurement value of the amount of incident light to the display state manager 441 , and the electronic device 101 (eg, the display state manager 441 ) generates a value based on the measured value of the amount of light. The third distance CC itself or the position of the second distance BB for calculating the third distance CC may be calculated.

Or, for example, a part of the display 160 is embedded inside the housing 210 of the electronic device 101 , and the embedded part when the display 160 is expanded is the housing 210 of the electronic device 101 . ) In the case of the electronic device 101 of a type exposed to the outside, the illuminance sensor 432 is positioned in a portion of the display 160 that is built into the housing 210 to be incident on the illuminance sensor 432 when exposed to the outside. The amount of light can be measured. The illuminance sensor 432 may transmit a measured value of the amount of incident light to the display state manager 441 , and the electronic device 101 (eg, the display state manager 441 ) may presently based on the measured value of the amount of light. The area of the display 160 exposed to the outside of the housing 210 may be determined. The electronic device 101 may calculate the third distance CC or the second distance BB based on the determined value of the area of the display 160 currently exposed to the outside of the housing 210 .

According to an embodiment, the electronic device 101 measures the third distance CC itself of the display 160 or the second distance BB for calculating the third distance CC using the Hall sensor 433 . can do. For example, a magnet is located inside the housing 210 corresponding to the side region 220 of the display 160 of the electronic device 101, and a pole is located on the leftmost surface of the housing 210 corresponding to the position of the magnet. A sensor 433 may be located. The hall sensor 433 may measure the magnetic force of the magnet and transmit it to the display state manager 441 , and the display state manager 441 may calculate the current position of the housing 210 based on the magnetic force measurement value. The display state manager 441 may calculate the area of the display 160 exposed to the outside based on the calculated position of the housing 210 .

According to an embodiment, the electronic device 101 measures the third distance CC itself of the display 160 or the second distance BB for calculating the third distance CC using the proximity sensor 434 . can do. For example, a part of the display 160 is embedded inside the housing 210 of the electronic device 101 , and the embedded part when the display 160 is expanded is the housing 210 of the electronic device 101 . The electronic device 101 may be exposed to the outside. In this case, the electronic device 101 may include a plurality of proximity sensors 434 in a lower portion of an area that can be embedded inside the housing 210 of the display 160 . According to an embodiment, the plurality of proximity sensors 434 may be positioned in a line along a direction in which the display 160 is expanded (eg, an x-axis direction). When a part of the display 160 is exposed to the outside of the housing 210, the proximity object (housing) detected by the proximity sensor 434 may not be detected, and depending on the degree of expansion of the display 160, a plurality of proximity sensors ( 434), some may detect a proximity object (housing), and some may not detect a proximity object (housing). The plurality of proximity sensors 434 may transmit a measurement value of a proximity object to the display state manager 441 , and the display state manager 441 may transmit the measurement value of the proximity object to the display 160 currently exposed to the outside of the housing 210 based on the received measurement value. ) area can be calculated. The electronic device 101 may calculate the area of the display 160 currently exposed to the outside of the housing 210 based on the proximity object detection results of the plurality of proximity sensors 434 , and based on the calculated value, the display 160 . A third distance CC or a second distance BB of may be calculated.

According to an embodiment, the electronic device 101 calculates the third distance CC itself of the display 160 or the second distance BB for calculating the third distance CC using the touch screen panel 435 . can be measured The electronic device 101 may include a touch screen panel 435 corresponding to the front surface of the display 160 . For example, a part of the display 160 is embedded inside the housing 210 of the electronic device 101 , and the embedded part when the display 160 is expanded is the housing 210 of the electronic device 101 . In the case of the electronic device 101 of a type exposed to the outside, when a part of the display 160 exposed to the outside of the housing 210 is rebuilt into the housing 210 , the part of the display 160 to be built in A hovering event may occur due to the housing 210 . In this case, in order to distinguish it from a hovering event caused by a user's finger, the touch screen panel 435 may generate a hovering event in a line in the y-axis direction. The touch screen panel 435 may transmit the coordinates at which the hovering event occurs to the display state manager 441, and the display state manager 441 may calculate the area of the display 160 currently exposed to the outside of the housing 210 based on this. there is. Also, the display state manager 441 may calculate the third distance CC or the second distance BB based on a calculated value of the area of the display 160 currently exposed to the outside of the housing 210 .

According to an embodiment, the electronic device 101 measures the third distance CC itself of the display 160 or the second distance BB for calculating the third distance CC using the rotation encoder 436 . can do. For example, if a portion of the display 160 is rolled through the rotation encoder 436 and the display 160 is expanded while the rolled portion is unfolded when the display 160 is expanded, the electronic device 101 is , through the amount of rotation of the rotation encoder 436, the currently unfolded display 160 can calculate the area. The rotation encoder 456 may transmit the rotation amount to the display state manager 441, and the display state manager 441 calculates the currently expanded display 160 area based on the received rotation amount of the encoder 456. Also, the third distance CC or the second distance BB may be calculated based on the calculated value.

According to an embodiment, the electronic device 101 measures the third distance CC itself of the display 160 or the second distance BB for calculating the third distance CC using the bending sensor 437 . can do. For example, the bending sensor 437 in the x-axis direction may be included under the display 160 . When the display 160 is bent and reduced in the form of the electronic device 101, the bending sensor 437 may transmit a bending position to the display state manager 441, and the display state manager 441 receives the received position. Based on the measured value, the third distance CC itself or the second distance BB for calculating the third distance CC may be calculated.

At least one sensor of the sensor module 176 may transmit a value measured by the sensor to the display state manager 441 . The display state manager 441 may acquire the third distance CC of the display 160 based on the received measurement value. The display state manager 441 obtains the third distance CC directly based on the received measurement value or acquires the second distance BB based on the received measurement value and adds the first distance AA to the second distance AA. 3 distance (CC) may be obtained. The display state manager 441 may transmit the obtained third distance CC to the display manager 442 . Also, the display state manager 441 may transmit the obtained third distance CC to the soft key manager 443 .

The display manager 442 may map logical coordinates of a display to output a graphical user interface (GUI) to physical coordinates of an actual display. According to an embodiment, reference coordinates in which the third distance CC indicating the physical size of the display 160 is an x-axis coordinate and the fourth distance DD is a y-axis coordinate may be referred to as physical coordinates, and the display 160 may be referred to as a physical coordinate. Pixel coordinates indicating a position of a pixel (pixel) for displaying a graphical user interface (GUI) among a plurality of pixels included in 160 may be referred to as logical coordinates.

The display manager 442 may change the third distance CC received from the display state manager 441 into logical coordinates. According to an embodiment, the display manager 442 may determine the position of a pixel (pixel) in the x-axis direction by multiplying the third distance CC, which is the x-axis coordinate of the reference coordinate, by the pixel density (display density or pixel density). Alternatively, the display manager 442 may determine the pixel (pixel) position in the x-axis direction by adding the pixel position corresponding to the first distance AA to a value obtained by multiplying the second distance BB by the pixel density.

The soft key manager 443 may change pixel coordinates (eg, logical coordinates) of the soft key 230 to correspond to the pixel (pixel) position determined by the display manager 442 . The soft key manager 443 may transmit pixel coordinates (eg, logical coordinates) of the soft key to the window manager 444 .

The window manager 444 manages one or more graphic user interface (GUI) resources used on the screen, and displays the soft key 230 based on the pixel coordinates (eg, logical coordinates) received from the soft key manager 443 (eg, logical coordinates). 160) can be displayed.

Hereinafter, an electronic device according to an exemplary embodiment will be described with reference to FIGS. 4 and 5 .

5 is a flowchart 500 illustrating an operation of an electronic device according to an exemplary embodiment.

4 and 5 , in operation 501, the processor 120 of the electronic device may detect a user's touch input for a soft key. For example, the electronic device may detect a user's input for a soft key through the touch screen panel 435 . As the display position of the soft key is changed as the side area of the display is changed, the coordinates at which the touch input is identified even for the same soft key (eg, a home key) may be different depending on the state of the display. The sensor module 176 may transmit input coordinates for the soft key to the soft key manager 443 .

In operation 503, the processor 120 of the electronic device may identify a key event value of the touched soft key. The soft key manager 443 may store a key event value corresponding to at least one soft key. The soft key manager 443 may identify a key event value corresponding to the touched soft key based on the input coordinates for the soft key received from the sensor module 176 . The soft key manager 443 may transmit the identified key event value to the input manager 445 . According to an embodiment, a key event value corresponding to a soft key may be changeable by a user's setting.

In operation 505, the processor 120 of the electronic device may generate a virtual key event. Based on the received key event value, the input manager 445 may generate a virtual key event for the soft key in the same way that the physical key is pressed. The electronic device may execute a virtual key event to perform an operation corresponding to the touched soft key. Also, according to an embodiment, the electronic device may provide sound or haptic feedback as a soft key is touched.

Hereinafter, an electronic device according to an embodiment will be described with reference to FIG. 6 .

6 is a diagram 610 and 620 of an electronic device according to an exemplary embodiment. The same components as those of the above-described embodiment may be referred to with the same reference numerals, and a description thereof may be omitted.

Referring to FIG. 6 , the electronic device 101 (eg, the electronic device 101 of FIG. 1 ) includes a housing 210 constituting a body of the electronic device 101 and a display 160 coupled to the housing 210 . (eg, the display module 160 of FIG. 1 ). The display 160 may include a side area 220 corresponding to one side bent so as to cover the outer surface of the housing 210 among the four side surfaces. The display 160 may be a variable display in which the position of the side area 220 among the entire screen can be changed.

If the electronic device 101 shown in FIG. 2 is in a portrait mode in which the vertical direction (y-axis direction) is longer than the horizontal direction (x-axis direction), the electronic device 101 is displayed vertically as shown in FIG. 6 . The display 160 may be expanded/reduced in a landscape mode in which a horizontal direction (x-axis direction) is longer than a direction (y-axis direction).

Referring to the first case 610 of FIG. 6 , in the landscape mode, as the display area of the display 160 expands in the +y-axis direction, in the entire screen of the display 160, the side area 220 moves in the +y-axis direction. can move When the side area 220 of the display 160 moves by a in the +y-axis direction, the electronic device 101 may change the y-axis coordinate of the reference coordinate of the soft key 230 by +a, and the changed reference coordinate The soft key 230 may be displayed by changing the pixel coordinates according to the .

Referring to the second case 620 of FIG. 6 , while the display area of the display 160 is reduced in the -y-axis direction in the landscape mode, the side area 220 in the entire screen of the display 160 is moved in the -y-axis direction. can move When the side area 220 of the display 160 moves by a in the -y-axis direction, the electronic device 101 may change the y-axis coordinate of the display coordinate of the soft key 230 by -a, and the changed reference coordinate The soft key 230 may be displayed by changing the pixel coordinates according to the .

Hereinafter, an operation of the electronic device according to an exemplary embodiment will be described with reference to FIGS. 7 and 8 .

7 is a flowchart 700 illustrating an operation of an electronic device according to an exemplary embodiment. 8 is a diagram 810 and 820 illustrating an electronic device according to an exemplary embodiment. The same components as those of the above-described embodiment may be referred to with the same reference numerals, and a description thereof may be omitted.

7 and 8 , in operation 701 , the processor 120 of the electronic device 101 may display the soft key 230 . A method of determining pixel coordinates for displaying the soft key 230 may be the same as the method described above with reference to FIGS. 2 to 4 .

In operation 703 , the processor 120 of the electronic device 101 may detect a grip on the electronic device 101 . The electronic device 101 may further include a grip sensor (not shown), and may sense gripping and gripping positions of the electronic device 101 through the grip sensor.

In operation 705, the processor 120 of the electronic device 101 may adjust the display position of the soft key in consideration of the grip detection position. For example, when the user's grip position is changed from the first state 810 to the second state 820 of FIG. 8 , the electronic device 101 displays the y-axis coordinate of the reference coordinate of the soft key 230 (= second state). 4 distance DD) may be changed to correspond to the gripping position. According to an embodiment, the electronic device 101 may display the soft key 230 at a position where the user can easily touch the soft key 230 in response to the grip position. For example, the electronic device 101 displays the soft key 230 by changing the y-axis coordinate (= fourth distance (DD)) of the reference coordinate at a specified interval (eg, 2 cm from the top) from the coordinate where the grip is detected. can do.

Hereinafter, an operation of the electronic device according to an exemplary embodiment will be described with reference to FIGS. 9 and 10 .

9 is a flowchart 900 illustrating an operation of an electronic device according to an exemplary embodiment. 10 is a diagram 1010 and 1020 illustrating an electronic device according to an exemplary embodiment.

9 and 10 , in operation 901 , the processor (eg, the processor 120 of FIG. 2 ) of the electronic device 1030 (eg, the electronic device 101 of FIG. 1 ) displays the display 1031 (eg: A state change event of the display 160 of FIG. 1 may be identified. The display 1031 of the electronic device 1030 may be a variable display in which a front area and a side area of the display are changed. Referring to the first drawing 1010 of FIG. 10 , for example, when a surface including the gaze area viewed by the user is defined as the front of the electronic device 101 , the display 160 is + It is slidable in the x-axis or -axis direction, and as it slides, the front area and the side area can be changed. For example, when the display 1031 is slid all the way in the +x-axis direction, the first area 1011 belongs to the front area, the second area 1012 becomes the side area, and the display 1031 moves in the -x-axis direction. In the case of sliding to the end, the second area 1012 may be a front area and the first area 1011 may be a side area.

In operation 903, the processor of the electronic device 1030 may determine whether the position of the side area is changed. Referring to the first drawing 1010 of FIG. 10 , according to an exemplary embodiment, the processor of the electronic device 1030 controls the side area of the display 160 as the display 160 slides in the +x-axis direction with respect to the front surface. It may be determined whether the first area 1011 is changed to the second area 1012 . The definition of the side area of the display 160 may be the same as in the embodiment described above with reference to FIG. 2 . Referring to the second drawing 1020 of FIG. 10 , as the display 160 is slid in the +x-axis direction with respect to the front surface, the side area of the entire screen of the display 160 moves from the first area 1011 to the second In the area 1012 , for example, it can be confirmed that the movement has been moved from left to right.

In operation 905 , the processor 120 of the electronic device 101 may change the display position of the soft key 230 to correspond to the changed position of the side area of the display 160 . Referring to the first drawing 1010 and the second drawing 1020 of FIG. 10 , in the electronic device 101 , the side area of the display 160 is changed from the first area 1011 to the second area 1012 . Accordingly, the soft key 230 displayed in the first area 1011 may be moved to be displayed in the second area 1012 . Also, the electronic device 101 may change at least one of a display size and a display direction of the soft key 230 to correspond to the changed side area.

Hereinafter, an operation of the electronic device according to an exemplary embodiment will be described with reference to FIGS. 11 and 12 .

11 is a flowchart 1100 illustrating an operation of an electronic device according to an exemplary embodiment. 12 is a diagram 1210 and 1220 of an electronic device according to an exemplary embodiment. The same components as those of the above-described embodiment may be referred to with the same reference numerals, and a description thereof may be omitted.

11 and 12 , in operation 1101 , the processor 120 of the electronic device 101 may identify an event related to the change of the gaze area of the display 160 . Referring to the first drawing 1210 of FIG. 12 , the display 160 of the electronic device 101 may completely surround the electronic device 101 or at least cover the front and rear surfaces of the electronic device 101 . As in the first drawing 1210 of FIG. 12 , the gaze area of the electronic device 101 is changed from the first area 1211 to the second area ( 1212) can be changed. According to an embodiment, the gaze area of the electronic device 101 may include a grip sensor (detecting the user's gripping direction), an iris sensor (detecting the user's iris), or a face recognition sensor (detecting the user's face) built in the electronic device 101 . can be identified through

In operation 1103 , the processor 120 of the electronic device 101 may identify a side area based on the gaze area of the display 160 . Referring to the first drawing 1210 of FIG. 12 , when the gaze area of the electronic device 101 is the first area 1211 , the first side area 1231 is located to the left of the first area 1211 and the right side area 1231 is located on the right side of the first area 1211 . As a result, the second side area 1232 may be located. When the gaze area of the electronic device 101 is changed to the second area 1212 , the first side area 1231 is located to the left of the second area 1212 and the second side area 1232 is located to the right of the second area 1212 . can be changed to The second drawing 1220 of FIG. 12 is a view showing only the entire screen of the display 160 of the electronic device. When the gaze area of the electronic device 101 is the first area 1211 , the left side of the first area 1211 . , when the first side area 1231 is located and the second side area 1232 is located to the right, and the gaze area of the electronic device 101 is changed to the second area 1212 , the It can be seen that the first side area 1231 is positioned to the left and the second side area 1232 is positioned to the right.

In operation 1105 , the processor 120 of the electronic device 101 may change the position of the soft key to correspond to the changed gaze area and side area of the display. Referring to the first drawing 1210 and the second drawing 1220 of FIG. 12 , the electronic device 101 views the first area 1211 when the gaze area of the electronic device 101 is the first area 1211 . The first soft key 1221 is displayed on the first side area 1231 on the left as a reference, and the second soft key 1222 on the second side area 1232 on the right side with respect to the first area 1211 . ) can be displayed. Also, when the gaze area of the electronic device 101 is changed to the second area 1212 , the electronic device 101 displays the first soft key in the first side area 1231 on the left side with respect to the second area 1212 . 1221 may be displayed, and the second soft key 1222 may be displayed on the second side area 1232 on the right side with respect to the second area 1212 .

Hereinafter, an operation of the electronic device according to an exemplary embodiment will be described with reference to FIGS. 13, 14 and 15 .

13 is a flowchart 1300 illustrating an operation of an electronic device according to an exemplary embodiment. 14 is a diagram 1410 and 1420 of an electronic device according to an exemplary embodiment. 15 is a diagram 1510 and 1520 of an electronic device according to an exemplary embodiment. The same components as those of the above-described embodiment may be referred to with the same reference numerals, and a description thereof may be omitted.

Referring to FIG. 13 , in operation 1301 , the processor 120 of the electronic device may display a soft key on the display. The method of calculating the display coordinates of the soft key may be the same as in the embodiment described above with reference to FIGS. 2 to 4 .

In operation 1303, the processor 120 of the electronic device may identify occurrence of a screen off event. According to an embodiment, the screen-off event may occur by any one of a press of a physical key, a touch of a soft key, a gesture input corresponding to a screen-off, a screen-off request from an application, or the lapse of a predetermined time without a user's input.

In operation 1305, after identifying the occurrence of the screen-off event, the processor 120 of the electronic device determines whether to activate an always on display (AOD) mode that displays a user interface such as date, clock, and missed contact even in a low power mode. can Whether to activate the AOD mode when the screen is turned off may be set by at least one of a design of the electronic device or a user setting. AOD mode is an example of a low-power technology, which displays AOD content on the display in response to a screen-off request, but limits the display's backlight to use minimal or no power to display AOD content. there is. In addition, in the AOD mode, the electronic device stops the operation of a system resource such as the CPU (eg, the processor 120 in FIG. 1 ) and activates the system operation only for a certain period at a certain period to refresh the screen and stop the operation of the system resource again. there is.

If it is determined that the AOD mode is not activated, in operation 1309, the electronic device may remove the screen display elements except for the soft key while switching the electronic device to the low power mode. According to an embodiment, in the low power mode, the electronic device stops access to a network, limits the operation of a CPU clock or an active core, stops searching for Wi-Fi in the system, ignores a wake request in the system, At least one of the following actions may be performed: a part of the AlarmManager event delays a certain time from the request period, the system disables synchronization execution, and the system disables the execution of the scheduler (JobScheduler).

If it is determined that the AOD mode is activated, in operation 1307, the electronic device switches the electronic device to the low power mode, removes screen display elements except for soft keys, and displays AOD content (eg, date, time, missed contact). there is.

Referring to the first drawing 1410 of FIG. 14 , the electronic device 101 may display various contents and soft keys 230 on the display 160 in a screen-on state. The electronic device 101 identifies the occurrence of the screen-off event and, as it is determined that the AOD mode is not activated, switches to a low power mode and removes all displayed contents of the screen except for the soft key 230 . there is. The electronic device 101 may still display the soft key 230 even in the low power mode.

Referring to the second drawing 1420 of FIG. 14 , even when the display 160 is expanded in the low power mode, the electronic device 101 may change the display position of the soft key 230 according to the changed position of the side area. . A method of calculating the display coordinates of the soft key 230 in the low power mode of the electronic device 101 may be the same as the method described above with reference to FIGS. 2 to 4 .

Referring to the first drawing 1510 of FIG. 15 , the electronic device 101 may display various contents and soft keys 230 on the display 160 in a screen-on state. The electronic device 101 identifies the occurrence of the screen-off event and, upon determining that the AOD mode is activated, switches to the low-power mode, removes all displayed content on the screen except for the soft key 230, and displays the AOD content 1511 can be displayed The electronic device 101 may still display the soft key 230 in the low power mode and the AOD mode.

Referring to the second drawing 1520 of FIG. 15 , even when the display 160 is extended in the low power mode and the AOD mode, the electronic device 101 determines the display position of the soft key 230 according to the changed position of the side area. can be changed A method of calculating the display coordinates of the soft key 230 in the low power mode and the AOD mode of the electronic device 101 may be the same as the method described above with reference to FIGS. 2 to 4 .

Hereinafter, an operation of the electronic device according to an exemplary embodiment will be described with reference to FIGS. 16, 17 and 18 .

16 is a flowchart 1600 illustrating an operation of an electronic device according to an exemplary embodiment. 17 is a diagram 1710 and 1720 of an electronic device according to an exemplary embodiment. 18 is a diagram 1810 and 1580 illustrating an electronic device according to an exemplary embodiment. The same components as those of the above-described embodiment may be referred to with the same reference numerals, and a description thereof may be omitted.

Referring to FIG. 16 , in operation 1601 , the processor 120 of the electronic device may display a screen off and a soft key. The method of calculating the display coordinates of the soft key may be the same as the method described above with reference to FIGS. 2 to 4 .

In operation 1603, the processor 120 of the electronic device may start a timer after turning off the screen. The timer may be a timer for a soft key hiding operation.

In operation 1605, the processor 120 of the electronic device may determine whether N time has elapsed. N time may be an arbitrary threshold value, which may be set by a design of an electronic device or a user setting.

In operation 1607, the processor 120 of the electronic device may remove the soft key when it is determined that N time has elapsed. When the soft key is removed, the electronic device may switch from a low power mode to a screen off state. Referring to the first drawing 1710 of FIG. 17 showing the electronic device 101 in the low power mode, the electronic device 101 removes the display of the soft key 230 when N hours have elapsed after entering the low power mode. It can be switched to a completely screen-off state. Referring to the second drawing 1720 of FIG. 17 showing the electronic device 101 in the low power AOD mode, the electronic device 101 displays the soft key 230 when N hours have elapsed after entering the low power AOD mode. It can be removed and switched to a complete screen off state.

Referring again to FIG. 16 , in operation 1609 , the processor 120 of the electronic device may determine whether a soft key display event has occurred. According to an embodiment, the soft key display event may be performed in response to at least one of a user's gesture input corresponding to the soft key display, a motion detection of the electronic device, a grip detection on the electronic device, a user touch detection on the display, or a system request. may be caused by

In operation 1611, the processor 120 of the electronic device may display the soft key on the display as it is determined that the soft key display event has occurred. Referring to the first drawing 1810 of FIG. 18 showing the electronic device 101 in the low power mode, when the electronic device 101 identifies occurrence of a soft key display event in a state in which the soft key is not displayed, the display ( A soft key 230 may be displayed on 160 . Referring to the second drawing 1820 of FIG. 18 showing the electronic device 101 in the low-power AOD mode, the electronic device 101 displays the AOD content 1511 and displays the soft key while not displaying the soft key. Upon identification of the occurrence of the display event, the soft key 230 may be displayed on the display 160 .

Referring back to FIG. 16 , if it is determined that N time has not elapsed in operation 1605 , the processor 120 of the electronic device may determine whether a soft key display event has occurred in operation 1613 . According to an embodiment, the soft key display event may be performed in response to at least one of a user's gesture input corresponding to the soft key display, a motion detection of the electronic device, a grip detection on the electronic device, a user touch detection on the display, or a system request. may be caused by

If it is determined in operation 1613 that a soft key display event has occurred, in operation 1615 , the processor 120 of the imputation device may restart the timer. After restarting the timer in operation 1615, the processor 120 of the electronic device may return to operation 1605 to determine whether N time for removing the soft key display has elapsed.

If it is determined in operation 1613 that the soft key display event has not occurred, the processor 120 of the electronic device returns to operation 1605 again without restarting the timer to determine whether N time for removing the soft key display has elapsed.

For example, if an event for displaying a soft key occurs before N time for removing the soft key has elapsed, the electronic device starts a timer for removing the soft key while continuing to display the soft key. can be restarted from

An electronic device according to an embodiment of the present disclosure includes a housing including a first side, a second side, a third side, and a fourth side, and is coupled to the housing, and the position of the side area surrounding the first side is changed A possible variable display, a sensor for detecting a change in the position of the side area, and a processor electrically connected to the variable display and the sensor, wherein the processor obtains data on a change in state of the variable display through the sensor, , calculate a change amount of the position of the side area based on the data on the state change, and change the display coordinates of the soft key based on the calculated change amount.

According to an embodiment of the present disclosure, the sensor is a time-of-flight (ToF) sensor, and the ToF sensor detects the amount of reflection of light emitted toward the first side to obtain data on the state change. can

According to an embodiment of the present disclosure, the sensor is an illuminance sensor, and the illuminance sensor is located below the variable display and detects the amount of light incident through the variable display to obtain data on the state change. can

According to an embodiment of the present disclosure, the sensor is a proximity sensor, and the proximity sensor is located under the variable display and detects a range in which a proximity object is detected when at least a part of the variable display is located inside the housing. Thus, it is possible to obtain data on the state change.

According to an embodiment of the present disclosure, the sensor is a touch screen panel, and the touch screen panel is located under the variable display, and detects hovering when at least a part of the variable display is located inside the housing to determine the state. You can get data about changes.

According to an embodiment of the present disclosure, the sensor is a rotation encoder, at least a part of the variable display is rolled through the rotation encoder, and the rotation encoder detects the rotation amount of the rotation encoder to change the state data can be obtained.

According to an embodiment of the present disclosure, the sensor is a bending sensor, and the bending sensor is located below the variable display and detects a bending amount to acquire data on the state change.

According to an embodiment of the present disclosure, further comprising a grip sensor, the processor detects a grip position with respect to the electronic device through the grip sensor, and controls the soft key based on the sensed grip position. It may be configured to change the display coordinates.

According to an embodiment of the present disclosure, the sensor is a sensor for detecting at least one of an iris, a face, and a grip, and the sensor detects at least one of an iris, a face, and a grip, and the state including data on a gaze area You can get data about changes.

According to an embodiment of the present disclosure, the processor displays the soft key on the variable display when starting the low power mode, and when there is no specific input until a specific time elapses after starting the low power mode, the soft key is set to the It may be configured to remove on the variable display and restart the timer if there is the specific input before the specific time elapses after starting the low power mode.

In a method of displaying a soft key on a variable display of an electronic device according to an embodiment of the present disclosure, data on a state change of the variable display is obtained through a sensor of the electronic device, and the state change a change amount of the position of the side area of the variable display is calculated based on the data for Based on the display coordinates of the soft key can be changed.

According to an embodiment of the present disclosure, the sensor is a time-of-flight (ToF) sensor, and the ToF sensor detects the amount of reflection of light emitted toward the first side to obtain data on the state change. can

According to an embodiment of the present disclosure, the sensor is an illuminance sensor, and the illuminance sensor is located below the variable display and detects the amount of light incident through the variable display to obtain data on the state change. can

According to an embodiment of the present disclosure, the sensor is a proximity sensor, and the proximity sensor is located under the variable display and detects a range in which a proximity object is detected when at least a part of the variable display is located inside the housing. Thus, it is possible to obtain data on the state change.

According to an embodiment of the present disclosure, the sensor is a touch screen panel, and the touch screen panel is located under the variable display, and detects hovering when at least a part of the variable display is located inside the housing to determine the state. You can get data about changes.

According to an embodiment of the present disclosure, the sensor is a rotation encoder, at least a part of the variable display is rolled through the rotation encoder, and the rotation encoder detects the rotation amount of the rotation encoder to change the state data can be obtained.

According to an embodiment of the present disclosure, the sensor is a bending sensor, and the bending sensor is located below the variable display and detects a bending amount to acquire data on the state change.

According to an embodiment of the present disclosure, the electronic device further includes a grip sensor, detects a gripping position with respect to the electronic device through the grip sensor, and based on the sensed gripping position, the soft key can change the display coordinates of

According to an embodiment of the present disclosure, the sensor is a sensor for detecting at least one of an iris, a face, and a grip, and the sensor detects at least one of an iris, a face, and a grip, and the state including data on a gaze area You can get data about changes.

According to an embodiment of the present disclosure, the soft key is displayed on the variable display when the low power mode is started, and the soft key is removed from the variable display when there is no specific input until a specific time elapses after starting the low power mode and restart the timer when there is the specific input before the specific time elapses after starting the low power mode.

Claims (20)

In an electronic device,
a housing comprising a first side, a second side, a third side, and a fourth side;
a variable display coupled to the housing and capable of changing a position of a side area surrounding the first side;
a sensor for detecting a change in the position of the side area; and
a processor electrically connected to the variable display and the sensor;
The processor is:
Obtaining data about the state change of the variable display through the sensor,
Calculate the amount of change in the position of the side area based on the data on the state change,
and change the display coordinates of the soft key based on the calculated amount of change. The method according to claim 1,
The sensor is a time-of-flight (ToF) sensor,
The ToF sensor detects a reflection amount of light emitted toward the first side surface to acquire data on the state change. The method according to claim 1,
The sensor is an illuminance sensor,
The illuminance sensor is located under the variable display and detects an amount of light incident through the variable display to obtain data on the state change. The method according to claim 1,
The sensor is a proximity sensor,
wherein the proximity sensor is located under the variable display, and detects a range in which a proximity object is sensed when at least a part of the variable display is located inside the housing to obtain data on the state change. The method according to claim 1,
The sensor is a touch screen panel,
wherein the touch screen panel is located under the variable display, and detects hovering when at least a part of the variable display is located inside the housing to acquire data on the state change. The method according to claim 1,
The sensor is a rotation encoder (encoder), at least a portion of the variable display is rolled through the rotation encoder,
The electronic device of claim 1, wherein the rotary encoder detects a rotation amount of the rotary encoder to obtain data on the state change. The method according to claim 1,
The sensor is a bending sensor,
The bending sensor is located under the variable display and detects a bending amount to obtain data on the state change. The method according to claim 1,
It further includes a grip sensor,
and the processor is configured to detect a grip position with respect to the electronic device through the grip sensor, and change the display coordinates of the soft key based on the sensed grip position. The method according to claim 1,
The sensor is a sensor for detecting at least one of the iris, face, or grip,
and the sensor detects at least one of red, face, and grip to acquire data on the state change including data on a gaze area. The method according to claim 1,
The processor displays the soft key on the variable display when starting the low power mode, and removes the soft key on the variable display if there is no specific input until a specific time elapses after starting the low power mode, and restart the timer if there is the specific input before the specific time has elapsed after starting. A method for displaying a soft key on a variable display of an electronic device, the method comprising:
acquiring data on a state change of the variable display through a sensor of the electronic device;
a change amount of a position of a side area of the variable display is calculated based on the data on the state change, wherein the side area corresponds to a portion of the variable display that surrounds a first side surface of the housing of the electronic device;
and changing display coordinates of the soft key based on the calculated amount of change. 12. The method of claim 11,
The sensor is a time-of-flight (ToF) sensor,
The method of displaying a soft key, wherein the ToF sensor detects a reflection amount of light emitted toward the first side and acquires data on the state change. 12. The method of claim 11,
The sensor is an illuminance sensor,
wherein the illuminance sensor is located under the variable display and detects an amount of light incident through the variable display to obtain data on the state change. 12. The method of claim 11,
The sensor is a proximity sensor,
The proximity sensor is located under the variable display, and when at least a part of the variable display is located inside the housing, detecting a range in which a proximity object is detected to obtain data on the state change, displaying a soft key Way. 12. The method of claim 11,
The sensor is a touch screen panel,
wherein the touch screen panel is located under the variable display, and detects hovering when at least a part of the variable display is located inside the housing to obtain data on the state change. 12. The method of claim 11,
The sensor is a rotation encoder (encoder), at least a portion of the variable display is rolled through the rotation encoder,
The method of displaying a soft key, wherein the rotary encoder detects a rotation amount of the rotary encoder to obtain data on the state change. 12. The method of claim 11,
The sensor is a bending sensor,
wherein the bending sensor is located below the variable display, and detects a bending amount to obtain data on the state change. 12. The method of claim 11,
The electronic device further includes a grip sensor,
A method of displaying a soft key by detecting a grip position with respect to the electronic device through the grip sensor, and changing the display coordinates of the soft key based on the detected grip position. 12. The method of claim 11,
The sensor is a sensor for detecting at least one of the iris, face, or grip,
The method of displaying a soft key, wherein the sensor detects at least one of a red face, a face, and a grip to obtain data on the state change including data on a gaze area. 12. The method of claim 11,
When starting the low power mode, the soft key is displayed on the variable display, and if there is no specific input until a specific time elapses after starting the low power mode, the soft key is removed from the variable display, and after starting the low power mode, the specific A method for displaying a soft key that restarts a timer if there is said specific input before the time has elapsed.

Images