KR102164453B1 - Object control method performed in device including transparent display, the device, and computer readable recording medium thereof - Google Patents

Abstract

An object control method performed by a device including a display, and a device and a recording medium thereof are provided. The object control method includes: displaying at least one object on the display; Receiving an input through an input interface of the display, the input interface including at least one of a first input interface and a second input interface; And in response to receiving the input, if the input interface is the first input interface and the predetermined input interface is the first input interface, performing a first function related to the predetermined input interface of the device, and the input And if the interface is the second input interface and the predetermined input interface is the second input interface, performing a second function related to the predetermined input interface.

Description

Object control method performed in device including transparent display, the device, and computer readable recording medium thereof in a device including a transparent display

The present invention relates to a device including a transparent display, and more particularly, to a method for controlling an object in a device including a transparent display, and a device and a recording medium therefor.

Recently, a next-generation display device including a transparent display has been developed. The transparent display has transparency through which a user can see through the display and objects behind the display.

A device including a transparent display may be a device that performs operations including information recognition, information processing, and information transfer. These operations can also be performed using a transparent electronic device.

The availability of transparent displays is creating a need for new and innovative ways of interacting with devices that include a transparent display and methods for extending the functionality of user interfaces for devices that include a transparent display.

An object to be solved by the present invention is to provide a method for controlling an object based on an input such as a touch to the front, rear, and both sides of a device including a transparent display, and a device and a recording medium thereof.

Another problem to be solved by the present invention is to provide a guide display method according to the state of an object or a display state in a device including a transparent display, and a device and a recording medium thereof.

Another problem to be solved by the present invention is to provide a method of controlling an object using a touch interface set for each object in a device including a transparent display, and a device and a recording medium thereof.

An object control method performed by a device including a display according to a preferred embodiment for achieving the above object includes: displaying at least one object on the display; Receiving an input through an input interface of the display, the input interface including at least one of a first input interface and a second input interface; If the input interface is the first input interface and the predetermined input interface is the first input interface in response to receiving the input, performing a first function related to the predetermined input interface of the device; And if the input interface is a second input interface and the predetermined input interface is the second input interface, performing a second function related to the predetermined input interface.

In the performing step, if the input interface is two of the first input interface and the second input interface, and the predetermined input interface is two of the first input interface and the second input interface, a second input interface related to the predetermined input interface It may further include the step of performing three functions.

The display may be a transparent display.

The first input interface may be at least one of a front surface and a rear surface of the transparent display, and the second input interface may be another one of the front surface and the rear surface of the transparent display.

The input may correspond to at least one object, and the predetermined input interface may correspond to the at least one object, and at least one of a display environment of the at least one object and an operation of the at least one object Can be set accordingly.

The at least one object may include a first object and a second object displayed in front of the first object, and the second function may include changing a display state of the second object. .

The at least one object may include a first window displayed in a first layer partially overlapping a second window displayed in a second layer under a first layer, The second function may include moving the second window.

The at least one object may include a first window displayed in a first layer partially overlapping a second window displayed in a second layer under a first layer, The second function may include displaying the second window on the first window.

The input may include touching more than a predetermined area of the second input interface, and the second function may include setting the transparency of the display to a maximum value. When an input of the second input interface touched for more than a predetermined time is not maintained for a predetermined time or longer, the second function includes returning to a screen before the touch step of the second input interface.

The method includes the steps of: in response to a request to edit an input interface related to the at least one object, displaying an edit window including information on the predetermined input interface related to the at least one object; And updating information about the predetermined input interface regarding the at least one object according to the user's input in the edit window.

The at least one object may include a first object and a second object, and the input is a first input regarding the first object received through the first input interface and a first input received through the second input interface. A second input relating to the second object, wherein the first function includes controlling the first object according to the first input received through the first input interface simultaneously with the second object, and , The second function may include controlling the second object according to the second input received through the second input interface simultaneously with the first object.

The input may include a gesture regarding the at least one object. The gesture may include moving a predetermined length or more in one direction through the predetermined input interface. The first function may include changing the predetermined input interface to the second input interface by rotating the at least one object in a direction corresponding to the gesture. The second function may include changing the predetermined input interface to the first input interface by rotating the at least one object in a direction corresponding to the gesture.

The second function may include variably displaying a guide corresponding to the input based on the state of the at least one object or the state of the display.

The second function may include displaying a guide corresponding to the position of the input.

The method may be embodied in a non-transitory computer-recordable medium containing instructions for performing the method.

A device for performing an object control method according to a preferred embodiment for achieving the above task includes: a display for displaying at least one object; An input interface for receiving an input through at least one of a first input interface and a second input interface; A storage device for storing a predetermined input interface; In response to receiving the input, if the input interface is the first input interface and the predetermined input interface is the first input interface, a first function related to the predetermined input interface is performed, and the input interface is a second input interface. If it is an input interface and the predetermined input interface is the second input interface, a processor that performs a second function related to the predetermined input interface may be included.

The processor further comprises a second input interface related to the predetermined input interface if the input interface is two of the first input interface and the second input interface, and the predetermined input interface is two of the first input interface and the second input interface. Can perform 3 functions.

The display may be a transparent display.

The first input interface may be at least one of a front surface and a rear surface of the transparent display, and the second input interface may be another one of the front surface and the rear surface of the transparent display.

The second function may include variably displaying a guide corresponding to the input based on a state of the at least one object or a state of the display.

The second function may include displaying a guide corresponding to the position of the input.

The input may include touching more than a predetermined area of the second input interface, and the second function may include setting the transparency of the display to a maximum value. When an input of the second input interface touching the predetermined area or more is not maintained for a predetermined time or longer, the second function includes returning to a screen before the touch of the second input interface.

The input may correspond to at least one object, and the predetermined input interface may be at least one of a first input interface and a second input interface, may correspond to the at least one object, and may correspond to the at least one object. It may be set according to at least one of a display environment and an operation of the at least one object.

The at least one object may include a first object and a second object, and the second function may include changing a display state of the second object displayed on the front surface of the first object.

The at least one object may include a first window displayed in a first layer partially overlapping a second window displayed in a second layer under a first layer, The second function may include moving the second window.

The at least one object may include a first window displayed in a first layer partially overlapping a second window displayed in a second layer under a first layer, The second function may include moving the second window from the second layer below the first layer, and displaying the second window above the first window.

The processor, in response to a request to edit an input interface for the at least one object, displays an edit window including information on the predetermined input interface for the at least one object; And updating information on the predetermined input interface regarding the at least one object in the edit window according to the user's input.

1 is a diagram for describing objects and a user input interface setting of an object according to an exemplary embodiment.
2A is a block diagram of a device including a transparent display according to an exemplary embodiment.
2B is a cross-sectional view of a detailed structure of the transparent display of FIG. 2A.
3 is a flowchart of an object control method performed by a device including a transparent display according to an exemplary embodiment.
4 to 20 illustrate examples of screens displayed in the object control method of FIG. 3.
21 is a flowchart of an object control method performed by a device including a transparent display according to another exemplary embodiment.
22A and 22B illustrate examples of screens displayed while changing the user input interface using the object control method of FIG. 21.
23 is a flowchart illustrating a method of displaying a guide display in a device including a transparent display according to another exemplary embodiment.
24 illustrates examples of a screen displaying a guide.
25 is a block diagram of a device including a transparent display according to an exemplary embodiment.
26 is a diagram illustrating software stored in a storage unit of a device including a transparent display according to an exemplary embodiment.

Since the present invention can apply various transformations and have various embodiments, specific embodiments are illustrated in the drawings and will be described in detail in the detailed description. However, this is not intended to limit the present invention to a specific embodiment, it should be understood to include all conversions, equivalents, or substitutes included in the spirit and scope of the present invention. In describing the present invention, when it is determined that a detailed description of a related known technology may obscure the subject matter of the present invention, a detailed description thereof will be omitted.

Terms such as first and second may be used to describe various components, but the components should not be limited by terms. The terms are only used for the purpose of distinguishing one component from another component.

The terms used in the present application are used only to describe specific embodiments, and are not intended to limit the present invention. The terms used in the present application have selected general terms that are currently widely used as possible while considering the functions of the present invention, but this may vary according to the intention of a technician working in the field, precedents, or the emergence of new technologies. In addition, in certain cases, there are terms arbitrarily selected by the applicant, and in this case, the meaning of the terms will be described in detail in the description of the corresponding invention. Therefore, the terms used in the present invention should be defined based on the meaning of the term and the overall contents of the present invention, not a simple name of the term.

Singular expressions include plural expressions unless the context clearly indicates otherwise. In the present application, terms such as "comprise" or "have" are intended to designate the presence of features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, but one or more other features. It is to be understood that the presence or addition of elements or numbers, steps, actions, components, parts, or combinations thereof, does not preclude in advance.

An object referred to throughout the specification refers to all components or information being displayed, and means components or information that can be selected by a user input and controlled according to a user's intention. The object is, for example, a first image, a second image included in the first image, a background image, an icon, a folder icon, an icon included in the folder icon, text, and a pop-up window. -up window), an application execution window, a content included in the application execution window, a list, an item included in the list, content, etc., but are not limited thereto.

The object may be displayed by a user input (a user input such as a touch input) through a user input interface (a user input interface such as a touch input surface) set for the object. For example, the object may be hidden according to a function being performed in a device including a transparent display, and then activated according to a user's input and displayed.

1 is a diagram for describing objects and a user input interface set for each of the objects according to an exemplary embodiment.

Referring to FIG. 1, when a list is displayed, objects include a scroll box, an item box, and an icon included in the item box. A user can input a command for an object using a user input interface.

1 shows an example of a hierarchical relationship between objects. The scroll box shown in FIG. 1 is an object of the highest layer, and an icon is an object of the lowest layer. In other words, FIG. 1 shows a case where a plurality of item box objects are included in a sub-layer of a scroll box object, and an icon object is included in a sub-layer of each item box object.

As shown in FIG. 1, a user input interface for each object may be set in advance. The user input interface of each object is set when a program for operating each object is produced, but may be set according to a user input interface set for an object of a higher layer.

For example, when the user input interface of the icon shown in FIG. 1 is not set when a program including each object is produced, the user input interface of the icon may be set according to the user input interface set in the item box. This is because an item box is a parent layer of the icon, and the icon is a sub-layer of the item box. Setting of the user input interface of the icon may be performed when the icon is displayed. When the user input interface of the item box and the icon is not set when a program including each object is produced, the user input interface of the scroll box and the icon may be set according to a preset user input interface of the scroll box. This is because the item box and the icon are sub-layers of the scroll box. Even in this case, when the item box and icon are displayed, the user input interface of the item box and icon may be set according to the user input interface of the scroll box.

As shown in Fig. 1, when the user input interface for each object is set, the scroll box operates according to the user's input through the rear input interface, and the item box_1 is the user's input through the front input interface. It operates according to the input, the icon is operated by a user's input input through the front input interface or/and the rear input interface, and the text is operated by the user's input input through the front input interface.

Accordingly, when the user input interface of each object is set as shown in FIG. 1, an operation error in which the item box or icon is incorrectly selected as the item box or icon is positioned at the position where scrolling is stopped does not occur. This is because the scroll box is operated by receiving input through the rear input interface, and the item box can receive input only through the rear user interface, and the icon can receive input through both the front and rear input interfaces. to be.

Also, the user input interface set for the object of FIG. 1 may be set based on at least one of a display environment of the object and an operation of the object, but the preferred embodiment is not limited thereto.

For example, an object with a narrow user input area (or touch area) is set as a front input interface that is relatively easy for the user to control, and an object with a large user input area is set as a rear input interface that is relatively difficult for the user to control. Can be set. Objects that perform actions frequently used by the user are set as the front input interface that is relatively easy for the user to control, and objects that perform actions that are not frequently used by the user are rear input interfaces that are relatively difficult for the user to control. Can be set to

When an object is set to perform multiple operations according to a user input interface, a double-sided input interface (for example, a front input interface and a rear input interface) may be set as the user input interface of the object. . The object based on the execution operation may be executed according to a user's input through a front input interface, a rear input interface, and a dual side input interface. Each of the objects shown in FIG. 1 may be referred to as a widget. The above-described front input interface, rear input interface, and dual input interface may be referred to as a front input mode, a back input mode, and a dual-side input mode, respectively. have.

The input interface or input mode referred to throughout the specification may be a touch surface of a transparent display. Therefore, the front input interface or the front input mode may be the front touch surface of the transparent display. The rear input interface or rear input mode can be the rear touch surface of the transparent display.

The user's input referred to throughout the specification may be input information provided according to the user's gesture. The user's gesture may be defined according to the position, direction, speed, number of touch points, and the like of the user's input. That is, when the input device is based on a touch screen, the user's gesture is, for example, tap, touch and hold, double tap, drag, panning, and flick. (flick), drag and drop (drag and drop), and may include a sweep (sweep), and the like, but is not limited thereto.

The user's gesture on the touch screen may be performed using a user's finger or a touch tool (eg, a stylus pen). The user's gesture may be received from a camera or other gesture detection device. In this case, the user's gesture may include a chess gesture performed in a space determined based on an image captured using a camera. The gesture performed in a space based on a user's motion may be determined by a device movement (eg, device shaking).

When the input device includes at least one of a physical button, a dial, a slider switch, a joystick, a click wheel, or the like, the user's input may depend on the user's physical control of the input device. When the input device is based on an audio input device, the user input may rely on natural language based user's speech recognition.

2A is a functional block diagram of a device 200 including a transparent display 201 according to an exemplary embodiment. The device 200 including the transparent display 201 shown in FIG. 2A is, for example, a smart phone, a notebook, a tablet PC, a phablet, and a mobile terminal. (Mobile Device), a handheld device or a handheld PC, PDA (Personal Digital Assistant), etc. may be, but is not limited thereto. The transparent display 201 is a smart window, high value-added glass, glass as a functional automobile component, a vehicle dashboard, a navigator, a security electronic device, a solar cell, a game console, a toy, a show window. It can be applied to various fields such as.

2A, the device 200 includes a transparent display 201, a user input unit 205, a sensing unit 206, a camera 207, an audio input unit 208, an audio signal processing unit 209, and an audio output unit. 210, a storage unit 211, a communication unit 212, a port 213, a power supply unit 214, and a processor 215. The configuration of the device 200 including the transparent display is not limited as shown in FIG. 2A.

The transparent display 201 may be referred to as a touch screen. The transparent display 201 includes a display unit 202 that displays objects, a front input unit 203 installed on the front of the display unit 202 to receive a touch-based user input (for example, a front input interface), and a display unit. It includes a rear input unit 204 (for example, a rear input interface) installed on the rear side of 202 and receiving a touch-based user input. The front input interface and rear input interface may also receive input from many other users based on imaging sensors, infrared sensors, proximity sensors, and the like.

The display unit 202 outputs information including an object processed by the device 200 including a transparent display. The information may include information other than objects. Although information other than the object is displayed, it may be set so that it cannot be selected by user input.

The display unit 202 is composed of a transparent device, and the transparency may be adjusted by adjusting the light transmittance of the transparent device or by adjusting the RGB (Red, Green, Blue) value of each pixel. In addition, the display unit 202 may have a structure in which an organic light-emitting diode and a liquid crystal display are combined. The organic light emitting diode may be positioned adjacent to the front input unit 203, and the liquid crystal display may be cooperatively positioned on the rear input unit 204. In the case of having the above-described combined structure with the display unit 202, the display unit 202 may be set to an opaque state because the liquid crystal display blocks light when power is supplied while maintaining a transparent glass-like state. .

The display unit 202 is a user's gesture or touch pattern through the front input unit 203 or/and the rear input unit 204, or a user input input through the user input unit 205 or a user input through the sensing unit 206 A screen that responds to an input, a user input input through the camera 207, and a user input input through the audio input unit 208 is displayed. The screen displayed on the display unit 202 includes a user interface (UI) screen or a graphical user interface (GUI) screen.

The display unit 202 includes a liquid crystal display, a thin film transistor-liquid crystal display, an organic light-emitting diode, a flexible display, and a three-dimensional display. 3D display), an Active-Matrix Organic Light-Emitting Diode (AM OLED), a volumetric display, and a holographic display may be implemented, but are not limited thereto. There may be two or more transparent displays 201 depending on the implementation type of the device 200.

The front input unit 203 and the rear input unit 204 include a touch sensor for receiving a touch based on a user's input. Alternatively, there may be two or more touch screens or other types of sensors for receiving user input. Sensors are designed to provide an unlimited number of input interfaces that are used to perform different functions by separating the user's input into each input interface or by simultaneously combining the user's input into multiple input interfaces. Can be used.

According to a preferred embodiment, a sensor for detecting a touch of the front input unit 203 and the rear input unit 204 includes a sensor for detecting a user's gesture or pattern in the front input unit 203 and the rear input unit 204. The sensor for sensing touch senses at least one of drag, flick, tap, touch and hold, double tap, pan, and sweep recognized as the user's gesture described above by the front input unit 203 and the rear input unit 204 One sensing signal can be generated. A tactile sensor may be used as an example of a sensor that senses a touch of the front input unit 203 and the rear input unit 204. The tactile sensor can detect various information such as roughness of a contact surface, hardness of a contact object, and temperature of a contact point.

The front input unit 203 and the rear input unit 204 may be configured as a pressure-sensitive (resistive film) type or a capacitive (or capacitance) type. Coordinate information can be used to recognize a user's input in the front input unit 203 and the rear input unit 204. The x-axis coordinates and the y-axis coordinates of the location where the touch is generated can be recognized as a user input.

The user's input may be generated according to a user's request or a user's selection based on a user gesture. The user's gesture may be determined according to various combinations of various methods, for example, the number of touches, a touch pattern, a touch area, and a touch intensity. The user's input may be input by a user's finger or a touch tool such as a stylus pen. The user's finger or stylus pen may be referred to as an external input device.

The touch of the front input unit 203 and the rear input unit 204 is a case where a pointer is touched on the touch panel. The touch input may include multi-touch or touch points. In the proximity-touch of the front input unit 203 and the rear input unit 204, the pointer does not actually touch the front input unit 203 and the rear input unit 204, and the front input unit 203 and the rear input unit ( 204).

The pointer is a tool for touching or near-touching specific portions of the front input unit 203 and the rear input unit 204, and the above-described user's finger or a stylus pen may be used.

The transparent display unit 201 may be implemented in various forms such as a transparent liquid crystal display (LCD) type, a transparent thin-film electroluminescent panel (TFEL) type, a transparent OLED type, a projection type, and the like, according to an embodiment. Hereinafter, various embodiments of the structure of the transparent display unit 201 will be described.

The transparent LCD type is a transparent display implemented by removing a backlight unit from an LCD device currently being used and using a pair of polarizing plates, optical films, transparent thin film transistors, and transparent electrodes.

The transparent TFEL type means a device using an AC type inorganic thin film EL display (AC-TFEL) composed of a transparent electrode, an inorganic phosphor, and an insulating film. AC-TFEL is a display that excites the phosphor to emit light as the accelerated electrons pass inside the inorganic phosphor. When the transparent display unit 201 is implemented in the form of a transparent TFEL, the processor 215 may determine an information display position by adjusting the electrons to be projected to an appropriate position. Since the inorganic phosphor and the insulating film have transparent properties, a very transparent display can be implemented.

The transparent OLED type means a transparent display device using an OLED capable of self-emission. Since the organic emission layer is transparent, if both electrodes are used as transparent electrodes, it can be implemented as a transparent display device. The OLED emits light by injecting electrons and holes from both sides of the organic emission layer and as they are combined in the organic emission layer. Transparent OLED devices use this principle to inject electrons and holes at a desired location to display information.

2B is a diagram showing a detailed configuration example of a transparent display unit implemented in a transparent OLED type. For convenience of explanation, a reference numeral 140-1 is assigned to a transparent display implemented in a transparent OLED (Organic Light-Emitting Diodes) type.

2B, the transparent display 140-1 includes a transparent substrate 141-1, a transparent transistor layer 142-1, a first transparent electrode 143-1, and a transparent organic light-emitting layer. layer) 144-1, a second transparent electrode 145-1, and a connection electrode 146-1.

The transparent substrate 141-1 may be made of glass or a polymer material such as plastic having a transparent property. The material of the transparent substrate 141-1 may be determined according to an environment in which the transparent display device 100 is to be used. For example, polymer materials can be used in portable display devices because they are light and flexible, and glass can be used in store showcase windows or general windows.

The transparent transistor layer 142-1 refers to a layer including a transistor manufactured by replacing opaque silicon of an existing thin film transistor with a transparent material such as transparent zinc oxide or titanium oxide. In the transparent transistor layer 142-1, a source, a gate, a drain, and various dielectric layers 147-1 and 148-1 are provided, and a connection electrode electrically connecting the drain and the first transparent electrode 143-1 ( 146-1) may also be provided. In FIG. 2B, only one transparent transistor including a source, a gate, and a drain is shown in the transparent transistor layer 142-1, but includes a plurality of transparent transistors evenly distributed over the entire area of the display surface. The processor 215 may display information by applying a control signal to the gates of each of the transistors in the transparent transistor layer 142-1 to drive the corresponding transparent transistor.

The first transparent electrode 143-1 and the second transparent electrode 145-1 are disposed in opposite directions with respect to the transparent organic emission layer 144-1. The first transparent electrode 143-1, the transparent organic emission layer 144-1, and the second transparent electrode 145-1 form transparent organic light-emitting diodes (Organic Light-Emitting Diodes).

Transparent organic light emitting diodes are classified into passive (Passive Matrix OLED) and active (Active Matrix OLED) according to the display driving method. The PMOLED has a structure in which a portion where the first and second transparent electrodes 143-1 and 145-1 cross each other forms a pixel. On the other hand, AMOLED has a structure with a thin film transistor (TFT) driving each pixel. In Fig. 2b, the active type is shown.

The first transparent electrode 143-1 and the second transparent electrode 145-1 each include a plurality of line electrodes, and alignment directions of the line electrodes are formed perpendicular to each other. For example, if the line electrodes of the first transparent electrode 143-1 are arranged in the horizontal direction, the line electrodes of the second transparent electrode 145-1 are arranged in the vertical direction. Accordingly, a plurality of crossing regions are formed between the first transparent electrode 143-1 and the second transparent electrode 145-1. A transparent transistor is connected to each crossing region as shown in FIG. 2B.

The processor 215 uses a transparent transistor to form a potential difference for each cross region. Electrons and holes from each electrode flow into the transparent organic light-emitting layer 144-1 in the cross region where the potential difference is formed, and are combined to emit light. The crossing region in which the potential difference is not formed does not emit light, so that the rear background is transparently reflected.

Indium tin oxide (ITO) may be used as the first and second transparent electrodes 143-1 and 145-1. Alternatively, a new material such as graphene may be used. Graphene refers to a material having a transparent property with carbon atoms connected to each other to form a honeycomb-shaped planar structure. In addition, the transparent organic light-emitting layer 144-1 may also be implemented with various materials.

As described above, the transparent display 201 may be implemented in a transparent liquid crystal display (LCD) type, a transparent thin-film electroluminescent panel (TFEL) type, a transparent OLED type, and a projection type. Projection type refers to a method of projecting and displaying an image on a transparent screen, such as a HUD (Head Up Display).

The user input unit 205 generates input data (or control data) for controlling the operation of the device 200 including a transparent display and a user input. The user input unit 205 includes a key pad, a dome switch, a touch pad, a jog wheel, a jog switch, a hardware (H/W) button, and the like. Can include.

The sensing unit 206 detects the current state of the device 200, such as the location of the device 200, the presence of a user's contact, the orientation of the device 200, and acceleration or deceleration of the device 200. It generates a sensing signal to control the operation.

The sensing unit 206 includes sensors other than sensors for sensing a touch of the front input unit 203 and the rear input unit 204. For example, the sensing unit 206 may include a proximity sensor. The proximity sensor detects the presence or absence of an external object approaching a predetermined detection surface (for example, a front input interface, a rear input interface, etc.) or an external object near the detection surface using the force of an electromagnetic field or infrared rays. It refers to a sensor that detects without physical contact. The external object is an object located outside the device 200. Examples of the proximity sensor include a transmission type photoelectric sensor, a diffuse reflection type photoelectric sensor, a mirror emission type photoelectric sensor, a high frequency oscillation type proximity sensor, a capacitive type proximity sensor, a magnetic type proximity sensor, and an infrared proximity sensor.

The camera 207 processes image frames such as still images or moving pictures obtained by an image sensor in a video call mode or a photographing mode. In addition, the processed image frame may be displayed on the display unit 202. The image frame processed by the camera 207 may be stored in the storage unit 211 or transmitted to an external device through the communication unit 212 or the port 213.

The camera 207 may also be configured to move the photographing angle of the lens in the left or right direction according to the user's gesture corresponding to the drag through the rear input unit 204. Two or more cameras 207 may be provided depending on the configuration of the device 200. In addition, the camera 207 may be used as an input device for recognizing a user's spatial gesture.

The audio input unit 208 receives an external sound signal in a call mode, a recording mode, or a voice recognition mode. The audio input unit 208 may be configured as a microphone, for example. The audio input unit 208 may be configured to include various noise removal algorithms for removing noise generated in a process of receiving an external sound signal.

The sound signal input using the audio input unit 208 may be a user's voice input used to control an object displayed on the transparent display 201 according to a preferred embodiment. That is, the sound signal input using the audio input unit 208 may be user input information based on a user's voice recognition based on a user's natural language or a user's spoken language. An external sound signal input through the audio input unit 208 may be stored in the storage unit 211 or transmitted to an external device through the communication unit 212 or the port 213.

The audio signal processing unit 209 provides an interface between the audio input unit 208 and the processor 215 and between the audio output unit 210 and the processor 215. For example, the audio signal processing unit 209 converts an audio signal received from the audio input unit 208 into audio data that can be processed by the processor 215 and transmits it to the processor 215. In addition, the audio signal processing unit 209 converts the audio data transmitted from the processor 215 into an electrical sound signal and transmits the converted audio data to the audio output unit 210.

The audio output unit 210 outputs an audio signal or an audio signal received from the audio signal processing unit 209 in a call mode or an audio reproduction mode. The audio output unit 210 may be configured as a speaker. The audio input unit 208 and the audio output unit 210 may be configured as a headset.

Depending on the user interface function between the device 200 and the user, the transparent display 201, the user input unit 205, the sensing unit 206, the camera 207, and the audio input unit 208 will be referred to as an input device or an input/output device. I can. For example, when the user interface function between the device 200 and the user includes a touch screen function, a voice recognition function, and a spatial gesture recognition function, the user input unit 205, the sensing unit 206, and the camera 207 , And the audio input unit 208 may be referred to as an input device, and the transparent display 201 may be referred to as an input/output device.

The storage unit 211 stores at least one program and a resource configured to be executable by the processor 215. At least one program is a program that executes an object control method or a guide display method according to a preferred embodiment, an operating system program of the device 200, an application set in the device 200, and a device 200. Includes a program necessary to perform various functions of (for example, communication functions and display functions).

The resource includes information necessary to execute the above-described programs, user interface screen information for object control mentioned in the preferred embodiments, object control function information corresponding to a user's input, and the like. The user interface screen information for object control may include, for example, an edit window for the input interface of each object. In addition, the resource includes information on a predetermined input interface set for each object.

A program that executes an object control method or a guide display method stored in the storage unit 211 may be configured to be included in an application program including an object. A program executing the guide display method may be configured as a program independent from the application program. That is, the program that executes the guide display method may be executed when the operation mode of the device 200 is set to execute the guide display method, or may be executed by default when the device 200 is powered on, regardless of the application being executed. For example, a program for executing the guide display method may be a part of the operating system of the device 200.

According to various exemplary embodiments described below, the program may be executed after executing any application, and may not be executed when the execution of any application is terminated. For example, when a program that executes a control method for objects included in a phone book application or a program that executes a guide display method is executed as part of the phone book application, and the execution of the phone book application ends , The operation of the program executing the control method for objects included in the phonebook application and the program executing the guide display method may be terminated. However, regardless of whether the phonebook application is executed, the guide display method may be executed.

The storage unit 211 includes a storage unit for storing at least one program required to perform various functions of the device 200 including the operating system of the device 200, and an object control method or guide display method according to various preferred embodiments. One or more programs and resources to be executed, and a storage unit for storing the applications may be independently configured.

The storage unit 211 may include a high-speed random access memory, a magnetic disk storage device, a nonvolatile memory such as a flash memory, or other nonvolatile semiconductor memory. Therefore, the storage unit 211 may be referred to as a memory.

The communication unit 212 includes a wireless Internet, a wireless intranet, a wireless telephone network, a wireless LAN (LAN), a Wi-Fi network, a Wi-Fi Direct (WFD) network, a 3G (Generation) network, and a 4G (4) network. Generation) LTE network, Bluetooth network, infrared communication (IrDA, Infrared Data Association) network, RFID (Radio Frequency Identification) network, UWB (Ultra WideBand) network, Zigbee network, NFC (Near Field Communication) network It is configured to transmit and receive data with an external electronic device (not shown) and a server (not shown) through a wireless network such as.

The communication unit 212 may include at least one of a broadcast reception module, a mobile communication module, a wireless Internet module, a wired Internet module, a short-range communication module, and a location information module, but is not limited thereto.

The broadcast receiving module receives a broadcast signal and/or broadcast-related information from an external broadcast management server through a broadcast channel. Broadcast channels may include satellite channels and terrestrial channels. The mobile communication module transmits and receives a wireless signal with at least one of a base station, an external electronic device (not shown), and a server (not shown) on a mobile communication network. The wireless signal may include a voice call signal, a video call call, or various types of data according to transmission/reception of text/multimedia messages. The wireless Internet module refers to a module for communicating through a wireless Internet connection. The wired internet module means a module for wired internet access.

The short-range communication module refers to a module for short-range communication. As a short-range communication technology, Bluetooth, RFID, infrared communication, UWB, Zigbee, WFD, NFC, etc. may be used. The location information module is a module for checking or obtaining the location of the device 200. An example is a Global Position System (GPS) module. The GPS module receives location information from a plurality of satellites. The location information may include coordinate information expressed in latitude and longitude.

The port 213 may transmit and receive data to and from an external device (not shown) using a plug and play interface such as a Universal Serial Bus (USB) port (not shown). The plug-and-play interface refers to a module that automatically plays when an external device is inserted into the plug of the device 200.

The power supply 214 supplies power to various components of the device 200. The power supply unit 214 includes one or more power sources such as a battery and an AC power source. The device 200 does not include the power supply unit 214 and may include a connection unit (not shown) that can be connected to an external power supply unit (not shown).

The processor 215 controls the overall operation of the device 200 and may include one or more processors. For example, the processor 215 may be divided into a plurality of processors according to the function of the device 200.

The processor 215 includes a transparent display 201, a user input unit 205, a sensing unit 206, a camera 207, an audio input unit 208, an audio output unit 210, an audio signal processing unit 209, and a storage unit. (211), the communication unit 212, and the port 213 can be controlled overall. Therefore, the processor 215 may be referred to as a controller, a microprocessor, a digital signal processor, or the like. In addition, the processor 215 is a transparent display 201 corresponding to the input device, the user input unit 205, the sensing unit 206, the camera 207, the user input information input using the audio input unit 208 and transparent A user interface based on the display 201 may be provided.

The processor 215 may execute at least one program related to an object control method or a guide display method according to an exemplary embodiment. The processor 215 reads the program from the storage unit 211 and executes it, or downloads and executes the program from an external device such as an application providing server (not shown) or a market server (not shown) connected through the communication unit 212 I can make it. A program related to the object control method may be included in an application-specific program as described above.

The processor 215 may include an interface function unit between the various functional modules in the device 200 and the processor 215. The operation of the processor 215 related to the method for controlling an object or a method for displaying a guide according to an exemplary embodiment may be performed as in the flowcharts illustrated in FIGS. 3, 21 and 23 to be described later.

3 is a flowchart of an object control method performed by the device 200 according to an exemplary embodiment. The object control method of FIG. 3 may be performed by the processor 215 of FIG. 2.

The processor 215 displays at least one object according to an application execution or an operation of the device 200 (S301).

The processor 215 receives a user's input through at least one of the front input unit 203 and the rear input unit 204 for the displayed object (S302).

The processor 215 determines whether the received user's input is input to at least one object being displayed through a predetermined input interface (S303). According to a preferred embodiment, the processor 215 determines whether a user's input is received through a front input interface, a rear input interface, and both a front input interface and a rear input interface. Next, the processor 215 compares a predetermined input interface of the object with the input interface detected from the user input.

The predetermined input interface may be set in advance for each object as described in FIG. 1 or may be set when an object is displayed according to a predetermined input interface of an object of an upper layer (parent layer). A predetermined input interface for the object being displayed may be mapped when the object is displayed.

The processor 215 controls the object based on the determination result in step S303 (S304). According to a preferred embodiment, the processor 215 performs a function corresponding to a predetermined input interface matching the detected input interface based on a comparison between the predetermined input interface of the object and the input interface detected from the user input. Can be done.

4 is an example of a screen for an object control process in FIG. 3. Referring to FIG. 4, when a screen as shown in 410 is displayed on the transparent display 201, the touch position of the rear input unit 204 corresponding to the folder icon 404 is As a long press (for example, a touch and hold gesture) is performed for a predetermined time longer than the time required to detect a touch, the screen 410 is changed to the screen 420. That is, the folder icon 404 on the screen 410 is changed to the screen 420 including the folder icon 404' enlarged in the direction of the front input unit 203. When the press on the touch position is released, the screen 420 having the enlarged folder icon 404' returns to the screen 410. When a plurality of objects 402 to 410 are displayed in step S301 of FIG. 3, the screens 410 and 420 illustrated in FIG. 4 are the first object 401 whose input interface is the rear input unit 204 in step S304. ) (Window), the display state of the second object (folder icon 404) displayed on the front surface of the first object 401 is changed according to the user's input 411.

At this time, if the predetermined input interface for the folder icon 404 corresponding to the second object is the front input unit 203, the user may select the expanded folder icon 404' based on the user's input through the front input unit 203. You can control the execution of icons included in ).

5 is an example of another screen of the object control process in FIG. 3. After a predetermined input interface touches the touch position of the rear input unit 204 corresponding to the folder icon 404 on the window 501, which is the rear input unit 204, the folder icon is dragged in the direction of 502. This is the case where 404 is moved to the position shown in 520. Therefore, when moving the icon 404, it is possible to prevent the icon 404, the adjacent icon 407, and the entire window 501 from being covered by the user's hand. This is because the user's hand performs a gesture behind the transparent display 201. In addition, icons on the screen 520 may be automatically rearranged due to the movement of the folder icon 404.

5, the display state of the second object 404 displayed on the front surface of the first object 501, which is the rear input unit 204, has a predetermined input interface as shown in FIG. This is an example that changes according to user input.

6 is an example of another screen of the object control process in FIG. 3. When a predetermined input interface is touched on the window 401, which is the rear input unit 204, the touch position of the rear input unit 204 corresponding to the icon 402 is touched, indicating the touch time or the touch strength (for example, touch pressure). A progress bar is displayed as 601 of the screen 620. As the touch time is maintained or the touch intensity increases, the screen 620 is changed to the screen 630 and the screen 630 is changed to the screen 640. When the progress bar 601 is all filled as shown in the screen 640, the icon 402 may be deleted as shown in the screen 650.

As shown in FIG. 4, in the screens 610 to 650 of FIG. 6, the display state of the second object 402 displayed on the front of the first object 401 is This is an example of being changed (deleted) according to the input. That is, the second object 402 in which the rear input unit 204 is set as a predetermined input interface is deleted according to an input through the rear input unit 204.

7 is an example of another screen of the object control process in FIG. 3.

The screen 710 and screen 720 of FIG. 7 are examples of moving the window 711 from a lower layer to an upper layer. When the plurality of windows 711 and 712 overlap each other and are displayed, the window 712 is moved to a lower layer according to a user's input through the rear input unit 204. Accordingly, on the screen 720, the window 712 is displayed in a lower layer of the window 711 and is hidden.

The predetermined input interface of the lower layer window 711 is the front input unit 203, the predetermined input interface of the background layer (not shown) of the lower layer window 711 is the rear input unit 204, and the background layer A lower layer window 711 may be moved to an upper layer and displayed by a user's gesture to the rear input unit 204 (not shown).

The screen 730 and screen 740 of FIG. 7 are examples of screens in which a transparent state is converted into an opaque state according to a user's input through the rear input unit 204. That is, when the screen as shown in 730 is displayed and the predetermined input interface of the object 731 is the rear input unit 204, the user's input through the rear input unit 204 starts at the top of the screen and As it is dragged to the bottom, the top of the screen changes from a transparent state to an opaque state like the screen 740.

The screen 730 and the screen 740 of FIG. 7 will be understood as an example in which the state of the display unit 202 is changed according to a user's input to the first object 731, which is the rear input unit 204 in a predetermined input interface. I can.

8 is an example of another screen of the object control process in FIG. 3.

8 is a case in which a plurality of objects are overlapped and displayed, and a predetermined input interface of the object 802 on the rear is the rear input unit 204, and the predetermined input interface of the object 803 on the front is the front input unit 203. .

Therefore, after touching the touch position of the rear input unit 204 corresponding to the object 802, and dragging in the direction 805, the object 802 on the rear surface is moved from the screen 810 to the screen 820 Is moved.

After touching the touch position of the front input unit 203 corresponding to the object 803, the object 803 in the front is moved from the screen 810 to the screen 830 as shown in the screen 830 by dragging in the direction of 831 do.

The front input unit 203 is set as a predetermined input interface for a plurality of objects 802 and 802, and the object 802 is a user's input to the rear input unit 204, which is a predetermined input interface of the background object 801. It can be moved according to the input. When performed in this way, the screens 810 and 820 of FIG. 8 are the front of the first object (for example, the background object 801) whose input interface is the rear input unit 204 as in FIG. 4 described above. The display state of the second object (for example, the object 802) being displayed in is changed according to the user's input to the first object 801.

9 is an example of another screen of the object control process in FIG. 3.

9 is an example of a device lock screen. That is, in FIG. 9, another home screen (or menu screen) may be displayed when unlocking according to a user input to the front input unit 203 and unlocking according to a user input to the rear input unit 204. .

The screen 910 and the screen 920 of FIG. 9 are examples of screens when unlocked according to a user's input to the front input unit 203. The screen 930 and the screen 940 of FIG. 9 are examples of screens when unlocked according to a user's input to the rear input unit 203.

9 may be applied even when the front unlock user and the rear unlock user are different.

The front input unit 203 and the rear input unit 204 are a predetermined input interface of an object (a screen capable of performing an unlock function, 911), and the screens 910 and 920 are inputted through the front input unit 203. In response to an input, a first function set in advance for the object 911 is performed, and the screens 930 and 940 are displayed in response to a user's input through the rear input unit 204. 2 performs a function.

10 is an example of another screen of the object control process in FIG. 3.

10 shows the type and size of the selected text by dividing the rear input unit 204 into three areas 1021, 1022, and 1023 as shown in 1020 as the text 1001 is selected through the front input unit 203. , And color can be set. A text type, text size, and text color that can be selected by performing a gesture such as a scrolling gesture based on a back touch in each of the three areas 1021, 1022, and 1023 may be reflected in the selected text 1001 and displayed.

In the screens 1010 and 1020 of FIG. 10, as in FIG. 9, a predetermined input interface for the object 1001 is a front input unit 203 and a rear input unit 204, and the front input unit 203 of the screen 1010. ) In response to the user's input through the object 1001 (for example, the function of dividing into three areas as described above) is performed, and the user through the rear input unit 204 of the screen 1020 In response to an input of, a second function (eg, text type setting, text size setting, text color setting) of the object 1001 is performed.

11 is an example of another screen of the object control process in FIG. 3.

11 shows a picture object 1101 is photographed with a camera 207, and a screen 1110 is a zoom-in gesture for the front input unit 203 of the object 1101 of the screen 1110 ( 1102) is performed and enlarged to the screen 1111, and a zoom-out gesture 1103 for the front input unit 203 of the object 1101 of the screen 1120 is performed to display the screen 1121. It is reduced.

In addition, after touching the rear input unit 204 of the object 1101, the shooting angle of the lens of the camera 207 is adjusted by dragging in the direction of (1104) so that the screen to be photographed is displayed on the screen 1100 of FIG. The screen is changed to the screen 1130, after touching the rear input unit 204 of the object 1101, and dragging in the direction (1105), the shooting angle of the lens of the camera 207 is adjusted and the screen to be captured is shown in FIG. The screen 1100 of is changed to the screen 1140.

In the screens 1110 to 1140 of FIG. 11, as in FIG. 9, a predetermined input interface for the object 1101 is a front input unit 203 and a rear input unit 204, and the front input unit 203 is In response to a user's input through the object 1101, a preset first function (zoom-in or zoom-out) is performed, and in response to a user's input through the rear input unit 204, the dictionary of the object 1101 The second function (changing the shooting screen or changing the shooting area) set in is performed.

11 may be applied to a panorama function. The angle of the camera lens can be changed by a gesture input to the rear input unit 204 used to capture a panoramic view in the picture object 1101. In addition, a panoramic photo can be viewed while moving a panoramic photo or a view angle of the photo by using a gesture input through the rear input unit 204.

12 is an example of another screen of the object control process in FIG. 3. 12 is an example of a screen related to a volume control function when the device 200 is playing media content and a system volume control function such as a ring tone control.

When volume control is requested through the user input unit 205, a screen 1200 is displayed, and system volume control is performed by a user's input to the front input unit 203 (screen 1210), and media volume control (e.g. For example, volume control when playing a media file) is performed by a user's input to the rear input unit 204.

In the screens 1200 to 1220 of FIG. 12, as in FIG. 9, a predetermined user input interface of the object 1201 is a front input unit 203 and a rear input unit 204, and the front input unit 203 is In response to the user's input through the object 1201, a first function (system volume control function) set in advance is executed, and in response to the user's input through the rear input unit 204, the object 1201 is It can be understood as an example of executing a 2 function (media volume control function). When a user's input to the front input unit 203 is detected, a message indicating system volume control is displayed on the display unit 202, and when a user's input to the rear input unit 204 is detected, the display unit 202 A message 1221 indicating volume may be displayed.

13 is another example of a screen of an object control process in FIG. 3. 13 is an example of a screen that performs different functions according to the front input unit 203 and the rear input unit 204 which are predetermined input interfaces of the object 1301.

That is, the surface of the object 1301 becomes opaque due to the user's touch on the front input unit 203 of the object 1301 displayed on the screen 1300 of FIG. 13 (1310 ). A background surface other than the object 1301 becomes opaque due to a user's touch on the rear input unit 204 for the object 1301 on the screen 1300 (1320). As the front input unit 203 and the rear input unit 204 for the object 1301 are simultaneously touched on the screen 1300, both the object 1301 itself and the background of the object 1301 become opaque (1330 ).

FIG. 13 can be applied to a case where an object is an image editing window and is displayed so that a user can intuitively know that an object to be edited varies according to a touch surface. For example, by a user touch on the front input unit 203 of the object 1301, the screen 1310 displays that the inner area of the object 1310 is set as an edit target, and the rear input unit 204 of the object 1301 The screen 1320 with a user touch to the screen 1320 displays that the external area of the object 1301 has been set as an editing target, and the screen 1330 with a user touch to the double-sided input units 203 and 204 of the object 1301 is the entire screen. This may mean screens that display that they have been set for editing.

14 is an example of another screen of the object control process in FIG. 3. 14, a predetermined input interface of an object is a front input unit 203 and a rear input unit 204, and perform different functions according to respective inputs to the front and rear surfaces.

That is, the screen 1400 of FIG. 14 is a screen on which a video is being performed according to a user's input through the front input unit 203 to the object 1401. Therefore, when playing a video, the front input unit 203 can execute basic functions (fast forward function, rewind function, playback function, etc.) related to video playback. When the video is being performed in this way, the video is stopped 1410 according to the user's touch to the rear input unit 204, and the still image is captured according to the user's gesture to the rear input unit 204 (1420). An example of the gesture may be a touch on the rear input unit 204 and drag upward.

15 is an example of another screen for the object control process in FIG. 3. In FIG. 15, predetermined input interfaces of the object 1511 are the front input unit 203 and the rear input unit 204, and perform different functions according to inputs to the front and rear surfaces.

That is, the page is switched according to the user's touch & drag on the front input unit 203 of the object 1511 (1510 and 1520). According to the user's touch & drag on the rear input unit 204 of the object 1511, the status bar at the top is moved in the dragging direction, and the contents of the status bar are changed (1510, 1530).

According to a preferred embodiment, the screen 1540 is shown according to the user's touch & drag gesture to the rear input unit 204, or the number of pages skipped by the page change function is applied to the user's touch & drag gesture to the front input unit 203. Accordingly, the screen 1520 may be different as shown. For example, when touching and dragging the front input unit 203, the page can be changed one by one (screen 1520), and when touching and dragging the rear input unit 204, the page can be changed by two pages (screen (1540)). Alternatively, a page provided when touching & dragging the front input unit 203 and a page provided when touching & dragging the rear input unit 204 may be set differently.

16 is another screen example of the object control process in FIG. 3. In FIG. 16, predetermined input interfaces to the object 1611 are the front input unit 203 and the rear input unit 204, and may perform different functions according to the front and rear surfaces.

That is, when the alarm goes off, the alarm can be stopped by a user's input to the front input unit 203 of the object 1611 of FIG. 16 (screen 1610) and the user's input to the rear input unit 204 It is possible to set the snooze time by (screen 1620). The reminder time can be set by touching and dragging up or down.

17 is an example of another screen of an object control process in FIG. 3. 17 also includes a front input unit 203 and a rear input unit 204 as a predetermined input interface for the object 1713, and may perform different functions depending on the front and rear surfaces.

In the case of FIG. 17, information according to a virtual key arranged according to a user's input to the front input unit 203 of the object 1713 is input, and a virtual key according to the user's input 1714 to the rear input unit 204 This is the case to convert the mode of. That is, when the rear input unit 204 is touched once on the screen (1710), the object 1713 is changed to a numeric key arrangement as shown in 1720, and when the rear input unit 204 is touched twice ( 1730, the object 1713 is changed to an English key sequence.

18 is an example of another screen of the object control process in FIG. 3. 18 is an example in which the transparency of the object 1811 is set to a maximum value in response to a user's input being touched over a certain area of the rear input unit 204.

That is, when a screen such as 1810 of FIG. 18 is displayed on the display unit 202, when a predetermined area or more of the rear input unit 204 is touched as shown in 1820, the transparency of the display unit 202 is the maximum value. It may be set to a value between and a minimum value (for example, maximum opaqueness). Although the transparency can be set to a maximum value by temporarily turning off the power of the display unit 202, the preferred embodiment is not limited thereto. According to a preferred embodiment, the proximity sensor may be used to determine whether or not to touch a predetermined area or more.

As shown in 1820, if the touch gesture is not maintained for a predetermined time or longer, the display unit 202 returns to the screen displayed before the touch is generated, as shown in 1830. However, when the touch gesture is maintained for a predetermined time or longer, the transparency of the display unit 202 maintains a state set to a maximum value (1840). The predetermined time or more may be set to 3 minutes or more, for example, but the preferred embodiment is not limited thereto.

19 is an example of another screen of the object control process in FIG. 3. 19, a predetermined input interface of an object is changed.

That is, in response to a request to edit an input interface for the object 1911 being displayed, the edit window 1921 including information on a predetermined input interface is displayed on the object 1911. The input interface editing request may be input using a predetermined user gesture or menu through the front input unit 203 or the rear input unit 204.

Information on a predetermined input interface for the object 1911 may be updated with the input interface information 1921 changed according to the user's input received based on the edit window 1921 (screen 1920 and screen 1930 )). In FIG. 19, a predetermined front input unit 203 for the object 1911 is changed to a rear input unit 204.

20 is an example of another screen of the object control process in FIG. 3. FIG. 20 is a case in which objects 2012 and 2013 for which different input interfaces are set are simultaneously controlled according to a user's input, and may be understood as multitasking.

That is, the user's input for the object 2012 on the screen 2010 through the front input unit 203 and the user’s input for the object 2013 through the rear input unit 204 are shown in (2020) of FIG. If it is dragged to the desired position as described above, the display position is simultaneously moved as shown in 2030 accordingly.

20(2014) is screen information showing a predetermined input interface of the object 2012 being displayed. By (2014), the user can intuitively know that the predetermined input interface of the object 2012 is the front input unit 203.

Examples of the screens of FIGS. 4 to 20 described above are provided by the operation of the processor 215. In addition, the user's gesture described in the example of the screens of FIGS. 4 to 20 is not limited to the above description.

21 is a flowchart of an object control method performed by the device 200 including a transparent display according to another exemplary embodiment. The object control method of FIG. 21 is also performed by the processor 215.

21 is a case in which the object control step shown in FIG. 3 is changed to the step of rotating the object and updating a predetermined input interface of the object accordingly. Accordingly, steps S2101 to S2103 of FIG. 21 are the same as steps S301 to S303, and thus a description thereof will be omitted.

As a result of determination in step S2103, the user's input is received through a predetermined input interface (for example, the front input unit 203 or the rear input unit 204) of the object, and the user's gesture is moved by a predetermined length or more in one direction. When is generated, the processor 215 changes a predetermined input interface of the object to another input interface while rotating the object in a direction corresponding to the user's gesture (S2104). Therefore, if the predetermined input interface of the object is the front input unit 203, the input unit of the object may be changed to the rear input unit 204 according to the above-described user input and user gesture. In this way, since the input interface for each object can be independently specified, more various types of user interfaces are possible and the complexity of implementation due to the use of dual surfaces can be reduced. In addition, it goes without saying that different input interfaces may be set for each of a plurality of objects displayed on one screen as shown in FIG. 1.

22A is an example of a screen for changing an input interface of an object by rotating an object 2210 among a plurality of objects 2202 to 2210. That is, as a gesture of the user moving in the direction of (2212) with respect to the object 2210 is generated on the screen 2240 of FIG. 22A, the object 2210 is rotated in the order shown in the screen 2250 and the screen 2260 do. The screen 2260 is a state in which the object 2210 is rotated 180 degrees so that the image of the object 2210 is changed between the rear and the front.

When the display surface of the object 2210 is changed according to the rotation of the object 2210, the processor 215 changes a predetermined input interface for the object 2210 to the rear input unit 204 (S2104).

22B shows that when the object 2210 performs different functions by setting different objects for the front input unit 203 and the rear input unit 204, respectively, the front input unit 203 and the rear surface are rotated according to the rotation of the object 2210. This is an example of a screen in which a function corresponding to the input unit 204 is changed.

That is, FIG. 22B shows a case in which an application in which the front input unit 203 of the object 2241 is set as an input interface is a mini diary, and an application in which the rear input unit 204 is set as an input interface is a store. As the screens 2240, 2250, and 2260 rotate in the direction 2242 as shown in, the store application is displayed on the front side of the device 200, and the input interface of the store application is the front input unit ( 203), the mini diary application is displayed on the rear side of the device 200, and the input interface of the mini diary is set to the rear input unit 204. Accordingly, the device 200 can execute the function of a store by input through the front input unit 203 of the object 2241, and the device 200 is a mini-diary by input through the rear input unit 204 of the object 2241 The function of can be executed.

23 is a flowchart illustrating a method of displaying a guide in a device 200 including a transparent display according to another exemplary embodiment. The guide display method is performed by the processor 215. The guide may include information indicating a position (touch position) at which a user's input to the rear input unit 204 is sensed.

The processor 215 displays at least one object on the display unit 203 according to the execution of an arbitrary application (S2301). When a user's input is received from the rear input unit 204 regardless of a predetermined input interface for the object being displayed (S2302), the guide is displayed at the received position (S2303). The guide may be variably displayed based on the state of the object being displayed at the point where the user's input is received or the state of the display unit 203. The guide may be referred to as an indicator.

The state of the object or the state of the display unit 203 may include a state based on transparency. Accordingly, the guide may display guides having different images or display guides having different transparency according to the transparency of the object or the transparency of the display unit 203. The transparency of the object may take into account the transparency of a region adjacent to the object. The state of the display unit 203 may include a touch intensity.

24 is an example of a screen displaying a guide. In the screen 2400 of FIG. 24, a virtual guide 2402 is displayed at the position of the text 4 as the user's input 2401 through the rear input unit 204 is detected for the text 4 object included in the list. It can be intuitively known that the text 4 object is being touched through the input unit 204. In addition, according to the change in the transparency of the guide, it is possible to intuitively know whether the user is touching the rear input unit 204 with a level of touch strength.

In the case of the screen 2410 of FIG. 24, a virtual guide is displayed on all surfaces on which the selected object is displayed. According to the screen 2410, the user can intuitively know that a user touch has occurred through the rear input unit 204 on the object of text 4. In the case of the screen 2410, the color of the object field of text 4 may be changed according to the touch intensity.

In addition, when moving the object 2421 in the direction of (2422) by touching and dragging the rear input unit 204 like the screen 2420, the object 2421 is moved through the front input unit 203 while the touch is maintained. When touched, the object 2421 may be continued with the operation performed by the rear input unit 204. That is, as illustrated in the screen 2430, an operation of moving the object 2421 in the direction 2431 by touching and dragging the object 2421 may be continuously performed.

25 is another functional block diagram of a device according to an exemplary embodiment.

Referring to FIG. 25, the device includes a transparent display 2501, a processor 2502, and a storage unit 2503.

The transparent display 2501 includes a front input unit 203, a rear input unit 204, and a display unit 202, like the transparent display 201 of FIG. 2A, among the front input unit 203 and the rear input unit 204 A user's touch input is received through at least one input unit, and an object is displayed in response to the user's touch input.

The storage unit 2503 stores information on a predetermined input interface for at least one object and a program for performing an object control method and a guide display method according to an exemplary embodiment. That is, data according to the software structure diagram shown in FIG. 26 may be stored in the storage unit 2503. The storage unit 211 of FIG. 2A may also store programs and resources according to the software structure diagram shown in FIG. 26.

In particular, touch mode data 2612 including setting values for a touch mode of each object or a predetermined user input interface shown in FIG. 26, and a touch input classification module 2622 for classifying an input touch. And a touch mode estimation module 2664 may be mainly used.

26 is a diagram for explaining a software layer stored in a storage unit of a device including a transparent display according to an exemplary embodiment.

Referring to Figure 26, the software layer is a storage module (storage module) 2600, a sensor recognition module (sensor & recognition module) 2620, a messaging module (messaging module) 2650, a telephony module (telephony module) 2640 ), a multimedia module 2665, a UI & graphics module 2661, and a legend module 2670, but are not limited thereto.

The storage module 2600 includes a system database 2611, which is a general data storage such as address book and environment information, and a touch mode data area 2612 for storing setting values for a touch mode of each object according to the present invention.

The sensor recognition module 2620 includes a module 2621 for recognizing a touch at the front input unit 203 and the rear input unit 204, and a module 2622 for identifying an input touch. The module 2622 for classifying an input touch is a front input mode 2623 that delivers an input to the front input unit 203 to the event processor 2662, and an input to the rear input unit 204 to the event processor 2662. It includes a rear input mode 2624 and a dual mode 2625 which transfers inputs for double-sided touches 203 and 204 to the event processor 2662.

The message module 2650 includes an instant module 2651, which is a module capable of conversation between users through an Internet connection, a module 2652 for short text and multimedia messages, and a module 2603 for sending an e-mail.

The telephony module 2640 includes an information collection module 2642 for telephone connection, and a voice service module 2641 for transmitting voice through the Internet based on Voice over Internet Protocol (VoIP).

The multimedia module 2665 includes a moving picture playback module 2666, a moving picture and still picture taking module 2667, and a voice playback module 2668.

The UI & graphic module 2661 is a module that receives touch input through a window manager, X11 (2662), a module (2663) that outputs all objects displayed to the user on the screen, and mode setting values stored for each object and current touch input. And an estimation module 2664 for.

As described above, programs having various structures may be stored in the storage unit 2503, and the display in FIG. 26 is not limited thereto.

The processor 2502 may perform operations according to various embodiments described above by using a program stored in the storage unit 2503. The processor 2502 controls the transparent display so that at least one object is displayed, determines whether a user's input received through at least one input interface is input to the object through a predetermined input interface, and based on the determination, the object Control.

One or more programs including instructions for performing a method of implementing an object control method or a guide display method according to embodiments of the present invention with a computer are recorded as computer-readable codes on a computer-readable recording medium. It is possible. The computer-readable recording medium includes all types of storage devices storing data that can be read by a computer system. Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data storage. In addition, the computer-readable recording medium can be distributed over a network-connected computer system, and stored and executed as computer-readable codes in a distributed manner.

So far, the present invention has been looked at around its preferred embodiments. Those of ordinary skill in the art to which the present invention pertains will be able to understand that the present invention can be implemented in a modified form without departing from the essential characteristics of the present invention. Therefore, the disclosed embodiments should be considered from an illustrative point of view rather than a limiting point of view. The scope of the present invention is shown in the claims rather than the foregoing description, and all differences within the scope equivalent thereto should be construed as being included in the present invention.

Claims (30)

In the object control method performed by a device including a transparent display,
Displaying a first object and a second object on the transparent display;
While the first object and the second object are being displayed, as a touch input for the first object is received through the first surface of the transparent display, a function corresponding to the first object is performed, and the first A surface being one of a front surface and a rear surface of the transparent display; And
While the first object and the second object are being displayed, as a touch input for the second object is received through the second surface of the transparent display, a function corresponding to the second object is performed, and the second object The surface being the other one of the front and rear surfaces of the transparent display; And
As a touch input for the first object through the first surface of the transparent display and a touch input for the second object through the second surface are simultaneously received, a function of controlling the first object and the second Object control method comprising the step of simultaneously performing a function of controlling the object. delete delete delete delete delete The method of claim 1, wherein the function of controlling the second object comprises changing a display state of the second object. The method of claim 1, wherein the function of controlling the second object comprises moving the second object. The method of claim 1, wherein the function of controlling the second object comprises displaying the second object on the first object. The method of claim 1, wherein the method
The first surface of the transparent display on which the touch input activating more than the preset area is received as a touch input for activating at least one of the first and second surfaces is received, and And setting the transparency of at least one of the second surfaces to a maximum value. The method of claim 10, wherein the method comprises:
When the touch input activating the preset area or more is maintained for a period less than a preset time, the first surface of the transparent display and the first surface of the transparent display before the touch input activating the preset area or more is received The object control method further comprising the step of returning the transparency of at least one of the second surface. The method of claim 1, wherein the method comprises:
At least one of the first surface and the second surface for at least one of the first object and the second object in response to a request for editing a surface of the transparent display for at least one of the first object and the second object Displaying an edit window including information for selecting one;
Allocating a surface corresponding to at least one of the first object and the second object according to a user input through the edit window; And
And updating information on the surface corresponding to the first object and the second object based on the allocation. delete The method of claim 1, wherein the touch input to the first object moves in one direction through the first surface by a predetermined length or more,
The function of controlling the first object includes rotating the first object in a direction corresponding to the touch input to change from the first surface to the second surface,
Including that the touch input for the second object moves more than a preset length in one direction through the second surface,
The function of controlling the second object includes changing the second surface to the first surface by rotating the second object in a direction corresponding to the touch input. The method of claim 1, wherein the function of controlling the second object includes variably displaying an indicator corresponding to a touch input based on a state of the second object. delete A computer recordable recording medium in which instructions executed by a computer for performing an object control method performed by a device including a transparent display are recorded,
The object control method,
Displaying a first object and a second object on the transparent display;
While the first object and the second object are being displayed, as a touch input for the first object is received through the first surface of the transparent display, a function corresponding to the first object is performed, and the first A surface being one of a front surface and a rear surface of the transparent display; And
While the first object and the second object are being displayed, as a touch input for the second object is received through the second surface of the transparent display, a function corresponding to the second object is performed, and the second object The surface being the other one of the front and rear surfaces of the transparent display; And
As a touch input for the first object through the first surface of the transparent display and a touch input for the second object through the second surface are simultaneously received, a function of controlling the first object and the second A recording medium comprising the step of simultaneously performing a function of controlling an object. In the device,
A transparent display displaying a first object and a second object;
A touch input is received through at least one of a first surface and a second surface, wherein the first surface is one of a front surface of the transparent display and a rear surface of the transparent display, and the second surface is a front surface and a rear surface of the transparent display. The other one of the input interface;
A storage unit that stores information on the first object, the second object, the first surface, and the second surface; And
While displaying the first object and the second object,
When a touch input for the first object is received through the first surface of the transparent display, a function corresponding to the first object is performed,
When a touch input for the second object is received through the second surface of the transparent display, a function corresponding to the second object is performed,
A device comprising a processor that simultaneously performs a function of controlling the first object and a function of controlling the second object as a touch input for the first object and a touch input for the second object are simultaneously received. delete delete delete The device of claim 18, wherein the function of controlling the second object includes variably displaying an indicator corresponding to a touch input based on a state of the second object. delete The method of claim 18, wherein the processor receives the touch input for activating the preset area or more as a touch input for activating at least one of the first and second surfaces is received. A device further configured to control the device to set the transparency of at least one of the first side and the second side of the transparent display to a maximum value while being used. The method of claim 24, wherein the processor,
When the touch input activating the preset area or more is maintained for a period less than a preset time, the first surface of the transparent display and the first surface of the transparent display before the touch input activating the preset area or more is received A device further configured to control the device to return to the transparency of at least one of the second sides. delete 19. The device of claim 18, wherein the function of controlling the second object comprises changing a display state of the second object. 19. The device of claim 18, wherein the function of controlling the second object comprises moving the second object. 19. The device of claim 18, wherein the function of controlling the second object comprises displaying the second object over the first object. The method of claim 18, wherein the processor,
In response to a request for editing a surface of the transparent display for at least one of the first object and the second object, the first surface and the second surface for at least one of the first object and the second object Display an edit window including information for selecting at least one of,
Allocating a surface corresponding to at least one of the first object and the second object according to a user input through the edit window,
The device further configured to update information on the plane corresponding to the first object and the second object based on the assignment.

Images