KR20160033495A - Apparatus and method for arranging furniture using augmented reality - Google Patents

Abstract

The present invention relates to a technology for arranging furniture by using augmented reality. According to a method of a simulator, having at least one processor, for arranging virtual furniture in a real space, an image of the real space, in which predetermined markers are located, is captured by using a camera and is displayed in a display device in real time; a user selects the virtual furniture that the user desires to arrange in the real space; and the virtual furniture is arranged on the real time image with respect to the markers as reference points according to manipulation by the user. The present invention is to eliminate inconvenience that furniture and space should have been actually measured and a complicated modeling tool should have been used.

Description

[0001] Apparatus and method for arranging furniture using augmented reality [0002]

The present invention relates to a virtual reality modeling technique capable of arranging furniture virtually without arranging furniture directly in a real space, and more particularly, to a virtual reality modeling technique capable of easily arranging virtual furniture in a real space using an augmented reality And a storage medium storing the method.

The biggest characteristic of furniture is that the replacement cycle is long. According to the Trade Commission's "Competitiveness of Furniture Industry Survey, Korea Living Environment Testing Institute, Competitiveness of Furniture Industry Survey, 2004. 10", it is generally said that the replacement period of furniture is generally 10 years. And it can be seen in the survey that it is a common practice to replace all furniture separately, not only at one time, but also in the case of backward furniture. Therefore, in order to increase sales substantially, furniture makers are more likely to purchase furniture and buy more furniture than those who do not want to replace it, and are in the late twenties who are independent of newly- It is desirable that the main goal is to be a college student who has visited in the provinces or who has to leave home because of the distance from the school.

Because newly married couples, socioeconomic students, and college students have a narrower residential environment than elderly people, when they move to a relatively small house while living in an existing large house, There is a possibility that the furniture may not fit into the house by purchasing the furniture considering the size of the residence. Even if it is not for this reason, I often do not like the design and size of the furniture, and often it takes a lot of effort to return it when I buy it, or to change the interior.

When planning such a furniture replacement, it is usually necessary to check the actual condition of the furniture and then measure the space of the present place of residence again so as to determine whether the furniture desired to be purchased can be properly positioned in the space, And so on. Therefore, the user needs a means for confirming the arrangement of the furniture by a simpler method before purchasing the furniture.

In the following prior art documents, technical means for designing a three-dimensional virtual space such as a cyber model house is introduced in response to such a demand of a consumer. However, this technique is still inconvenient in that it is not a means to verify the furniture layout with a simple operation in the field where the user wants to arrange the furniture immediately.

Korean Patent Publication No. 2003-0071944, published on September 13, 2003, Samsung SDS Co., Ltd.

The technical problem to be solved by the embodiments of the present invention is to solve the conventional inconvenience that the user had to utilize furniture and space measurement or complicated modeling tool in reviewing and verifying the furniture arrangement in the actual space in which the furniture is to be arranged .

According to an aspect of the present invention, there is provided a method of arranging a virtual furniture in a real space by a simulator having at least one processor according to an embodiment of the present invention includes the steps of: (a) A step of photographing an image of a real space located on a display device using a camera and displaying the captured image on a real time basis; (b) the simulator selecting a virtual household to be placed in the real space from a user; And (c) arranging the virtual furniture selected by the simulator on the real-time image with reference to the marker according to an operation of the user.

(D) storing, in advance, a real image of a marker, which is a reference for displaying the space information given to the virtual furniture on the real-time image of the real space, by the simulator; Wherein the simulator determines a relative size, a separation distance, and a projection direction of the virtual furniture based on the marker, and displays the image.

According to an embodiment of the present invention, there is provided a method of arranging a furniture, the method comprising: (e) storing, in advance, the space information including the size information of the virtual furniture in the real space and the virtual furniture formation direction; Wherein the simulator reads the pre-stored spatial information for the selected virtual furniture, converts the pre-stored spatial information into a suitable scale within the real-time image, and displays the converted real-time image on the real-time image.

In the furniture arrangement method according to an embodiment, the simulator may display at least one virtual furniture by matching the one marker with an image, and the relative positional relationship between the markers Can be stored.

In the furniture arrangement method according to an embodiment, the step (c) includes the steps of: (c1) identifying a marker in the real-time image by the simulator; (c2) recognizing a photographing direction and a ratio representing a viewing direction of the user with respect to the real space by comparing the image of the marker registered by the simulator with an identifier of the registered marker; (c3) converting the size and basic layout direction of the virtual furniture selected from the user based on the photographing direction and the ratio recognized by the simulator; And (c4) arranging the converted virtual furniture on the real-time image based on the marker according to an operation of the user.

The present invention also provides a computer-readable recording medium having recorded thereon a program for causing a computer to execute the methods of controlling the furniture arrangement method described above.

According to an aspect of the present invention, there is provided an apparatus for arranging virtual furniture in a real space according to an embodiment of the present invention includes: a camera for photographing an image of a real space in which a preset marker is located; A display unit for displaying an image photographed through the camera in real time; An input unit for selecting a virtual household to be placed in the real space from a user and inputting a position and a direction for arranging the selected virtual furniture; And a processor for arranging the virtual furniture selected according to the operation of the user on the real time image based on the marker.

The furniture arrangement apparatus according to an embodiment of the present invention may further include: a real image of a marker, which is a reference for displaying on the real-time image the spatial information given to the virtual furniture, a size information of the virtual furniture in the real space, And a storage unit for storing in advance the spatial information including the forming direction of the virtual furniture in relation to the virtual furniture, wherein the processing unit is configured to determine, based on the marker, the relative size, The projection direction is determined and displayed as an image. The spatial information stored in advance in the selected virtual furniture can be read out and converted into a suitable ratio within the real-time image and displayed on the real-time image.

In the furniture arrangement apparatus according to an embodiment, the processing unit identifies a marker in the real-time image, compares an image of the identified marker with an image of a previously registered marker, And converts the size and basic layout direction of the virtual furniture selected from the user based on the recognized photographing direction and the ratio, and displays the converted virtual furniture on the basis of the marker It can be placed on real time image.

Embodiments of the present invention recognize a positional relationship and a scale of a real space using markers and arrange virtual furniture on the basis of a marker using furniture information, It is possible to freely arrange the virtual furniture according to the converted ratio at the same time while confirming the real space through the camera in real time without needing to do.

1 is a flowchart illustrating a method of arranging a house using an augmented reality according to an embodiment of the present invention.
FIG. 2 is a diagram illustrating a process of arranging furniture using markers in a real-time image displayed on a display screen in a furniture arrangement method according to an embodiment of the present invention.
FIG. 3 is a diagram for explaining a process of checking and selecting furniture information registered in advance in the furniture arrangement method according to an embodiment of the present invention.
FIG. 4 is a flowchart illustrating a process of arranging virtual furniture on a real-time image in the furniture arrangement method of FIG. 1 according to an embodiment of the present invention.
FIG. 5 is a block diagram illustrating a furniture arrangement apparatus using an augmented reality according to an embodiment of the present invention. Referring to FIG.
FIGS. 6A to 6G are diagrams illustrating operations of a software prototype implementing a method of arranging a house using an augmented reality adopted by embodiments of the present invention.

Augmented reality is a technology that superimposes a virtual object on the real world that the user sees. It is called Mixed Reality (MR) because it combines real world and virtual world with additional information in real time and displays it as one image. The hybrid VR system is a hybrid VR system that combines the real environment with the virtual environment. Since the late 1990s, research and development have been conducted mainly in the United States and Japan.

Augmented reality, a concept that complements the real world with a virtual world, uses a virtual environment created by computer graphics, but the protagonist is a real environment. Computer graphics serve to provide additional information needed for the real world. By overlapping the three-dimensional virtual image on the real image that the user is viewing, it means that the distinction between the real environment and the virtual screen becomes blurred.

Virtual reality technology allows a user to be immersed in a virtual environment, and the real environment can not be seen. However, augmented reality technology, which is a mixture of real environment and virtual objects, allows the user to see the real environment and provides better realism and additional information. For example, if you use a smartphone camera to illuminate the surroundings, information such as the location of a nearby store, phone number, and the like is displayed as a stereoscopic image.

These augmented reality technologies can be used for remote medical diagnosis, broadcasting, architectural design, and manufacturing process management. Recently, as smart phones have become widespread, they have entered into commercialization stage. In the game and mobile solution industry and education field, .

The embodiments of the present invention, which are proposed below, introduce the concept of the Augmented Reality to enable a user to operate a simulator for furniture placement through a portable terminal such as a smart phone, It helps to show real time and to arrange mock batches to place virtual furniture in real space. Hereinafter, embodiments of the present invention will be described in more detail with reference to the accompanying drawings.

1 is a flowchart illustrating a method of arranging a house using an augmented reality according to an embodiment of the present invention. As described above, the augmented reality is based on computing technology, and the present invention is intended to implement a simulated arrangement in which virtual furniture can be arranged in advance in a real space (which means actual indoor space). Therefore, An example may be implemented via a simulator with at least one processor.

In step S110, the simulator may register markers and furniture information in advance. Here, the marker is a key tool for realizing the augmented reality as a means for providing a connection relation for matching a virtual object (here, it means a virtual household) to a real space. That is, the marker is a means for associating the scale and the positional relation when arranging the virtual furniture in the real space. For example, when a virtual object has a floor width of 1 m ² , the size of the virtual object can be adjusted by comparing the size of the marker in the currently used image through the marker captured in the real space, thereby displaying the image. In addition, when the virtual object is displayed in the image, the marker becomes a reference point for determining the position of the virtual object disposed on the screen when the position and direction of the camera change in real time. In other words, even if the position and orientation of the camera are changed, the simulator can detect the change in the size and orientation of the marker, and adjust the relative size and arrangement position of the virtual object.

In step S120, the simulator photographs an image of a real space in which a predetermined marker is located, using a camera, and displays the image on a display device in real time. The simulator starts to collect information about one of the two elements (real and virtual space) that constitute the augmented reality while displaying the image of the real space on the display device. At this time, the simulator can recognize the actual size and direction of the real space in the currently photographed image by recognizing the markers included in the image.

In step S130, the simulator selects virtual furniture to be placed in the real space from the user. Such virtual furniture may be registered in advance in the simulator, and it is particularly desirable to provide specific values for the size and shape of the furniture to realize the augmented reality. Of course, since the relative size of such a virtual furniture can also be determined in comparison with a previously registered marker, the simulator can prepare a change in the shape of the furniture to be displayed on the actual screen based on the size information of the furniture.

In step S140, the simulator places the selected virtual furniture on the real-time image with reference to the marker according to the operation of the user. That is, the simulator can recognize the size and direction of the real space through the size and direction of the marker included in the real-time image, and adjust the size to fit the recognized real space based on the size information of the virtual furniture . Now, when the adjusted furniture is displayed in the image, the user can freely place the furniture in the real space by using an input means such as a touch screen operation or a button input. Through this process, the user can easily imagine the furniture arrangement by directly arranging the virtual furniture in the real-time image of the real space without actually measuring the furniture or moving it directly.

FIG. 2 is a diagram illustrating a process of arranging furniture using markers in a real-time image displayed on a display screen in a furniture arrangement method according to an embodiment of the present invention.

As described above with reference to step S110 of FIG. 1, the simulator can previously store the real image of the marker, which is a reference for displaying the spatial information given to the virtual furniture on the real-time image on the real space. Accordingly, the simulator can determine the relative size, spacing distance, and projection direction of the virtual furniture based on the marker and display it as an image.

Referring to FIG. 2, a real-time image 10 of an actual real space is displayed on a display screen 23, and a marker 220 is disposed in the real space. That is, in the augmented reality technology adopted in the embodiments of the present invention, a marker, which is a criterion for realizing an augmented reality, must exist in a real-time image of a real space photographed through a camera.

It is preferable that such a marker is photographed in advance and registered in the simulator. That is, the simulator can prepare for placing virtual furniture by recognizing the registered markers 210 in the image.

Meanwhile, it is preferable that the simulator displays at least one or more virtual furniture in a single marker and displays it as an image. Thus, it is possible to arrange a plurality of furniture using only one marker. In addition, the simulator stores information on the relative positional relationship with the markers for each household. As an example of implementing the simulator, the simulator calculates the relative positional relationship with respect to each of the furniture placed on the basis of the markers as 1: N (N May be stored in the form of a relational database model having a relationship of a positive integer).

FIG. 3 is a diagram for explaining a process of checking and selecting furniture information registered in advance in the furniture arrangement method according to an embodiment of the present invention.

As described above with reference to step S110 of FIG. 1, the simulator may previously store the spatial information including the size information occupied by the virtual furniture in the real space and the formation direction of the virtual furniture in advance for the virtual furniture. Accordingly, the simulator can read the pre-stored spatial information for the selected virtual furniture, convert it into a suitable scale within the real-time image, and display it on the real-time image.

3, when the furniture list 310 is displayed on the display screen 23 and one furniture (for example, a refrigerator is selected) is selected according to the user's operation, Specific items (various refrigerator information) may be displayed in the remaining area 320 of the refrigerator 23. In FIG. 3, two pieces of furniture information about the refrigerator are registered. After confirming the image and size of the furniture, the user can select an item desired to be arranged. Of course, it is preferable that the selected furniture item stores more detailed information (for example, three-dimensional modeling information) than the brief information on the size of the furniture displayed on the screen. Using this specific information, the simulator will be able to more accurately depict the furniture displayed in the actual photographed reality space. On the other hand, by marking the prices of the households together, the user can use them as basic data for selecting households or induce direct online purchase.

FIG. 4 is a flowchart illustrating a process of arranging virtual furniture on a real-time image in the furniture arrangement method of FIG. 1 according to an embodiment of the present invention. Here, S140) will be described.

In step S141, the simulator identifies the marker in the real-time image. To this end, the simulator utilizes an image of a previously registered marker to determine whether a marker image exists in the currently photographed image, and if so, how it is deformed. Considering the environment in which the camera is used, it is common that the marker is linearly deformed by the perspective, so that the image detection technique according to this linear deformation can be utilized.

In step S142, the simulator recognizes the photographing direction and the ratio indicating the observation direction of the user with respect to the real space by comparing the image of the identified marker with the image of the previously registered marker. Assuming that the rectangle marker is located on the bottom of the real space as described above, the perspective closer to the camera will be seen to be relatively larger relative to the far side of the camera, and by sensing this linear deformation, And discriminates the difference from the marker. Then, the simulator determines the direction and the ratio to the real space in consideration of the observation point of the user (that is, the photographing direction of the camera).

In step S143, the simulator converts the size and basic layout direction of the virtual furniture selected from the user based on the recognized photographing direction and the ratio. Now, the simulator transforms the virtual furniture in consideration of the photographing direction and the ratio recognized in step S142. This transformation produces an output reflecting the projection direction and scale of the furniture to be displayed on the actual screen.

In step S144, the simulator places the converted virtual furniture on the real-time image with reference to the marker according to a user's operation. As described above, since the marker adopted by the embodiments of the present invention can be matched with one or more furniture, each virtual furniture can be a relative position (e.g., three-dimensional coordinate information) Can be recorded on the real-time image.

FIG. 5 is a block diagram illustrating a furniture arrangement apparatus using an augmented reality according to an embodiment of the present invention, and is a diagram describing the furniture arrangement method of FIG. 1 described above from a hardware point of view. Therefore, in order to avoid duplication of explanation, the feature is outlined centering on the separated device configuration.

The camera 21 captures an image of the physical space 10 in which the preset marker is located.

The display unit 23 displays the image photographed through the camera 21 in real time under the control of the processing unit 22.

The input unit 25 selects virtual furniture to be placed in the real space from the user, and receives a position and a direction for arranging the selected virtual furniture. In consideration of a recent smartphone, such an input unit 25 may employ a touch screen technology, and may be integrated with the display unit 23 and implemented in one configuration.

The processing unit 22 arranges the virtual furniture selected according to the operation of the user on the real-time image with reference to the marker.

The storage unit 24 stores the real image of the marker, which is a reference for displaying the spatial information given to the virtual furniture on the real-time image on the real space, the size information that the virtual furniture occupies in the real space, And the spatial information including the forming direction can be stored in advance in the virtual furniture.

Accordingly, the processing unit 22 determines the relative size, spacing distance, and projection direction of the virtual furniture based on the markers in the real-time image and displays the virtual furniture on the display unit 23 as an image, The spatial information stored in advance in the storage unit 24 can be read and converted into a suitable ratio within the real time image and displayed on the real time image.

In addition, the processing unit 22 identifies the marker in the real-time image and compares the identified marker with the image of the marker registered in advance in the storage unit 24, Recognizing the photographing direction and the ratio, converting the size and basic arrangement direction of the virtual furniture selected from the user based on the recognized photographing direction and the ratio, and converting the converted virtual furniture to the reference Can be placed on the real-time image.

FIGS. 6A to 6G are diagrams illustrating operations of a software prototype implementing a method of arranging a house using an augmented reality adopted by embodiments of the present invention. The software prototype was implemented as an app for smartphones, and markers and virtual furniture were registered in advance in the software. Hereinafter, the process of arranging the virtual furniture according to the operation of the software will be described sequentially.

6A illustrates a process in which a simulator initially receives a selection from a user for an indoor space (e.g., a bedroom, a living room, a kitchen, an office, etc.) Is displayed in one screen, there may be inconvenience in selection of the user, and therefore, here is a screen introduced to limit the setting object. However, it is not always necessary to specify such a place, and the illustrated screen is merely one example proposed for convenience of use as home-use software.

6B is a screen for displaying the type of living room furniture and individual items, assuming that the simulator has selected 'living room' from the user. And a detailed furniture item is displayed in the right space when the left furniture type is selected.

FIG. 6C illustrates a concrete screen in which a user selects 'sofa' and selects one of various sofas. After confirming the detailed information displayed on the display screen, the user can decide to place it in the living room in advance.

FIG. 6D illustrates a process of selecting a matching means, that is, a marker, for realizing a virtual furniture selected by a user as an augmented reality in an actual living room space. For example, one or more of these markers may be utilized, and each marker may again be positioned to match one or more households.

FIG. 6E illustrates a screen for arranging a virtual furniture (sofa) selected by a user while photographing an actual living room through a smart phone. Markers are arranged at the center of the display screen, and these markers are the same as the markers previously registered and selected by the user through FIG. 6D. However, since the image of the marker photographed in the image is not a pre-registered rectangular shape, the simulator detects this shape change and transforms the virtual furniture in a form suitable for the observation time. Referring to FIG. 6E, it can be seen that the virtual furniture of the modified form adapted to the viewpoint is overlapped with the image of the real space. The user can change the arrangement of furniture by using various layout tools (horizontal movement button, rotation button, vertical movement button, etc.) displayed in the screen.

FIG. 6F illustrates how furniture locations are determined and stored. As described above, virtual furniture stores the relative distance, position, and orientation with respect to the marker based on the marker.

FIG. 6E illustrates how three virtual furniture are arranged in an actual indoor space (living room) according to the same method. 6F and the viewpoint of the observer (that is, the camera shooting direction) are different from each other, the relationship between the marker and the placed furniture remains the same. That is, in the incremental reality technique adopted by the simulator of the present invention, since the real-time image displayed on the display screen is reconstructed based on the relative position and direction information of the marker and the virtual furniture, It is advantageous that the previously determined furniture arrangement can be maintained regardless of the movement.

In summary, according to the embodiments of the present invention described above, the positional relationship and the scale of the real space are recognized using the marker, and the virtual furniture is arranged on the basis of the marker by using the furniture information, It is possible to freely arrange the virtual furniture according to the converted ratio at the same time while checking the real space through the camera in real time in real time without actually measuring the furniture and the indoor space.

Meanwhile, the embodiments of the present invention can be embodied as computer readable codes on a computer readable recording medium. A computer-readable recording medium includes all kinds of recording apparatuses in which data that can be read by a computer system is stored.

Examples of the computer-readable recording medium include a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device and the like, and also a carrier wave (for example, transmission via the Internet) . In addition, the computer-readable recording medium may be distributed over network-connected computer systems so that computer readable codes can be stored and executed in a distributed manner. In addition, functional programs, codes, and code segments for implementing the present invention can be easily deduced by programmers skilled in the art to which the present invention belongs.

The present invention has been described above with reference to various embodiments. It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. Therefore, the disclosed embodiments should be considered in an illustrative rather than a restrictive sense. The scope of the present invention is defined by the appended claims rather than by the foregoing description, and all differences within the scope of equivalents thereof should be construed as being included in the present invention.

10: reality space
20: Furniture arrangement device
21: Camera
22:
23:
24:
25:

Claims (8)

A method of arranging virtual furniture in a real space by a simulator having at least one processor,
(a) photographing an image of a real space in which the preset marker is located, using the camera, and displaying the captured image on a display device in real time;
(b) the simulator selecting a virtual household to be placed in the real space from a user; And
(c) arranging the virtual furniture selected by the simulator on the real time image based on the marker according to an operation of the user. The method according to claim 1,
(d) storing in advance a real image of a marker, which is a reference for displaying the spatial information given to the virtual furniture on the real-time image of the real space by the simulator,
Wherein the simulator determines a relative size, a separation distance, and a projection direction of the virtual furniture based on the marker, and displays the image. The method according to claim 1,
(e) storing, in advance, the spatial information including the size information of the virtual furniture and the formation direction of the virtual furniture in the virtual space in the virtual space,
Wherein the simulator reads the pre-stored spatial information for the selected virtual furniture, converts the pre-stored spatial information into a suitable scale within the real-time image, and displays the converted real-time image on the real-time image. The method according to claim 1,
The step (c)
Wherein the simulator matches at least one or more virtual furniture to one marker and displays the image as an image, and stores information on a relative positional relationship with the marker for each furniture. The method according to claim 1,
The step (c)
(c1) identifying the marker in the real-time image by the simulator;
(c2) recognizing a photographing direction and a ratio representing a viewing direction of the user with respect to the real space by comparing the image of the marker registered by the simulator with an identifier of the registered marker;
(c3) converting the size and basic layout direction of the virtual furniture selected from the user based on the photographing direction and the ratio recognized by the simulator; And
(c4) arranging the converted virtual furniture on the real-time image with reference to the marker according to an operation of the user by the simulator. An apparatus for arranging virtual furniture in a real space,
A camera for capturing an image of a real space in which a preset marker is located;
A display unit for displaying an image photographed through the camera in real time;
An input unit for selecting a virtual household to be placed in the real space from a user and inputting a position and a direction for arranging the selected virtual furniture; And
And a processor for arranging the virtual furniture selected in accordance with the operation of the user on the real time image based on the marker. The method according to claim 6,
A virtual space for storing the virtual space; a real image of the marker, which is a reference for displaying the space information allocated to the virtual space on the real time image of the space, a size information of the virtual room, And a storage unit for storing information in advance with respect to the virtual household,
Wherein,
Determining a relative size, a separation distance, and a projection direction of the virtual furniture on the basis of the marker,
And reads the space information stored in advance for the selected virtual furniture, converts the space information into an appropriate ratio within the real time image, and displays the real space image on the real time image. The method according to claim 6,
Wherein,
Recognizing a marker in the real-time image, comparing the identified marker with an image of a previously registered marker, recognizing a photographing direction and a ratio indicating a viewing direction of the user with respect to the real space, and based on the recognized photographing direction and ratio Wherein the converting unit converts the size and basic layout direction of the virtual furniture selected by the user and arranges the converted virtual furniture on the real time image based on the marker based on the user's operation.

Images