Nothing Special   »   [go: up one dir, main page]

WO2021228200A1 - 用于实现三维空间场景互动的方法、装置和设备 - Google Patents

用于实现三维空间场景互动的方法、装置和设备 Download PDF

Info

Publication number
WO2021228200A1
WO2021228200A1 PCT/CN2021/093628 CN2021093628W WO2021228200A1 WO 2021228200 A1 WO2021228200 A1 WO 2021228200A1 CN 2021093628 W CN2021093628 W CN 2021093628W WO 2021228200 A1 WO2021228200 A1 WO 2021228200A1
Authority
WO
WIPO (PCT)
Prior art keywords
user
dimensional model
user terminal
pixel
information
Prior art date
Application number
PCT/CN2021/093628
Other languages
English (en)
French (fr)
Inventor
白杰
姚锟
贾松林
郑深圳
张蕾
Original Assignee
贝壳技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from CN202010401813.7A external-priority patent/CN111562845B/zh
Priority claimed from CN202010698810.4A external-priority patent/CN111885398B/zh
Application filed by 贝壳技术有限公司 filed Critical 贝壳技术有限公司
Publication of WO2021228200A1 publication Critical patent/WO2021228200A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/239—Interfacing the upstream path of the transmission network, e.g. prioritizing client content requests

Definitions

  • the present disclosure relates to virtual reality panoramic technology and streaming media technology, and in particular to a method for realizing three-dimensional space scene interaction, a device for realizing three-dimensional space scene interaction, a storage medium, and electronic equipment.
  • VR panoramic technology is an emerging rich media technology. Because VR panoramic technology can present users with three-dimensional space scenes without blind angles at 720 degrees, and bring users an immersive visual experience, VR panoramic technology is widely used in various fields such as online shopping malls, travel services, and real estate services. How to enable VR panoramic technology to bring users a richer experience is a technical issue worthy of attention.
  • three-dimensional models can give people a stronger visual perception.
  • any view of the object can be presented to the user, and the correct projection relationship can be maintained between the views.
  • the user terminal while the user terminal is presenting a three-dimensional model, it can support real-time voice on-screen interaction between user terminals, that is, in the process of the user terminal presenting the three-dimensional model, the voice of the opposite user of the user terminal can be transmitted in real time To the user terminal, and the voice acquired by the user terminal can also be transmitted to the opposite terminal in real time.
  • a method for realizing interaction in a three-dimensional space scene including: in response to detecting a user operation of setting footprint information in the three-dimensional space scene, determining the user's current position in the three-dimensional space scene The first pixel in the current view corresponding to the perspective; determine the three-dimensional model corresponding to the first pixel; determine the position of the user's footprint information in the three-dimensional model, where the footprint information is used to display when the three-dimensional space scene is viewed ; And set the user's footprint information at the location.
  • an interaction method based on a three-dimensional model including: at a first user terminal presenting a user interface: in response to detecting a user's target interaction operation on the user interface, The server that provides page data in the user interface sends an interaction request for the target interaction operation, where the user interface is used to present a three-dimensional model, and the three-dimensional model establishes an association relationship with the user account logged in the second user terminal in advance; Streaming video obtained by the terminal; and presenting the streaming video and three-dimensional model on the user interface.
  • an interaction method based on a three-dimensional model including: at a second user terminal: in response to receiving an interaction request sent by a server, acquiring a streaming video, wherein the interaction request indicates The first user terminal detects the user's target interaction operation on the user interface presented by the first user terminal, the user interface is used to present a three-dimensional model, and the three-dimensional model establishes a pre-association relationship with the user account logged in by the second user terminal; and sends a stream to the server Media video, where the server is used to send the streaming video to the first user terminal, so that the first user terminal presents the streaming video and the three-dimensional model on the user interface.
  • a device for realizing interaction of a three-dimensional space scene including: a device for executing the method described in any one of the above methods.
  • an interaction device based on a three-dimensional model which is provided in a first user terminal, and the device includes: a device for executing the method described in any one of the above methods.
  • an interaction device based on a three-dimensional model which is provided in a second user terminal, and the device includes: a device for executing the method described in any one of the above methods.
  • an interactive system based on a three-dimensional model, including: a first user terminal for presenting a user interface; a second user terminal; and a server.
  • the second user terminal is in communication connection.
  • the first user terminal is configured to: in response to detecting the user's target interaction operation on the user interface, send an interaction request for the target interaction operation to the server, and the user interface is used to present the three-dimensional model, the three-dimensional model and the second 2.
  • the user account logged in by the user terminal establishes an association relationship in advance; the second user terminal is configured to: obtain the streaming video; and send the streaming video to the server; the server is configured to: send the streaming media to the first user terminal Video; and the first user terminal is configured to: present a streaming video and a three-dimensional model on the user interface.
  • a non-transitory computer-readable storage medium stores a computer program. method.
  • an electronic device including: a processor; and a memory for storing processor-executable instructions.
  • the processor-executable instructions implement any of the above methods when executed by the processor. The method described in one item.
  • a computer program product including a computer program, which when executed by a computer causes the computer to implement the method described in any one of the above methods.
  • FIG. 1 is a schematic diagram of an embodiment of an applicable scenario of the present disclosure
  • FIG. 2 is a flowchart of an embodiment of a method for realizing interaction of a three-dimensional space scene of the present disclosure
  • FIG. 3 is a flowchart of an embodiment of determining a three-dimensional model corresponding to a first pixel in the present disclosure
  • FIG. 4 is a flowchart of another embodiment of determining a three-dimensional model corresponding to a first pixel point of the present disclosure
  • FIG. 5 is a flowchart of an embodiment of presenting footprint information for browsing users in the present disclosure
  • FIG. 6 is a schematic structural diagram of an embodiment of an apparatus for realizing interaction in a three-dimensional space scene of the present disclosure
  • FIG. 7 is a flowchart of an embodiment of the first interaction method based on a three-dimensional model of the present disclosure.
  • 8A-8C are schematic diagrams of application scenarios for the embodiment of FIG. 7.
  • Fig. 9 is a flowchart of another embodiment of the first three-dimensional model-based interaction method of the present disclosure.
  • FIG. 10 is a flowchart of another embodiment of the first interaction method based on a three-dimensional model of the present disclosure.
  • FIG. 11 is a flowchart of an embodiment of the second three-dimensional model-based interaction method of the present disclosure.
  • FIG. 12 is a flowchart of another embodiment of the second three-dimensional model-based interaction method of the present disclosure.
  • FIG. 13 is a flowchart of an embodiment of the first interactive device based on a three-dimensional model of the present disclosure.
  • FIG. 14 is a flowchart of an embodiment of the second interactive device based on a three-dimensional model of the present disclosure.
  • FIG. 15 is a schematic diagram of interaction of an embodiment of the interactive system based on a three-dimensional model of the present disclosure.
  • Fig. 16 is a structural diagram of an electronic device provided by an exemplary embodiment of the present disclosure.
  • plural may refer to two or more than two, and “at least one” may refer to one, two, or more than two.
  • the term "and/or" in the present disclosure is only an association relationship that describes associated objects, indicating that there can be three relationships, such as A and/or B, which can mean: A alone exists, and A and B exist at the same time. There are three cases of B alone.
  • the character "/" in the present disclosure generally indicates that the associated objects before and after are in an "or" relationship.
  • the embodiments of the present disclosure can be applied to electronic devices such as terminal devices, computer systems, servers, etc., which can operate with many other general-purpose or special-purpose computing system environments or configurations.
  • Examples of well-known terminal devices, computing systems, environments and/or configurations suitable for use with electronic devices such as terminal devices, computer systems, or servers include, but are not limited to: personal computer systems, server computer systems, thin clients, and thick clients Computers, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network personal computers, small computer systems, large computer systems, and distributed cloud computing technology environments including any of the above systems, etc.
  • Electronic devices such as terminal devices, computer systems, and servers can be described in the general context of computer system executable instructions (such as program modules) executed by the computer system.
  • program modules may include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types.
  • the computer system/server can be implemented in a distributed cloud computing environment.
  • tasks can be performed by remote processing devices linked through a communication network.
  • program modules may be located on a storage medium of a local or remote computing system including a storage device.
  • the inventor found that in the process of a user experiencing a three-dimensional space scene by adjusting his current perspective, some feelings such as emotions and thoughts are often generated. If the user can set the footprint information used to characterize his feelings into the three-dimensional space scene, it will not only help improve the user’s sense of participation, but the footprint information left by the user can also provide other users watching the three-dimensional space scene. Come for a richer VR panoramic experience.
  • VR panoramic technology can be used to set a three-dimensional space scene for a house to be rented or a house to be sold. Any user can access through the network and watch the three-dimensional space scene of the corresponding house anytime and anywhere.
  • the present disclosure allows the user to leave his own footprint information for the house he is browsing, and the present disclosure can target the user’s own footprint information and other information about the house. The footprint information left by the user for the house is presented to the user together.
  • the footprint information 120 left by other users for the three-dimensional space scene of the two-bedroom and one-living house includes: "I like this group of sofas, it's great”, “This decorative partition is good”, “This sofa is good, high-end class”, “Combination and collocation are very diligent, praise and praise", "The design of the tea table is very unique ⁇ the longest copywriting is 20 characters” and the three-dimensional model 110 shown in the upper right corner of Figure 1.
  • the user who browses the three-dimensional space scene of the house is presented with the footprint information 120 left by other users with respect to the three-dimensional space scene of the house, which helps the user understand other users’ feelings about the house, thereby helping to deepen the user’s experience of the house. Cognition, which in turn helps to improve the user’s browsing experience of the house.
  • the user can also express his own feelings about the house, that is, leave his own footprint information in the three-dimensional space scene.
  • the user can set footstep information such as "this pillar makes the house look more distinctive" at the position of the pillar shown in FIG.
  • the footprint information set by the user can be instantly displayed in the three-dimensional space scene shown in Figure 1, that is, the user can see the footprint information left by himself during the process of viewing the three-dimensional space scene of the house, which is conducive to enhancing user participation feel.
  • all other footprint information set by the user for the house that does not belong to the view shown in FIG. 1 can be presented to the user in the form of a bullet screen 130, which is beneficial to improve the user's ability to browse three-dimensional space scenes in other locations of the house. interest.
  • the technology for realizing the interaction of three-dimensional space scenes provided by the present disclosure can also be applied to various other scenes. For example, when a user browses a three-dimensional space scene of a library, it can target a book or chair or coffee in the library. Set the corresponding footprint information.
  • the footprint information set by the user for the book may be the user's impression of the book or the number of pages currently read by the user.
  • the scenarios where the technology for realizing the interaction of three-dimensional space scenes provided by the present disclosure can be applied will not be illustrated one by one.
  • FIG. 2 is a flowchart of an embodiment of a method for realizing interaction of a three-dimensional space scene of the present disclosure.
  • the method 200 of the embodiment shown in FIG. 2 includes steps 210 to 240. Each step is described separately below.
  • step 210 in response to detecting the user operation of setting the footprint information in the three-dimensional space scene, determine the first pixel in the current view corresponding to the user's current perspective in the three-dimensional space scene.
  • a three-dimensional space scene may refer to a space scene with a three-dimensional sense that is presented to the user by using a preset panoramic image and a three-dimensional model.
  • the three-dimensional space scene may be a three-dimensional space scene set for a library, a three-dimensional space scene set for a house, a three-dimensional space scene set for a cafe, or a three-dimensional space scene set for a shopping mall.
  • the embodiment of the present disclosure when the user triggers the function of setting the footprint information in the three-dimensional space scene, it can be detected that the user needs to set the footprint information in the three-dimensional space scene. For example, when the user clicks a button for setting footstep information or a corresponding option on a menu, the embodiment of the present disclosure can detect that the user needs to set footstep information in a three-dimensional space scene. For another example, the user can use a preset shortcut to trigger the function of setting footprint information in the three-dimensional space scene.
  • the user's footprint information may be information that can indicate that the user has visited the three-dimensional space scene. The footprint information can be considered as the visit trace information of the user.
  • the current perspective of the user in the three-dimensional space scene may refer to the position and angle at which the user currently views the three-dimensional space scene.
  • the user's current perspective in the three-dimensional space scene usually changes with the user's operation. For example, the user can control his current perspective in the three-dimensional scene by performing operations such as dragging on the touch screen.
  • the user's current perspective in the three-dimensional space scene determines the content/area of the panorama that the user can currently see, that is, the user's current perspective in the three-dimensional space scene determines the current view.
  • the first pixel point is one pixel point in the current view.
  • the first pixel point can be obtained according to a preset default rule.
  • the first pixel may be a specific pixel in the current view, or it may be any pixel in the current view.
  • step 220 the three-dimensional model corresponding to the first pixel is determined.
  • a three-dimensional space scene is generally formed by a plurality of three-dimensional models.
  • the three-dimensional space scene may also be formed by a three-dimensional model.
  • a pixel point in the current view seen by the user may be a representation of a point in the three-dimensional model.
  • a pixel in the current view that the user sees may not be a representation of any point in the three-dimensional model. That is to say, under normal circumstances, any point in any three-dimensional model in the three-dimensional space scene can be presented in the panorama, and the points in the panoramic image may not be all three-dimensional models in the three-dimensional scene. In the point.
  • the present disclosure does not exclude the possibility that some points in the three-dimensional model in the three-dimensional space scene are not presented in the panoramic image.
  • the three-dimensional model where the point is located is the three-dimensional model corresponding to the first pixel.
  • the three-dimensional model corresponding to the first pixel may be: the current view that is similar to the first pixel and is used to present the points in the three-dimensional model.
  • the three-dimensional model corresponding to other pixels that is to say, when the first pixel is used to present the points in the non-three-dimensional model, and the first pixel is not updated, the three-dimensional model corresponding to other pixels in the current view can be used as the three-dimensional model corresponding to the first pixel. Model.
  • step 230 the position of the user's footprint information in the three-dimensional model is determined, where the footprint information is used to display when the three-dimensional space scene is browsed.
  • the positions of the first pixel or the other pixels in the three-dimensional model can be obtained. This location is the location of the user's footprint information.
  • all three-dimensional models in a three-dimensional space scene may be respectively provided with their own three-dimensional coordinate systems, or may have the same three-dimensional coordinate system.
  • the position of the user's footprint information in the three-dimensional model can be represented by (x, y, z). That is, the user's footprint information can be deep.
  • step 240 the user's footprint information is set at the location.
  • setting the user's footprint information at the location may include: setting a three-dimensional model identifier and three-dimensional coordinates for the user's footprint information, and storing the three-dimensional model identifier, three-dimensional coordinates, and user's footprint information. Correspondence.
  • the user's footprint information may be used to display to browsing users (such as all browsing users or some browsing users) of the three-dimensional space scene.
  • the browsing user of the three-dimensional space scene may include the user who sets the footprint information.
  • the three-dimensional model corresponding to the first pixel and the position of the footprint information in the three-dimensional model are obtained, so that the user can set
  • the footprint information of can be associated with the corresponding position of the corresponding 3D model.
  • the footprint information includes: at least one of text, picture, audio, video, and a three-dimensional model.
  • the text can be considered as a message in the form of characters (such as text, letters, numbers, or symbols, etc.).
  • a picture can be considered as a message in the form of an image (such as a photo or emoticon, etc.).
  • Audio can be thought of as a voice message (also called memo, etc.).
  • Video can be thought of as a message in the form of an image.
  • the three-dimensional model can be considered as a three-dimensional message.
  • the user's footprint information may be referred to as the user's message.
  • a piece of footprint information set by the user may include one or more of text, picture, audio, video, and three-dimensional model at the same time.
  • the user's footprint information By enabling the user's footprint information to include at least one of text, pictures, audio, video, and a three-dimensional model, it is beneficial to enrich the expression form of the user's footprint information, thereby helping to enrich the way for the user to interact with the three-dimensional space scene.
  • obtaining the first pixel point in the current view corresponding to the current perspective of the user in the three-dimensional space scene may be: obtaining the central pixel point of the current view corresponding to the current perspective of the user in the three-dimensional space scene , And use the central pixel as the first pixel.
  • the user’s current perspective in the 3D space scene triggers the function of setting footprint information in the 3D space scene by clicking a button or an option on the menu.
  • the center pixel can be considered as the default pixel set for the user's footprint information, and the user can change the default pixel by dragging and other methods.
  • the central pixel can be considered as a pixel in the central area of the current view.
  • the central area of the current view may include one pixel or multiple pixels.
  • obtaining the first pixel in the current view corresponding to the current perspective of the user in the three-dimensional space scene may be: setting the footprint information in the current view corresponding to the current perspective of the user in the three-dimensional space scene
  • the operation of the target position obtains the pixel point in the current view corresponding to the target position of the footprint information, and the pixel point is regarded as the first pixel point. That is, when the user performs the operation of setting the target position of the footprint information, the pixel point where the target position of the footprint information formed by the operation in the current view is located may be used as the first pixel point.
  • the operation of setting the target position of the footprint information may be an operation used to determine the starting target position of the footprint information, an operation used to determine the ending target position of the footprint information, or an operation used to determine the footprint information. Operation of the center target position.
  • the operation of setting the target location of the footprint information may specifically be a click operation or a scroll operation or drag operation based on a tool such as a mouse or a keyboard, and may also be a click operation or a drag operation based on a touch screen.
  • the present disclosure does not limit the specific operation of setting the target position of the footprint information.
  • the footprint information set by the user By determining the first pixel point according to the user's operation of setting the target position of the footprint information, it is beneficial to make the footprint information set by the user be located at the desired position of the user, thereby improving the flexibility of setting the footprint information and making the footprint information more stable. The location is more appropriate.
  • the user triggers the function of setting footprint information in the 3D space scene by clicking a button or an option on the menu.
  • the user can use the left mouse button to click, the keyboard up, down, left, and right keys to move the cursor or click the corresponding position on the touch screen to set the position of the desired footprint information in the current view.
  • the pixel at this position can be used as the first pixel.
  • the user triggers the function of setting footprint information in the 3D space scene by clicking a button or an option on the menu.
  • the pixel point is regarded as the first pixel point.
  • step 220 the implementation of determining the three-dimensional model corresponding to the first pixel (step 220) may be as shown in FIG. 3. As shown in FIG. 3, step 220 further includes steps 310 to 340.
  • step 310 the central pixel point of the front view is determined as the first pixel point.
  • the central pixel may be considered as the default pixel set for the user's footprint information.
  • the current view is an image of (2n+1) ⁇ (2m+1) (where n and m are both integers greater than 1), then the pixels in the current view (n+1 , M+1) as the central pixel.
  • the pixel point (n, m) and pixel point (n+1) in the current view can be , M), pixel (n, m+1), and pixel (n+1, m+1) are used as the central area of the current view, so that any pixel in the central area can be used as the central pixel.
  • step 320 it is determined whether a three-dimensional model is set for the first pixel. If a three-dimensional model is set for the first pixel point, go to step 330. If the three-dimensional model is not set for the first pixel point, go to step 340.
  • step 330 in response to the determination that the three-dimensional model is set for the first pixel, the three-dimensional model set for the first pixel is used as the three-dimensional model corresponding to the first pixel.
  • step 340 in response to the determination that the three-dimensional model is not set for the first pixel, the three-dimensional model set for other pixels in the current view is used as the three-dimensional model corresponding to the first pixel.
  • pixels in the current view are pixels in the current view where the three-dimensional model is set.
  • the pixels where the three-dimensional model is set can be found according to preset rules.
  • the other pixels found may be the pixels closest to the first pixel in a certain direction (such as the left direction, the right direction, the upper direction, or the lower direction).
  • the first pixel point can be used as a starting point, and according to a preset inspection rule, the pixel point in the current view corresponding to the current perspective in the three-dimensional space scene can be checked. If it is determined that the pixel point with the three-dimensional model is checked , The three-dimensional model corresponding to the first pixel is obtained, and the inspection process is stopped. For example, you can use the first pixel as a starting point to check the pixels in the current view to the left, and determine whether a three-dimensional model is set for the currently checked pixel.
  • the inspection process is stopped, and the three-dimensional model obtained by the current inspection is used as the three-dimensional model corresponding to the first pixel.
  • the first pixel point may be updated by using the detected pixel point provided with the three-dimensional model.
  • the first pixel may not be updated.
  • step 220 may be as shown in FIG. 4.
  • step 220 may include step 410 to step 450.
  • step 410 in response to the user's operation of setting the target position of the footprint information in the current view, a pixel point in the current view corresponding to the target position of the footprint information is determined as the first pixel point.
  • the user may be allowed to set the specific location of the footprint information (that is, the target location of the footprint information) in the current view.
  • the user can set the target position of the footprint information in the current view by tapping, sliding, dragging and other operations on the touch screen.
  • the target position of the footprint information may be the upper left vertex, the lower left vertex, the upper right vertex, or the lower right vertex of the text box.
  • the target location of the footprint information may be the upper left vertex, the lower left vertex, the upper right vertex, or the lower right vertex of the picture.
  • the target location of the footprint information may be a pixel point in the current view, and the pixel point is the first pixel point.
  • step 420 it is determined whether a three-dimensional model is set for the first pixel. If a three-dimensional model is set for the first pixel point, go to step 430. If a three-dimensional model is not set for the first pixel point, go to step 440.
  • step 430 in response to the determination that the three-dimensional model is set for the first pixel, the three-dimensional model set for the first pixel is used as the three-dimensional model corresponding to the first pixel.
  • step 440 in response to the determination that the three-dimensional model is not set for the first pixel point, prompt information for updating the target position of the footprint information is output.
  • the prompt information is used to prompt the user to update the target location of the footprint information currently set. That is, the prompt information is used to prompt the user that the current target location of the footprint information cannot be set to the footprint information, and the user should reset the target location of the footprint information.
  • the prompt information can be output in the form of text, audio, or graphics. After outputting the prompt information, wait for the user's subsequent operations. If the user triggers the function of canceling the setting of the footprint information at this time, the process shown in Figure 4 ends.
  • step 450 in response to the determination that the pixel in the current view corresponding to the target position of the updated footprint information is set with a three-dimensional model, the pixel set with the three-dimensional model is taken as the first pixel. Then, the flow returns to step 420.
  • the target position of the footprint information obtained again may also be a pixel in the current view, and this pixel is the first pixel. That is, the first pixel point obtained last time is updated by the similarity of the target position of the currently obtained footprint information.
  • the first pixel when the first pixel is provided with a three-dimensional model, since the first pixel in the current view has a mapping relationship with the point in the three-dimensional model, the first pixel can be obtained based on the mapping relationship.
  • the point corresponding to the point in the three-dimensional model the position of the point is the position of the first pixel point in the three-dimensional model.
  • the position of the first pixel in the three-dimensional model can be directly used as the position of the user's footprint information in the three-dimensional model, which is beneficial to quickly and accurately obtain the position of the user's footprint information in the three-dimensional model.
  • the browsing user in the process of viewing the three-dimensional space scene by the browsing user, the browsing user may be presented with at least one user's footprint information left in the three-dimensional space scene.
  • An example is shown in Figure 5.
  • step 510 for any browsing user browsing the three-dimensional space scene, the footprint area corresponding to the current perspective of the browsing user in the three-dimensional space scene is determined.
  • the browsing user includes a user who sets his footprint information in the three-dimensional space scene.
  • the footprint area can be considered as an area set for the footprint information that needs to be displayed.
  • the footprint area can be a footprint area based on the current view, or a footprint area based on a three-dimensional model.
  • the size of the footprint area can be preset.
  • the shape of the footprint area can be rectangle, circle, triangle, etc.
  • an implementation manner of determining the footprint area may be: First, obtain the current view corresponding to the current viewing angle of the browsing user in the three-dimensional space scene. The center pixel of the view, and then the center pixel is the center of the circle, and the predetermined length (such as 1.5 meters in a three-dimensional scene, etc., and 1.5 meters can be converted to the length in the current view) is used as the radius to determine the current view Footprint area. Since at least some of the pixels in the footprint area in the current view have a mapping relationship with points in the three-dimensional model, the footprint information that currently needs to be displayed can be easily obtained by using the footprint area in the current view. In addition, the footprint area in the current view can be regarded as a circle, that is, the footprint area in the current view does not have depth information.
  • an implementation manner for determining the footprint area may be: First, obtain the current view corresponding to the current perspective of the browsing user in the three-dimensional space scene. View the center pixel of the view, and determine whether the center pixel is set with a three-dimensional model. If the center pixel is set with a three-dimensional model, determine the position of the center pixel in the three-dimensional model, and then use that position as the center of the circle, Use a predetermined length (such as 1.5 meters in a three-dimensional space scene, etc.) as a radius to determine the footprint area in the three-dimensional model.
  • the footprint area may be completely in one 3D model, or it may span multiple 3D models.
  • the footprint area in the three-dimensional model can be considered as a cylinder, that is, the footprint area in the three-dimensional model has depth information.
  • step 520 the footprint information belonging to the footprint area in the three-dimensional model is determined.
  • the embodiment of the present disclosure can check whether each pixel point in the footprint area has a mapping relationship with a point in the three-dimensional model. If there is a mapping relationship, then it is determined whether the points in the three-dimensional model that have a mapping relationship with the pixel points are provided with footprint information. If the footprint information is set, the footprint information can be regarded as the footprint information belonging to the footprint area.
  • the embodiments of the present disclosure can check whether each point in the footprint area is provided with footprint information. If the footprint information is set, the footprint information can be regarded as the footprint information belonging to the footprint area.
  • step 530 in the current view corresponding to the current perspective of the browsing user in the three-dimensional space scene, the footprint information belonging to the footprint area is displayed.
  • the location of each footprint information belonging to the footprint area in the current view can be determined according to the location of each footprint information, so that each footprint information can be displayed according to the location of each footprint information in the current view.
  • the process of displaying footprint information it is possible to avoid overlapping display of different footprint information in the current view.
  • the obtained multiple footprint information may have different positions, or may have the same position (that is, the position of the footprint information conflicts).
  • each footprint information may be displayed in the current view directly according to the image positions of the multiple footprint information in the current view.
  • the displayed footprint information can be allowed to partially overlap, and the location control can also be used to make the footprint information not overlap each other.
  • different image positions may be assigned to different footprint information in the current view, and the image positions described above may be displayed in the current view according to the assigned image positions. Having different footprint information at the same position helps to avoid overlapping display of different footprint information in the current view.
  • all the footprint information belonging to the footprint area can be displayed, or part of the footprint information belonging to the footprint area can be displayed.
  • part of the footprint information can be selected from it according to a predetermined rule, and the selected part of the footprint information can be displayed in the current view.
  • a predetermined number of footprint information can be randomly selected from all the footprint information belonging to the footprint area, and part of the randomly selected footprint information can be displayed in the current view.
  • the form of a bullet screen may be used to display the footprint information outside the current view for the browsing user. For example, you can first determine all the footprint information in the three-dimensional model that does not belong to the current view, and display all the above-mentioned footprint information in the current view corresponding to the current perspective of the browsing user in the three-dimensional scene in the form of a bullet screen. Part of the footprint information.
  • the form of a bullet screen may be used to display the footprint information outside the footprint area for the browsing user. For example, you can first determine all the footprint information in the three-dimensional model that does not belong to the footprint area, and display all the above-mentioned footprint information in the current view corresponding to the current perspective of the browsing user in the three-dimensional scene in the form of a bullet screen. Part of the footprint information.
  • FIG. 6 is a schematic structural diagram of an embodiment of an apparatus for realizing interaction in a three-dimensional space scene of the present disclosure.
  • the device of this embodiment can be used to implement the foregoing method embodiments of the present disclosure.
  • the device of this embodiment includes: a pixel point acquiring module 600, a three-dimensional model determining module 601, a position determining module 602, and a footprint information setting module 603.
  • the device may further include: a footprint area determination module 604, a footprint information determination module 605, a footprint information display module 606, and a bullet screen display module 607.
  • the pixel obtaining module 600 is configured to determine the first pixel in the current view corresponding to the current perspective of the user in the three-dimensional space scene in response to detecting the user operation of setting the footprint information in the three-dimensional space scene.
  • the footprint information may include: at least one of text, picture, audio, video, and a three-dimensional model.
  • the pixel point acquiring module 600 may include: a first sub-module 6001.
  • the first sub-module 6001 is used to determine the center pixel of the current view as the first pixel.
  • the pixel point obtaining module 600 may include: a fifth sub-module 6002.
  • the fifth sub-module 6002 is configured to determine the pixel points in the current view corresponding to the target position of the footprint information in response to the user's operation of setting the target position of the footprint information in the current view corresponding to the current perspective in the three-dimensional space scene.
  • the fifth sub-module 6002 can use the pixel as the first pixel.
  • the three-dimensional model determining module 601 is used to determine the three-dimensional model corresponding to the first pixel obtained by the pixel obtaining module 600.
  • the determining three-dimensional model module 601 may include: the second sub-module 6011, the third sub-module 6012, and the fourth sub-module 6013.
  • the second sub-module 6011 is used to determine whether a three-dimensional model is set for the first pixel.
  • the third sub-module 6012 is configured to, if the determination result of the second sub-module 6011 is that a three-dimensional model is set for the first pixel, use the three-dimensional model set for the first pixel as the three-dimensional model corresponding to the first pixel.
  • the fourth sub-module 6013 is configured to, if the judgment result of the second sub-module 6011 is that no three-dimensional model is set for the first pixel, use the three-dimensional model set for other pixels in the current view as the three-dimensional model corresponding to the first pixel. Model. For example, if the judgment result of the second sub-module 6011 is that a three-dimensional model is not set for the first pixel, the fourth sub-module 6013 can take the first pixel as a starting point and perform a check on the three-dimensional scene according to the preset inspection rules. Check other pixels in the current view corresponding to the current angle of view. If a pixel with a three-dimensional model is detected, the first pixel is updated to a pixel with a three-dimensional model, the three-dimensional model corresponding to the first pixel is obtained, and this inspection is stopped.
  • the determining three-dimensional model module 601 may include: a sixth sub-module 6014, a seventh sub-module 6015, and an eighth sub-module 6016.
  • the sixth sub-module 6014 is used to determine whether a three-dimensional model is set for the first pixel. If the determination result of the sixth sub-module 6014 is that a three-dimensional model is set for the first pixel, the seventh sub-module 6015 uses the three-dimensional model set for the first pixel as the three-dimensional model corresponding to the first pixel.
  • the eighth sub-module 6016 may output prompt information for updating the target position of the footprint information, and the sixth sub-module 6014 determines When the pixel in the current view corresponding to the target location of the footprint information is set with a three-dimensional model, the pixel with the three-dimensional model is used as the first pixel.
  • the eighth sub-module 6016 obtains the three-dimensional model corresponding to the first pixel.
  • the position determining module 602 is used to determine the position of the user's footprint information in the three-dimensional model determined by the three-dimensional model determining module 601. For example, the position determining module 602 may obtain the position of the first pixel in the three-dimensional model, and the position determining module 602 may use the position of the first pixel in the three-dimensional model as the position of the user's footprint information in the three-dimensional model.
  • the setting footprint information module 603 is used for setting the user's footprint information at the location determined by the location determining module 602.
  • the user's footprint information set by the setting footprint information module 603 is used to be displayed to users who browse the three-dimensional space scene.
  • the footprint area determination module 604 is used for determining the footprint area corresponding to the current perspective of the browsing user in the three-dimensional space scene for any browsing user who browses the three-dimensional space scene. For example, the module 604 for determining the footprint area may first determine the center pixel of the current view corresponding to the current perspective of the browsing user in the three-dimensional space scene, and then the module 604 for determining the footprint area 604 takes the center pixel as the center of the circle and the predetermined length as The radius determines the footprint area in the current view.
  • the footprint information determining module 605 is used to determine the footprint information belonging to the footprint area determined by the footprint area determining module 604 in the three-dimensional model.
  • the footprint information display module 606 is configured to display the footprint information that belongs to the footprint area determined by the determination footprint information module 605 in the current view corresponding to the current perspective of the browsing user in the three-dimensional space scene.
  • the display footprint information module 606 may respectively locate the multiple footprint information in the current view according to the image positions of the multiple footprint information in the current view. Display the multiple footprint information.
  • the footprint information display module 606 may assign different image positions for different footprint information in the current view, and according to the assigned image Position, display different footstep information in the current view.
  • the barrage display module 607 is used to determine at least one piece of footprint information in the three-dimensional model that does not belong to the footprint area/current view.
  • the barrage display module 607 displays the at least one footprint information in the current view corresponding to the current perspective of the browsing user in the three-dimensional space scene in the form of a barrage.
  • FIG. 7 shows a process 700 of an embodiment of the first three-dimensional model-based interaction method according to the present disclosure.
  • the three-dimensional model-based interaction method is applied to a first user terminal, and the first user terminal is presented with a user interface, and the three-dimensional model-based interaction method includes:
  • Step 710 In response to detecting the user's target interaction operation on the user interface, send an interaction request for the target interaction operation to the server that provides page data for the user interface, where the user interface is used to present the three-dimensional model, the three-dimensional model and the second user The user account logged in by the terminal establishes an association relationship.
  • the user can use the first user terminal to interact with the server through the network.
  • the first user terminal may be various electronic devices, including but not limited to smart phones, tablet computers, laptop portable computers, desktop computers, and so on.
  • the first user terminal may be installed with various client applications, such as real estate transaction software.
  • the aforementioned user interface may be a page in an application installed by the first user terminal.
  • the user can interact with the server through the user interface, thereby realizing interaction with other user terminals (for example, the second user terminal).
  • the first user terminal may send an interaction request for the target interaction operation to a server that provides page data for the user interface.
  • the aforementioned user interface is used to present a three-dimensional model.
  • the three-dimensional model establishes an association relationship with the user account logged in by the second user terminal in advance.
  • the aforementioned target interaction operation may be various operations for instructing the first user terminal to request interaction (information interaction) with the second user terminal.
  • the target interaction operation may indicate video communication with the second user terminal.
  • the foregoing interaction request may be used to indicate a user request of the first user terminal to interact with the second user terminal.
  • the foregoing interaction request may be used to instruct the user of the first user terminal to request video communication with the second user terminal.
  • the user interface of the first user terminal may present the above-mentioned three-dimensional model, or may not present the three-dimensional model.
  • each three-dimensional model it can be associated with a user account in advance. Therefore, for a specific three-dimensional model, the user account that is associated with the three-dimensional model can be determined to determine the user terminal logging in to the user account, and then the user terminal used to interact with the first user terminal is determined (Ie the second user terminal).
  • the above-mentioned three-dimensional model may be a three-dimensional model of any object.
  • the three-dimensional model may be a three-dimensional model inside a cell, or a three-dimensional model of a house interior.
  • Step 720 Receive the streaming video obtained by the server from the second user terminal.
  • the above-mentioned first user terminal may receive the streaming video obtained by the server from the second user terminal.
  • the aforementioned interaction confirmation information may be used to instruct the user of the second user terminal to confirm (agree) to perform the interaction indicated by the aforementioned interaction request with the first user terminal.
  • the foregoing interactive confirmation information may be used to instruct the user of the second user terminal to confirm (agree) to conduct video communication with the first user terminal.
  • the aforementioned streaming video may include images and/or voice.
  • the image acquisition device and/or the voice acquisition device of the second user terminal can be used to acquire the aforementioned streaming video.
  • the server may use streaming media technology to continuously send the images and/or voice (ie streaming media video) collected by the second user terminal to the first user terminal.
  • streaming media technology refers to a media format that uses streaming technology to be continuously played in real time on the network.
  • Streaming media technology is also called streaming media technology.
  • the second user terminal may send the continuous image and sound information collected by it to the server after compression processing.
  • the server transmits each compressed package to the first user terminal sequentially or in real time, so that users who use the first user terminal can watch and listen while downloading.
  • the server may send the streaming video collected by the second user terminal to the first user terminal, and may also perform image processing (such as beauty) and voice processing (such as denoising) on the streaming video collected by the second user terminal. ), after operations such as transcoding, recording, and pornography, send the processed streaming video to the first user terminal.
  • image processing such as beauty
  • voice processing such as denoising
  • the first user terminal may perform step 720 again.
  • the first user terminal in the case of the interaction confirmation information sent by the second user terminal in response to the interaction request, can present the streaming video through subsequent steps; however, the second user terminal does not send the above
  • the first user terminal does not present the streaming video. Therefore, the streaming video and the three-dimensional model can be presented on the user interface of the first user terminal only after obtaining the permission of the user of the second user terminal (for example, connecting to the video call initiated by the first user terminal). This helps to improve the privacy protection of the user of the second user terminal, and provides preparation time for the user of the second user terminal to present the streaming media video to the user of the first user terminal.
  • the first user terminal may also directly execute the foregoing step 720 (without the interaction confirmation information sent by the second user terminal in response to the interaction request).
  • the user of the second user terminal may be in a state of shooting a streaming video (for example, a live broadcast) to users of other user terminals.
  • a streaming video for example, a live broadcast
  • the first user terminal can receive the streaming video obtained by the server from the second user terminal at any time, thereby improving the real-time performance of the streaming video presentation.
  • the first user terminal may adopt the following steps to receive the streaming video obtained by the server from the second user terminal:
  • the current network speed value of the first user terminal is sent to the server.
  • the streaming media video obtained and sent by the server from the second user terminal is received, and the streaming media video has a resolution matching the current network speed value.
  • the resolution can be positively correlated with the network speed value.
  • the resolution of the streaming video received by the first user terminal can be reduced when the network is poor by receiving the streaming video sent by the server whose resolution matches the current network speed value and obtained by the second user terminal. Rate to improve the real-time performance of streaming video transmission.
  • Step 730 Present the streaming video and the three-dimensional model on the user interface.
  • the first user terminal may present the streaming video and the three-dimensional model on the same screen on the user interface.
  • the above-mentioned user interface of the first user terminal may be divided into two parts, and the above-mentioned two parts may respectively present a streaming video and a three-dimensional model.
  • the three-dimensional model can also be used as the background of the aforementioned user interface, and the streaming video is presented in a part of the page area of the user interface.
  • FIGS. 8A-8C are schematic diagrams of application scenarios for the embodiment of FIG. 7.
  • the first user terminal may provide the user interface with The server of the page data sends an interaction request for the target interaction operation 810.
  • the user interface presents a three-dimensional model of the house of XX home.
  • the three-dimensional model has a pre-established association relationship with the user account logged in by the second user terminal.
  • the first user terminal presents a streaming video 830 and a three-dimensional model on the user interface.
  • the interaction method based on the three-dimensional model provided by the above-mentioned embodiments of the present disclosure can send an interaction request for the target interaction operation to a server that provides page data for the user interface when the user's target interaction operation for the user interface is detected.
  • the user interface is used for presenting a three-dimensional model, and the three-dimensional model establishes an association relationship with the user account logged in by the second user terminal in advance.
  • the streaming video obtained by the server from the second user terminal is received.
  • the streaming video and 3D model are presented on the user interface.
  • the first user terminal may also perform the following steps:
  • the model adjustment information sent by the server is received, where the model adjustment information indicates an adjustment operation of the user who uses the second user terminal on the three-dimensional model presented on the second user terminal.
  • the adjustment operation includes at least one of the following: zoom, rotate, move, and switch viewpoints.
  • the user can perform at least one operation of zooming, rotating, moving, and switching viewpoints on the three-dimensional model.
  • the same adjustment operation is performed on the three-dimensional model presented on the user interface.
  • the operations performed by the user of the second user terminal on the three-dimensional model can be synchronized to the first user terminal. Therefore, when the streaming video collected by the second user terminal is related to the three-dimensional model (for example, the user of the second user terminal explains or introduces the three-dimensional model), it is convenient for the user of the first user terminal to refer to the second user The same three-dimensional model presented by the terminal acquires the information in the streaming video, thereby improving the pertinence of information acquisition.
  • the first user terminal may also perform the following steps:
  • the feedback information may include but is not limited to at least one of the following: likes, ratings, comments, and so on.
  • the feedback information may be used to characterize the evaluation of the user of the first user terminal on the streaming video of the user of the second user terminal.
  • the feedback information is sent to the server, where the server is used to establish an association relationship between the feedback information and the user account.
  • the server is used to establish an association relationship between the feedback information and the user account.
  • an associative storage method can be used to establish an association relationship between the feedback information and the user account.
  • establishing an association relationship between the feedback information and the user account can reflect the user's satisfaction with the object indicated by the three-dimensional model and the user of the second user terminal by the user of the first user terminal, and thus can be more targeted for the first user The terminal pushes information.
  • FIG. 9 is a flow 900 of another embodiment of the first three-dimensional model-based interaction method of the present disclosure.
  • the three-dimensional model-based interaction method is applied to a first user terminal, and the first user terminal is presented with a user interface, and the method includes:
  • Step 910 In response to detecting the user's target interaction operation on the user interface, send an interaction request for the target interaction operation to a server that provides page data for the user interface.
  • Step 920 Receive the streaming video obtained by the server from the second user terminal.
  • Step 930 Present the streaming video and the three-dimensional model on the user interface.
  • step 910 to step 930 are basically the same as step 710 to step 730 in the embodiment corresponding to FIG. 7, and will not be repeated here.
  • Step 940 In response to the current network speed value of the first user terminal being less than or equal to the preset network speed threshold, adjust the target user image based on each frame of voice in the streaming video to generate a new video different from the streaming video.
  • the first user terminal may adjust the target user image based on each frame of voice in the streaming video, Generate a new video.
  • the new video characterizes the actions of the user indicated by the target user's image to perform each frame of voice instructions.
  • the user indicated by the target user image may be a user using the second user terminal.
  • the new video may be a streaming video that is sent in segments and instantly transmitted based on the network, or it may be a video that is generated locally without being based on the network.
  • the first user terminal may generate a new video in the following manner: For each frame of voice in the streaming video, input the frame of voice into a predetermined image frame generation model to obtain a target user that matches the frame of voice The image of the user indicated by the image. Thereby, the obtained frames of images that match each frame of voice in the streaming video and the frames of voice are merged to obtain a new video. The user's action in the user's image indicated by the target user image that matches the voice matches the voice.
  • the mouth shape of the user in the user's image indicated by the target user image matching the audio may be the voice "ah” Lip shape
  • the action can be an action in a state of fright.
  • the aforementioned image frame generation model may be a recurrent neural network model or a convolutional neural network model obtained by training using a machine learning algorithm based on training samples including voice frames, target user images, and image frames matching the voice frames.
  • An image frame generation model can be trained for each user, and the target user image in each training sample used to train the user’s image frame generation model can be the same.
  • For each voice frame of the user it is determined that the voice frame corresponds to the voice frame. Matched image frames, and then obtain a training sample set used to train the image frame generation model of the user.
  • the image frame generation model may also be a two-dimensional table or database that stores the voice frame, the target user image, and the image frame matching the voice frame in association with each other.
  • each record of the database may include voice frames, target user images and matching voice frames Image frame.
  • the target user image in each record can be the same.
  • Frame database that is, image frame generation model.
  • the first user terminal may also determine the target user image by any of the following methods:
  • the target user image is generated.
  • the image whose ratio is greater than the preset threshold is regarded as the target user image.
  • the user can upload an image through the user account he uses as the target user image; or after logging in the account he uses, select an image from a predetermined image set as the target user image.
  • the above-mentioned optional implementation methods can automatically generate a target user image from the images in the streaming video, or the user manually sets the target user image, so that based on multiple target user image determination methods, the new video
  • the generation method is more diversified.
  • step 950 the new video is used to replace the streaming video for presentation.
  • the first user terminal may use a new video to replace the streaming video for presentation.
  • the streaming video can be hidden (that is, no longer presented).
  • the first The user terminal can locally generate a new video to replace the streaming video presentation. Therefore, the first user terminal only needs to continuously obtain voice from the server, but does not need to continuously obtain video, thereby reducing the occupation of network resources. In a case where the current network speed value of the first user terminal is relatively small, the real-time performance of the video presentation of the first user terminal can be improved.
  • the first user terminal may also send a camera shutdown confirmation to the server. information.
  • the camera closing confirmation information is used to determine whether the second user terminal closes the camera.
  • the server may send to the second user terminal information for determining whether the second user terminal turns off the camera. Therefore, the user of the second user terminal can reduce the occupation of network resources by the second user terminal by turning off the camera.
  • FIG. 10 is a flowchart of another embodiment of the first three-dimensional model-based interaction method of the present disclosure.
  • the interaction method based on the three-dimensional model is applied to a first user terminal, and the first user terminal presents a user interface.
  • the process 1000 of the interaction method based on the three-dimensional model includes:
  • Step 1010 In response to detecting the user's target interaction operation on the user interface, send an interaction request for the target interaction operation to a server that provides page data for the user interface.
  • the user interface is used for presenting a three-dimensional model, and the three-dimensional model establishes an association relationship with the user account logged in by the second user terminal in advance.
  • Step 1020 Receive the streaming video obtained by the server from the second user terminal.
  • Step 1030 Present the streaming video and the three-dimensional model on the user interface.
  • step 1010 to step 1030 are basically the same as step 710 to step 730 in the embodiment corresponding to FIG. 7, and will not be repeated here.
  • the three-dimensional model includes three-dimensional sub-models of multiple sub-space scenes, and the sub-space scenes in the multiple sub-space scenes correspond to keywords in a predetermined keyword set.
  • Step 1040 Perform voice recognition on the voice in the streaming video to obtain a voice recognition result.
  • the first user terminal may perform voice recognition on the voice in the streaming video to obtain the voice recognition result.
  • the voice recognition result can represent the text corresponding to the voice in the streaming video.
  • Step 1050 In response to the determination that the voice recognition result includes keywords in the keyword set, present on the user interface a three-dimensional sub-model of the corresponding sub-space scene among the multiple sub-space scenes corresponding to the keywords included in the voice recognition result .
  • the first user terminal may display the subspace scene corresponding to the keywords contained in the voice recognition result on the aforementioned user interface.
  • the above-mentioned three-dimensional model is a three-dimensional model inside a house.
  • the house includes a bedroom, a living room, a kitchen, and a bathroom, with a total of four sub-space scenes. That is, the above-mentioned three-dimensional model includes a three-dimensional sub-model of a bedroom, a three-dimensional sub-model of a living room, a three-dimensional sub-model of a kitchen, and a three-dimensional sub-model of a bathroom.
  • the keyword set includes bedroom, living room, kitchen, bathroom.
  • the keyword corresponding to the subspace scene bedroom can be "bedroom”; the keyword corresponding to the subspace scene kitchen can be “kitchen”; the keyword corresponding to the subspace scene living room can be “living room” ; The keyword corresponding to the bathroom in the subspace scene can be "toilet”. Further, as an example, if the voice recognition result includes the keyword "bedroom", then the first user terminal may present a three-dimensional sub-model of the bedroom on the aforementioned user interface.
  • the embodiment of the present application may also include the same or similar features and effects as the embodiment corresponding to FIG. 7 and/or FIG. 9, and details are not described herein again.
  • the viewpoint switching of the three-dimensional model can be realized by voice, thereby presenting the subspace scene corresponding to the keywords contained in the voice recognition result The three-dimensional sub-model.
  • the convenience of browsing the three-dimensional model is improved, and the matching between the presented three-dimensional model and the voice acquired by the second user terminal is improved.
  • FIG. 11 shows a process 1100 of an embodiment of the second three-dimensional model-based interaction method according to the present disclosure.
  • the three-dimensional model-based interaction method is applied to a second user terminal, and the user account logged in by the second user terminal establishes an association relationship with the three-dimensional model in advance.
  • the interactive method based on the 3D model includes:
  • Step 1110 In response to receiving the interactive request sent by the server, obtain the streaming video.
  • the user can use the second user terminal to interact with the server and the first user terminal through the network.
  • the second user terminal may be various electronic devices, including but not limited to smart phones, tablet computers, laptop computers, desktop computers, and so on.
  • the second user terminal may be installed with various client applications, such as real estate transaction software.
  • the streaming video upon receiving the interaction request sent by the server, the streaming video is acquired.
  • the interaction request indicates that the first user terminal detects the user's target interaction operation on the user interface presented by the first user terminal.
  • the aforementioned interaction request may be used to instruct the user of the first user terminal to request video communication with the second user terminal.
  • the user interface is used to present the three-dimensional model.
  • Streaming videos can contain images and/or voice.
  • the image acquisition device and/or the voice acquisition device of the second user terminal can be used to acquire the aforementioned streaming video.
  • the first user terminal may send an interaction request for the target interaction operation to a server that provides page data for the user interface.
  • the user interface is used to present the three-dimensional model.
  • the three-dimensional model establishes an association relationship with the user account logged in by the second user terminal in advance.
  • the aforementioned target interaction operation may be various operations for instructing the first user terminal to request interaction (information interaction) with the second user terminal.
  • the target interaction operation may indicate video communication with the second user terminal.
  • the user interface of the first user terminal may present the above-mentioned three-dimensional model, or may not present the three-dimensional model.
  • Step 1120 Send the streaming video to the server.
  • the second user terminal may send the streaming video to the server.
  • the server is used to send the streaming video to the first user terminal, so that the first user terminal presents the streaming video and the three-dimensional model on the user interface.
  • the server can use streaming media technology to continuously send the images and/or voice (that is, streaming video) collected by the second user terminal to the first user terminal.
  • Streaming media technology refers to a media format that uses streaming technology to continuously play on the network in real time.
  • the second user terminal may send the continuous image and sound information collected by it to the server after compression processing.
  • the server transmits each compressed package to the first user terminal sequentially or in real time, so that users who use the first user terminal can watch and listen while downloading.
  • the server may send the streaming video collected by the second user terminal to the first user terminal, and may also perform image processing (such as beauty) and voice processing (such as denoising) on the streaming video collected by the second user terminal. ), after operations such as transcoding, recording, and pornography, send the processed streaming video to the first user terminal.
  • image processing such as beauty
  • voice processing such as denoising
  • the second three-dimensional model-based interaction method provided by the foregoing embodiment of the present disclosure is applied to a second user terminal, and the user account logged in by the second user terminal establishes an association relationship with the three-dimensional model in advance.
  • the second user terminal may determine whether the user's confirmation operation for the interaction request is detected in the case of receiving the interaction request sent by the server.
  • the interaction request indicates that the first user terminal detects the user's target interaction operation on the user interface presented by the first user terminal, and the user interface is used to present a three-dimensional model. Afterwards, if the confirmation operation is detected, the streaming video is obtained.
  • the streaming video is sent to the server, where the server is used to send the streaming video to the first user terminal, so that the first user terminal presents the streaming video and the three-dimensional model on the user interface.
  • the server is used to send the streaming video to the first user terminal, so that the first user terminal presents the streaming video and the three-dimensional model on the user interface.
  • the streaming media video by presenting the streaming media video and the three-dimensional model on the same page of the terminal device, it is helpful to use the streaming media video to present information related to the three-dimensional model to the user, thereby increasing the diversity of interaction modes.
  • users can browse the three-dimensional model more calmly, improve the user's browsing time, and help meet the users' more diversified interactive needs.
  • the foregoing step 1110 may include the following steps:
  • the confirmation operation indicates that the user of the second user terminal confirms (agrees) to interact with the first user terminal (for example, video communication).
  • the first user terminal may present the streaming video; and the second user terminal does not send the interaction confirmation information.
  • the first user terminal does not present the streaming video. Therefore, the streaming video and the three-dimensional model can be presented on the user interface of the first user terminal only after the permission of the user of the second user terminal is obtained (for example, the video call initiated by the first user terminal is connected). This helps to improve the privacy protection of the user of the second user terminal, and provides preparation time for the user of the second user terminal to present the streaming media video to the user of the first user terminal.
  • the second user terminal may also directly obtain the streaming video, and send the streaming video to the first user terminal through the server, without the need for a second user terminal. 2. Interaction confirmation information sent by the user of the user terminal in response to the interaction request.
  • the user of the second user terminal may be in a state of shooting a streaming video (for example, a live broadcast) to users of other user terminals.
  • a streaming video for example, a live broadcast
  • the first user terminal can receive the streaming video obtained by the server from the second user terminal at any time, thereby improving the real-time performance of the streaming video presentation.
  • the second user terminal may receive the camera shutdown confirmation message from the server, and display The camera is turned off confirmation message.
  • the camera close confirmation information is used to determine whether the second user terminal closes the camera.
  • the server may send to the second user terminal information for determining whether the second user terminal turns off the camera. Therefore, the user of the second user terminal can reduce the occupation of network resources by the second user terminal by turning off the camera.
  • the second user terminal may send model adjustment information indicating the adjustment operation to the server , So that the server controls the first user terminal to perform the same adjustment operation on the three-dimensional model presented on the user interface according to the adjustment operation indicated by the model adjustment information.
  • the adjustment operation includes at least one of the following: zoom, rotate, move, and switch viewpoints.
  • the user can perform at least one operation of zooming, rotating, moving, and switching viewpoints on the three-dimensional model.
  • the operations performed by the user of the second user terminal on the three-dimensional model can be synchronized to the first user terminal. Therefore, when the streaming video collected by the second user terminal is related to the three-dimensional model (for example, the user of the second user terminal explains or introduces the three-dimensional model), it is convenient for the user of the first user terminal to refer to the second user The same three-dimensional model presented by the terminal acquires the information in the streaming video, thereby improving the pertinence of information acquisition.
  • the second user terminal may follow the adjustment operation indicated by the model adjustment information , Perform the same adjustment operation on the three-dimensional model presented by the second user terminal.
  • the adjustment operation includes at least one of the following: zoom, rotate, move, and switch viewpoints.
  • the user can perform at least one operation of zooming, rotating, moving, and switching viewpoints on the three-dimensional model.
  • the operations performed by the user of the first user terminal on the three-dimensional model can be synchronized to the second user terminal.
  • the second user terminal may perform the same method as the feedback information. Matching operation.
  • the feedback information may include but is not limited to at least one of the following: likes, ratings, comments, and so on.
  • the feedback information may be used to characterize the evaluation of the user of the first user terminal on the streaming video of the user of the second user terminal.
  • the second user terminal may present an operation that matches the feedback information, for example, “XX gave you a like!” .
  • FIG. 12 is a flow 1200 of another embodiment of the second three-dimensional model-based interaction method of the present disclosure.
  • the three-dimensional model-based interaction method is applied to a first user terminal, and the first user terminal presents a user Interface, the method includes:
  • Step 1210 In response to receiving the interactive request sent by the server, obtain the streaming video.
  • Step 1220 Send the streaming video to the server.
  • step 1210 to step 1220 are basically the same as step 1110 to step 1120 in the embodiment corresponding to FIG. 11, and will not be repeated here.
  • the three-dimensional model includes three-dimensional sub-models of multiple sub-space scenes, and the sub-space scenes in the multiple sub-space scenes correspond to keywords in a predetermined keyword set.
  • Step 1230 Perform voice recognition on the voice acquired by the first user terminal to obtain a voice recognition result.
  • the second user terminal may perform voice recognition on the voice acquired by the first user terminal to obtain a voice recognition result.
  • the voice recognition result can represent the text corresponding to the voice in the streaming video.
  • Step 1240 in response to the determination that the voice recognition result contains keywords in the keyword set, present on the user interface a three-dimensional sub-model of the corresponding sub-space scene among the multiple sub-space scenes corresponding to the keywords contained in the voice recognition result .
  • the second user terminal may present on the user interface a three-dimensional subspace scene corresponding to the keywords contained in the voice recognition result. Model.
  • the above-mentioned three-dimensional model is a three-dimensional model inside a house.
  • the house includes a bedroom, a living room, a kitchen, and a bathroom, with a total of four sub-space scenes, that is, the above-mentioned three-dimensional model includes a three-dimensional sub-model of the bedroom, a three-dimensional sub-model of the living room, a three-dimensional sub-model of the kitchen, and a three-dimensional sub-model of the bathroom.
  • the keyword set includes bedroom, living room, kitchen, bathroom.
  • the keyword corresponding to the subspace scene bedroom can be "bedroom”; the keyword corresponding to the subspace scene kitchen can be “kitchen”; the keyword corresponding to the subspace scene living room can be “living room” ; The keyword corresponding to the bathroom in the subspace scene can be "toilet”. Further, as an example, if the voice recognition result includes the keyword "bedroom", then the second user terminal may present a three-dimensional sub-model of the bedroom on the aforementioned user interface.
  • the viewpoint switching of the three-dimensional model can be realized by voice, thereby presenting the subspace scene corresponding to the keywords contained in the voice recognition result.
  • the three-dimensional sub-model As a result, the convenience of browsing the three-dimensional model is improved, and the matching between the presented three-dimensional model and the voice acquired by the second user terminal is improved.
  • the present disclosure provides an embodiment of an interaction device based on a three-dimensional model.
  • the device embodiment may also include the same or corresponding features as the method embodiment shown in FIGS. 7, 9, and 10, and produce the same as those in the method embodiment shown in FIGS.
  • the method embodiments shown have the same or corresponding effects.
  • the interaction apparatus 1300 based on the three-dimensional model of this embodiment is set in a first user terminal, and the first user terminal presents a user interface.
  • the device 1300 includes: a first sending unit 1310 configured to send an interaction request for the target interaction operation to a server that provides page data for the user interface in response to detecting a user's target interaction operation on the user interface, wherein the user interface For presenting the three-dimensional model, the three-dimensional model is pre-associated with the user account logged in the second user terminal; the first receiving unit 1320 is configured to receive the streaming video obtained by the server from the second user terminal; the first presenting unit 1330 is It is configured to present streaming video and 3D models on the user interface.
  • the first sending unit 1310 of the interactive device 1300 based on the three-dimensional model may send the information about the target interaction operation to the server that provides page data for the user interface.
  • the user interface is used for presenting a three-dimensional model, and the three-dimensional model establishes an association relationship with the user account logged in by the second user terminal in advance.
  • the first receiving unit 1320 may receive the streaming video obtained by the server from the second user terminal.
  • the first presentation unit 1330 may present the streaming video and the three-dimensional model on the user interface.
  • the first receiving unit is further configured to: in response to the server receiving the interaction confirmation information sent by the second user terminal in response to the interaction request, receive the stream obtained by the server from the second user terminal. Media video.
  • the device 1300 further includes: a first adjustment unit (not shown in the figure) configured to respond to the current network speed value of the first user terminal being less than or equal to a preset Network speed threshold, based on each frame of voice in the streaming video, adjust the target user's image to generate a new video, where the new video characterizes the user indicated by the target user's image to perform the actions of each frame of voice instruction; the second presentation unit ( Figure Not shown in ), is configured to use new video instead of streaming video for presentation.
  • a first adjustment unit (not shown in the figure) configured to respond to the current network speed value of the first user terminal being less than or equal to a preset Network speed threshold, based on each frame of voice in the streaming video, adjust the target user's image to generate a new video, where the new video characterizes the user indicated by the target user's image to perform the actions of each frame of voice instruction
  • the second presentation unit ( Figure Not shown in ), is configured to use new video instead of streaming video for presentation.
  • the device 1300 further includes: a first generating unit (not shown in the figure) configured to generate a target user image based on an image in the streaming video; or A determining unit (not shown in the figure) is configured to determine the user image associated with the user account as the target user image.
  • the device 1300 further includes: a second sending unit (not shown in the figure), configured to send camera shutdown confirmation information to the server in response to a new video presented on the user interface , Wherein the camera closing confirmation information is used to determine whether the second user terminal closes the camera.
  • a second sending unit (not shown in the figure), configured to send camera shutdown confirmation information to the server in response to a new video presented on the user interface , Wherein the camera closing confirmation information is used to determine whether the second user terminal closes the camera.
  • the first receiving unit is further configured to: send the current network speed value of the first user terminal to the server; receive the streaming video that the server obtains and sends from the second user terminal , The streaming video has a resolution that matches the current network speed value.
  • the device 1300 further includes: a second receiving unit (not shown in the figure) configured to receive model adjustment information sent by the server, wherein the model adjustment information indicates the use of the first 2.
  • the user of the user terminal adjusts the three-dimensional model presented on the second user terminal.
  • the adjustment operation includes at least one of the following: zoom, rotate, move, and switch viewpoints; the second adjustment unit (not shown in the figure) is configured According to the adjustment operation indicated by the model adjustment information, the same adjustment operation is performed on the three-dimensional model presented on the user interface.
  • the three-dimensional model includes three-dimensional sub-models of multiple sub-space scenes, and the sub-space scenes in the multiple sub-space scenes correspond to keywords in a predetermined keyword set; and,
  • the device 1300 also includes: a first recognition unit (not shown in the figure), configured to perform voice recognition on the voice in the streaming video to obtain a voice recognition result; and a third presentation unit (not shown in the figure), configured In response to determining that the voice recognition result contains the keywords in the keyword set, a three-dimensional sub-model of the subspace scene corresponding to the keywords contained in the voice recognition result is presented on the user interface.
  • the device 1300 further includes: a first acquiring unit (not shown in the figure), configured to acquire user feedback information for streaming media videos; and a third sending unit (not shown in the figure). Not shown in ), is configured to send feedback information to a server, where the server is used to establish an association relationship between the feedback information and the user account.
  • the interaction device based on the three-dimensional model provided by the above-mentioned embodiments of the present disclosure is set in a first user terminal, and the first user terminal presents a user interface.
  • the first sending unit 1310 may send an interaction request for the target interaction operation to a server that provides page data for the user interface, where the user interface is used for The three-dimensional model is presented.
  • the three-dimensional model is pre-associated with the user account logged in the second user terminal.
  • the first receiving unit 1320 receives the streaming video obtained by the server from the second user terminal.
  • the first presenting unit 1330 displays on the user interface Streaming videos and 3D models are presented on the website.
  • streaming media videos and 3D models can be presented on the same page of the terminal device, which helps to use streaming media videos to present information related to the 3D model to users, which improves the diversity of interaction methods.
  • Users browse the 3D model more calmly, increase the user's browsing time, and help meet users' more diversified interactive needs.
  • the present disclosure provides an embodiment of a second interaction device based on a three-dimensional model.
  • the device embodiment may also include the same or corresponding features as the method embodiment shown in Figs. 11 and 12, and the method implementation shown in Figs. The same or corresponding effect.
  • the interaction device 1400 based on the three-dimensional model of this embodiment is set in the second user terminal.
  • the device 1400 includes: a second determining unit 1410 configured to obtain a streaming video in response to receiving an interaction request sent by a server, wherein the interaction request indicates that the first user terminal detects that the user interface presented to the first user terminal
  • the user interface is used to present a three-dimensional model, and the three-dimensional model has a pre-established association relationship with the user account logged in the second user terminal
  • the fourth sending unit 1420 is configured to send streaming video to the server, where the server It is used to send the streaming video to the first user terminal, so that the first user terminal presents the streaming video and the three-dimensional model on the user interface.
  • the second determining unit 1410 may obtain the streaming video.
  • the interaction request indicates that the first user terminal detects a user's target interaction operation on the user interface presented by the first user terminal, and the user interface is used to present a three-dimensional model.
  • the fourth sending unit 1420 may be configured to send the streaming video to the server, where the server is used to send the streaming video to the first user terminal, so that the first user terminal displays the streaming video on the user interface. And three-dimensional models.
  • the second determining unit 1410 is further configured to: in response to receiving the interaction request sent by the server, determine whether a confirmation operation of the user for the interaction request is detected; in response to detecting the confirmation Operation to obtain streaming video.
  • the device 1400 further includes: a third receiving unit (not shown in the figure), configured to respond to the current network speed value of the first user terminal being less than or equal to a preset The network speed threshold, receiving camera closing confirmation information from the server, and presenting camera closing confirmation information, where the camera closing confirmation information is used to determine whether the second user terminal closes the camera.
  • a third receiving unit (not shown in the figure), configured to respond to the current network speed value of the first user terminal being less than or equal to a preset The network speed threshold, receiving camera closing confirmation information from the server, and presenting camera closing confirmation information, where the camera closing confirmation information is used to determine whether the second user terminal closes the camera.
  • the device 1400 further includes: a fifth sending unit (not shown in the figure), configured to respond to receiving from the server the three-dimensional information presented on the second user terminal by the user.
  • the adjustment operation of the model is to perform the same adjustment operation on the three-dimensional model presented on the user interface according to the adjustment operation indicated by the model adjustment information, where the adjustment operation includes at least one of the following: zoom, rotate, move, and switch viewpoints.
  • the device 1400 further includes: a fifth sending unit (not shown in the figure), configured to respond to detecting that the user has received the three-dimensional model presented on the second user terminal
  • the adjustment operation is to send model adjustment information indicating the adjustment operation to the server, so that the server controls the first user terminal to perform the same adjustment operation on the three-dimensional model presented on the user interface according to the adjustment operation indicated by the model adjustment information.
  • the adjustment operation includes the following At least one item: zoom, rotate, move, viewpoint switch.
  • the three-dimensional model includes three-dimensional sub-models of multiple sub-space scenes, and the sub-space scenes in the multiple sub-space scenes correspond to keywords in a predetermined keyword set; and,
  • the device 1400 further includes: a second recognition unit (not shown in the figure), configured to perform voice recognition on the voice acquired by the first user terminal to obtain a voice recognition result; and a fourth presentation unit (not shown in the figure), It is configured to, in response to determining that the voice recognition result contains the keywords in the keyword set, present on the user interface a three-dimensional sub-model of the subspace scene corresponding to the keywords contained in the voice recognition result.
  • the apparatus 1400 further includes: an execution unit (not shown in the figure), configured to respond to receiving a stream media message sent by the server and sent by the user using the first user terminal. Video feedback information, perform operations that match the feedback information.
  • the interaction device based on the three-dimensional model provided by the above-mentioned embodiment of the present disclosure is set in the second user terminal, and the user account logged in by the second user terminal establishes an association relationship with the three-dimensional model in advance.
  • the second determining unit 1410 may obtain the streaming video, where the interaction request indicates that the first user terminal detects the user's target interaction operation on the user interface presented by the first user terminal, and the user interface is used to present the three-dimensional model.
  • the fourth sending unit 1420 may send the streaming video to the server, where the server is used to send the streaming video to the first user terminal, so that the first user terminal can present the streaming video and the three-dimensional model on the user interface.
  • streaming videos and 3D models can be presented on the same page of the terminal device, which helps to use streaming videos to present information related to the 3D model to users, which improves the diversity of interaction methods. Users browse the 3D model more calmly, increase the user's browsing time, and help meet users' more diversified interactive needs.
  • FIG. 15 is a schematic diagram of interaction of an embodiment 1500 of the interactive system based on a three-dimensional model of the present disclosure.
  • the interactive system based on the three-dimensional model includes a first user terminal, a second user terminal, and a server.
  • the first user terminal presents a user interface
  • the server is respectively communicatively connected with the first user terminal and the second user terminal.
  • the first user terminal, the second user terminal, and the server in the interactive system based on the three-dimensional model can perform the following steps:
  • Step 1501 The first user terminal detects the user's target interaction operation on the user interface.
  • the first user terminal detects the user's target interaction operation on the user interface.
  • the user interface is used for presenting a three-dimensional model, and the three-dimensional model establishes an association relationship with the user account logged in by the second user terminal in advance.
  • Step 1502 The first user terminal sends an interaction request for the target interaction operation to the server.
  • the first user terminal may send an interaction request for a target interaction operation to the server.
  • Step 1503 The second user terminal obtains the streaming video.
  • the second user terminal can obtain the streaming video.
  • Step 1504 The second user terminal sends the streaming video to the server.
  • the second user terminal may send the streaming video to the server.
  • Step 1505 The server sends the streaming video to the first user terminal.
  • the server may send the streaming video to the first user terminal.
  • Step 1506 The first user terminal presents the streaming video and the three-dimensional model on the user interface.
  • the first user terminal may present the streaming video and the three-dimensional model on the user interface.
  • step 1501 to step 1506 can also refer to the implementation of the first three-dimensional model-based interaction method described above.
  • the technical features in each embodiment of the second three-dimensional model-based interaction method, and the third three-dimensional model-based interaction method will be explained.
  • this embodiment may also include the same or corresponding features as the above-mentioned embodiment of the interaction method based on the three-dimensional model, and produce the same or corresponding effects, which will not be repeated here.
  • the interactive system based on the three-dimensional model provided by the above-mentioned embodiments of the present disclosure includes a first user terminal, a second user terminal, and a server.
  • the first user terminal presents a user interface
  • the server is in communication connection with the first user terminal and the second user terminal. .
  • the first user terminal is configured to: in response to detecting the user's target interaction operation on the user interface, send an interaction request for the target interaction operation to the server, wherein the user interface is used to present the three-dimensional model, the three-dimensional model and the second user
  • the user account logged in by the terminal establishes an association relationship in advance
  • the second user terminal is configured to: obtain the streaming video; send the streaming video to the server;
  • the server is also configured to: send the streaming video to the first user terminal;
  • the first user terminal It is configured to: present streaming video and three-dimensional model on the user interface.
  • streaming media videos and 3D models can be presented on the same page of the terminal device, which helps to use streaming media videos to present information related to the 3D model to users, which improves the diversity of interaction methods.
  • Users browse the 3D model more calmly, increase the user's browsing time, and help meet users' more diversified interactive needs.
  • FIG. 16 shows a block diagram of an electronic device 1600 according to an embodiment of the present disclosure.
  • the electronic device 1600 includes one or more processors 1611 and a memory 1612.
  • the processor 1611 may be a central processing unit (CPU) or another form of processing unit having the ability to implement three-dimensional scene interaction and/or instruction execution capabilities, and may control other components in the electronic device 1600 to perform desired functions .
  • CPU central processing unit
  • the memory 1612 may include one or more computer program products, and the computer program products may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory.
  • the volatile memory for example, may include: random access memory (RAM) and/or cache memory (cache).
  • the non-volatile memory for example, may include: read-only memory (ROM), hard disk, flash memory, and the like.
  • One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 1611 may run the program instructions to implement the various methods described above and/or other desired functions.
  • Various contents such as input signals, signal components, noise components, etc. can also be stored in the computer-readable storage medium.
  • the electronic device 1600 may further include: an input device 1613, an output device 1614, etc., and these components are interconnected by a bus system and/or other forms of connection mechanisms (not shown).
  • the input device 1613 may also include, for example, a keyboard, a mouse, and so on.
  • the output device 1614 can output various information to the outside.
  • the output device 1614 may include, for example, a display, a speaker, a printer, a communication network and a remote output device connected to it, and so on.
  • the electronic device 1600 may also include any other appropriate components.
  • the embodiments of the present disclosure may also be computer program products, which include computer program instructions that, when run by a processor, cause the processor to perform operations according to various embodiments of the present disclosure. Steps in various methods.
  • the computer program product may use any combination of one or more programming languages to write program codes for performing the operations of the embodiments of the present disclosure.
  • the programming languages include object-oriented programming languages, such as Java, C++, etc. , Also includes conventional procedural programming languages, such as "C" language or similar programming languages.
  • the program code can be executed entirely on the user's computing device, partly on the user's device, executed as an independent software package, partly on the user's computing device and partly executed on the remote computing device, or entirely on the remote computing device or server Executed on.
  • embodiments of the present disclosure may also be a computer-readable storage medium on which computer program instructions are stored.
  • the processor executes each of the various embodiments of the present disclosure. Steps in a method.
  • the computer-readable storage medium may adopt any combination of one or more readable media.
  • the readable medium may be a readable signal medium or a readable storage medium.
  • the readable storage medium may include, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the above, for example. More specific examples (non-exhaustive list) of readable storage media may include: electrical connections with one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), Erasable programmable read only memory (EPROM or flash memory), optical fiber, portable compact disk read only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.
  • the method and apparatus of the present disclosure may be implemented in many ways.
  • the method and apparatus of the present disclosure can be implemented by software, hardware, firmware or any combination of software, hardware, and firmware.
  • the above-mentioned order of the steps for the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above, unless specifically stated otherwise.
  • the present disclosure can also be implemented as programs recorded in a recording medium, and these programs include machine-readable instructions for implementing methods according to embodiments of the present disclosure.
  • the present disclosure also covers a recording medium storing a program for executing a method according to an embodiment of the present disclosure.
  • each component or each step can be decomposed and/or recombined. These decomposition and/or recombination should be regarded as equivalent solutions of the present disclosure.

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

一种用于实现三维空间场景互动的方法，包括：响应于检测到在三维空间场景中设置足迹信息的用户操作，确定用户在三维空间场景中的当前视角所对应的当前视图中的第一像素点；确定第一像素点对应的三维模型；确定用户的足迹信息在所述三维模型中的位置，其中足迹信息用于在三维空间场景被浏览时显示；以及在位置处设置用户的足迹信息。

Description

用于实现三维空间场景互动的方法、装置和设备 技术领域
本公开涉及虚拟现实全景技术和流媒体技术，尤其是涉及一种用于实现三维空间场景互动的方法、用于实现三维空间场景互动的装置、存储介质以及电子设备。
背景技术
VR(Virtual Reality虚拟现实)全景技术是一种新兴的富媒体技术。由于VR全景技术可以720度无死角的为用户呈现三维空间场景，给用户带来浸入式视觉体验，因此，VR全景技术被广泛应用在网上商城、旅游服务以及房产服务等多种领域中。如何使VR全景技术给用户带来更丰富的使用体验，是一个值得关注的技术问题。
相对于二维画面，三维模型可以给人以更为强烈的视觉观感。有了物体的三维数据，便可以向用户呈现该物体的任意视图，并且，视图间可以保持正确的投影关系。
现有技术中，在用户终端呈现三维模型的同时，可以支持用户终端间的实时语音同屏交互，也即，在用户终端呈现三维模型的过程中，用户终端的对端用户的语音可以实时传输至该用户终端，并且，用户终端获取的语音也可以实时传输至对端。
然而，现有技术中的上述交互方式较为单一，语音交互通常存在局限性。
发明内容
根据本公开实施例的一个方面，提供了一种用于实现三维空间场景互动的方法，包括：响应于检测到在三维空间场景中设置足迹信息的用户操作，确定用户在三维空间场景中的当前视角所对应的当前视图中的第一像素点；确定第一像素点对应的三维模型；确定用户的足迹信息在所述三维模型中的位置，其中足迹信息用于在三维空间场景被浏览时显示；以及在位置处设置用户的足迹信息。
根据本公开实施例的另一个方面，提供了一种基于三维模型的交互方法，包括：在呈现有用户界面的第一用户终端处：响应于检测到用户针对用户界面的目标交互操作，向为用户界面提供页面数据的服务器发送针对目标交互操作的交互请求，其中，用户界面用于呈现三维模型，三维模型与第二用户终端登录的用户账号预先建立关联关系；接收所述服务器从第二用户终端获取的流媒体视频；以及在用户界面上呈现流媒体视频和三维模型。
根据本公开实施例的另一方面，提供了一种基于三维模型的交互方法，包括：在第二用户终端处：响应于接收到服务器发送的交互请求，获取流媒体视频，其中，交互请求指示第一用户终端检测到用户针对第一用户终端呈现的用户界面的目标交互操作，用户界面用于呈现三维模型，三维模型与第二用户终端登录的用户账号预先建立关联关系；以及向服务器发送流媒体视频，其中，服务器用于将流媒体视频发送至所述第一用户终端，以使第一用户终端在用户界面上呈现流媒体视频和三维模型。
根据本公开实施例的另一个方面，提供了一种用于实现三维空间场景互动的装置，包括：用于执行上述方法中任一项所述的方法的装置。
根据本公开实施例的另一方面，提供了一种基于三维模型的交互装置，设置于第一用户终端，装置包括：用于执行上述方法中任一项所述的方法的装置。
根据本公开实施例的另一方面，提供了一种基于三维模型的交互装置，设置于第二用户终端，装置包括：用于执行上述方法中中任一项所述的方法的装置。
根据本公开实施例的另一方面，提供了一种基于三维模型的交互系统，包括：第一用户终端，用于呈现用户界面；第二用户终端；和服务器，服务器与第一用户终端和第二用户终端通信连接，第一用户终端被配置成：响应于检测到用户针对用户界面的目标交互操作，向服务器发送针对目标交互操作的交互请求，用户界面用于呈现三维模型，三维模型与第二用户终端登录的用户账号预先建立关联关系；第二用户终端被配置成：获取流媒体视频；并且向服务器发送所述流媒体视频；服务器被配置成：向第一用户终端发送所述流媒体视频；并且第一用户终端被配置成：在用户界面上呈现流媒体视频和三维模型。
根据本公开实施例的另一个方面，提供了一种非暂态性计算机可读存储介质，存储介质存储有计算机程序，计算机程序当被计算机执行时使计算机实现上述方法中任一项所述的方法。
根据本公开实施例的另一个方面，提供了一种电子设备，包括：处理器；以及用于存储处理器可执行指令的存储器，处理器可执行指令在被处理器执行时实现上述方法中任一项所述的方法。
根据本公开的实施例的另一个方面，提供了一种计算机程序产品，包括计算机程序，计算机程序当被计算机执行时使计算机实现上述方法中任一项所述的方法。
下面通过附图和实施例，对本公开的技术方案做进一步的详细描述。
附图说明
构成说明书的一部分的附图描述了本公开的实施例，并且连同描述一起用于解释本公开的 原理。
参照附图，根据下面的详细描述，可以更加清楚地理解本公开，其中：
图1为本公开的适用场景的一个实施例的示意图；
图2为本公开的用于实现三维空间场景互动的方法一个实施例的流程图；
图3为本公开的确定第一像素点对应的三维模型一实施例的流程图；
图4为本公开的确定第一像素点对应的三维模型另一实施例的流程图；
图5为本公开的为浏览用户呈现足迹信息一实施例的流程图；
图6为本公开的用于实现三维空间场景互动的装置一个实施例的结构示意图；
图7为本公开的第一个基于三维模型的交互方法的一个实施例的流程图。
图8A-图8C是针对图7的实施例的应用场景示意图。
图9为本公开的第一个基于三维模型的交互方法的另一个实施例的流程图。
图10为本公开的第一个基于三维模型的交互方法的又一个实施例的流程图。
图11为本公开的第二个基于三维模型的交互方法的一个实施例的流程图。
图12为本公开的第二个基于三维模型的交互方法的另一个实施例的流程图。
图13为本公开的第一个基于三维模型的交互装置的一个实施例的流程图。
图14为本公开的第二个基于三维模型的交互装置的一个实施例的流程图。
图15为本公开的基于三维模型的交互系统的一个实施例的交互示意图。
图16为本公开一示例性实施例提供的电子设备的结构图。
具体实施方式
下面将参考附图详细地描述根据本公开的示例实施例。显然，所描述的实施例仅仅是本公开的一部分实施例，而不是本公开的全部实施例，应理解，本公开不受这里描述的示例实施例的限制。
应注意到：除非另外具体说明，否则在这些实施例中阐述的部件和步骤的相对布置、数字表达式和数值不限制本公开的范围。
本领域技术人员可以理解，本公开实施例中的“第一”、“第二”等术语仅用于区别不同步骤、设备或模块等，既不代表任何特定技术含义，也不表示它们之间的必然逻辑顺序。
还应理解，在本公开实施例中，“多个”可以指两个或者两个以上，“至少一个”可以指一个、两个或两个以上。
还应理解，对于本公开实施例中提及的任一部件、数据或结构，在没有明确限定或者在前 后文给出相反启示的情况下，一般可以理解为一个或多个。
另外，本公开中术语“和/或”，仅是一种描述关联对象的关联关系，表示可以存在三种关系，如A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。另外，本公开中字符“/”，一般表示前后关联对象是一种“或”的关系。
还应理解，本公开对各个实施例的描述着重强调各个实施例之间的不同之处，其相同或相似之处可以相互参考，为了简洁，不再一一赘述。
同时，应当明白，为了便于描述，附图中所示出的各个部分的尺寸并不是按照实际的比例关系绘制的。
以下对至少一个示例性实施例的描述实际上仅仅是说明性的，决不作为对本公开及其应用或使用的任何限制。
对于相关领域普通技术人员已知的技术、方法和设备可能不作详细讨论，但在适当情况下，所述技术、方法和设备应当被视为说明书的一部分。
应注意到：相似的标号和字母在下面的附图中表示类似项，因此，一旦某一项在一个附图中被定义，则在随后的附图中不需要对其进行进一步讨论。
本公开的实施例可以应用于终端设备、计算机系统、服务器等电子设备，其可与众多其它通用或者专用计算系统环境或配置一起操作。适于与终端设备、计算机系统或者服务器等电子设备一起使用的众所周知的终端设备、计算系统、环境和/或配置的例子包括但不限于：个人计算机系统、服务器计算机系统、瘦客户机、厚客户机、手持或膝上设备、基于微处理器的系统、机顶盒、可编程消费电子产品、网络个人电脑、小型计算机系统、大型计算机系统和包括上述任何系统的分布式云计算技术环境等等。
终端设备、计算机系统、服务器等电子设备可以在由计算机系统执行的计算机系统可执行指令(诸如程序模块)的一般语境下描述。通常，程序模块可以包括例程、程序、目标程序、组件、逻辑、数据结构等等，它们执行特定的任务或者实现特定的抽象数据类型。计算机系统/服务器可以在分布式云计算环境中实施。在分布式云计算环境中，任务可以是由通过通信网络链接的远程处理设备执行的。在分布式云计算环境中，程序模块可以位于包括存储设备的本地或远程计算系统存储介质上。
在实现本公开的过程中，发明人发现，在用户通过调整其当前视角来感受三维空间场景的过程中，往往会产生一些情绪以及想法等感受。如果用户能够将用于表征其感受的足迹信息设置到三维空间场景中，则不仅有利于提高用户自身的参与感，而且，用户留下的足迹信息还可以为观看该三维空间场景的其他用户带来更丰富的VR全景体验。
下面结合图1，对本公开提供的用于实现三维空间场景互动的技术的应用场景的一个例子，进行说明。
在房产领域，可以利用VR全景技术为待出租房屋或者待出售房屋，设置三维空间场景。任一用户均可以通过网络访问，随时随地的观看相应房屋的三维空间场景。在用户观看相应房屋的三维空间场景的过程中，本公开允许该用户针对其所浏览的房屋留下自己的足迹信息，而且，本公开可以将该用户自己针对该房屋留下的足迹信息和其他用户针对该房屋留下的足迹信息，一并呈现给该用户。
在一个具体的例子中，假设用户正在浏览房屋的三维空间场景，基于该用户的当前视角所看到的当前视图如图1所示。
其他用户针对该两室一厅的房屋的三维空间场景留下的足迹信息120包括：“喜欢这组沙发，棒棒哒”、“这个装饰隔断不错”、“这个沙发不错，高级上档次”、“组合搭配很用心，赞赞赞”、“茶几造型设计的很独特～文案最长二十个字啊”以及图1右上角所示的三维模型110。为浏览该房屋的三维空间场景的用户呈现其他用户针对该房屋的三维空间场景所留下的足迹信息120，有利于使该用户了解其他用户对该房屋的感受，从而有利于加深用户对该房屋的认知，进而有利于提高用户对该房屋的浏览体验。
另外，用户在观看该房屋的三维空间场景的过程中，也可以发表自己对该房屋的感受，即在该三维空间场景中留下自己的足迹信息。例如，用户可以在图1所示的柱子的位置处设置“这个柱子让房屋看起来更具特色”等足迹信息。用户自己设置的足迹信息可以即时显示在图1所示的三维空间场景中，即用户可以在观看该房屋的三维空间场景的过程中看到自己留下的足迹信息，从而有利于提升用户的参与感。
再有，所有用户针对该房屋设置的不属于图1所示的视图的其他足迹信息，可以以弹幕130的形式呈现给用户，从而有利于提升用户浏览该房屋的其他位置的三维空间场景的兴趣。
本公开提供的用于实现三维空间场景互动的技术还可以应用在其他多种场景中，例如，用户在浏览图书馆的三维空间场景时，可以针对图书馆中的一本书或者座椅或者咖啡机等设置相应的足迹信息。用户针对书设置的足迹信息可以是用户针对书的观感或者用户当前阅读到的页数等。在此对本公开提供的用于实现三维空间场景互动的技术所能应用的场景，不再一一举例说明。
图2为本公开的用于实现三维空间场景互动的方法一个实施例的流程图。图2所示的实施例的方法200包括步骤210至240。下面对各步骤分别进行说明。
在步骤210，响应于检测到在三维空间场景中设置足迹信息的用户操作，确定用户在该三 维空间场景中的当前视角所对应的当前视图中的第一像素点。
根据本公开的示范性实施例，三维空间场景可以是指利用预先设置的全景图和三维模型，为用户呈现出的具有三维立体感的空间场景。例如，三维空间场景可以为针对图书馆设置的三维空间场景、针对房屋设置的三维空间场景、针对咖啡馆设置的三维空间场景、或者针对商场设置的三维空间场景等。
根据本公开的示范性实施例，可以在用户触发在三维空间场景中设置足迹信息的功能时，检测到用户需要在该三维空间场景中设置足迹信息。例如，在用户点击用于设置足迹信息的按钮或者菜单上的相应选项时，本公开的实施例可以检测到用户需要在三维空间场景中设置足迹信息。再例如，用户可以使用预先设置的快捷方式触发在三维空间场景中设置足迹信息的功能。本公开的实施例中，用户的足迹信息可以为能够表示出用户曾经到访过该三维空间场景的信息。该足迹信息可以认为是用户的来访痕迹信息。
根据本公开的示范性实施例，用户在三维空间场景中的当前视角可以是指：用户当前观看三维空间场景的位置和角度。用户在三维空间场景中的当前视角通常会随着用户的操作，而发生变化。例如，用户通过在触摸屏上执行拖拽等操作，可以控制其在三维空间场景中的当前视角。用户在三维空间场景中的当前视角决定了用户当前所能看到的全景图的内容/区域，即用户在三维空间场景中的当前视角决定了当前视图。
根据本公开的示范性实施例，第一像素点为当前视图中的一个像素点。可以根据预先设置的缺省规则，获得第一像素点。例如，第一像素点可以为当前视图中的一个特定像素点，也可以为当前视图中的任意一个像素点。
在步骤220，确定该第一像素点对应的三维模型。
根据本公开的示范性实施例，三维空间场景通常由多个三维模型形成。在一些实施例中，三维空间场景也可以由一个三维模型形成。用户看到的当前视图中的一个像素点可以是三维模型中的一个点的呈现。用户看到的当前视图中的一个像素点也可以不是三维模型中的任一点的呈现。也就是说，在通常情况下，三维空间场景中的任一三维模型中的任一点均可以呈现在全景图中，而全景图中的点所呈现的可能并不是三维空间场景中的所有三维模型中的点。当然，本公开也不排除三维空间场景中的三维模型中的部分点未被呈现在全景图中的可能性。
在一些示范性实施例中，在第一像素点用于呈现三维模型中的点时，该点所在的三维模型即为第一像素点对应的三维模型。
在一些示范性实施例中，在第一像素点用于呈现非三维模型中的点时，可以利用当前视图中的其他像素点来更新第一像素点。在一些实施例中，也可以不对第一像素点进行更新处理， 此时，第一像素点对应的三维模型可以为：当前视图中与第一像素点相近的用于呈现三维模型中的点的其他像素点所对应的三维模型。也就是说，在第一像素点用于呈现非三维模型中的点，且不更新第一像素点的情况下，可以将当前视图中其他像素点对应的三维模型作为第一像素点对应的三维模型。
在步骤230，确定该用户的足迹信息在该三维模型中的位置，其中足迹信息用于在三维空间场景被浏览时显示。
由于全景图中的至少部分像素点与三维模型中的点存在映射关系，因此，可以获得上述第一像素点或者上述其他像素点在三维模型中的位置。该位置即为用户的足迹信息的位置。
根据本公开的示范性实施例，三维空间场景中的所有三维模型可以分别设置有各自的三维坐标系，也可以具有同一的三维坐标系。用户的足迹信息在三维模型中的位置可以通过(x，y，z)来表示。即，用户的足迹信息可以是具有深度的。
在步骤240，在位置处设置用户的足迹信息。
根据本公开的示范性实施例，在该位置处设置该用户的足迹信息可以包括：为该用户的足迹信息设置三维模型标识以及三维坐标，并存储三维模型标识、三维坐标以及用户的足迹信息的对应关系。
根据本公开的示范性实施例，用户的足迹信息可以用于显示给该三维空间场景的浏览用户(如所有浏览用户或者部分浏览用户)。本公开的实施例中，三维空间场景的浏览用户可以包括设置该足迹信息的用户。
根据本公开的示范性实施例，通过利用需要设置足迹信息的用户的当前视图中的第一像素点，获得第一像素点对应的三维模型以及足迹信息在该三维模型中的位置，使用户设置的足迹信息可以与相应三维模型的相应位置产生关联。这样，在基于用户的当前视角，利用全景图来呈现三维模型，以形成三维空间场景时，可以使用户的足迹信息呈现在三维空间场景的适当位置处，从而有利于使用户对三维空间场景中的具体部位的感受，精准的呈现在三维空间场景的相应位置处。由此，实现了用户与三维空间场景的互动。这不仅有利于提高用户自身的参与感以及沉浸感，并提升用户在三维空间场景中的停留时长，而且，用户留下的足迹信息还可以为该三维空间场景的至少一浏览用户带来更丰富的VR全景体验。
在一个可选示例中，足迹信息包括：文本、图片、音频、视频以及三维模型中的至少一个。文本可以认为是字符(如文字、字母、数字或者符号等)形式的留言。图片可以认为是图像(如照片或者表情符号等)形式的留言。音频可以认为是声音形式的留言(也可以称为留音等)。视频可以认为是影像形式的留言。三维模型可以认为是立体形式的留言。本公开的实施例中， 用户的足迹信息可以称为用户的留言。用户设置的一条足迹信息可以同时包括：文本、图片、音频、视频以及三维模型中的一种或者多种。通过使用户的足迹信息包括文本、图片、音频、视频以及三维模型中的至少一个，有利于使用户的足迹信息的表现形式更加丰富，从而有利于丰富用户与三维空间场景的互动方式。
在一个可选示例中，获取用户在三维空间场景中的当前视角所对应的当前视图中的第一像素点可以为：获取用户在三维空间场景中的当前视角所对应的当前视图的中心像素点，并将该中心像素点作为第一像素点。例如，假设用户在三维空间场景中的当前视角，通过点击按钮或者菜单上的选项等方式触发了在三维空间场景中设置足迹信息的功能，此时，可以直接将当前视图的中心像素点作为第一像素点。中心像素点可以认为是针对用户的足迹信息设置的缺省像素点，用户可以通过拖拽等方式对该缺省像素点进行更改。在一个例子中，中心像素点可以认为是当前视图的中心区域中的一个像素点。当前视图的中心区域可以包括一个像素点或者多个像素点。通过直接将当前视图的中心像素点作为第一像素点，不仅有利于快速获得第一像素点，而且，有利于使用户设置的足迹信息在当前视图中位于较为显著的位置。
在一个可选示例中，获取用户在三维空间场景中的当前视角所对应的当前视图中的第一像素点可以为：根据用户在三维空间场景中的当前视角所对应的当前视图中设置足迹信息目标位置的操作，获取足迹信息目标位置对应的当前视图中的像素点，该像素点被作为第一像素点。也就是说，在用户执行设置足迹信息目标位置的操作的情况下，可以将该操作在当前视图中形成的足迹信息目标位置所在的像素点，作为第一像素点。
可选地，设置足迹信息目标位置的操作可以为用于确定足迹信息的起始目标位置的操作，也可以为用于确定足迹信息的终止目标位置的操作，还可以为用于确定足迹信息的中心目标位置的操作。
可选地，设置足迹信息目标位置的操作可以具体为基于鼠标或者键盘等工具的点击操作或者滚动操作或者拖动操作等，也可以具体为基于触摸屏的点击操作或者拖拽操作等。本公开对设置足迹信息目标位置的具体操作不作限定。
通过根据用户的设置足迹信息目标位置的操作来确定第一像素点，有利于使用户设置的足迹信息位于用户期望的位置，从而有利于提高设置足迹信息的灵活性，并有利于使足迹信息的位置更为恰当。
可选地，假设在用户基于其在三维空间场景中的当前视角观看当前视图浏览的过程中，用户通过点击按钮或者菜单上的选项等方式触发了在三维空间场景中设置足迹信息的功能，此时，用户可以利用鼠标左键点击、键盘中的上下左右键移动光标或者点击触摸屏中的相应位置 等方式，来设置其所希望的足迹信息在当前视图中的位置。可以将该位置所在的像素点作为第一像素点。
可选地，假设在用户基于其在三维空间场景中的当前视角观看当前视图浏览的过程中，用户通过点击按钮或者菜单上的选项等方式触发了在三维空间场景中设置足迹信息的功能，此时，可以先将当前视图的中心像素点作为第一像素点。如果用户不对该第一像素点进行更改，则将该中心像素点作为最终的第一像素点。如果用户利用鼠标左键的拖动操作、键盘中的上下左右键移动光标操作或者用手指在触摸屏上的拖动操作等方式，来更改第一像素点，则将操作结果获得的具体位置所在的像素点作为第一像素点。
在一个可选示例中，确定第一像素点对应的三维模型(步骤220)的实现方式可以如图3所示。如图3所示，步骤220还包括步骤310至340。
在步骤310，确定点前视图的中心像素点作为第一像素点。
可选地，中心像素点可以认为是针对用户的足迹信息设置的缺省像素点。在一个例子中，假设当前视图为(2n+1)×(2m+1)的图像(其中的n和m均为大于1的整数)，则可以直接将当前视图中的像素点(n+1，m+1)作为中心像素点。在另一个例子中，假设当前视图为2n×2m的图像(其中的n和m均为大于1的整数)，则可以将当前视图中的像素点(n，m)、像素点(n+1，m)、像素点(n，m+1)以及像素点(n+1，m+1)作为当前视图的中心区域，从而可以将中心区域中的其中任一像素点作为中心像素点。
在步骤320，确定针对第一像素点是否设置有三维模型。如果针对第一像素点设置有三维模型，则到步骤330。如果针对第一像素点未设置有三维模型，则到步骤340。
在图3的示例中，由于并不是当前视图中的所有像素点均为三维模型中的相应点的呈现，因此需要判断针对第一像素点是否设置有三维模型，即需要判断第一像素点是否为用于呈现三维模型中的相应点的像素点，以使得可以将用户的足迹信息设置在三维模型中的相应位置处。
在步骤330，响应于针对第一像素点设置有三维模型的确定，将针对第一像素点设置的三维模型作为第一像素点对应的三维模型。
在步骤340，响应于针对第一像素点未设置有三维模型的确定，将针对当前视图中的其他像素点设置的三维模型作为第一像素点对应的三维模型。
可选地，当前视图中的其他像素点为当前视图中的设置有三维模型的像素点。可以根据预设规则来寻找设置有三维模型的像素点。在一个例子中，寻找到的其他像素点可以为在某一个方向(如左方向、右方向、上方向或者下方向)上距离第一像素点最近的像素点。
可选地，可以以第一像素点为起点，根据预设检查规则，对三维空间场景中的当前视角所 对应的当前视图中的像素点进行检查，如果确定检查到设置有三维模型的像素点，则获得第一像素点对应的三维模型，并停止本次检查过程。例如，可以以第一像素点为起点，向左检查当前视图中的像素点，并判断是否针对当前检查到的像素点设置了三维模型。如果判断结果为针对当前检查到的像素点设置有三维模型，则停止本次检查过程，并将当前检查获得的三维模型作为第一像素点对应的三维模型。另外，可以利用检查到的设置有三维模型的像素点更新第一像素点。当然，也可以不更新第一像素点。
通过判断第一像素点是否设置有三维模型，并根据判断结果执行不同的操作，有利于避免在针对第一像素点未设置有三维模型的情况下，存在的无法将用户的足迹信息设置在三维模型中的相应位置处的现象。进一步，通过利用预设检查规则，获得设置有三维模型的其他像素点，并将针对该其他像素点设置的三维模型作为第一像素点对应的三维模型，有利于快速获得第一像素点对应的三维模型。
在一个可选示例中，确定第一像素点对应的三维模型(步骤220)的实现方式可以如图4所示。如图4所示，步骤220可以包括步骤410至步骤450。
在步骤410，响应于用户在当前视图中设置足迹信息的目标位置的操作，确定足迹信息的目标位置对应的当前视图中的像素点，以作为第一像素点。
可选地，可以允许用户在当前视图中自行设置足迹信息的具体位置(即足迹信息的目标位置)。例如，在用户触发在三维空间场景中设置足迹信息的功能后，用户可以通过在触摸屏上点击、滑动、拖拽等操作，在当前视图中设置足迹信息的目标位置。该足迹信息的目标位置可以为文本框的左上顶点、左下顶点、右上顶点或者右下顶点等。该足迹信息的目标位置可以为图片的左上顶点、左下顶点、右上顶点或者右下顶点等。根据本公开的示范性实施例，足迹信息的目标位置可以是当前视图中的一个像素点，该像素点即为第一像素点。
在步骤420，判断针对第一像素点是否设置有三维模型。如果针对第一像素点设置有三维模型，则到步骤430。如果针对第一像素点未设置有三维模型，则到步骤440。
在步骤430，响应于针对第一像素点设置有三维模型的确定，将针对第一像素点设置的三维模型作为第一像素点对应的三维模型。
在步骤440，响应于针对第一像素点未设置有三维模型的确定，输出更新足迹信息的目标位置的提示信息。
可选地，提示信息用于提示用户更新其当前设置的足迹信息目标位置。即，提示信息用于提示用户其当前设置的足迹信息目标位置不能设置足迹信息，用户应重新设置足迹信息目标位置。可以通过文字或者音频或者图形等形式输出提示信息。在输出提示信息后，等待用户的后 续操作，如果用户此时触发了取消设置足迹信息的功能，则图4所示的流程结束。
在步骤450，响应于针对更新后的足迹信息的目标位置对应的当前视图中的像素点设置有三维模型的确定，将该设置有三维模型的像素点作为第一像素点。然后，流程返回步骤420。
如果用户当前执行了更新足迹信息目标位置的操作，则再次获得足迹信息目标位置。再次获得的足迹信息目标位置同样可以是当前视图中的一个像素点，该像素点即为第一像素点。也就是说，前次获得的第一像素点被当前获得的足迹信息目标位置所在的相似度更新。
通过判断用户自行设置的足迹信息目标位置是否设置有三维模型，并根据判断结果执行不同的操作，有利于避免在针对足迹信息目标位置未设置有三维模型的情况下，存在的无法将用户的足迹信息设置在三维模型中的相应位置处的现象。通过利用步骤420至步骤450的循环过程，有利于促使用户将其足迹信息最终设置在三维模型的相应位置处，从而有利于使足迹信息的位置更为恰当。
在一个可选示例中，在第一像素点设置有三维模型的情况下，由于当前视图中的第一像素点与三维模型中的点存在映射关系，因此，可以基于该映射关系获得第一像素点所对应的三维模型中的点，该点的位置即为第一像素点在三维模型中的位置。可以直接将第一像素点在三维模型中的位置作为用户的足迹信息在三维模型中的位置，从而有利于快捷准确地获得用户的足迹信息在三维模型中的位置。
在一个可选示例中，在浏览用户观看三维空间场景的过程中，可以为该浏览用户呈现至少一用户在该三维空间场景中留下的足迹信息。一个例子如图5所示。
图5中，在步骤510，对于浏览三维空间场景的任一浏览用户，确定该浏览用户在三维空间场景中的当前视角所对应的足迹区域。
可选地，浏览用户包括在该三维空间场景中设置其足迹信息的用户。足迹区域可以认为是针对需要显示的足迹信息设置的区域。该足迹区域可以为基于当前视图的足迹区域，也可以为基于三维模型的足迹区域。足迹区域的大小可以是预先设置的。足迹区域的形状可以为长方形或者圆形或者三角形等。
在足迹区域为基于当前视图的足迹区域时，根据本公开的一些示范性实施例，确定足迹区域的一种实现方式可以为：首先，获取浏览用户在三维空间场景中的当前视角所对应的当前视图的中心像素点，之后，以该中心像素点为圆心，以预定长度(如三维空间场景中的1.5米等，且1.5米可以被换算为当前视图中的长度)为半径，确定当前视图中的足迹区域。由于当前视图中的足迹区域中的至少部分像素点与三维模型中的点存在映射关系，因此，利用当前视图中的足迹区域，可以便捷的获得当前需要显示的足迹信息。另外，当前视图中的足迹区域可以认 为是一个圆形，即当前视图中的足迹区域不具有深度信息。
在足迹区域为基于三维模型的足迹区域时，根据本公开的一些示范性实施例，确定足迹区域的一种实现方式可以为：首先，获取浏览用户在三维空间场景中的当前视角所对应的当前视图的中心像素点，并确定该中心像素点是否设置有三维模型，如果该中心像素点设置有三维模型，则确定该中心像素点在该三维模型中的位置，之后，以该位置为圆心，以预定长度(如三维空间场景中的1.5米等)为半径，确定三维模型中的足迹区域。该足迹区域可能会完全位于一个三维模型中，也可能会跨多个三维模型。另外，三维模型中的足迹区域可以认为是一个圆柱体，即三维模型中的足迹区域具有深度信息。
在步骤520，确定三维模型中的属于足迹区域的足迹信息。
在足迹区域为基于当前视图的足迹区域时，本公开的实施例可以检查足迹区域中的每一个像素点是否与三维模型中的点存在映射关系。如果存在映射关系，再判断与像素点存在映射关系的三维模型中的点是否设置有足迹信息。如果设置有足迹信息，则可以将该足迹信息认为是属于该足迹区域的足迹信息。
在足迹区域为基于三维模型的足迹区域时，本公开的实施例可以检查足迹区域中的每一个点是否设置有足迹信息。如果设置有足迹信息，则可以将该足迹信息认为是属于该足迹区域的足迹信息。
在步骤530，在该浏览用户在三维空间场景中的当前视角所对应的当前视图中，显示属于该足迹区域的足迹信息。
可选地，可以根据各足迹信息的位置，确定出属于该足迹区域的各足迹信息分别在当前视图中的位置，从而可以根据各足迹信息分别在当前视图中的位置显示各足迹信息。在显示足迹信息的过程中，可以尽量避免不同足迹信息在当前视图中的重叠显示现象。
可选地，获得的多个足迹信息可能具有不同的位置，也可能具有相同的位置(即足迹信息的位置冲突)。响应于确定所述属于所述足迹区域的足迹信息具有不同位置的多个足迹信息，可以直接根据多个足迹信息分别在当前视图中的图像位置，在当前视图中显示各足迹信息。而且，可以允许显示的各足迹信息部分重叠，也可以通过位置控制使各足迹信息互不重叠。响应于确定所述属于所述足迹区域的足迹信息包括相同位置的不同足迹信息，可以在当前视图中为不同足迹信息分别分配不同的图像位置，并根据分配的图像位置，在当前视图中显示上述具有相同位置的不同足迹信息，从而有利于避免不同足迹信息在当前视图中的重叠显示现象。
可选地，可以显示属于该足迹区域的所有足迹信息，也可以显示属于该足迹区域的部分足迹信息。例如，在属于该足迹区域的所有足迹信息的数量过于庞大(例如，数量超过预定数量) 时，可以按照预定规则，从中选取部分足迹信息，并在当前视图中显示选取出的部分足迹信息。
可选地，可以从属于该足迹区域的所有足迹信息中随机选取出预定数量的足迹信息，并在当前视图中显示随机选取出的部分足迹信息。
可选地，可以从属于该足迹区域的所有足迹信息中优先选取浏览用户自己设置的足迹信息，还可以优先选取质量好的足迹信息等，并在当前视图中显示选取出的部分足迹信息。
在一个可选示例中，可以采用弹幕的形式，为浏览用户显示当前视图之外的足迹信息。例如，可以先确定三维模型中的不属于当前视图的所有足迹信息，并以弹幕的形式，在该浏览用户在三维空间场景中的当前视角所对应的当前视图中，显示上述所有足迹信息或者部分足迹信息。
在一个可选示例中，可以采用弹幕的形式，为浏览用户显示足迹区域之外的足迹信息。例如，可以先确定三维模型中的不属于足迹区域的所有足迹信息，并以弹幕的形式，在该浏览用户在三维空间场景中的当前视角所对应的当前视图中，显示上述所有足迹信息或者部分足迹信息。
通过采用弹幕的形式，显示不属于足迹区域/当前视图的足迹信息，不仅有利于促使浏览用户探查该三维空间场景的其他部分，提高浏览用户的沉浸感，而且有利于进一步提升浏览用户的VR全景体验。
图6为本公开的用于实现三维空间场景互动的装置一个实施例的结构示意图。该实施例的装置可用于实现本公开上述各方法实施例。
如图6所示，本实施例的装置包括：获取像素点模块600、确定三维模型模块601、确定位置模块602以及设置足迹信息模块603。另外，装置还可以包括：确定足迹区域模块604、确定足迹信息模块605、显示足迹信息模块606以及弹幕显示模块607。
获取像素点模块600用于响应于检测到在三维空间场景中设置足迹信息的用户操作，确定用户在三维空间场景中的当前视角所对应的当前视图中的第一像素点。
可选地，足迹信息可以包括：文本、图片、音频、视频以及三维模型中的至少一个。
可选地，获取像素点模块600可以包括：第一子模块6001。该第一子模块6001用于确定所述当前视图的中心像素点作为第一像素点。
可选地，获取像素点模块600可以包括：第五子模块6002。该第五子模块6002用于响应于用户在三维空间场景中的当前视角所对应的当前视图中设置足迹信息的目标位置的操作，确定足迹信息的目标位置对应的当前视图中的像素点。第五子模块6002可以将该像素点做为第一像素点。
确定三维模型模块601用于确定获取像素点模块600获取到的第一像素点对应的三维模型。
可选地，在获取像素点模块600包括第一子模块6001的情况下，确定三维模型模块601可以包括：第二子模块6011、第三子模块6012以及第四子模块6013。第二子模块6011用于确定针对第一像素点是否设置有三维模型。第三子模块6012用于如果第二子模块6011的确定结果为针对第一像素点设置有三维模型，则将针对第一像素点设置的三维模型作为第一像素点对应的三维模型。第四子模块6013用于如果第二子模块6011的判断结果为针对第一像素点未设置有三维模型，则将针对当前视图中的其他像素点设置的三维模型作为第一像素点对应的三维模型。例如，如果第二子模块6011的判断结果为针对第一像素点未设置有三维模型，则第四子模块6013可以以第一像素点为起点，根据预设检查规则，对三维空间场景中的当前视角所对应的当前视图中的其他像素点进行检查。如果检查到设置有三维模型的像素点，则将第一像素点更新为设置有三维模型的像素点，获得第一像素点对应的三维模型，并停止本次检查。
在获取像素点模块600包括第五子模块6002的情况下，确定三维模型模块601可以包括：第六子模块6014、第七子模块6015以及第八子模块6016。第六子模块6014用于确定针对第一像素点是否设置有三维模型。如果第六子模块6014的确定结果为针对第一像素点设置有三维模型，则第七子模块6015将针对第一像素点设置的三维模型作为第一像素点对应的三维模型。如果第六子模块6014的确定结果为针对第一像素点未设置有三维模型，则第八子模块6016可以输出更新足迹信息目标位置的提示信息，并在第六子模块6014判断出针对更新后的足迹信息目标位置对应的当前视图中的像素点设置有三维模型时，将该设置有三维模型的像素点作为第一像素点。第八子模块6016获得第一像素点对应的三维模型。
确定位置模块602用于确定用户的足迹信息在确定三维模型模块601确定出的三维模型中的位置。例如，确定位置模块602可以获取第一像素点在三维模型中的位置，且确定位置模块602可以将第一像素点在三维模型中的位置作为用户的足迹信息在三维模型中的位置。
设置足迹信息模块603用于在确定位置模块602确定出的位置处设置用户的足迹信息。设置足迹信息模块603设置的用户的足迹信息用于显示给三维空间场景的浏览用户。
确定足迹区域模块604用于对于浏览三维空间场景的任一浏览用户，确定该浏览用户在三维空间场景中的当前视角所对应的足迹区域。例如，确定足迹区域模块604可以先确定该浏览用户在三维空间场景中的当前视角所对应的当前视图的中心像素点，然后，确定足迹区域模块604以该中心像素点为圆心，以预定长度为半径，确定当前视图中的足迹区域。
确定足迹信息模块605用于确定三维模型中的属于确定足迹区域模块604确定出的足迹 区域的足迹信息。
显示足迹信息模块606用于在该浏览用户在三维空间场景中的当前视角所对应的当前视图中，显示确定足迹信息模块605确定出的属于足迹区域的足迹信息。
可选地，响应于确定所述属于所述足迹区域的足迹信息具有不同位置的多个足迹信息，显示足迹信息模块606可以根据多个足迹信息分别在当前视图中的图像位置，在当前视图中显示所述多个足迹信息。
可选地，响应于确定所述属于所述足迹区域的足迹信息具有相同位置的不同足迹信息，显示足迹信息模块606可以在当前视图中为不同足迹信息分配不同的图像位置，并根据分配的图像位置，在当前视图中显示不同足迹信息。
弹幕显示模块607用于确定三维模型中的不属于足迹区域/当前视图的至少一个足迹信息。弹幕显示模块607以弹幕的形式，在该浏览用户在三维空间场景中的当前视角所对应的当前视图中，显示该至少一个足迹信息。
上述各模块及其包括的子模块具体执行的操作可以参见上述方法实施例中针对图2-图5的描述，在此不再详细说明。
请参考图7，示出了根据本公开的第一个基于三维模型的交互方法的一个实施例的流程700。该基于三维模型的交互方法应用于第一用户终端，第一用户终端呈现有用户界面，该基于三维模型的交互方法包括：
步骤710，响应于检测到用户针对用户界面的目标交互操作，向为用户界面提供页面数据的服务器发送针对目标交互操作的交互请求，其中，用户界面用于呈现三维模型，三维模型与第二用户终端登录的用户账号建立关联关系。
在本实施例中，用户可以使用第一用户终端通过网络与服务器进行交互。第一用户终端可以是各种电子设备，包括但不限于智能手机、平板电脑、膝上型便携计算机和台式计算机等等。第一用户终端可以安装有各种客户端应用，例如房产交易软件等。上述用户界面可以是第一用户终端所安装的应用中的页面。实践中，用户可以通过该用户界面与服务器进行交互，进而实现与其他用户终端(例如第二用户终端)之间的交互。
在本实施例中，在检测到用户针对用户界面的目标交互操作的情况下，第一用户终端可以向为用户界面提供页面数据的服务器发送针对目标交互操作的交互请求。
上述用户界面用于呈现三维模型。三维模型与第二用户终端登录的用户账号预先建立关联关系。上述目标交互操作可以是各种用于指示第一用户终端请求与第二用户终端进行交互(信息交互)的操作。作为示例，该目标交互操作可以指示与第二用户终端进行视频通信。上述交 互请求可以用于指示第一用户终端的用户请求与第二用户终端进行交互。示例性的，上述交互请求可以用于指示第一用户终端的用户请求与第二用户终端进行视频通信。
在这里，在执行上述步骤710时，第一用户终端的用户界面可以呈现有上述三维模型，也可以未呈现三维模型。
实践中，对于每个三维模型，其可以与一个用户账号预先建立有关联关系。由此，对于一个特定的三维模型，可以通过确定与该三维模型建立有关联关系的用户账号，从而确定登录该用户账号的用户终端，进而确定出用于与第一用户终端进行交互的用户终端(即第二用户终端)。
上述三维模型可以是任意物体的三维模型。示例性的，该三维模型可以是细胞内部的三维模型，也可以是房屋室内三维模型。
步骤720，接收服务器从第二用户终端获取的流媒体视频。
在本实施例中，上述第一用户终端可以接收服务器从第二用户终端获取的流媒体视频。
其中，上述交互确认信息可以用于指示所述第二用户终端的用户确认(同意)与第一用户终端进行上述交互请求指示的交互。例如，上述交互确认信息可以用于指示所述第二用户终端的用户确认(同意)与第一用户终端进行视频通信。
上述流媒体视频可以包含图像和/或语音。实践中，第二用户终端的图像获取装置和/或语音获取装置，可以用于获取上述流媒体视频。
实践中，服务器可以采用流媒体技术，将第二用户终端采集的图像和/或语音(即流媒体视频)，持续发送至第一用户终端。其中，流媒体技术是指采用流式传输技术在网络上连续实时播放的媒体格式。流媒体技术也称流式媒体技术。这里，第二用户终端可以将其所采集的连续的影像和声音信息经过压缩处理后发送至服务器。由服务器向第一用户终端顺序或实时地传送各个压缩包，让使用第一用户终端的用户一边下载一边观看、收听。
可选地，服务器可以将第二用户终端采集的流媒体视频发送至第一用户终端，也可以对第二用户终端采集的流媒体视频进行图像处理(例如美颜)、语音处理(例如去噪)、转码、录制、鉴黄等操作后，将处理后的流媒体视频发送至第一用户终端。
在本实施例的一些可选的实现方式中，可以在服务器接收到第二用户终端针对交互请求发送的交互确认信息的情况下，第一用户终端再执行上述步骤720。
可以理解，上述可选的实现方式中，在第二用户终端针对交互请求发送的交互确认信息的情况下，通过后续步骤第一用户终端可以呈现流媒体视频；而在第二用户终端未发送上述交互确认信息的情况下，第一用户终端则不呈现流媒体视频。由此，可以在获得第二用户终端的用 户的允许(例如接通第一用户终端发起的视频通话)后，才在第一用户终端的用户界面上呈现流媒体视频和三维模型。这有助于提高对第二用户终端的用户的隐私保护性，为第二用户终端的用户向第一用户终端的用户呈现流媒体视频提供准备时间。
在本实施例的一些可选的实现方式中，在服务器接收到交互请求之后，第一用户终端也可以直接执行上述步骤720(而无需第二用户终端针对交互请求发送的交互确认信息)。
可以理解，上述可选的实现方式中，第二用户终端的用户可以处于向其他用户终端的用户拍摄流媒体视频(例如直播)的状态。由此，在服务器接收到交互请求之后，第一用户终端可以随时接收服务器从第二用户终端获取的流媒体视频，从而提高了流媒体视频呈现的实时性。
在本实施例的一些可选的实现方式中，第一用户终端可以采用如下步骤接收服务器从第二用户终端获取的流媒体视频：
首先，将第一用户终端的当前网速值发送至服务器。
然后，接收服务器从所述第二用户终端获取并发送的的流媒体视频，该流媒体视频具有与当前网速值相匹配的分辨率。
在这里，分辨率可以与网速值成正相关。
可以理解，通过接收服务器发送的分辨率与当前网速值相匹配的、第二用户终端获取的流媒体视频，可以在网络较差的情况下，降低第一用户终端接收的流媒体视频的分辨率，以提高流媒体视频传输的实时性。
步骤730，在用户界面上呈现流媒体视频和三维模型。
在本实施例中，第一用户终端可以在用户界面上，同屏呈现流媒体视频和三维模型。
在这里，第一用户终端的上述用户界面可以被划分为两个部分，上述两个部分可以分别呈现流媒体视频和三维模型。可选地，也可以将三维模型作为上述用户界面的背景，在用户界面的一部分页面区域呈现流媒体视频。
请参考图8A-8C，图8A-8C是针对图7的实施例的应用场景示意图。如图8A所示，在第一用户终端检测到用户针对用户界面的目标交互操作810(图示中目标交互操作810指示开启视频实时交互)的情况下，第一用户终端可以向为用户界面提供页面数据的服务器发送针对目标交互操作810的交互请求。其中，图8A中，用户界面呈现有XX家园的房屋室内的三维模型。该三维模型与第二用户终端登录的用户账号预先建立关联关系。在图8B中，第二用户终端接收到上述交互请求之后，第二用户终端的用户执行了开始交互的操作820。之后，第二用户终端向服务器发送针对交互请求的交互确认信息，以及第二用户终端采集的流媒体视频。最后，如图8C所示，第一用户终端在用户界面上呈现了流媒体视频830和三维模型。
本公开的上述实施例提供的基于三维模型的交互方法，可以在检测到用户针对用户界面的目标交互操作的情况下，向为用户界面提供页面数据的服务器发送针对目标交互操作的交互请求。用户界面用于呈现三维模型，三维模型与第二用户终端登录的用户账号预先建立关联关系。之后，接收服务器从第二用户终端获取的流媒体视频。最后，在用户界面上呈现流媒体视频和三维模型。通过在终端设备的同一页面呈现流媒体视频和三维模型，有助于采用流媒体视频向用户呈现三维模型相关的信息，提高了交互方式的多样性。通过多维度信息交互，让用户更加沉静地浏览三维模型，提升用户的浏览时长，有助于满足用户更多元化的交互需求。
在本实施例的一些可选的实现方式中，第一用户终端还可以执行如下步骤：
首先，接收服务器发送的模型调整信息，其中，模型调整信息指示使用第二用户终端的用户对呈现于第二用户终端的三维模型的调整操作。调整操作包括以下至少一项：缩放、旋转、移动、视点切换。
这里，通常情况下，用户可以对三维模型进行缩放、旋转、移动、视点切换中的至少一项操作。
然后，按照模型调整信息指示的调整操作，对用户界面上呈现的三维模型进行相同调整操作。
可以理解，上述可选的实现方式中，第二用户终端的用户对三维模型所执行的操作，可以同步到第一用户终端。由此，在第二用户终端采集的流媒体视频与三维模型相关(例如，第二用户中终端的用户对三维模型进行讲解、介绍等)时，方便第一用户终端的用户参考与第二用户终端呈现的相同三维模型，对流媒体视频中的信息进行获取，从而提高了信息获取的针对性。
在本实施例的一些可选的实现方式中，第一用户终端还可以执行如下步骤：
首先，获取用户针对流媒体视频的反馈信息。该反馈信息可以包括但不限于以下至少一项：点赞、评分、评论等等。该反馈信息可以用于表征第一用户终端的用户对第二用户终端的用户的流媒体视频的评价。
然后，将反馈信息发送至服务器，其中，服务器用于将反馈信息与用户账号建立关联关系。例如，可以采用关联存储的方式，将反馈信息与用户账号建立关联关系。
可以理解，将反馈信息与用户账号建立关联关系，可以反映第一用户终端的用户对三维模型指示的物体、对第二用户终端的用户的满意程度，进而可以更具针对性地为第一用户终端推送信息。
进一步参考图9，图9是本公开的第一个基于三维模型的交互方法的又一个实施例的流程900。该基于三维模型的交互方法应用于第一用户终端，第一用户终端呈现有用户界面，该方 法包括：
步骤910，响应于检测到用户针对用户界面的目标交互操作，向为用户界面提供页面数据的服务器发送针对目标交互操作的交互请求。
步骤920，接收服务器从第二用户终端获取的流媒体视频。
步骤930，在用户界面上呈现流媒体视频和三维模型。
在本实施例中，步骤910至步骤930分别与图7对应实施例中的步骤710至步骤730基本一致，这里不再赘述。
步骤940，响应于第一用户终端的当前网速值小于或等于预设网速阈值，基于流媒体视频中的各帧语音，对目标用户图像进行调整，生成不同于流媒体视频的新视频。
在本实施例中，在第一用户终端的当前网速值小于或等于预设网速阈值的情况下，第一用户终端可以基于流媒体视频中的各帧语音，对目标用户图像进行调整，生成新视频。新视频表征目标用户图像指示的用户执行各帧语音指示的动作。目标用户图像指示的用户可以是使用第二用户终端的用户。可选地，新视频可以是基于网络分段发送、即时传输的流媒体视频，也可以是无需基于网络在本地生成的或视频。
具体地，第一用户终端可以采用如下方式生成新视频：对于流媒体视频中的每帧语音，将该帧语音输入至预先确定的图像帧生成模型，得到与该帧语音相匹配的、目标用户图像指示的用户的图像。从而将所得到的与流媒体视频中的各帧语音相匹配的各帧图像，以及该各帧语音进行融合，从而得到新视频。与语音相匹配的、目标用户图像指示的用户的图像中用户的动作与该语音相吻合。例如，如果语音为“啊”的音频，并且该音频表征用户处于惊吓状态，那么，与该音频相匹配的目标用户图像指示的用户的图像中的用户的口型可以是发出语音“啊”的口型，动作可以是处于惊吓状态下的动作。
在这里，上述图像帧生成模型可以是采用机器学习算法，基于包括语音帧、目标用户图像和与语音帧相匹配的图像帧的训练样本，训练得到的循环神经网络模型或卷积神经网络模型。针对每个用户可以训练得到一个图像帧生成模型，用以训练该用户的图像帧生成模型的各个训练样本中的目标用户图像可以相同，针对该用户的每个语音帧确定出与该语音帧相匹配的图像帧，进而得到用以训练该用户的图像帧生成模型的训练样本集合。
可选地，图像帧生成模型还可以是关联存储有语音帧、目标用户图像和与语音帧相匹配的图像帧的二维表或数据库。在图像帧生成模型还是关联存储有语音帧、目标用户图像和与语音帧相匹配的图像帧的数据库的情况下，该数据库的每条记录可以包括语音帧、目标用户图像和与语音帧相匹配的图像帧。各条记录中的目标用户图像可以相同，针对该用户的每个语音帧确 定出与该语音帧相匹配的图像帧，进而得到关联存储有语音帧、目标用户图像和与语音帧相匹配的图像帧的数据库，即图像帧生成模型。
在本实施例的一些可选的实现方式中，第一用户终端还可以通过以下任一方式，确定出目标用户图像：
(1)基于流媒体视频中的图像，生成目标用户图像。
这里，可以从流媒体视频中的各帧图像中，随机选取一张图像作为目标用户图像，也可以从流媒体视频中的各帧图像中，选取一张面部图像区域与整张图像帧的面积之比大于预设阈值的图像，作为目标用户图像。
(2)将与用户账号相关联的用户图像确定为目标用户图像。
这里，用户可以通过其所使用的用户账号上传一张图像，作为目标用户图像；也可以在登录其所使用的账号之后，从预先确定的图像集合中选取一张图像，作为目标用户图像。
可以理解，上述可选的实现方式可以实现从流媒体视频中的图像中，自动生成目标用户图像，或者，用户手动设置目标用户图像，从而基于多种目标用户图像的确定方式，使得新视频的生成方式更为多样化。
步骤950，采用新视频替代流媒体视频进行呈现。
在本实施例中，第一用户终端可以采用新视频替代流媒体视频进行呈现。换言之，在第一用户终端呈现新视频时，可以对流媒体视频进行隐藏(即不再呈现)。
需要说明的是，除上面所记载的内容外，本申请实施例还可以包括与图7对应的实施例相同或类似的特征、效果，在此不再赘述。
从图9中可以看出，本实施例中的基于三维模型的交互方法的流程900在第一用户终端的当前网速值较小(小于或等于预设网速阈值)的情况下，第一用户终端可以在本地生成用以替代流媒体视频呈现的新视频。由此，第一用户终端只需从服务器持续获取语音，而无需持续获取视频，从而降低了对网络资源的占用。在第一用户终端的当前网速值较小的情况下，可以提高第一用户终端的视频呈现的实时性。
在本实施例的一些可选的实现方式中，在用户界面呈现有新视频(未呈现第二用户终端获取的流媒体视频)的情况下，第一用户终端还可以向上述服务器发送摄像头关闭确认信息。其中，摄像头关闭确认信息用于确定第二用户终端是否关闭摄像头。
可以理解，在服务器接收到摄像头关闭确认信息之后，服务器可以向第二用户终端发送用于确定第二用户终端是否关闭摄像头的信息。由此，第二用户终端的用户可以通过关闭摄像头，来降低第二用户终端对网络资源的占用。
请继续参考图10，图10是本公开的第一个基于三维模型的交互方法的又一个实施例的流程图。该基于三维模型的交互方法应用于第一用户终端，第一用户终端呈现有用户界面。该基于三维模型的交互方法的流程1000，包括：
步骤1010，响应于检测到用户针对用户界面的目标交互操作，向为用户界面提供页面数据的服务器发送针对目标交互操作的交互请求。用户界面用于呈现三维模型，三维模型与第二用户终端登录的用户账号预先建立关联关系。
步骤1020，接收服务器从第二用户终端获取的流媒体视频。
步骤1030，在用户界面上呈现流媒体视频和三维模型。
在本实施例中，步骤1010至步骤1030分别与图7对应实施例中的步骤710至步骤730基本一致，这里不再赘述。
需要说明的是，在本实施例中，三维模型包括多个子空间场景的三维子模型，多个子空间场景中的子空间场景与预先确定的关键词集合中的关键词相对应。
步骤1040，对流媒体视频中的语音进行语音识别，得到语音识别结果。
在本实施例中，第一用户终端可以对流媒体视频中的语音进行语音识别，得到语音识别结果。
这里，语音识别结果可以表征流媒体视频中的语音对应的文字。
步骤1050，响应于确定语音识别结果包含关键词集合中的关键词的确定，在用户界面上呈现与语音识别结果包含的关键词相对应的多个子空间场景中的对应子空间场景的三维子模型。
在本实施例中，在确定语音识别结果包含关键词集合中的关键词的情况下，第一用户终端可以在上述用户界面上，呈现与语音识别结果包含的关键词相对应的子空间场景的三维子模型。
作为示例，如果上述三维模型为房屋室内的三维模型。该房屋包括卧室、客厅、厨房、卫生间，共四个子空间场景。也即，上述三维模型包括卧室的三维子模型、客厅的三维子模型、厨房的三维子模型、卫生间的三维子模型。关键词集合包括卧室、客厅、厨房、卫生间。由此，与子空间场景卧室相对应的关键词可以是“卧室”；与子空间场景厨房相对应的关键词可以是“厨房”；与子空间场景客厅相对应的关键词可以是“客厅”；与子空间场景卫生间相对应的关键词可以是“卫生间”。进一步地，作为示例，如果语音识别结果包含关键词“卧室”，那么，第一用户终端可以在上述用户界面上，呈现卧室的三维子模型。
在这里，可以通过切换三维模型的视点，实现呈现与语音识别结果包含的关键词相对应的 子空间场景的三维子模型。
需要说明的是，除上面所记载的内容外，本申请实施例还可以包括与图7和/或图9对应的实施例相同或类似的特征、效果，在此不再赘述。
从图10中可以看出，本实施例中的基于三维模型的交互方法的流程1000中，可以通过语音实现三维模型的视点切换，从而呈现与语音识别结果包含的关键词相对应的子空间场景的三维子模型。由此，提高了三维模型浏览的便利性，提高了所呈现的三维模型与第二用户终端获取的语音之间的匹配性。
请继续参考图11，示出了根据本公开的第二个基于三维模型的交互方法的一个实施例的流程1100。该基于三维模型的交互方法应用于第二用户终端，第二用户终端登录的用户账号与三维模型预先建立关联关系。该基于三维模型的交互方法包括：
步骤1110，响应于接收到服务器发送的交互请求，获取流媒体视频。
在本实施例中，用户可以使用第二用户终端通过网络与服务器、第一用户终端进行交互。第二用户终端可以是各种电子设备，包括但不限于智能手机、平板电脑、膝上型便携计算机和台式计算机等等。第二用户终端可以安装有各种客户端应用，例如房产交易软件等。
在本实施例中，在接收到服务器发送的交互请求的情况下，获取流媒体视频。
交互请求指示第一用户终端检测到用户针对第一用户终端呈现的用户界面的目标交互操作。示例性地，上述交互请求可以用于指示第一用户终端的用户请求与第二用户终端进行视频通信。用户界面用于呈现三维模型。流媒体视频可以包含图像和/或语音。实践中，第二用户终端的图像获取装置和/或语音获取装置，可以用于获取上述流媒体视频。
实践中，在在检测到第一用户终端的用户针对用户界面的目标交互操作的情况下，第一用户终端可以向为用户界面提供页面数据的服务器发送针对目标交互操作的交互请求。用户界面用于呈现三维模型。三维模型与第二用户终端登录的用户账号预先建立关联关系。上述目标交互操作可以是各种用于指示第一用户终端请求与第二用户终端进行交互(信息交互)的操作。作为示例，该目标交互操作可以指示与第二用户终端进行视频通信。
在这里，在执行步骤1110时，第一用户终端的用户界面可以呈现有上述三维模型，也可以未呈现三维模型。
步骤1120，向服务器发送流媒体视频。
在本实施例中，第二用户终端可以向服务器发送流媒体视频。服务器用于将流媒体视频发送至第一用户终端，以使第一用户终端在用户界面上呈现流媒体视频和三维模型。
实践中，服务器可以采用流媒体技术，将第二用户终端采集的图像和/或语音(即流媒体将 视频)，持续发送至第一用户终端。流媒体技术是指采用流式传输技术在网络上连续实时播放的媒体格式。这里，第二用户终端可以将其所采集的连续的影像和声音信息经过压缩处理后发送至服务器。由服务器向第一用户终端顺序或实时地传送各个压缩包，让使用第一用户终端的用户一边下载一边观看、收听。
可选地，服务器可以将第二用户终端采集的流媒体视频发送至第一用户终端，也可以对第二用户终端采集的流媒体视频进行图像处理(例如美颜)、语音处理(例如去噪)、转码、录制、鉴黄等操作后，将处理后的流媒体视频发送至第一用户终端。
本公开的上述实施例提供的第二种基于三维模型的交互方法，应用于第二用户终端，第二用户终端登录的用户账号与三维模型预先建立关联关系。第二用户终端可以在接收到服务器发送的交互请求的情况下，确定是否检测到用户针对交互请求的确认操作。交互请求指示第一用户终端检测到用户针对第一用户终端呈现的用户界面的目标交互操作，用户界面用于呈现三维模型。之后，在检测到确认操作的情况下，获取流媒体视频。最后向服务器发送流媒体视频，其中，服务器用于将流媒体视频发送至第一用户终端，以使第一用户终端在用户界面上呈现流媒体视频和三维模型。本公开实施例通过在终端设备的同一页面呈现流媒体视频和三维模型，有助于采用流媒体视频向用户呈现三维模型相关的信息，提高了交互方式的多样性。通过多维度信息交互，让用户更加沉静地浏览三维模型，提升用户的浏览时长，有助于满足用户更多元化的交互需求。
在本实施例的一些可选的实现方式中，上述步骤1110可以包括以下步骤：
首先，在接收到服务器发送的交互请求的情况下，确定是否检测到用户针对交互请求的确认操作响应于检测到确认操作。确认操作表征第二用户终端的用户确认(同意)与第一用户终端进行交互(例如视频通信)。
然后，在检测到确认操作的情况下，获取流媒体视频。
可以理解，上述可选的实现方式中，在第二用户终端针对交互请求发送的交互确认信息的情况下，第一用户终端可以呈现流媒体视频；而在第二用户终端未发送上述交互确认信息的情况下，第一用户终端则不呈现流媒体视频。由此，可以在获得第二用户终端的用户的允许(例如接通第一用户终端发起的视频通话)后，才在第一用户终端的用户界面上呈现流媒体视频和三维模型。这有助于提高对第二用户终端的用户的隐私保护性，为第二用户终端的用户向第一用户终端的用户呈现流媒体视频提供准备时间。
在本实施例的一些可选的实现方式中，在服务器接收到交互请求之后，第二用户终端也可以直接获取流媒体视频，并将流媒体视频通过服务器发送至第一用户终端，而无需第二用户终 端的用户针对交互请求发送的交互确认信息。
可以理解，上述可选的实现方式中，第二用户终端的用户可以处于向其他用户终端的用户拍摄流媒体视频(例如直播)的状态。由此，在服务器接收到交互请求之后，第一用户终端可以随时接收服务器从第二用户终端获取的流媒体视频，从而提高了流媒体视频呈现的实时性。
在本实施例的一些可选的实现方式中，在第一用户终端的当前网速值小于或等于预设网速阈值的情况下，第二用户终端可以从服务器接收摄像头关闭确认信息，以及呈现摄像头关闭确认信息。摄像头关闭确认信息用于确定第二用户终端是否关闭摄像头。
可以理解，在服务器接收到第一用户终端的当前网速值小于或等于预设网速阈值的信息之后，服务器可以向第二用户终端发送用于确定第二用户终端是否关闭摄像头的信息。由此，第二用户终端的用户可以通过关闭摄像头，来降低第二用户终端对网络资源的占用。
在本实施例中的一些可选的实现方式中，在检测到用户对呈现于第二用户终端的三维模型的调整操作的情况下，第二用户终端可以向服务器发送指示调整操作的模型调整信息，以使服务器控制第一用户终端按照模型调整信息指示的调整操作，对用户界面上呈现的三维模型进行相同调整操作。调整操作包括以下至少一项：缩放、旋转、移动、视点切换。
这里，通常情况下，用户可以对三维模型进行缩放、旋转、移动、视点切换中的至少一项操作。
可以理解，上述可选的实现方式中，第二用户终端的用户对三维模型所执行的操作，可以同步到第一用户终端。由此，在第二用户终端采集的流媒体视频与三维模型相关(例如，第二用户中终端的用户对三维模型进行讲解、介绍等)时，方便第一用户终端的用户参考与第二用户终端呈现的相同三维模型，对流媒体视频中的信息进行获取，从而提高了信息获取的针对性。
在本实施例中的一些可选的实现方式中，在从服务器接收到用户对呈现于第一用户终端的三维模型的调整操作的情况下，第二用户终端可以按照模型调整信息指示的调整操作，对第二用户终端呈现的三维模型进行相同调整操作。调整操作包括以下至少一项：缩放、旋转、移动、视点切换。
这里，通常情况下，用户可以对三维模型进行缩放、旋转、移动、视点切换中的至少一项操作。
可以理解，上述可选的实现方式中，第一用户终端的用户对三维模型所执行的操作，可以同步到第二用户终端。由此，可以方便第一用户终端的用户参考与第二用户终端呈现的相同三维模型，对流媒体视频中的信息进行获取，从而提高了信息获取的针对性。
在本实施例中的一些可选的实现方式中，在接收到服务器发送的、使用第一用户终端的用 户针对流媒体视频的反馈信息的情况下，上述第二用户终端可以执行与反馈信息相匹配的操作。该反馈信息可以包括但不限于以下至少一项：点赞、评分、评论等等。该反馈信息可以用于表征第一用户终端的用户对第二用户终端的用户的流媒体视频的评价。
作为示例，如果第一用户终端的用户针对流媒体视频的反馈信息为点赞，那么，第二用户终端可以呈现与该反馈信息相匹配的操作，例如呈现“XX为您点了一赞！”。
可以理解，上述可选的实现方式可以提高互动的真实性、多样性。
进一步参考图12，图12是本公开的第二个基于三维模型的交互方法的又一个实施例的流程1200，该基于三维模型的交互方法应用于第一用户终端，第一用户终端呈现有用户界面，该方法包括：
步骤1210，响应于接收到服务器发送的交互请求，获取流媒体视频。
步骤1220，向服务器发送流媒体视频。
在本实施例中，步骤1210至步骤1220分别与图11对应实施例中的步骤1110至步骤1120基本一致，这里不再赘述。
需要说明的是，在本实施例中，三维模型包括多个子空间场景的三维子模型，多个子空间场景中的子空间场景与预先确定的关键词集合中的关键词相对应。
步骤1230，对第一用户终端获取的语音进行语音识别，得到语音识别结果。
在本实施例中，第二用户终端可以对第一用户终端获取的语音进行语音识别，得到语音识别结果。
这里，语音识别结果可以表征流媒体视频中的语音对应的文字。
步骤1240，响应于确定语音识别结果包含关键词集合中的关键词的确定，在用户界面上呈现与语音识别结果包含的关键词相对应的多个子空间场景中的对应子空间场景的三维子模型。
在本实施例中，在确定语音识别结果包含关键词集合中的关键词的情况下，第二用户终端可以在用户界面上呈现与语音识别结果包含的关键词相对应的子空间场景的三维子模型。
作为示例，如果上述三维模型为房屋室内的三维模型。该房屋包括卧室、客厅、厨房、卫生间，共四个子空间场景，也即上述三维模型包括卧室的三维子模型、客厅的三维子模型、厨房的三维子模型、卫生间的三维子模型。关键词集合包括卧室、客厅、厨房、卫生间。由此，与子空间场景卧室相对应的关键词可以是“卧室”；与子空间场景厨房相对应的关键词可以是“厨房”；与子空间场景客厅相对应的关键词可以是“客厅”；与子空间场景卫生间相对应的关键词可以是“卫生间”。进一步地，作为示例，如果语音识别结果包含关键词“卧室”，那么，第二 用户终端可以在上述用户界面上，呈现卧室的三维子模型。
在这里，可以通过切换三维模型的视点，实现呈现与语音识别结果包含的关键词相对应的子空间场景的三维子模型。
需要说明的是，除上面所记载的内容外，本申请实施例还可以包括与图11对应的实施例相同或类似的特征、效果，在此不再赘述。
从图12中可以看出，本实施例中的基于三维模型的交互方法的流程1200中，可以通过语音实现三维模型的视点切换，从而呈现与语音识别结果包含的关键词相对应的子空间场景的三维子模型。由此，提高了三维模型浏览的便利性，提高了所呈现的三维模型与第二用户终端获取的语音之间的匹配性。
进一步参考图13，作为对上述第一个基于三维模型的交互方法的实现，本公开提供了一种基于三维模型的交互装置的一个实施例，该装置实施例与图7、9、10所示的方法实施例相对应，除下面所记载的特征外，该装置实施例还可以包括与图7、9、10所示的方法实施例相同或相应的特征，以及产生与图7、9、10所示的方法实施例相同或相应的效果。
如图13所示，本实施例的基于三维模型的交互装置1300设置于第一用户终端，第一用户终端呈现有用户界面。该装置1300包括：第一发送单元1310，被配置成响应于检测到用户针对用户界面的目标交互操作，向为用户界面提供页面数据的服务器发送针对目标交互操作的交互请求，其中，用户界面用于呈现三维模型，三维模型与第二用户终端登录的用户账号预先建立关联关系；第一接收单元1320，被配置成接收服务器从第二用户终端获取的流媒体视频；第一呈现单元1330，被配置成在用户界面上呈现流媒体视频和三维模型。
在本实施例中，在检测到用户针对用户界面的目标交互操作的情况下，基于三维模型的交互装置1300的第一发送单元1310可以向为用户界面提供页面数据的服务器发送针对目标交互操作的交互请求。用户界面用于呈现三维模型，三维模型与第二用户终端登录的用户账号预先建立关联关系。
在本实施例中，第一接收单元1320可以接收服务器从第二用户终端获取的流媒体视频。
在本实施例中，第一呈现单元1330可以在用户界面上呈现流媒体视频和三维模型。
在本实施例的一些可选的实现方式中，第一接收单元进一步被配置成：响应于服务器接收到第二用户终端针对交互请求发送的交互确认信息，接收服务器从第二用户终端获取的流媒体视频。
在本实施例的一些可选的实现方式中，该装置1300还包括：第一调整单元(图中未示出)，被配置成响应于第一用户终端的当前网速值小于或等于预设网速阈值，基于流媒体视频中的各 帧语音，对目标用户图像进行调整，生成新视频，其中，新视频表征目标用户图像指示的用户执行各帧语音指示的动作；第二呈现单元(图中未示出)，被配置成采用新视频替代流媒体视频进行呈现。
在本实施例的一些可选的实现方式中，该装置1300还包括：第一生成单元(图中未示出)，被配置成基于流媒体视频中的图像，生成目标用户图像；或者，第一确定单元(图中未示出)，被配置成将与用户账号相关联的用户图像确定为目标用户图像。
在本实施例的一些可选的实现方式中，该装置1300还包括：第二发送单元(图中未示出)，被配置成响应于用户界面呈现有新视频，向服务器发送摄像头关闭确认信息，其中，摄像头关闭确认信息用于确定第二用户终端是否关闭摄像头。
在本实施例的一些可选的实现方式中，第一接收单元进一步被配置成：将第一用户终端的当前网速值发送至服务器；接收服务器从第二用户终端获取并发送的流媒体视频，该流媒体视频具有与当前网速值相匹配的分辨率。
在本实施例的一些可选的实现方式中，该装置1300还包括：第二接收单元(图中未示出)，被配置成接收服务器发送的模型调整信息，其中，模型调整信息指示使用第二用户终端的用户对呈现于第二用户终端的三维模型的调整操作，调整操作包括以下至少一项：缩放、旋转、移动、视点切换；第二调整单元(图中未示出)，被配置成按照模型调整信息指示的调整操作，对用户界面上呈现的三维模型进行相同调整操作。
在本实施例的一些可选的实现方式中，三维模型包括多个子空间场景的三维子模型，多个子空间场景中的子空间场景与预先确定的关键词集合中的关键词相对应；以及，该装置1300还包括：第一识别单元(图中未示出)，被配置成对流媒体视频中的语音进行语音识别，得到语音识别结果；第三呈现单元(图中未示出)，被配置成响应于确定语音识别结果包含关键词集合中的关键词，在用户界面上呈现与语音识别结果包含的关键词相对应的子空间场景的三维子模型。
在本实施例的一些可选的实现方式中，该装置1300还包括：第一获取单元(图中未示出)，被配置成获取用户针对流媒体视频的反馈信息；第三发送单元(图中未示出)，被配置成将反馈信息发送至服务器，其中，服务器用于将反馈信息与用户账号建立关联关系。
本公开的上述实施例提供的基于三维模型的交互装置设置于第一用户终端，第一用户终端呈现有用户界面。该装置1300中，在检测到用户针对用户界面的目标交互操作的情况下，第一发送单元1310可以向为用户界面提供页面数据的服务器发送针对目标交互操作的交互请求，其中，用户界面用于呈现三维模型，三维模型与第二用户终端登录的用户账号预先建立关 联关系，然后，第一接收单元1320接收服务器从第二用户终端获取的流媒体视频，最后，第一呈现单元1330在用户界面上呈现流媒体视频和三维模型。由此，可以在终端设备的同一页面呈现流媒体视频和三维模型，有助于采用流媒体视频向用户呈现三维模型相关的信息，提高了交互方式的多样性，可以通过多维度信息交互，让用户更加沉静的浏览三维模型，提升用户的浏览时长，有助于满足用户更多元化的交互需求。
进一步参考图14，作为对上述第二个基于三维模型的交互方法的实现，本公开提供了第二种基于三维模型的交互装置的一个实施例，该装置实施例与图11、12所示的方法实施例相对应，除下面所记载的特征外，该装置实施例还可以包括与图11、12所示的方法实施例相同或相应的特征，以及产生与图11、12所示的方法实施例相同或相应的效果。
如图14所示，本实施例的基于三维模型的交互装置1400设置于第二用户终端。该装置1400包括：第二确定单元1410，被配置成响应于接收到服务器发送的交互请求，获取流媒体视频，其中，交互请求指示第一用户终端检测到用户针对第一用户终端呈现的用户界面的目标交互操作，用户界面用于呈现三维模型，三维模型与所述第二用户终端登录的用户账号预先建立关联关系；第四发送单元1420，被配置成向服务器发送流媒体视频，其中，服务器用于将流媒体视频发送至第一用户终端，以使第一用户终端在用户界面上呈现流媒体视频和三维模型。
在本实施例中，在接收到服务器发送的交互请求的情况下，第二确定单元1410可以获取流媒体视频。其中，交互请求指示第一用户终端检测到用户针对第一用户终端呈现的用户界面的目标交互操作，用户界面用于呈现三维模型。
在本实施例中，第四发送单元1420可以成向服务器发送流媒体视频，其中，服务器用于将流媒体视频发送至第一用户终端，以使第一用户终端在用户界面上呈现流媒体视频和三维模型。
在本实施例的一些可选的实现方式中，第二确定单元1410进一步被配置成：响应于接收到服务器发送的交互请求，确定是否检测到用户针对交互请求的确认操作；响应于检测到确认操作，获取流媒体视频。
在本实施例的一些可选的实现方式中，该装置1400还包括：第三接收单元(图中未示出)，被配置成响应于第一用户终端的当前网速值小于或等于预设网速阈值，从服务器接收摄像头关闭确认信息，以及呈现摄像头关闭确认信息，其中，摄像头关闭确认信息用于确定第二用户终端是否关闭摄像头。
在本实施例的一些可选的实现方式中，该装置1400还包括：第五发送单元(图中未示出)，被配置成响应于从服务器接收到用户对呈现于第二用户终端的三维模型的调整操作，按照模型 调整信息指示的调整操作，对用户界面上呈现的三维模型进行相同调整操作，其中，调整操作包括以下至少一项：缩放、旋转、移动、视点切换。
在本实施例的一些可选的实现方式中，该装置1400还包括：第五发送单元(图中未示出)，被配置成响应于检测到用户对呈现于第二用户终端的三维模型的调整操作，向服务器发送指示调整操作的模型调整信息，以使服务器控制第一用户终端按照模型调整信息指示的调整操作，对用户界面上呈现的三维模型进行相同调整操作，其中，调整操作包括以下至少一项：缩放、旋转、移动、视点切换。
在本实施例的一些可选的实现方式中，三维模型包括多个子空间场景的三维子模型，多个子空间场景中的子空间场景与预先确定的关键词集合中的关键词相对应；以及，该装置1400还包括：第二识别单元(图中未示出)，被配置成对第一用户终端获取的语音进行语音识别，得到语音识别结果；第四呈现单元(图中未示出)，被配置成响应于确定语音识别结果包含关键词集合中的关键词，在用户界面上呈现与语音识别结果包含的关键词相对应的子空间场景的三维子模型。
在本实施例的一些可选的实现方式中，该装置1400还包括：执行单元(图中未示出)，被配置成响应于接收到服务器发送的、使用第一用户终端的用户针对流媒体视频的反馈信息，执行与反馈信息相匹配的操作。
本公开的上述实施例提供的基于三维模型的交互装置设置于第二用户终端，第二用户终端登录的用户账号与三维模型预先建立关联关系，该装置1400中，在接收到服务器发送的交互请求的情况下，第二确定单元1410可以获取流媒体视频，其中，交互请求指示第一用户终端检测到用户针对第一用户终端呈现的用户界面的目标交互操作，用户界面用于呈现三维模型，之后，第四发送单元1420可以向服务器发送流媒体视频，其中，服务器用于将流媒体视频发送至第一用户终端，以使第一用户终端在用户界面上呈现流媒体视频和三维模型。由此，可以在终端设备的同一页面呈现流媒体视频和三维模型，有助于采用流媒体视频向用户呈现三维模型相关的信息，提高了交互方式的多样性，可以通过多维度信息交互，让用户更加沉静的浏览三维模型，提升用户的浏览时长，有助于满足用户更多元化的交互需求。
请继续参考图15，图15为本公开的基于三维模型的交互系统的一个实施例1500的交互示意图。该基于三维模型的交互系统包括第一用户终端、第二用户终端和服务器，第一用户终端呈现有用户界面，服务器与第一用户终端、第二用户终端分别通信连接。
图15所示，该基于三维模型的交互系统中的第一用户终端、第二用户终端和服务器可以执行如下步骤：
步骤1501，第一用户终端检测到用户针对用户界面的目标交互操作。
在本实施例中，第一用户终端检测到用户针对用户界面的目标交互操作。用户界面用于呈现三维模型，三维模型与第二用户终端登录的用户账号预先建立关联关系。
步骤1502，第一用户终端向服务器发送针对目标交互操作的交互请求。
在本实施例中，第一用户终端可以向服务器发送针对目标交互操作的交互请求。
步骤1503，第二用户终端获取流媒体视频。
在本实施例中，第二用户终端可以获取流媒体视频。
步骤1504，第二用户终端向服务器发送流媒体视频。
在本实施例中，第二用户终端可以向服务器发送流媒体视频。
步骤1505，服务器向第一用户终端发送流媒体视频。
在本实施例中，服务器可以向第一用户终端发送流媒体视频。
步骤1506，第一用户终端在用户界面上呈现流媒体视频和三维模型。
在本实施例中，第一用户终端可以在用户界面上呈现流媒体视频和三维模型。
在本实施例中，在不冲突的前提下，除本实施例中上述记载的内容之外，步骤1501至步骤1506中的技术特征还可以参考上述第一种基于三维模型的交互方法的各个实施例、第二种基于三维模型的交互方法的各个实施例，以及第三种基于三维模型的交互方法中的各个实施例中的技术特征进行解释。并且，该实施例还可以包括与上述基于三维模型的交互方法实施例相同或相应的特征，以及产生与其相同或相应的效果，在此不再赘述。
本公开的上述实施例提供的基于三维模型的交互系统包括第一用户终端、第二用户终端和服务器，第一用户终端呈现有用户界面，服务器与第一用户终端、第二用户终端分别通信连接。其中：第一用户终端被配置成：响应于检测到用户针对用户界面的目标交互操作，向服务器发送针对目标交互操作的交互请求，其中，用户界面用于呈现三维模型，三维模型与第二用户终端登录的用户账号预先建立关联关系；第二用户终端被配置成：获取流媒体视频；向服务器发送流媒体视频；服务器还被配置成：向第一用户终端发送流媒体视频；第一用户终端被配置成：在用户界面上呈现流媒体视频和三维模型。由此，可以在终端设备的同一页面呈现流媒体视频和三维模型，有助于采用流媒体视频向用户呈现三维模型相关的信息，提高了交互方式的多样性，可以通过多维度信息交互，让用户更加沉静的浏览三维模型，提升用户的浏览时长，有助于满足用户更多元化的交互需求。
下面参考图16来描述根据本公开实施例的电子设备。图16示出了根据本公开实施例的电子设备1600的框图。如图16所示，电子设备1600包括一个或多个处理器1611和存储器1612。
处理器1611可以是中央处理单元(CPU)或者具有用于实现三维空间场景互动的能力和/或指令执行能力的其他形式的处理单元，并且可以控制电子设备1600中的其他组件以执行期望的功能。
存储器1612可以包括一个或多个计算机程序产品，所述计算机程序产品可以包括各种形式的计算机可读存储介质，例如易失性存储器和/或非易失性存储器。所述易失性存储器，例如，可以包括：随机存取存储器(RAM)和/或高速缓冲存储器(cache)等。所述非易失性存储器，例如，可以包括：只读存储器(ROM)、硬盘以及闪存等。在所述计算机可读存储介质上可以存储一个或多个计算机程序指令，处理器1611可以运行所述程序指令，以实现上文所述的各种方法以及/或者其他期望的功能。在所述计算机可读存储介质中还可以存储诸如输入信号、信号分量、噪声分量等各种内容。
在一个示例中，电子设备1600还可以包括：输入装置1613以及输出装置1614等，这些组件通过总线系统和/或其他形式的连接机构(未示出)互连。此外，该输入设备1613还可以包括例如键盘、鼠标等等。该输出装置1614可以向外部输出各种信息。该输出设备1614可以包括例如显示器、扬声器、打印机、以及通信网络及其所连接的远程输出设备等等。
为了简化，图16中仅示出了该电子设备1600中与本公开实施例有关的组件中的一些，省略了诸如总线、输入/输出接口等等的组件。除此之外，根据具体应用情况，电子设备1600还可以包括任何其他适当的组件。除了上述方法和设备以外，本公开的实施例还可以是计算机程序产品，其包括计算机程序指令，所述计算机程序指令在被处理器运行时使得所述处理器执行根据本公开各种实施例的各种方法中的步骤。
所述计算机程序产品可以以一种或多种程序设计语言的任意组合来编写用于执行本公开实施例操作的程序代码，所述程序设计语言包括面向对象的程序设计语言，诸如Java、C++等，还包括常规的过程式程序设计语言，诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算设备上执行、部分地在用户设备上执行、作为一个独立的软件包执行、部分在用户计算设备上部分在远程计算设备上执行、或者完全在远程计算设备或服务器上执行。
此外，本公开的实施例还可以是计算机可读存储介质，其上存储有计算机程序指令，所述计算机程序指令在被处理器运行时使得所述处理器执行根据本公开各种实施例的各种方法中的步骤。
所述计算机可读存储介质可以采用一个或多个可读介质的任意组合。可读介质可以是可读信号介质或者可读存储介质。可读存储介质例如可以包括但不限于电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。可读存储介质的更具体的例子(非穷举 的列举)可以包括：具有一个或者多个导线的电连接、便携式盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、光纤、便携式紧凑盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。
以上结合具体实施例描述了本公开的基本原理，但是，需要指出的是，在本公开中提及的优点、优势、效果等仅是示例而非限制，不能认为这些优点、优势以及效果等是本公开的各个实施例必须具备的。另外，上述公开的具体细节仅是为了示例的作用和便于理解的作用，而非限制，上述细节并不限制本公开为必须采用上述具体的细节来实现。
本说明书中各个实施例均采用递进的方式描述，每个实施例重点说明的都是与其它实施例的不同之处，各个实施例之间相同或相似的部分相互参见即可。对于系统实施例而言，由于其与方法实施例基本对应，所以描述的比较简单，相关之处参见方法实施例的部分说明即可。
本公开中涉及的器件、装置、设备、系统的方框图仅作为例示性的例子并且不意图要求或暗示必须按照方框图示出的方式进行连接、布置、配置。如本领域技术人员将认识到的，可以按任意方式连接、布置、配置这些器件、装置、设备以及系统。诸如“包括”、“包含、“具有”等等的词语是开放性词汇，指“包括但不限于”，且可与其互换使用。这里所使用的词汇“或”和“和”指词汇“和/或”，且可与其互换使用，除非上下文明确指示不是如此。这里所使用的词汇“诸如”指词组“诸如但不限于”，且可与其互换使用。
可能以许多方式来实现本公开的方法和装置。例如，可通过软件、硬件、固件或者软件、硬件、固件的任何组合来实现本公开的方法和装置。用于所述方法的步骤的上述顺序仅是为了进行说明，本公开的方法的步骤不限于以上具体描述的顺序，除非以其它方式特别说明。此外，在一些实施例中，还可将本公开实施为记录在记录介质中的程序，这些程序包括用于实现根据本公开实施例的方法的机器可读指令。因而，本公开还覆盖存储用于执行根据本公开实施例的方法的程序的记录介质。
还需要指出的是，在本公开的装置、设备和方法中，各部件或各步骤是可以分解和/或重新组合的。这些分解和/或重新组合应视为本公开的等效方案。
提供所公开的方面的以上描述，以使本领域的任何技术人员能够做出或者使用本公开。对这些方面的各种修改等对于本领域技术人员而言，是非常显而易见的，并且在此定义的一般原理可以应用于其他方面，而不脱离本公开的范围。因此，本公开不意图被限制到在此示出的方面，而是按照与在此公开的原理和新颖的特征一致的最宽范围。
为了例示和描述的目的已经给出了以上描述。此外，此描述不意图将本公开的实施例限制到在此公开的形式中。尽管以上已经讨论了多个示例方面以及实施例，但是本领域技术人员将 认识到其某些变型、修改、改变、添加和子组合。

Claims (35)

  1. 一种用于实现三维空间场景互动的方法，包括：
    响应于检测到在三维空间场景中设置足迹信息的用户操作，确定所述用户在所述三维空间场景中的当前视角所对应的当前视图中的第一像素点；
    确定所述第一像素点对应的三维模型；
    确定所述用户的足迹信息在所述三维模型中的位置，其中所述足迹信息用于在所述三维空间场景被浏览时显示；以及
    在所述位置处设置所述用户的足迹信息。
  2. 根据权利要求1所述的方法，其中，所述足迹信息包括由以下各项所组成的组中的至少一项：
    文本、图片、音频、视频以及三维模型。
  3. 根据权利要求1或2所述的方法，其中，所述确定所述用户在所述三维空间场景中的当前视角所对应的当前视图中的第一像素点，包括：
    确定所述当前视图的中心像素点作为所述第一像素点。
  4. 根据权利要求3所述的方法，其中，所述确定所述第一像素点对应的三维模型，包括：
    确定针对所述第一像素点是否设置有三维模型；
    响应于针对所述第一像素点设置有三维模型的确定，将所述针对所述第一像素点设置的三维模型作为所述第一像素点对应的三维模型；以及
    响应于针对所述第一像素点未设置有三维模型的确定，将针对所述当前视图中的其他像素点设置的三维模型作为所述第一像素点对应的三维模型。
  5. 根据权利要求4所述的方法，其中，所述将针对所述当前视图中的其他像素点设置的三维模型作为所述第一像素点对应的三维模型，包括：
    以所述第一像素点为起点，根据预设检查规则，对所述三维空间场景中的当前视角所对应的当前视图中的其他像素点进行检查；
    响应于确定检查到设置有三维模型的像素点，将所述第一像素点更新为所述设置有三维模型的像素点；
    获得所述第一像素点对应的三维模型；和
    停止所述检查。
  6. 根据权利要求1或2所述的方法，其中，所述确定所述用户在所述三维空间场景中的当前视角所对应的当前视图中的第一像素点，包括：
    响应于所述用户在所述当前视图中设置所述足迹信息的目标位置的操作，确定所述足迹信息的目标位置对应的当前视图中的像素点，以作为所述第一像素点。
  7. 根据权利要求6所述的方法，其中，所述确定所述第一像素点对应的三维模型，包括：
    确定针对所述第一像素点是否设置有三维模型；
    响应于针对所述第一像素点设置有三维模型的确定，将所述针对所述第一像素点设置的三维模型作为所述第一像素点对应的三维模型；
    响应于针对所述第一像素点未设置有三维模型的确定，输出更新所述足迹信息的目标位置的提示信息；
    响应于针对更新后的所述足迹信息的目标位置对应的当前视图中的像素点设置有三维模型的确定，将该设置有三维模型的像素点作为第一像素点；以及
    获得所述第一像素点对应的三维模型。
  8. 根据权利要求5或7所述的方法，其中，所述确定所述用户的足迹信息在所述三维模型中的位置，包括：
    获取所述第一像素点在所述三维模型中的位置以作为所述用户的足迹信息在所述三维模型中的位置。
  9. 根据权利要求1至8中任一项所述的方法，还包括：
    对于浏览所述三维空间场景的任一浏览用户，确定该浏览用户在所述三维空间场景中的当前视角所对应的足迹区域；
    确定所述三维模型中的属于所述足迹区域的足迹信息；以及
    在该浏览用户在所述三维空间场景中的当前视角所对应的当前视图中，显示所述属于所述足迹区域的足迹信息。
  10. 根据权利要求9所述的方法，其中，所述确定该浏览用户在所述三维空间场景中的当前视角所对应的足迹区域，包括：
    确定该浏览用户在所述三维空间场景中的当前视角所对应的当前视图的中心像素点；以及
    以所述中心像素点为圆心，以预定长度为半径，确定所述当前视图中的足迹区域。
  11. 根据权利要求9或10所述的方法，其中，所述显示所述属于所述足迹区域的足迹信息，包括：
    响应于确定所述属于所述足迹区域的足迹信息具有不同位置的多个足迹信息，根据所述多个足迹信息分别在所述当前视图中的图像位置，在所述当前视图中显示所述多个足迹信息；以及
    响应于确定所述属于所述足迹区域的足迹信息包括相同位置的不同足迹信息，在所述当前视图中为所述不同足迹信息分配不同的图像位置，并根据所述分配的图像位置，在所述当前视图中显示所述不同足迹信息。
  12. 根据权利要求9至11中任一项所述的方法，还包括：
    确定所述三维模型中的不属于所述足迹区域或当前视图的至少一个足迹信息；以及
    以弹幕的形式，在该浏览用户在所述三维空间场景中的当前视角所对应的当前视图中，显示所述至少一个足迹信息。
  13. 一种基于三维模型的交互方法，包括：
    在呈现有用户界面的第一用户终端处：
    响应于检测到用户针对所述用户界面的目标交互操作，向为所述用户界面提供页面数据的服务器发送针对所述目标交互操作的交互请求，其中，所述用户界面用于呈现三维模型，所述三维模型与第二用户终端登录的用户账号预先建立关联关系；
    接收所述服务器从所述第二用户终端获取的流媒体视频；以及
    在所述用户界面上呈现所述流媒体视频和所述三维模型。
  14. 根据权利要求13所述的方法，其中，所述接收所述服务器从所述第二用户终端获取的流媒体视频，包括：
    响应于所述服务器接收到所述第二用户终端针对所述交互请求发送的交互确认信息，接收所述服务器从所述第二用户终端获取的流媒体视频。
  15. 根据权利要求13或14所述的方法，还包括：
    响应于所述第一用户终端的当前网速值小于或等于预设网速阈值，基于所述流媒体视频中的各帧语音，对目标用户图像进行调整，生成不同于所述流媒体视频的新视频，其中，所述新视频表征所述目标用户图像指示的用户执行所述各帧语音指示的动作；以及
    采用所述新视频替代所述流媒体视频进行呈现。
  16. 根据权利要求15所述的方法，还包括：
    基于所述流媒体视频中的图像，生成所述目标用户图像；或者
    将与所述用户账号相关联的用户图像确定为所述目标用户图像。
  17. 根据权利要求15或16所述的方法，还包括：
    响应于所述用户界面呈现有所述新视频，向所述服务器发送摄像头关闭确认信息，其中，所述摄像头关闭确认信息用于确定所述第二用户终端是否关闭摄像头。
  18. 根据权利要求13所述的方法，其中，所述接收所述服务器从所述第二用户终端获取 的流媒体视频，包括：
    将所述第一用户终端的当前网速值发送至所述服务器；以及
    接收所述服务器从所述第二用户终端获取并发送的流媒体视频，该流媒体视频具有与所述当前网速值相匹配的分辨率。
  19. 根据权利要求13-18之一所述的方法，还包括：
    接收所述服务器发送的模型调整信息，其中，所述模型调整信息指示使用所述第二用户终端的用户对呈现于所述第二用户终端的所述三维模型的调整操作，所述调整操作包括由以下各项所组成的组中的至少一项：缩放、旋转、移动和视点切换；以及
    按照所述模型调整信息指示的调整操作，对所述用户界面上呈现的所述三维模型进行相同调整操作。
  20. 根据权利要求13-19之一所述的方法，其中，所述三维模型包括多个子空间场景的三维子模型，所述多个子空间场景中的相应子空间场景与预先确定的关键词集合中的相应关键词相对应，所述方法还包括：
    对所述流媒体视频中的语音进行语音识别，得到语音识别结果；以及
    响应于所述语音识别结果包含所述关键词集合中的关键词的确定，在所述用户界面上呈现与所述语音识别结果包含的关键词相对应的所述多个子空间场景中的对应子空间场景的三维子模型。
  21. 根据权利要求1320之一所述的方法，还包括：
    获取用户针对所述流媒体视频的反馈信息；以及
    将所述反馈信息发送至所述服务器，其中，所述服务器用于将所述反馈信息与所述用户账号建立关联关系。
  22. 一种基于三维模型的交互方法，包括：
    在第二用户终端处：
    响应于接收到服务器发送的交互请求，获取流媒体视频，其中，所述交互请求指示第一用户终端检测到用户针对所述第一用户终端呈现的用户界面的目标交互操作，所述用户界面用于呈现三维模型，所述三维模型与所述第二用户终端登录的用户账号预先建立关联关系；以及
    向所述服务器发送所述流媒体视频，其中，所述服务器用于将所述流媒体视频发送至所述第一用户终端，以使所述第一用户终端在所述用户界面上呈现所述流媒体视频和所述三维模型。
  23. 根据权利要求22所述的方法，其中，所述响应于接收到服务器发送的交互请求，获取流媒体视频，包括：
    响应于接收到服务器发送的交互请求，确定是否检测到用户针对所述交互请求的确认操作；以及
    响应于检测到所述确认操作，获取流媒体视频。
  24. 根据权利要求22或23所述的方法，还包括：
    响应于所述第一用户终端的当前网速值小于或等于预设网速阈值，从所述服务器接收摄像头关闭确认信息；以及
    呈现所述摄像头关闭确认信息，其中，所述摄像头关闭确认信息用于确定所述第二用户终端是否关闭摄像头。
  25. 根据权利要求22-24之一所述的方法，还包括：
    响应于从所述服务器接收到用户对呈现于所述第一用户终端的所述三维模型的调整操作，按照所述模型调整信息指示的调整操作，对所述第二用户终端呈现的所述三维模型进行相同调整操作，其中，所述调整操作包括由以下各项所组成的组中的至少一项：缩放、旋转、移动和视点切换。
  26. 根据权利要求22-25之一所述的方法，还包括：
    响应于检测到用户对呈现于所述第二用户终端的所述三维模型的调整操作，向所述服务器发送指示所述调整操作的模型调整信息，以使所述服务器控制所述第一用户终端按照所述模型调整信息指示的调整操作，对所述用户界面上呈现的所述三维模型进行相同调整操作，其中，所述调整操作包括由以下各项所组成的组中的至少一项：缩放、旋转、移动和视点切换。
  27. 根据权利要求22-26之一所述的方法，其中，所述三维模型包括多个子空间场景的三维子模型，所述多个子空间场景中的相应子空间场景与预先确定的关键词集合中的相应关键词相对应，
    所述方法还包括：
    对所述第一用户终端获取的语音进行语音识别，得到语音识别结果；以及
    响应于所述语音识别结果包含所述关键词集合中的关键词的确定，在所述用户界面上呈现与所述语音识别结果包含的关键词相对应的所述多个子空间场景中的对应子空间场景的三维子模型。
  28. 根据权利要求22-27之一所述的方法，还包括：
    响应于接收到所述服务器发送的、使用所述第一用户终端的用户针对所述流媒体视频的反 馈信息，执行与所述反馈信息相匹配的操作。
  29. 一种用于实现三维空间场景互动的装置，包括：用于执行权利要求1至12中任一项所述的方法的装置。
  30. 一种基于三维模型的交互装置，设置于第一用户终端，其中，所述装置包括：用于执行权利要求13至21中任一项所述的方法的装置。
  31. 一种基于三维模型的交互装置，设置于第二用户终端，其中，所述装置包括：用于执行权利要求22-28中任一项所述的方法的装置。
  32. 一种基于三维模型的交互系统，包括：
    第一用户终端，用于呈现用户界面；
    第二用户终端；和
    服务器，所述服务器与所述第一用户终端和所述第二用户终端通信连接，
    其中，所述第一用户终端被配置成：响应于检测到用户针对所述用户界面的目标交互操作，向所述服务器发送针对所述目标交互操作的交互请求，其中，所述用户界面用于呈现三维模型，所述三维模型与第二用户终端登录的用户账号预先建立关联关系；
    其中，所述第二用户终端被配置成：获取流媒体视频；并且向所述服务器发送所述流媒体视频；
    其中，所述服务器被配置成：向所述第一用户终端发送所述流媒体视频；并且
    其中，所述第一用户终端被配置成：在所述用户界面上呈现所述流媒体视频和所述三维模型。
  33. 一种非暂态性计算机可读存储介质，所述存储介质存储有计算机程序，所述计算机程序当被计算机执行时使所述计算机实现上述权利要求1-28中任一项所述的方法。
  34. 一种电子设备，包括：
    处理器；以及
    用于存储处理器可执行指令的存储器，所述处理器可执行指令在被所述处理器执行时实现上述权利要求1-28中任一项所述的方法。
  35. 一种计算机程序产品，包括计算机程序，所述计算机程序当被计算机执行时使所述计算机实现上述权利要求1-28中任一项所述的方法。
PCT/CN2021/093628 2020-05-13 2021-05-13 用于实现三维空间场景互动的方法、装置和设备 WO2021228200A1 (zh)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
CN202010401813.7A CN111562845B (zh) 2020-05-13 2020-05-13 用于实现三维空间场景互动的方法、装置和设备
CN202010401813.7 2020-05-13
CN202010698810.4 2020-07-20
CN202010698810.4A CN111885398B (zh) 2020-07-20 2020-07-20 基于三维模型的交互方法、装置、系统、电子设备和存储介质

Publications (1)

Publication Number Publication Date
WO2021228200A1 true WO2021228200A1 (zh) 2021-11-18

Family

ID=78525899

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/093628 WO2021228200A1 (zh) 2020-05-13 2021-05-13 用于实现三维空间场景互动的方法、装置和设备

Country Status (1)

Country Link
WO (1) WO2021228200A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114241132A (zh) * 2021-12-16 2022-03-25 北京字跳网络技术有限公司 场景内容展示控制方法、装置、计算机设备及存储介质
CN115499641A (zh) * 2022-09-20 2022-12-20 北京三月雨文化传播有限责任公司 一种快速构建数字展览文件的方法及智能终端

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107710284A (zh) * 2015-06-30 2018-02-16 奇跃公司 用于在虚拟图像生成系统中更有效地显示文本的技术
CN108874471A (zh) * 2018-05-30 2018-11-23 链家网（北京）科技有限公司 一种房源的功能间附加元素添加方法及系统
CN110531847A (zh) * 2019-07-26 2019-12-03 中国人民解放军军事科学院国防科技创新研究院 一种基于增强现实的新型社交方法及系统
CN110891167A (zh) * 2019-11-30 2020-03-17 北京城市网邻信息技术有限公司 一种信息交互方法、第一终端和计算机可读存储介质
CN110944140A (zh) * 2019-11-30 2020-03-31 北京城市网邻信息技术有限公司 远程展示方法、远程展示系统、电子装置、存储介质
CN111047717A (zh) * 2019-12-24 2020-04-21 北京法之运科技有限公司 一种对三维模型进行文字标注的方法
CN111562845A (zh) * 2020-05-13 2020-08-21 贝壳技术有限公司 用于实现三维空间场景互动的方法、装置和设备
CN111885398A (zh) * 2020-07-20 2020-11-03 贝壳技术有限公司 基于三维模型的交互方法、装置和系统

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107710284A (zh) * 2015-06-30 2018-02-16 奇跃公司 用于在虚拟图像生成系统中更有效地显示文本的技术
CN108874471A (zh) * 2018-05-30 2018-11-23 链家网（北京）科技有限公司 一种房源的功能间附加元素添加方法及系统
CN110531847A (zh) * 2019-07-26 2019-12-03 中国人民解放军军事科学院国防科技创新研究院 一种基于增强现实的新型社交方法及系统
CN110891167A (zh) * 2019-11-30 2020-03-17 北京城市网邻信息技术有限公司 一种信息交互方法、第一终端和计算机可读存储介质
CN110944140A (zh) * 2019-11-30 2020-03-31 北京城市网邻信息技术有限公司 远程展示方法、远程展示系统、电子装置、存储介质
CN111047717A (zh) * 2019-12-24 2020-04-21 北京法之运科技有限公司 一种对三维模型进行文字标注的方法
CN111562845A (zh) * 2020-05-13 2020-08-21 贝壳技术有限公司 用于实现三维空间场景互动的方法、装置和设备
CN111885398A (zh) * 2020-07-20 2020-11-03 贝壳技术有限公司 基于三维模型的交互方法、装置和系统

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114241132A (zh) * 2021-12-16 2022-03-25 北京字跳网络技术有限公司 场景内容展示控制方法、装置、计算机设备及存储介质
CN114241132B (zh) * 2021-12-16 2023-07-21 北京字跳网络技术有限公司 场景内容展示控制方法、装置、计算机设备及存储介质
CN115499641A (zh) * 2022-09-20 2022-12-20 北京三月雨文化传播有限责任公司 一种快速构建数字展览文件的方法及智能终端
CN115499641B (zh) * 2022-09-20 2023-09-12 广东鸿威国际会展集团有限公司 一种快速构建数字展览文件的方法及智能终端

Similar Documents

Publication Publication Date Title
US11899900B2 (en) Augmented reality computing environments—immersive media browser
WO2021109652A1 (zh) 文字虚拟礼物的赠送方法、装置、设备及存储介质
US10134364B2 (en) Prioritized display of visual content in computer presentations
CN111178191B (zh) 信息播放方法、装置、计算机可读存储介质及电子设备
JP5901151B2 (ja) 仮想環境におけるオブジェクトの選択方法
US8117281B2 (en) Using internet content as a means to establish live social networks by linking internet users to each other who are simultaneously engaged in the same and/or similar content
WO2022116751A1 (zh) 交互方法、装置、终端、服务器和存储介质
KR20220115824A (ko) 콘텐츠를 공간 3d 환경에 매칭
WO2018152455A1 (en) System and method for creating a collaborative virtual session
US20180160194A1 (en) Methods, systems, and media for enhancing two-dimensional video content items with spherical video content
US11908056B2 (en) Sentiment-based interactive avatar system for sign language
CN112596694B (zh) 一种房源信息的处理方法和装置
KR20190047144A (ko) 인터액티브 비디오 생성
WO2021228200A1 (zh) 用于实现三维空间场景互动的方法、装置和设备
US20220319063A1 (en) Method and apparatus for video conferencing
US20230409632A1 (en) Systems and methods for using conjunctions in a voice input to cause a search application to wait for additional inputs
US20240283994A1 (en) Display apparatus and method for person recognition and presentation
TW201403501A (zh) 虛擬社群建立系統及方法
CN112051956A (zh) 一种房源的交互方法和装置
CN111885398A (zh) 基于三维模型的交互方法、装置和系统
CN111562845B (zh) 用于实现三维空间场景互动的方法、装置和设备
CN116017082A (zh) 一种信息处理方法和电子设备
US20240185530A1 (en) Information interaction method, computer-readable storage medium and communication terminal
CN116762333A (zh) 将电话会议参与者的图像与共享文档叠加

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21805221

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21805221

Country of ref document: EP

Kind code of ref document: A1