CN108664300B

CN108664300B - Application interface display method and device in picture-in-picture mode

Info

Publication number: CN108664300B
Application number: CN201810290762.8A
Authority: CN
Inventors: 孙哲
Original assignee: Hisense Mobile Communications Technology Co Ltd
Current assignee: Hisense Mobile Communications Technology Co Ltd
Priority date: 2018-04-03
Filing date: 2018-04-03
Publication date: 2021-06-25
Anticipated expiration: 2038-04-03
Also published as: CN108664300A

Abstract

The invention discloses an application interface display method and device in a picture-in-picture mode, and belongs to the technical field of mobile terminals. The method comprises the following steps: when the application to be started is detected to be a second application which needs to be displayed by taking a picture-in-picture mode as a foreground application, the display direction of the application interface of the first application which is currently displayed is obtained, the interface display direction appointed in the application package of the second application is replaced by the display direction of the application interface of the first application, and the application interface of the second application is displayed on the upper layer of the application interface of the first application based on the interface display direction appointed in the application package of the second application after replacement. That is, in the embodiment of the present invention, the display direction of the interface of the second application is replaced with the display direction of the application interface of the first application, so that although the second application is a foreground application, the display direction of the application interface of the first application does not change along with the display direction of the interface of the second application, and the display direction of the application interface of the first application is more suitable for the user requirements.

Description

Application interface display method and device in picture-in-picture mode

Technical Field

The invention relates to the technical field of mobile terminals, in particular to a method and a device for displaying an application interface in a picture-in-picture mode.

Background

Currently, the picture-in-picture mode is increasingly applied on mobile terminals. The picture-in-picture mode is to display an application interface of a second application in a small area on an upper layer of the application interface while displaying the application interface of a first application. In order to solve the problem, when the second application is started, the mobile terminal may forcibly start the second application as the foreground application and forcibly convert the first application into the background application.

In the related art, according to the display rule of the android system, the display directions of the application interfaces of all background applications need to be consistent with the display direction of the application interface of the foreground application. However, although the first application is converted for the background application, the application interface of the first application is often the focus interface that the user focuses most, that is, the display direction of the application interface of the first application is the display direction that the user wants most. Therefore, in order to improve user experience, it is urgently needed to provide an application interface display method after the second application is started as the foreground application, so as to ensure that the display direction of the application interface of the first application as the background application does not change along with the change of the display direction of the application interface of the foreground application.

Disclosure of Invention

In order to solve the problem of complex operation when switching applications in the prior art, embodiments of the present invention provide a method and an apparatus for displaying an application interface in a picture-in-picture mode. The technical scheme is as follows:

in a first aspect, a method for displaying an application interface in a picture-in-picture mode is provided, the method including:

when detecting that the application to be started is a second application which needs to be displayed by taking a picture-in-picture mode as a foreground application, acquiring the display direction of an application interface of a first application which is currently displayed;

replacing the interface display direction specified in the application package of the second application with the display direction of the application interface of the first application;

and displaying the application interface of the second application on the upper layer of the application interface of the first application based on the interface display direction appointed in the replaced application package of the second application.

Optionally, before the obtaining the display direction of the application interface of the first application currently being displayed when it is detected that the application to be started is the second application that needs to be displayed in the picture-in-picture mode as the foreground application, the method further includes:

judging whether an application interface information packet of the application to be started comprises a first mark, wherein the first mark is used for indicating the application to be started to be displayed as a foreground application in a picture-in-picture mode;

and if the application interface information packet of the application to be started comprises the first mark, determining that the application to be started is a second application which needs to be displayed by taking a picture-in-picture mode as a foreground application.

Optionally, before replacing the interface display direction specified in the application package of the second application with the display direction of the application interface of the first application, the method further includes:

acquiring an interface display direction appointed in an application package of the second application;

judging whether the display direction of the interface is the same as the display direction of the application interface of the first application or not;

and if the interface display direction is different from the display direction of the application interface of the first application, executing a step of replacing the interface display direction specified in the application package of the second application with the display direction of the application interface of the first application.

Optionally, the displaying the application interface of the second application on the upper layer of the application interface of the first application based on the interface display direction specified in the replaced application package of the second application includes:

acquiring the width and height of an application interface of the second application when the second application is displayed according to the interface display direction appointed in the replaced application package of the second application;

and adjusting the width and the height of the application interface of the second application, and displaying the application interface of the second application on the upper layer of the application interface of the first application based on the adjusted width and the adjusted height of the application interface of the second application.

Optionally, the adjusting the width and the height of the application interface of the second application, and displaying the application interface of the second application on an upper layer of the application interface of the first application based on the adjusted width and the adjusted height of the application interface of the second application includes:

taking the width of the application interface of the second application as the height of the adjusted application interface of the second application, and taking the height of the application interface of the second application as the width of the adjusted application interface of the second application;

displaying the application interface of the second application on the upper layer of the application interface of the first application based on the adjusted width and height of the application interface of the second application.

In a second aspect, there is provided an application interface display apparatus in a picture-in-picture mode, the apparatus comprising:

the first obtaining module is used for obtaining the display direction of the application interface of the first application which is currently displayed when the application to be started is detected to be a second application which needs to be displayed by taking a picture-in-picture mode as a foreground application;

a replacing module, configured to replace a display direction of an interface specified in the application package of the second application with a display direction of an application interface of the first application;

and the display module is used for displaying the application interface of the second application on the upper layer of the application interface of the first application based on the interface display direction appointed in the replaced application package of the second application.

Optionally, the apparatus further comprises:

the first judging module is used for judging whether an application interface information packet of the application to be started comprises a first mark, and the first mark is used for indicating the application to be started to be displayed by taking a picture-in-picture mode as a foreground application;

and the determining module is used for determining that the application to be started is a second application which needs to be displayed by taking a picture-in-picture mode as a foreground application if the application interface information packet of the application to be started comprises the first mark.

Optionally, the apparatus further comprises:

the second acquisition module is used for acquiring the interface display direction appointed in the application package of the second application;

the second judgment module is used for judging whether the display direction of the interface is the same as the display direction of the application interface of the first application or not;

and the triggering module is used for triggering the replacing module to replace the interface display direction appointed in the application package of the second application with the display direction of the application interface of the first application if the interface display direction is different from the display direction of the application interface of the first application.

Optionally, the display module comprises:

the obtaining submodule is used for obtaining the width and the height of the application interface of the second application when the second application is displayed according to the interface display direction appointed in the replaced application package of the second application;

and the adjusting submodule is used for adjusting the width and the height of the application interface of the second application, and displaying the application interface of the second application on the upper layer of the application interface of the first application based on the adjusted width and the adjusted height of the application interface of the second application.

Optionally, the adjusting submodule is specifically configured to:

In a third aspect, a mobile terminal is provided, which includes:

a processor;

a memory for storing the processor-executable instructions;

wherein the processor is configured to perform the steps of any one of the methods of the first aspect.

In a fourth aspect, a computer-readable storage medium is provided, having instructions stored thereon, which when executed by a processor, implement the steps of any of the methods of the first aspect described above.

In a fifth aspect, there is provided a computer program product containing instructions which, when run on a computer, cause the computer to perform the application interface display method of the first aspect.

The technical scheme provided by the embodiment of the invention has the following beneficial effects: when the application to be started is detected to be a second application which needs to be displayed by taking a picture-in-picture mode as a foreground application, the display direction of the application interface of the first application which is currently displayed is obtained, the interface display direction appointed in the application package of the second application is replaced by the display direction of the application interface of the first application, and the application interface of the second application is displayed on the upper layer of the application interface of the first application based on the interface display direction appointed in the application package of the second application after replacement. Therefore, in the embodiment of the present invention, when the second application displayed in the picture-in-picture mode as the foreground application is started, the display direction of the application interface of the second application specified in the application package of the second application may be replaced by the display direction of the application interface of the first application currently being displayed, so that although the second application is the foreground application, the display direction of the application interface of the first application does not change along with the display direction of the application interface of the second application, that is, in the process of displaying the application interface of the second application in the picture-in-picture mode, while it is ensured that the second application can be edited and operated by the user as the foreground application, the display direction of the application interface of the first application as the user focus may be kept unchanged, and the display direction is more suitable for the user.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

FIG. 1 is a flowchart of an application interface display method in a picture-in-picture mode according to an embodiment of the present invention;

FIG. 2A is a flowchart of a method for displaying an application interface in a picture-in-picture mode according to an embodiment of the present invention;

FIG. 2B is a diagram illustrating a second application displayed in a picture-in-picture mode in an application interface of a first application using methods of the related art;

fig. 2C is a schematic diagram illustrating an application interface of a second application being displayed in an alternative interface display direction before a picture-in-picture layout is not adjusted according to an embodiment of the present invention;

fig. 2D is a schematic diagram illustrating an application interface of a second application being adjusted and displayed according to a picture-in-picture layout after the picture-in-picture layout is adjusted according to an embodiment of the present invention;

FIG. 3A is a block diagram of an application interface display device in a picture-in-picture mode according to an embodiment of the present invention;

FIG. 3B is a block diagram of an application interface display device in a picture-in-picture mode according to an embodiment of the present invention;

FIG. 3C is a block diagram of a display module according to an embodiment of the invention;

fig. 4 is a schematic structural diagram of a mobile terminal according to an embodiment of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the present invention more apparent, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

Before explaining the embodiments of the present invention in detail, an application scenario related to the embodiments of the present invention will be described.

Currently, the picture-in-picture mode is increasingly applied on various mobile terminals. The pip mode is actually a content presentation mode, and a relatively common mode includes a video pip mode, which means that another video picture is displayed in a small area region on a full-frequency broadcast picture of a certain video. With the continuous research on the picture-in-picture mode, the picture-in-picture mode has not been limited to the presentation of video contents, but can be applied to the presentation of application interfaces of other applications installed in the mobile terminal. For example, assuming that a current user is browsing a web page through a browser installed on a mobile terminal, at this time, a social application installed in the mobile terminal receives a friend message, and the user may not want to exit the currently browsing web page, in which case, the social application may be displayed in the currently browsing web page in a picture-in-picture mode.

It should be noted that, in the above-mentioned example, if the application interface of the social application is displayed in the web page of the browser in the picture-in-picture mode, the browser is actually a foreground application, and the social application is actually a background application, in this case, the application interface of the social application cannot be edited or operated, that is, even if the application interface of the social application is displayed in the web page in the picture-in-picture mode, the user cannot perform operations such as message reply on the application interface of the social application, which is far from meeting the user's requirement. To solve this problem, the mobile terminal may enforce the browser as a foreground application and enforce the browser as a background application when starting the social application. However, if the social application is forcibly converted into the foreground application, the display direction of the web page of the browser will also change along with the display direction of the application interface of the social application, however, actually, the web page of the browser is the focus page of the user, that is, when the application interface of the social application is displayed in the web page in the picture-in-picture mode, the user often does not want any change of the web page of the browser. Based on this, the embodiment of the invention provides an application interface display method in a picture-in-picture mode, which can ensure that a user can edit and operate a social application, and meanwhile, the display direction of a webpage serving as a user focus is kept unchanged, so that the application interface display method is more suitable for user requirements.

It should be noted that, in the application scenario, only the browser and the social application are taken as examples for description, in an actual application, the browser and the social application may be any other application installed in the mobile terminal, and the embodiment of the present invention is not limited to this specifically.

Next, a method for displaying an application interface in a picture-in-picture mode according to an embodiment of the present invention will be described.

Fig. 1 is a flowchart of an application interface display method in a picture-in-picture mode according to an embodiment of the present invention, which may be used in a mobile terminal, as shown in fig. 1, and the method includes the following steps:

step 101: when the application to be started is detected to be a second application which needs to be displayed in a picture-in-picture mode as a foreground application, the display direction of the application interface of the first application which is currently displayed is obtained.

The first application and the second application may be any applications installed in the mobile terminal, and the application interface of the first application is currently being displayed on the mobile terminal. In addition, the display orientation of the application interface may include landscape display and portrait display.

Step 102: and replacing the display direction of the interface designated in the application package of the second application with the display direction of the application interface of the first application.

The interface display direction specified in the application package of the second application refers to a display direction supported by the application interface of the second application. That is, the display direction of the interface may be horizontal screen, vertical screen, or display direction information indicating that both horizontal and vertical screens support.

Step 103: and displaying the application interface of the second application on the upper layer of the application interface of the first application based on the interface display direction specified in the replaced application package of the second application.

In the embodiment of the invention, when detecting that the application to be started is a second application which needs to be displayed by taking a picture-in-picture mode as a foreground application, the display direction of the application interface of the first application which is currently displayed is obtained, the interface display direction specified in the application package of the second application is replaced by the display direction of the application interface of the first application, and the application interface of the second application is displayed on the upper layer of the application interface of the first application based on the interface display direction specified in the application package of the second application after replacement. Therefore, in the embodiment of the present invention, when the second application displayed in the picture-in-picture mode as the foreground application is started, the display direction of the application interface of the second application specified in the application package of the second application may be replaced by the display direction of the application interface of the first application currently being displayed, so that although the second application is the foreground application, the display direction of the application interface of the first application does not change along with the display direction of the application interface of the second application, that is, in the process of displaying the application interface of the second application in the picture-in-picture mode, while it is ensured that the second application can be edited and operated by the user as the foreground application, the display direction of the application interface of the first application as the user focus may be kept unchanged, and the display direction is more suitable for the user.

Fig. 2A is a flowchart of an application interface display method in a picture-in-picture mode according to an embodiment of the present invention, where the method may be used in a mobile terminal, as shown in fig. 2A, and the method includes the following steps:

step 201: and judging whether the application interface information packet of the application to be started comprises a first mark.

In the embodiment of the present invention, when the mobile terminal is currently displaying the application interface of the first application, if the user wants to display an application in the picture-in-picture mode, the user may perform a selection operation for the application to trigger an instruction to start the application in the picture-in-picture mode. When the mobile terminal receives the instruction, the application may be started through startactivity (intent). The intent is an application interface information packet which encapsulates information of an application interface of the application. In the related art, the content in the application interface information packet is obtained by querying from a system of the mobile terminal or provided by a third-party application, and in the embodiment of the present invention, a mark for indicating that the application needs to be started in a picture-in-picture mode as a foreground application may be additionally added in the application interface information packet. That is, when receiving an instruction to start the application in the picture-in-picture mode, the mobile terminal may mark the starting mode of the application at the starting entrance. Therefore, in the process that the mobile terminal starts the application as the foreground application, all the tag information in the event can be acquired through the getFlags interface acquired by the tag information of the application interface information packet event, and whether the acquired all the tag information comprises the first tag or not is judged. The first mark is mark information which is additionally added in the intent at the entrance for starting the application and is used for indicating that the application needs to be displayed as a foreground application in a picture-in-picture mode.

Step 202: and if the application interface information packet of the application to be started comprises the first mark, determining that the application to be started is a second application which needs to be displayed by taking a picture-in-picture mode as a foreground application.

When the mobile terminal determines that the acquired mark information includes the first mark, the application to be started is the second application which needs to be displayed as a foreground application in a picture-in-picture mode.

Optionally, in this embodiment of the present invention, if the obtained tag information does not include the first tag, it indicates that the application to be started is not an application displayed in a picture-in-picture mode, and at this time, the mobile terminal may process the application to be started according to a method for starting the application and displaying an application interface in the related art, which is not described herein again in this embodiment of the present invention.

Step 203: the display direction of the application interface of the first application currently being displayed is obtained.

When the mobile terminal detects that the application to be started is a second application which needs to be displayed by taking a picture-in-picture mode as a foreground application, the mobile terminal can acquire the display direction of the application interface of the first application which is currently displayed, so that the display direction of the application interface of the second application can be adjusted according to the display direction of the application interface of the first application. The first application refers to an application currently displaying an application interface, and the second application displayed in the picture-in-picture mode refers to the application interface of the first application displayed in the picture-in-picture mode.

Specifically, in the embodiment of the present invention, before the second application is not started, the first application is a foreground application, and when the second application is started, the second application becomes a foreground application. At this time, the first application in the application stack queue is the second application, and the second application arranged behind the second application is the first application, so that the mobile terminal can acquire the second application, that is, the application information of the first application, from the application stack queue and acquire the display direction of the application interface of the first application from the acquired application information.

The display direction of the application interface of the first application refers to a display direction of the application interface currently being displayed by the first application, and the display direction may be a horizontal screen display or a vertical screen display.

Step 204: and replacing the display direction of the interface designated in the application package of the second application with the display direction of the application interface of the first application.

In an embodiment of the present invention, in a possible implementation manner, after the application interface display direction of the first application is obtained, the mobile terminal may first obtain an interface display direction specified in an application package of the second application, so as to determine whether to adjust a width and a height of an application interface of the second application by comparing the interface display direction with a display direction of an application interface of the first application in the following step. After that, the mobile terminal may directly replace the interface display direction specified in the application package of the second application with the display direction of the application interface of the first application.

The interface display direction specified in the application package of the second application refers to a display direction supported by the second application specified in the application package of the second application. For one application, an android manifest of the application contains a parameter screenorganization for describing a display direction supported by the application, if the screenorganization is Landscape, the second application is only supported by horizontal screen display, if the screenorganization is portal, the second application is only supported by vertical screen display, and if the screenorganization does not define a value, the second application is not only supported by horizontal screen display and vertical screen display. Based on this, after the mobile terminal acquires the display direction of the application interface of the first application, the mobile terminal can read the specified interface display direction from the android manifest.

For example, assuming that a parameter for describing a display direction in the android manifest.xml file is screen orientation (port), and a display direction of the acquired application interface of the first application is a horizontal screen (Landscape), the mobile terminal may directly replace a value of screen orientation in the android manifest.xml file with Landscape, that is, replace port with Landscape.

Alternatively, in another possible implementation manner, the interface display direction specified in the application package of the second application may be the same as the display direction of the application interface of the first application, in which case, the mobile terminal may not need to replace the specified interface display direction any more. Based on this, before replacing the interface display direction specified in the application package of the second application with the display direction of the application interface of the first application, the mobile terminal may first obtain the interface display direction specified in the application package of the second application, and determine whether the interface display direction is the same as the display direction of the application interface of the first application, if the interface display direction is different from the display direction of the application interface of the first application, the mobile terminal may perform the step of replacing the interface display direction specified in the application package of the second application with the display direction of the application interface of the first application, and if the interface display direction is the same as the display direction of the application interface of the first application, the mobile terminal may not replace the interface display direction specified in the application package of the second application.

It should be noted that, as can be seen from the foregoing description, the mobile terminal may read a parameter, screenorder, for indicating the interface display direction from an android manifest. Wherein, the value of ScreenOrientation may be Portrait, Landscape or undefined value. That is, the display direction of the interface can be horizontal screen display, vertical screen display or both horizontal and vertical screen display. The mobile terminal may compare a display orientation of an application interface of the first application with the interface display orientation. If the display direction of the application interface of the first application is horizontal screen display, and the display direction of the interface of the second application is vertical screen display or both horizontal and vertical screen display, it can be determined that the display direction of the application interface of the first application is different from the display direction of the interface. If the display direction of the application interface of the first application is vertical screen display, and the display direction of the interface of the second application is horizontal screen display or both horizontal and vertical screen display, it can be determined that the display direction of the application interface of the first application is different from the display direction of the interface.

When the display direction of the application interface of the first application is different from the display direction of the interface after comparison, the mobile terminal can replace the value of the screenorganization in the android manifest. If the two are the same, the mobile terminal may not modify the value of the screencreation in the android manifest.

It should be noted that, in the embodiment of the present invention, modifying and replacing a value of screenchange in an android manifest of an xml file of a second application is actually performed in a screen turn control code of a system of a mobile terminal, that is, before the screen turn control code is sent to the second application to start an application interface, so that since the code is directly modified, it is actually equivalent to forcibly converting a display direction of the application interface of the second application before the second application starts the application interface.

Step 205: and displaying the application interface of the second application on the upper layer of the application interface of the first application based on the interface display direction specified in the replaced application package of the second application.

After replacing the interface display direction specified in the application package of the second application with the display direction of the application interface of the first application, the mobile terminal may display the application interface of the second application on an upper layer of the application interface of the first application according to the interface display direction specified in the application package of the second application after replacement.

Based on the description in step 204, it can be known that the mobile terminal may replace the interface display direction specified in the application package of the second application through two different policies, and for different replacement modes, there may also be two different implementation modes in this step.

The first mode is as follows: if the mobile terminal acquires the display direction of the application interface of the first application and the display direction of the interface appointed in the application package of the second application, the display direction of the application interface of the first application is directly replaced by the display direction of the interface, after the replacement, the mobile terminal may compare the interface display direction specified by the application package of the second application after the replacement with the interface display direction specified in the second application package when the replacement is not performed, and if the two are the same, it means that the display direction of the interface designated in the application package of the second application when not replaced is consistent with the display direction of the application interface of the first application, that is, the application interface of the second application itself supports the display direction of the application interface of the first application, and at this time, the mobile terminal can directly display the application interface of the second application according to the interface display direction specified in the application package of the replaced second application.

Of course, if the two are different after the comparison, it is indicated that the display direction of the interface specified in the application package of the second application when not replaced is inconsistent with the display direction of the application interface of the first application, that is, the second application itself does not support the display direction of the application interface of the first application. In this case, since the interface display direction specified in the application package of the second application is replaced with the display direction of the application interface of the first application, the second application displays the application interface according to the replaced interface display direction, and the application interface of the second application does not support the replaced interface display direction, so that if the application interface of the second application is directly displayed, the display of the application interface of the second application is abnormal. In order to solve the problem, the mobile terminal may adjust the width and the height of the application interface of the second application, and then display the application interface of the second application according to the adjusted width and the adjusted height of the application interface of the second application, so that the display of the application interface of the second application can be more reasonable.

Specifically, when the application interface of the second application is adjusted, the mobile terminal may obtain the width and height of the application interface of the second application when displaying according to the interface display direction specified in the application package of the second application after replacement; and adjusting the width and the height of the application interface of the second application, and displaying the application interface of the second application on the upper layer of the application interface of the first application based on the adjusted width and the adjusted height of the application interface of the second application.

It should be noted that the width and height of the application interface of the second application determine the size of the area occupied by the application interface of the second application when displayed on the upper layer of the application interface of the first application. The values of the width and the height of the application interface of the second application displayed as the picture-in-picture may be two values that are set in advance, but which of the two values is the width and which is the height may be determined according to the interface display direction specified in the application package of the second application after the replacement. For example, if the interface display direction specified in the application package of the replaced second application (i.e., the display direction of the application interface of the first application) is a landscape screen, the width of the application interface of the second application is greater than the height, and the larger value of the two values is the width, and the smaller value is the height. If the interface display direction (i.e. the display direction of the application interface of the first application) specified in the application package of the replaced second application is vertical, the width of the application interface of the second application is smaller than the height, and in this case, the smaller value of the two values is the width, and the larger value is the height. Based on this, the mobile terminal can acquire the width and height of the application interface of the second application when displayed in the interface display direction specified in the application package of the second application after replacement.

After the width and the height of the application interface of the second application are obtained, since the interface display direction specified in the application package of the replaced second application is not the interface display direction supported by the second application, the mobile terminal may adjust the application interface of the second application, and display the application interface of the second application on the upper layer of the application interface of the first application based on the adjusted width and the adjusted height of the application interface of the second application.

The mobile terminal may use the width of the application interface of the second application as the height of the adjusted application interface of the second application, and use the height of the application interface of the second application as the width of the adjusted application interface of the second application; and displaying the application interface of the second application on the upper layer of the application interface of the first application based on the adjusted width and height of the application interface of the second application.

In addition, since the interface display direction specified in the application package of the replaced second application is not the interface display direction supported by the second application itself, the width and height of the application interface of the second application when displayed in the interface display direction specified in the application package of the replaced second application are different from the width and height of the application interface of the second application when displayed in the interface display direction supported by the second application itself. For example, assuming that the interface display direction specified in the application package of the replaced second application is a horizontal screen and the interface display direction supported by the second application itself is a vertical screen, when the second application is displayed according to the horizontal screen, the width of the application interface of the second application is greater than the height, and actually, when the second application is displayed according to the vertical screen supported by the second application, the width of the application interface of the second application is smaller than the height, so that the mobile terminal can change the width and the height of the application interface of the second application when adjusting the application interface of the second application. And then, the mobile terminal can draw the application interface of the second application according to the adjusted ratio of the width to the height of the application interface of the second application, and display the drawn application interface of the second application on the upper layer of the application interface of the first application.

It should be noted that the above is only an optional method for adjusting the pip layout provided in the embodiment of the present invention, and in practical applications, the mobile terminal may further calculate the width and height of the application interface of the second application according to a ratio between the width and height of the application interface of the first application, which is not described herein again in the embodiment of the present invention.

The second mode is as follows: if the mobile terminal determines whether to replace the interface display direction by judging the display direction of the application interface of the first application and the interface display direction after acquiring the display direction of the application interface of the first application and the interface display direction specified in the application package of the second application, because the replacement is performed under the condition that the display direction of the application interface of the first application and the interface display direction are different, it can be determined that the interface display direction after the replacement is different from the interface display direction before the replacement, that is, the second application does not support the interface display direction after the replacement. On the basis, in the implementation mode, whether the display direction of the interface after replacement is the same as the display direction of the interface before replacement does not need to be judged again, but the width and the height of the application interface of the second application can be directly adjusted according to the display direction of the interface after replacement, so that the application interface of the second application is displayed on the upper layer of the application interface of the first application according to the width and the height of the application interface of the second application after adjustment.

The specific implementation manner of the mobile terminal adjusting the width and the height of the application interface of the second application and displaying the application interface of the second application on the upper layer of the application interface of the first application according to the adjusted width and the adjusted height of the application interface of the second application may refer to the related description in the first manner, and the embodiment of the present invention is not described herein again.

In the embodiment of the present invention, when the application to be started is a second application that needs to be displayed in a picture-in-picture mode as a foreground application, a display direction of an application interface of a first application that is currently being displayed is obtained, a display direction of an interface specified in an application package of the second application is replaced with the display direction of the application interface of the first application, and the application interface of the second application is displayed on an upper layer of the application interface of the first application based on the interface display direction specified in the application package of the second application after replacement. Therefore, in the embodiment of the present invention, when a second application that needs to be displayed in a picture-in-picture mode as a foreground application is started, the display direction of the application interface of the second application specified in the application package of the second application may be replaced by the display direction of the application interface of the first application that is currently being displayed, so that although the started second application is the foreground application, the display direction of the application interface of the first application does not change along with the display direction of the application interface of the second application, that is, in the process of displaying the application interface of the second application in the picture-in-picture mode, while it is ensured that the second application can be edited and operated by a user as the foreground application, the display direction of the application interface of the first application that is the focus of the user may remain unchanged, and further meets the user requirements.

The following embodiments of the present invention will be described in brief and schematically with reference to fig. 2B-2D by the following examples.

The display direction of the application interface of the first application is assumed to be a landscape screen, and the display direction of the interface designated in the application package of the second application is a portrait screen, that is, the second application only supports portrait screen display. If the display rule in the related art is adopted to display the application interface of the second application in the application interface of the first application in the picture-in-picture mode, because the second application is foreground application, the display direction of the application interface of the first application changes along with the display direction of the interface of the second application, that is, as shown in fig. 2B, the application interface of the first application is converted into vertical screen display, and the application interface of the second application is displayed on the upper layer of the application interface of the first application in the vertical screen mode.

However, if the method provided by the embodiment of the present invention is used for displaying, after the interface display direction specified in the application package of the second application is replaced with the display direction of the application interface of the first application, the application interface of the first application will keep the landscape display unchanged, and without adjusting the width and height of the application interface of the second application, the second application will be displayed on the upper layer of the application interface of the first application as the landscape screen is as shown in fig. 2C.

Further, as can be seen from the interface display direction before replacement, the second application only supports vertical screen display and does not support horizontal screen display, so that, if the display is performed as shown in fig. 2C, the application interface of the second application is not adapted, is not suitable for user operation, and may be abnormal, therefore, the mobile terminal may change the width and height of the application interface of the second application, and then draw the application interface of the second application according to the changed width-height ratio, so as to display the application interface of the second application on the upper layer of the application interface of the first application, as shown in fig. 2D. At this time, although the second application is a foreground application and the first application is a background application, the display direction of the application interface of the first application is not changed along with the display direction of the interface specified in the application package of the second application, and the display direction of the application interface of the second application can be better adapted to the display direction of the application interface of the first application by adjusting the layout of the application interface of the second application, so that the user experience is greatly improved, and the user requirements are met.

Next, the application interface display device in the picture-in-picture mode according to the embodiment of the present invention will be described.

Fig. 3A is a block diagram of an application interface display apparatus 300 according to an embodiment of the present invention, and referring to fig. 3A, the apparatus 300 includes:

a first obtaining module 301, configured to obtain, when it is detected that an application to be started is a second application that needs to be displayed in a picture-in-picture mode as a foreground application, a display direction of an application interface of a first application that is currently being displayed;

a replacing module 302, configured to replace a display direction of an interface specified in an application package of a second application with a display direction of an application interface of a first application;

a display module 303, configured to display the application interface of the second application on an upper layer of the application interface of the first application based on the interface display direction specified in the replaced application package of the second application.

Optionally, the apparatus 300 further comprises:

the first judgment module is used for judging whether an application interface information packet of the application to be started comprises a first mark, and the first mark is used for indicating the application to be started to be displayed by taking a picture-in-picture mode as foreground application;

Optionally, referring to fig. 3B, the apparatus 300 further comprises:

a second obtaining module 304, configured to obtain an interface display direction specified in an application package of a second application;

a second judging module 305, configured to judge whether the display direction of the interface is the same as the display direction of the application interface of the first application;

the triggering module 306 is configured to, if the interface display direction is different from the display direction of the application interface of the first application, trigger the replacing module to replace the interface display direction specified in the application package of the second application with the display direction of the application interface of the first application.

Alternatively, referring to fig. 3C, the display module 303 includes:

the obtaining submodule 3031 is configured to obtain a width and a height of an application interface of the second application when the second application is displayed according to an interface display direction specified in the replaced application package of the second application;

and the adjusting submodule 3032 is configured to adjust the width and the height of the application interface of the second application, and display the application interface of the second application on the upper layer of the application interface of the first application based on the adjusted width and the adjusted height of the application interface of the second application.

Optionally, the adjusting submodule 3032 is specifically configured to:

and displaying the application interface of the second application on the upper layer of the application interface of the first application based on the adjusted width and height of the application interface of the second application.

In the embodiment of the invention, when detecting that the application to be started is a second application which needs to be displayed by taking a picture-in-picture mode as a foreground application, the display direction of the application interface of the first application which is currently displayed is obtained, the interface display direction specified in the application package of the second application is replaced by the display direction of the application interface of the first application, and the application interface of the second application is displayed on the upper layer of the application interface of the first application based on the interface display direction specified in the application package of the second application after replacement. Therefore, in the embodiment of the present invention, when the second application displayed in the picture-in-picture mode as the foreground application is started, the display direction of the application interface of the second application specified in the application package of the second application may be replaced by the display direction of the application interface of the first application currently being displayed, so that although the started second application is the foreground application, the display direction of the application interface of the first application does not change along with the display direction of the application interface of the second application, that is, in the process of displaying the application interface of the second application in the picture-in-picture mode, while it is ensured that the second application can be edited and operated by the user as the foreground application, the display direction of the application interface of the first application as the focus of the user may be kept unchanged, and the display direction of the application interface of the first application as the focus of the user may better meet the user requirements.

It should be noted that: in the application interface display device under the pd mode provided in the above embodiment, only the division of the above functional modules is used for illustration when displaying the application interface, and in practical applications, the above function distribution can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the above described functions. In addition, the application interface display device in the picture-in-picture mode provided by the above embodiment and the application interface display method in the picture-in-picture mode belong to the same concept, and the specific implementation process thereof is detailed in the method embodiment and is not described herein again.

Fig. 4 shows a block diagram of a mobile terminal 400 according to an exemplary embodiment of the present invention. Among them, the mobile terminal 400 may be: industrial computers, industrial personal computers, notebook computers, desktop computers, smart phones or tablet computers, and the like. The mobile terminal 400 may also be referred to by other names such as user equipment, portable terminal, laptop terminal, desktop terminal, and the like.

Generally, the mobile terminal 400 includes: a processor 401 and a memory 402.

Processor 401 may include one or more processing cores, such as a 4-core processor, an 8-core processor, or the like. The processor 401 may be implemented in at least one hardware form of a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), and a PLA (Programmable Logic Array). The processor 401 may also include a main processor and a coprocessor, where the main processor is a processor for Processing data in an awake state, and is also called a Central Processing Unit (CPU); a coprocessor is a low power processor for processing data in a standby state. In some embodiments, the processor 401 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content required to be displayed by the display screen. In some embodiments, the processor 401 may further include an AI (Artificial Intelligence) processor for processing computing operations related to machine learning.

Memory 402 may include one or more computer-readable storage media, which may be non-transitory. Memory 402 may also include high speed random access memory, as well as non-volatile memory, such as one or more magnetic disk storage devices, flash memory storage devices. In some embodiments, a non-transitory computer readable storage medium in memory 402 is used to store at least one instruction for execution by processor 401 to implement the application interface display method in picture-in-picture mode provided by method embodiments herein.

In some embodiments, the mobile terminal 400 may further optionally include: a peripheral interface 403 and at least one peripheral. The processor 401, memory 402 and peripheral interface 403 may be connected by bus or signal lines. Each peripheral may be connected to the peripheral interface 403 via a bus, signal line, or circuit board. Specifically, the peripheral device includes: at least one of radio frequency circuitry 404, touch screen display 405, camera 406, audio circuitry 407, positioning components 408, and power supply 409.

The peripheral interface 403 may be used to connect at least one peripheral related to I/O (Input/Output) to the processor 401 and the memory 402. In some embodiments, processor 401, memory 402, and peripheral interface 403 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 401, the memory 402 and the peripheral interface 403 may be implemented on a separate chip or circuit board, which is not limited by this embodiment.

The Radio Frequency circuit 404 is used for receiving and transmitting RF (Radio Frequency) signals, also called electromagnetic signals. The radio frequency circuitry 404 communicates with communication networks and other communication devices via electromagnetic signals. The rf circuit 404 converts an electrical signal into an electromagnetic signal to transmit, or converts a received electromagnetic signal into an electrical signal. Optionally, the radio frequency circuit 404 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a subscriber identity module card, and so forth. The radio frequency circuitry 404 may communicate with other terminals via at least one wireless communication protocol. The wireless communication protocols include, but are not limited to: the world wide web, metropolitan area networks, intranets, generations of mobile communication networks (2G, 3G, 4G, and 5G), Wireless local area networks, and/or WiFi (Wireless Fidelity) networks. In some embodiments, the rf circuit 404 may further include NFC (Near Field Communication) related circuits, which are not limited in this application.

The display screen 405 is used to display a UI (User Interface). The UI may include graphics, text, icons, video, and any combination thereof. When the display screen 405 is a touch display screen, the display screen 405 also has the ability to capture touch signals on or over the surface of the display screen 405. The touch signal may be input to the processor 401 as a control signal for processing. At this point, the display screen 405 may also be used to provide virtual buttons and/or a virtual keyboard, also referred to as soft buttons and/or a soft keyboard. In some embodiments, the display screen 405 may be one, providing the front panel of the mobile terminal 400; in other embodiments, the number of the display screens 405 may be at least two, and the two display screens are respectively disposed on different surfaces of the mobile terminal 400 or in a folding design; in still other embodiments, the display 405 may be a flexible display disposed on a curved surface or on a folded surface of the mobile terminal 400. Even further, the display screen 405 may be arranged in a non-rectangular irregular pattern, i.e. a shaped screen. The Display screen 405 may be made of LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode), and other materials.

The camera assembly 406 is used to capture images or video. Optionally, camera assembly 406 includes a front camera and a rear camera. Generally, a front camera is disposed at a front panel of the terminal, and a rear camera is disposed at a rear surface of the terminal. In some embodiments, the number of the rear cameras is at least two, and each rear camera is any one of a main camera, a depth-of-field camera, a wide-angle camera and a telephoto camera, so that the main camera and the depth-of-field camera are fused to realize a background blurring function, and the main camera and the wide-angle camera are fused to realize panoramic shooting and VR (Virtual Reality) shooting functions or other fusion shooting functions. In some embodiments, camera assembly 406 may also include a flash. The flash lamp can be a monochrome temperature flash lamp or a bicolor temperature flash lamp. The double-color-temperature flash lamp is a combination of a warm-light flash lamp and a cold-light flash lamp, and can be used for light compensation at different color temperatures.

The audio circuit 407 may include a microphone and a speaker. The microphone is used for collecting sound waves of a user and the environment, converting the sound waves into electric signals, and inputting the electric signals to the processor 401 for processing, or inputting the electric signals to the radio frequency circuit 404 for realizing voice communication. The microphones may be provided in a plurality for stereo sound collection or noise reduction, and are respectively disposed at different portions of the mobile terminal 400. The microphone may also be an array microphone or an omni-directional pick-up microphone. The speaker is used to convert electrical signals from the processor 401 or the radio frequency circuit 404 into sound waves. The loudspeaker can be a traditional film loudspeaker or a piezoelectric ceramic loudspeaker. When the speaker is a piezoelectric ceramic speaker, the speaker can be used for purposes such as converting an electric signal into a sound wave audible to a human being, or converting an electric signal into a sound wave inaudible to a human being to measure a distance. In some embodiments, audio circuitry 407 may also include a headphone jack.

The positioning component 408 is used to locate the current geographic Location of the mobile terminal 400 for navigation or LBS (Location Based Service). The Positioning component 408 may be a Positioning component based on the GPS (Global Positioning System) of the united states, the beidou System of china, or the galileo System of the european union.

The power supply 409 is used to supply power to various components in the mobile terminal 400. The power source 409 may be alternating current, direct current, disposable or rechargeable. When the power source 409 includes a rechargeable battery, the rechargeable battery may be a wired rechargeable battery or a wireless rechargeable battery. The wired rechargeable battery is a battery charged through a wired line, and the wireless rechargeable battery is a battery charged through a wireless coil. The rechargeable battery may also be used to support fast charge technology.

In some embodiments, the mobile terminal 400 also includes one or more sensors 410. The one or more sensors 410 include, but are not limited to: acceleration sensor 411, gyro sensor 412, pressure sensor 413, fingerprint sensor 414, optical sensor 415, and proximity sensor 416.

The acceleration sensor 411 may detect the magnitude of acceleration in three coordinate axes of a coordinate system established with the mobile terminal 400. For example, the acceleration sensor 411 may be used to detect components of the gravitational acceleration in three coordinate axes. The processor 401 may control the touch display screen 405 to display the user interface in a landscape view or a portrait view according to the gravitational acceleration signal collected by the acceleration sensor 411. The acceleration sensor 411 may also be used for acquisition of motion data of a game or a user.

The gyro sensor 412 may detect a body direction and a rotation angle of the mobile terminal 400, and the gyro sensor 412 may cooperate with the acceleration sensor 411 to collect a 3D motion of the user on the mobile terminal 400. From the data collected by the gyro sensor 412, the processor 401 may implement the following functions: motion sensing (such as changing the UI according to a user's tilting operation), image stabilization at the time of photographing, game control, and inertial navigation.

The pressure sensor 413 may be disposed at a side bezel of the mobile terminal 400 and/or at a lower layer of the touch display screen 405. When the pressure sensor 413 is disposed on a side frame of the mobile terminal 400, a user holding signal of the mobile terminal 400 can be detected, and the processor 401 performs left-right hand recognition or shortcut operation according to the holding signal collected by the pressure sensor 413. When the pressure sensor 413 is disposed at the lower layer of the touch display screen 405, the processor 401 controls the operability control on the UI interface according to the pressure operation of the user on the touch display screen 405. The operability control comprises at least one of a button control, a scroll bar control, an icon control and a menu control.

The fingerprint sensor 414 is used for collecting a fingerprint of the user, and the processor 401 identifies the identity of the user according to the fingerprint collected by the fingerprint sensor 414, or the fingerprint sensor 414 identifies the identity of the user according to the collected fingerprint. Upon recognizing that the user's identity is a trusted identity, processor 401 authorizes the user to perform relevant sensitive operations including unlocking the screen, viewing encrypted information, downloading software, paying, and changing settings, etc. The fingerprint sensor 414 may be disposed on the front, back, or side of the mobile terminal 400. When a physical key or vendor Logo is provided on the mobile terminal 400, the fingerprint sensor 414 may be integrated with the physical key or vendor Logo.

The optical sensor 415 is used to collect the ambient light intensity. In one embodiment, the processor 401 may control the display brightness of the touch display screen 405 based on the ambient light intensity collected by the optical sensor 415. Specifically, when the ambient light intensity is high, the display brightness of the touch display screen 405 is increased; when the ambient light intensity is low, the display brightness of the touch display screen 405 is turned down. In another embodiment, the processor 401 may also dynamically adjust the shooting parameters of the camera assembly 406 according to the ambient light intensity collected by the optical sensor 415.

A proximity sensor 416, also known as a distance sensor, is typically disposed on the front panel of the mobile terminal 400. The proximity sensor 416 is used to collect a distance between the user and the front surface of the mobile terminal 400. In one embodiment, when the proximity sensor 416 detects that the distance between the user and the front surface of the mobile terminal 400 gradually decreases, the processor 401 controls the touch display screen 405 to switch from the bright screen state to the dark screen state; when the proximity sensor 416 detects that the distance between the user and the front surface of the mobile terminal 400 becomes gradually larger, the processor 401 controls the touch display screen 405 to switch from the screen-on state to the screen-on state.

That is, not only is an application interface display apparatus provided in the mobile terminal 400, which includes a processor and a memory for storing executable instructions of the processor, wherein the processor is configured to execute the application interface display method in the picture-in-picture mode provided in the foregoing embodiments, but also a computer-readable storage medium is provided, in which a computer program is stored, and when the computer program is executed by the processor, the application interface display method in the picture-in-picture mode in the foregoing embodiments can be implemented.

It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, where the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.

The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents, improvements and the like that fall within the spirit and principle of the present invention are intended to be included therein.

Claims

1. A method for displaying an application interface in a picture-in-picture mode, the method comprising:

when detecting that an application to be started is a second application which needs to be displayed by taking a picture-in-picture mode as a foreground application, acquiring the display direction of an application interface of a first application which is currently displayed, wherein the display of the second application in the picture-in-picture mode refers to the display in the picture-in-picture mode in the application interface of the first application;

acquiring an interface display direction appointed in an application package of the second application, and judging whether the interface display direction is the same as the display direction of the application interface of the first application; if the interface display direction is different from the display direction of the application interface of the first application, replacing the interface display direction specified in the application package of the second application with the display direction of the application interface of the first application;

2. The method according to claim 1, wherein before the obtaining of the display direction of the application interface of the currently displayed first application when detecting that the application to be started is the second application that needs to be displayed in the picture-in-picture mode as the foreground application, further comprises:

3. The method according to any one of claims 1-2, wherein the displaying the application interface of the second application on the upper layer of the application interface of the first application based on the interface display direction specified in the replaced application package of the second application comprises:

4. The method according to claim 3, wherein the adjusting the width and the height of the application interface of the second application, and displaying the application interface of the second application on the upper layer of the application interface of the first application based on the adjusted width and the adjusted height of the application interface of the second application comprises:

5. An apparatus for displaying an application interface in a picture-in-picture mode, the apparatus comprising:

the device comprises a first obtaining module, a second obtaining module and a display module, wherein the first obtaining module is used for obtaining the display direction of an application interface of a first application which is currently displayed when detecting that an application to be started is a second application which needs to be displayed by taking a picture-in-picture mode as a foreground application, and the second application is displayed in the picture-in-picture mode, namely displayed in the picture-in-picture mode in the application interface of the first application;

a display module, configured to display an application interface of the second application on an upper layer of an application interface of the first application based on the interface display direction specified in the replaced application package of the second application;

the device further comprises:

6. The apparatus of claim 5, further comprising:

7. The apparatus of any of claims 5-6, wherein the display module comprises:

8. The apparatus of claim 7, wherein the adjustment submodule is specifically configured to:

9. A computer-readable storage medium having stored therein instructions which, when executed by a processor, implement the method of displaying an application interface in a picture-in-picture mode according to any one of claims 1 to 4.