WO2024058288A1

WO2024058288A1 - Electronic device and translation information providing method therefor

Info

Publication number: WO2024058288A1
Application number: PCT/KR2022/013802
Authority: WO
Inventors: 히진롱
Original assignee: 쿠팡 주식회사
Priority date: 2022-09-13
Filing date: 2022-09-15
Publication date: 2024-03-21
Also published as: TW202429330A; KR20240036294A

Abstract

A translation information providing method for an electronic device is provided. The translation information providing method may comprise the steps of: identifying whether information on an image corresponding to a first application has been obtained; if the information on the image has been obtained, obtaining translation information on the image; and providing the obtained translation information.

Description

Electronic devices and methods of providing translation information thereof

The present disclosure relates to an electronic device and a control method for translating text included in an image based on an image corresponding to an application so that a user can more conveniently use an application that does not provide multilingual services.

As the use of the Internet and the spread of mobile terminals have become widespread, users have become able to obtain information through various applications. Depending on the type of application that users use, there are frequent cases where users obtain information written in text in their native language, as well as information written in text in a foreign language that users cannot understand.

However, whenever users encounter foreign language text that they cannot understand, directly translating the foreign language text causes hassle and inconvenience. Therefore, in order to improve user convenience, there is a need for a method by which users can more conveniently obtain translation information for foreign language texts included in an application.

In relation to this, prior literature such as KR 10-2258004 B1 may be referred to.

The disclosed embodiments seek to provide an electronic device and a method of providing translation information thereof. More specifically, the purpose is to provide an electronic device and a control method for translating text included in an image based on an image corresponding to the application so that users can more conveniently use applications that do not provide multilingual services. Do it as

The technical challenges that this embodiment aims to achieve are not limited to the technical challenges described above, and other technical challenges can be inferred from the following embodiments.

One aspect of the present disclosure is a method for providing translation information, comprising: checking whether information about an image corresponding to a first application has been obtained; When information about the image is obtained, obtaining translation information about the image; and providing the obtained translation information.

In addition, in one embodiment of the present disclosure, the step of acquiring the translation information includes obtaining the translation information based on information about the image using a second application running in the background while the first application is activated. A method of providing translation information may be provided, including the step of obtaining.

In addition, in one embodiment of the present disclosure, the translation information may include a second text in a second language obtained based on the first text in the first language included in the image, and a method of providing translation information may be provided. there is.

In addition, in one embodiment of the present disclosure, the step of checking whether information about the image has been obtained may provide a method of providing translation information, including checking whether a new image file has been created in a set storage. there is.

Additionally, in one embodiment of the present disclosure, the step of checking whether information about the image has been obtained may provide a method of providing translation information, including checking whether an image capture function is activated.

In addition, in one embodiment of the present disclosure, the step of obtaining translation information for the image includes providing a user interface (UI) to inquire whether to translate the image. can be provided.

In addition, in one embodiment of the present disclosure, the step of obtaining translation information for the image may provide a method of providing translation information, including providing a user interface to inquire about the language into which the image is to be translated. .

Additionally, in one embodiment of the present disclosure, obtaining translation information for the image includes capturing the image and checking whether a first type of input requesting translation of the captured image has been obtained, The first type of input may provide a method of providing translation information that is different from the second type of input requesting capture of an image.

In addition, in one embodiment of the present disclosure, the method for providing translation information further includes obtaining a translation request for a target area in the image, and the translation information includes translation information for the target area. A method of provision can be provided.

In addition, in one embodiment of the present disclosure, a method of providing translation information may be provided, further comprising the step of checking a time varying area in the image.

In addition, in one embodiment of the present disclosure, the method for providing translation information includes: checking whether to translate the area that changes with time; and obtaining a translation request for the area that changes with time, wherein the translation information includes translation information about the area that changes with time.

Additionally, in one embodiment of the present disclosure, a method of providing translation information can be provided in which translation information for an area that changes with time is displayed in a set method.

In addition, in one embodiment of the present disclosure, the image includes an image captured by scrolling a scrollable page, and the step of providing translation information of the image corresponds to an area within the page displayed on the screen according to scrolling of the page. A method of providing translation information may be provided, including providing translation information together with the page.

In addition, in one embodiment of the present disclosure, the image includes an image capturing a page of the first application, and the translation information providing method includes checking whether the page has changed as the first application is updated. step; As the page is confirmed to have changed, checking whether to translate an image corresponding to the changed page; As the image corresponding to the changed page is confirmed to be translated, obtaining information about the image corresponding to the changed page; Obtaining translation information about the image corresponding to the changed page; and providing translation information for an image corresponding to the changed page.

Additionally, in one embodiment of the present disclosure, the obtained translation information may be displayed on the content of the first application, providing a method of providing translation information.

Additionally, in one embodiment of the present disclosure, obtaining a set input for a first region in the translation information; Based on the set input, confirming a second area within the first application corresponding to the first area; and providing a page corresponding to the second area.

In addition, in an embodiment of the present disclosure, when the size of the information about the image and the translation information are different, a method of providing translation information is provided in which elements set in the translation information are displayed in a set method.

Another aspect of the present disclosure includes a communication unit, a memory, and a controller, wherein the control unit determines whether information about an image corresponding to a first application has been obtained, and when information about the image is obtained, An electronic device may be provided that obtains translation information for the image and provides the obtained translation information.

Another aspect of the present disclosure may provide a computer-readable recording medium on which a program for implementing a method performed by an electronic device is recorded.

Specific details of other embodiments are included in the detailed description and drawings.

If you follow the proposed embodiment, one or more of the following effects can be expected.

According to an embodiment of the present specification, by automatically detecting whether an image has been acquired and providing translation information, users can obtain the technical effect of more conveniently obtaining translation information for an image corresponding to an application.

In addition, according to the embodiment of the present specification, by providing translation information about the area that changes with time in the image displayed on the screen of the application, the technology allows users to more conveniently obtain translation information about the image corresponding to the application. You can get the effect.

In addition, according to an embodiment of the present specification, by capturing an image and providing translation information based on a type of input requesting translation of the captured image, users can more conveniently provide translation information for the image corresponding to the application. You can obtain the technical effect of obtaining .

In addition, according to the embodiment of the present specification, by providing translation information for images captured by scrolling a scrollable page, users can obtain the technical effect of more conveniently obtaining translation information for images corresponding to the application. there is.

In addition, according to the embodiment of the present specification, by providing translation information about images on the application screen, users can obtain the technical effect of more conveniently obtaining translation information about images corresponding to the application.

The effect of the invention is not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description of the claims.

1 shows a system according to one embodiment.

FIGS. 2A to 2G are diagrams for explaining a process in which an electronic device provides translation information, according to an embodiment.

FIGS. 3A and 3B are diagrams for explaining a process in which an electronic device provides translation information, according to an embodiment.

FIGS. 4A to 4G are diagrams for explaining a process in which an electronic device provides translation information, according to an embodiment.

5A to 5E are diagrams for explaining a process in which an electronic device provides translation information, according to an embodiment.

FIGS. 6A to 6F are diagrams for explaining a process in which an electronic device provides translation information, according to an embodiment.

FIGS. 7A to 7E are diagrams for explaining a process in which an electronic device provides translation information, according to an embodiment.

FIGS. 8A and 8B are diagrams for explaining a process in which an electronic device provides translation information, according to an embodiment.

9A to 9C are diagrams for explaining a process in which an electronic device provides translation information, according to an embodiment.

FIGS. 10A to 10D are diagrams for explaining a process in which an electronic device provides translation information, according to an embodiment.

Figure 11 shows a flowchart of a method for providing translation information by an electronic device, according to an embodiment.

Figure 12 shows a block diagram of an electronic device according to an embodiment.

The terms used in the embodiments are general terms that are currently widely used as much as possible while considering the functions in the present disclosure, but this may vary depending on the intention or precedent of a person working in the art, the emergence of new technology, etc. In addition, in certain cases, there are terms arbitrarily selected by the applicant, and in this case, the meaning will be described in detail in the relevant description. Therefore, the terms used in this disclosure should be defined based on the meaning of the term and the overall content of this disclosure, rather than simply the name of the term.

When it is said that a part "includes" a certain element throughout the specification, this means that, unless specifically stated to the contrary, it does not exclude other elements but may further include other elements.

The expression “at least one of a, b, and c” used throughout the specification means ‘a alone’, ‘b alone’, ‘c alone’, ‘a and b’, ‘a and c’, ‘b and c ', or 'all of a, b, and c'.

The “terminal” mentioned below may be implemented as a computer or portable terminal that can connect to a server or other terminal through a network. Here, the computer includes, for example, a laptop, desktop, laptop, etc. equipped with a web browser, and the portable terminal is, for example, a wireless communication device that guarantees portability and mobility. , all types of communication-based terminals such as IMT (International Mobile Telecommunication), CDMA (Code Division Multiple Access), W-CDMA (W-Code Division Multiple Access), and LTE (Long Term Evolution), smartphones, tablet PCs, etc. It may include a handheld-based wireless communication device.

Below, with reference to the attached drawings, embodiments of the present disclosure will be described in detail so that those skilled in the art can easily practice them. However, the present disclosure may be implemented in many different forms and is not limited to the embodiments described herein.

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings.

1 shows a system according to one embodiment.

Referring to FIG. 1 , the system may include at least one of an electronic device 100, a first server 120, a second server 140, and a network 160. Meanwhile, the system shown in FIG. 1 shows only components related to this embodiment. Accordingly, those skilled in the art can understand that other general-purpose components may be included in addition to the components shown in FIG. 1.

The electronic device 100 is a device that configures and provides various information. The electronic device 100 may provide the configured information in the form of a web page or application screen, or may provide information in a form that can be displayed as a web page or application screen on the receiving terminal.

According to one embodiment, the electronic device 100 is a terminal used by each user, and the users can access services provided by the network 160 using their respective electronic devices 100. For example, the electronic device 100 may check whether information about the image corresponding to the application has been obtained, and if information about the image is obtained, obtain and provide translation information about the image.

The first server 120 is a device that provides services related to applications or web pages, and can provide information in a form that can be displayed on a web page or application screen, etc. based on the request received from the electronic device 100. there is. For example, the first server 120 may provide services related to applications that can order and deliver items or applications that can view various types of content.

The second server 140 is a device that provides translation services and can generate translation information based on information about images received from the electronic device 100 and transmit it to the electronic device 100 . For example, the second server 140 obtains an image from the electronic device 100, translates the text of the first language included in the image into the second language, and then translates the text into the second language. Translation information may be transmitted to the electronic device 100.

Meanwhile, hereinafter, an application related to a service provided by the first server 120 may be referred to as a first application, and an application related to a service provided by the second server 140 may be referred to as a second application. .

The first server 120, the second server 140, and the electronic device 100 may communicate with each other within the network 160. The network 160 includes a local area network (LAN), a wide area network (WAN), a value added network (VAN), a mobile radio communication network, a satellite communication network, and their respective networks. It is a comprehensive data communication network that includes a combination and allows each network constituent shown in FIG. 1 to communicate smoothly with each other, and may include wired Internet, wireless Internet, and mobile wireless communication network. Wireless communications include, for example, wireless LAN (Wi-Fi), Bluetooth, Bluetooth low energy, ZigBee, WFD (Wi-Fi Direct), UWB (ultra wideband), and infrared communication (IrDA, infrared Data Association). ), NFC (Near Field Communication), etc., but are not limited thereto.

FIGS. 2A to 2G are diagrams for explaining a process in which the electronic device 100 provides translation information, according to an embodiment.

According to one embodiment, the electronic device 100 may execute a first application and display a screen 200 of the first application. For example, referring to FIG. 2A , the electronic device 100 may display a portion of the search result page when searching for “vitamin d” on the screen 200 of the first application.

According to one embodiment, the electronic device 100 may check whether information about the image has been obtained. For example, referring to FIG. 2B, the electronic device 100 stores an image 210 that captures the screen 200 of the first application or an image 210 corresponding to the screen 200 of the first application. You can check whether information about the address, etc. has been obtained. At this time, the electronic device 100 can check whether information about the image has been obtained through various methods.

According to one embodiment, the electronic device 100 may check whether information about the image has been obtained based on whether a new image file has been created in the set storage. For example, when a new image file is created in the storage where the captured image is stored, the electronic device 100 may confirm that information about the image has been obtained.

According to one embodiment, the electronic device 100 may check whether information about the image has been obtained based on whether the image capture function is activated. For example, the electronic device 100 may check whether the image capture function is activated by hooking the capture input of the image, and if it is confirmed that the image capture function is activated, the electronic device 100 may determine whether the image capture function is activated. It can be confirmed that information regarding has been obtained.

According to one embodiment, the electronic device 100 can check whether to translate the image 210 by providing a user interface (UI) 220 that inquires whether to translate the image 210. there is. For example, referring to FIG. 2C, when information about the image 210 is obtained, the electronic device 100 may display a user interface 220 on the screen asking whether to translate the image 210. there is. Thereafter, when the electronic device 100 obtains a user input of selecting the Yes icon, the electronic device 100 may obtain translation information 240 for the image 210.

According to one embodiment, it is possible to capture an image and determine whether to translate the image 210 based on whether an input of a type requesting translation of the captured image is obtained. At this time, the type of input requesting capture of an image and the type of input requesting capture of the image and translation of the captured image may be set differently.

For example, the input requesting capture of an image is set to the input of simultaneously pressing the power button and the volume down button of the electronic device 100, and the input requesting capture of the image and translation of the captured image is set to the input of pressing the power button and the volume down button of the electronic device 100. ) can be set as an input by pressing the power button and volume up button simultaneously. As the electronic device 100 captures an image and obtains an input of a set type requesting translation of the captured image, it omits the process of asking the user whether to translate the image 210 and translates the image 210 Translation information 240 for can be obtained.

According to one embodiment, the electronic device 100 may provide a user interface 230 to inquire about the language into which the image 210 is to be translated. For example, referring to FIG. 2D, the electronic device 100 obtains a user input of selecting the Yes icon of the user interface 220, and then displays the user interface 230 to inquire about the language to translate the image 210 into. ) can be displayed on the screen. Thereafter, the electronic device 100 may obtain a user input for selecting a language and obtain translation information 240 for an image including text translated into the selected language.

Alternatively, the electronic device 100 may display the user interface 230 on the screen to inquire about the language to translate the image into, regardless of whether information about the image has been obtained. Accordingly, when the electronic device 100 obtains information about the image 210, it omits the process of inquiring about the language to be translated, and provides translation information about the image 210 including text translated into the set language. You can obtain (240).

According to one embodiment, the electronic device 100 uses a second application running in the background while the first application is activated, translation information for the image 210 based on information about the image 210 ( 240) can be obtained. For example, the electronic device 100 may provide translation information 240 for the image 210 by directly translating text from the first language included in the image 210 into the second language using the second application. It can be obtained. Alternatively, the electronic device 100 may use the second application to transmit a translation request for the image 210 to an external server and receive translation information for the image 210 from the external server, thereby transmitting a translation request for the image 210 to the external server. Translation information 240 can be obtained.

According to one embodiment, the translation information 240 for the image 210 may include a second text in the second language obtained based on the first text in the first language included in the image 210. For example, referring to FIG. 2E, translation information 240 for the image 210 may include English text obtained by translating the Korean text included in the image 210.

According to one embodiment, the electronic device 100 may provide the obtained translation information 240. For example, referring to FIG. 2F, the electronic device 100 may run the second application in the foreground and display translation information 240 for the image 210 on the screen 250 of the second application. there is. Alternatively, the electronic device 100 may transmit translation information 240 for the image 210 to a set email address. However, this is only an example, and the method by which the electronic device 100 provides the obtained translation information is not limited to the above.

According to one embodiment, when the size of the information about the image 210 and the translation information 240 about the image 210 are different, the electronic device 100 displays elements set in the translation information 240 in a set method. can do. For example, referring to FIG. 2F, in the process of translating Korean text into English text, the amount of text changes, and accordingly, the size of the information about the image 210 and the translation information 240 about the image 210 may be different. In this case, the electronic device 100 displays translation information 240 for the image 210 on the screen 250 of the second application and displays a bar icon 260 within the translation information 240 for the user's convenience. It can be displayed as a highlight. However, elements displayed in the translation information 240 and methods for displaying the elements may be set differently and may change depending on user input, etc.

Meanwhile, the electronic device 100 may check whether to provide the obtained translation information according to an embodiment. For example, referring to FIG. 2G, the electronic device 100 may display the user interface 270 on the screen to inquire whether to load the obtained translation information. Thereafter, when the electronic device 100 obtains a user input of selecting the Yes icon, translation information 240 for the image 210 may be displayed on the screen 250 of the second application.

FIGS. 3A and 3B are diagrams for explaining a process in which the electronic device 100 provides translation information, according to an embodiment. Content that overlaps with FIGS. 2A to 2G will be briefly described or omitted.

According to one embodiment, the electronic device 100 may execute a first application and display a screen of the first application. Afterwards, the electronic device 100 can check whether information about the image 300 has been obtained. When information about the image 300 is obtained, the electronic device 100 can confirm whether to translate the image 300.

According to one embodiment, the electronic device 100 may obtain a translation request for the target area 320 within the image 300. For example, referring to FIG. 3A, when it is confirmed that image 300 is being translated, electronic device 100 may obtain an input for selecting an area 320 within image 300 that the user wishes to translate. there is.

According to one embodiment, the electronic device 100 may acquire translation information 340 for the image 300 including translation information for the target area 320. For example, referring to FIG. 3B, the electronic device 100 includes “Related products that other customers have bought a lot” included in the target area 320. Translation information 340 can be obtained. At this time, text included in areas other than the target area 320 may not be translated.

According to one embodiment, the electronic device 100 may provide the obtained translation information 340. For example, the electronic device 100 may provide translation information 340 in which only the text included in the target area 320 is translated in various forms.

FIGS. 4A to 4G are diagrams for explaining a process in which the electronic device 100 provides translation information according to an embodiment. Content that overlaps with FIGS. 2A to 2G will be briefly described or omitted.

According to one embodiment, a specific area within the screen 400 of the first application may change over time. For example, referring to FIG. 4A, a coupon icon may be displayed in the area 410 of the screen 400 of the first application along with the text “Get a surprise coupon!” However, referring to FIG. 4B, over time, the text “Out of stock” and “One-day special price” may be displayed in area 410. As such, if a specific area within the screen 400 of the first application changes over time, additional translation of the area may be required.

According to one embodiment, the electronic device 100 may check whether information about the image has been obtained. For example, the electronic device 100 may check whether information about the image has been obtained based on whether a new image file has been created in the set storage or whether the image capture function has been activated.

According to one embodiment, the electronic device 100 may check a time varying area on the screen 400 of the first application. More specifically, the electronic device 100 may check the area that changes with time by comparing captured images before and after the area changes, or check the area that changes with time based on a user input.

For example, referring to FIGS. 4C and 4D, the electronic device 100 captures an image 420 of the screen 400 before the area 410 changes and the screen 400 after the area 410 changes. A captured image 430 may be obtained. Thereafter, the electronic device 100 may compare the image 420 and the image 430 to identify a different area and identify the area as the area 410 that changes with time.

Alternatively, the electronic device 100 may check the area that changes with time by monitoring the screen 400 of the first application. More specifically, the electronic device 100 may acquire the image 420 corresponding to the screen 400 of the first application and then monitor the screen 400 of the first application. The electronic device 100 may identify an area that is different from the screen 400 when the image 420 was acquired, and may identify the area as the area 410 that changes with time.

Alternatively, the electronic device 100 may check an area that changes with time based on user input. More specifically, the electronic device 100 obtains a user input for selecting a region that changes with time in the screen 400 of the first application or the image 420 corresponding to the screen 400 of the first application, and the selected The area can be identified as an area 410 that changes with time.

According to one embodiment, the electronic device 100 can check whether to translate an area that changes over time. For example, referring to FIG. 4E, the electronic device 100 may provide a user interface 440 to inquire whether to translate an area that changes over time. Thereafter, when the electronic device 100 obtains a user input for selecting the Yes icon, the electronic device 100 may obtain translation information 460 for the image 430.

According to one embodiment, the electronic device 100 may use a second application running in the background while the first application is activated to obtain translation information about the image based on information about the image. For example, referring to FIG. 4F , the electronic device 100 may obtain translation information 450 for the image 420 using the second application. Additionally, referring to FIG. 4G, the electronic device 100 may obtain translation information 460 for the image 430 using the second application.

According to one embodiment, the electronic device 100 may provide the obtained translation information. For example, the electronic device 100 runs the second application in the foreground and displays translation information 450 about the image 420 captured before the area 410 changes on the screen of the second application. You can. Alternatively, the electronic device 100 may run the second application in the foreground and display translation information 460 about the image 430 captured after the area 410 changes on the screen of the second application.

FIGS. 5A to 5E are diagrams for explaining a process in which the electronic device 100 provides translation information according to an embodiment. Content that overlaps with FIGS. 2A to 2G will be briefly described or omitted.

According to one embodiment, a specific area within the screen 500 of the first application may change over time. For example, referring to FIG. 5A, a coupon icon may be displayed in an area 520 within the screen 500 of the first application along with the text “Get a surprise coupon!” However, referring to FIG. 5B, over time, the text “Out of stock” and “One-day special price” may be displayed in area 520. As such, if a specific area within the screen 500 of the first application changes over time, additional translation of the area may be required.

According to one embodiment, the electronic device 100 may check an area that changes with time on the screen 500 of the first application. The specific operation of checking the area in which the electronic device 100 changes with time is omitted since it overlaps with the content described in relation to FIGS. 4A to 4G.

According to one embodiment, the electronic device 100 uses a second application running in the background while the first application is activated, and translates information about the image 540 based on information about the image 540 ( 560) can be obtained. For example, referring to FIG. 5D , the electronic device 100 may obtain translation information 560 for the image 540 using a second application.

According to one embodiment, the electronic device 100 may provide the obtained translation information. At this time, the electronic device 100 may display translation information about an area that changes over time in a set method. For example, referring to FIG. 5E, the electronic device 100 runs the second application in the foreground and displays translation information 560 for the image 540 on the screen 580 of the second application, The area 520 that changes with time can be displayed as a highlight. However, the method by which the electronic device 100 displays translation information for an area that changes over time is not limited to the above.

FIGS. 6A to 6F are diagrams for explaining a process in which the electronic device 100 provides translation information, according to an embodiment. Content that overlaps with FIGS. 2A to 2G will be briefly described or omitted.

According to one embodiment, the electronic device 100 may display a portion of a scrollable page on the screen 600 of the first application. For example, referring to FIG. 6A , the electronic device 100 may display the upper part of the search result page when searching for “vitamin d” on the screen 600 of the first application. Additionally, referring to FIG. 6B, as the user's input of lowering the scroll bar is obtained, the electronic device 100 displays the lower part of the search result page when searching for "vitamin d" on the screen 600 of the first application. It can be displayed.

According to one embodiment, the electronic device 100 may obtain a scroll-captured image of a scrollable page. At this time, scroll capture is an image acquisition method that captures a page while moving a scroll bar, and can be used to capture a page that is larger than the screen of the terminal.

For example, referring to FIG. 6C, the electronic device 100 may obtain an image 620 captured by scrolling a search result page when searching for “vitamin d.” Accordingly, the electronic device 100 may obtain an image 620 that is larger than the screen 600 of the first application.

According to one embodiment, the electronic device 100 uses a second application running in the background while the first application is activated, and translates information about the image 620 based on information about the image 620 ( 640) can be obtained. For example, referring to FIG. 6D, the electronic device 100 may obtain translation information 640 for the image 620 using the second application.

According to one embodiment, the electronic device 100 may provide the obtained translation information. At this time, the electronic device 100 may provide translation information corresponding to the area within the page displayed on the screen 600 of the first application to the screen 660 of the second application as the page is scrolled.

For example, referring to FIG. 6E, the electronic device 100 displays translation information corresponding to the upper part of the search result page when searching for "vitamin d" displayed on the screen 600 of the first application to the second application. It can be displayed on the application screen 660. In addition, referring to FIG. 6F, as the user's input of lowering the scroll bar included in the screen 600 of the first application is obtained, the electronic device 100 displays the message "vitamin d" displayed on the screen 600 of the first application. Translation information corresponding to the bottom portion of the search result page when searching for " may be displayed on the screen 660 of the second application.

At this time, the electronic device 100 may highlight the bar icon 680 in the translation information for the user's convenience. However, the elements displayed in the translation information and the methods for displaying the elements may be set differently and may change depending on user input, etc.

FIGS. 7A to 7E are diagrams for explaining a process in which the electronic device 100 provides translation information according to an embodiment. Content that overlaps with FIGS. 2A to 2G will be briefly described or omitted.

According to one embodiment, the electronic device 100 may execute a first application and display a screen 700 of the first application. For example, referring to FIG. 7A , the electronic device 100 may display the home page of the first application on the screen 700 of the first application.

According to one embodiment, the electronic device 100 may check whether the page corresponding to the acquired image has changed as the first application is updated. For example, referring to FIG. 7B, after it is confirmed that information about the image corresponding to the home page of the first application has been acquired, the electronic device 100 determines whether the home page has changed as the first application is updated. You can check. The electronic device 100 can confirm that as the first application is updated, the text written in the area 720 of the home page changes from “Chuseok up to 70%” to “New members receive a half-price coupon” and a coupon icon is added. there is.

According to one embodiment, as the page corresponding to the acquired image is confirmed to have changed, the electronic device 100 may check whether to translate the image corresponding to the changed page. For example, referring to FIG. 7C, the electronic device 100 displays a user interface 740 that notifies that a part of the home page has changed as the first application is updated and inquires whether to translate the image accordingly. You can do it.

According to one embodiment, the electronic device 100 may acquire an image 760 corresponding to the changed page. For example, referring to FIG. 7D, when the electronic device 100 obtains a user input of selecting the Yes icon in the user interface 740, the electronic device 100 captures an image 760 of the changed homepage. It can be obtained.

According to one embodiment, the electronic device 100 uses a second application running in the background while the first application is activated to provide translation information about the image 760 based on information about the image 760. It can be obtained. For example, the electronic device 100 may obtain translation information about the image 760 using the second application.

According to one embodiment, the electronic device 100 may provide the obtained translation information. At this time, the electronic device 100 may display translation information for the area that has changed as the first application is updated in a set method. For example, referring to FIG. 7E, the electronic device 100 runs the second application in the foreground, displays translation information about the image 760 on the screen 780 of the second application, and displays the first application. As this is updated, the changed area 720 can be displayed as a highlight.

FIGS. 8A and 8B are diagrams for explaining a process in which the electronic device 100 provides translation information according to an embodiment. Content that overlaps with FIGS. 2A to 2G will be briefly described or omitted.

According to one embodiment, the electronic device 100 may execute a first application and display a screen 800 of the first application. For example, referring to FIG. 8A , the electronic device 100 may display the home page of the first application on the screen 800 of the first application.

According to one embodiment, as it is confirmed that information about the image has been obtained, the electronic device 100 may check whether to translate the image. For example, the electronic device 100 may translate the image by providing a user interface that inquires whether to translate the image, or by capturing the image and checking whether an input of a type requesting translation of the captured image has been obtained. You can check whether to translate or not.

According to one embodiment, the electronic device 100 may use a second application running in the background while the first application is activated to obtain translation information about the image based on information about the image. For example, the electronic device 100 directly translates the text of the first language included in the image into the second language, transmits a translation request for the image to an external server, and receives translation information about the image from the external server. By doing so, translation information about the image can be obtained.

According to one embodiment, the electronic device 100 may provide translation information obtained together with the content of the first application. For example, referring to FIG. 8B, the electronic device 100 may display a homepage on the screen 800 of the first application and display text translated into the second language on top of the corresponding text in the first language. . At this time, the electronic device 100 may adjust the transparency of the text translated into the second language so that the user can check which text of the first language corresponds to the text translated into the second language.

FIGS. 9A to 9C are diagrams for explaining a process in which the electronic device 100 provides translation information, according to an embodiment. Content that overlaps with FIGS. 2A to 2G will be briefly described or omitted.

According to one embodiment, the electronic device 100 may execute a first application and display a screen 900 of the first application. For example, referring to FIG. 9A , the electronic device 100 may display the home page of the first application on the screen 900 of the first application.

According to one embodiment, the electronic device 100 may provide translation information obtained together with the content of the first application. For example, referring to FIG. 9B , the electronic device 100 may display a speech bubble icon 920 next to text in the first language while displaying the homepage on the screen 900 of the first application. However, the shape of the icon displayed next to the text in the first language is only an example, and is not limited to the above.

According to one embodiment, when a user input for selecting the speech bubble icon 920 is obtained, the electronic device 100 may provide translation information corresponding to the selected speech bubble icon 920. For example, referring to FIG. 9C, the electronic device 100 displays a speech bubble icon 940 containing text in a second language that is a translated text of the first language corresponding to the selected speech bubble icon 920 in the first language. It can be displayed next to the text. However, the shape of the icon displayed next to the text in the first language is only an example, and is not limited to the above.

FIGS. 10A to 10D are diagrams for explaining a process in which the electronic device 100 provides translation information, according to an embodiment. Content that overlaps with FIGS. 2A to 2G will be briefly described or omitted.

According to one embodiment, the electronic device 100 may execute a first application and display a screen 1000 of the first application. For example, referring to FIG. 10A , the electronic device 100 may display a portion of the search result page when searching for “vitamin d” on the screen 1000 of the first application.

According to one embodiment, the electronic device 100 may provide the obtained translation information. For example, referring to FIG. 10B, the electronic device 100 may run the second application in the foreground and display translation information about the image on the screen 1020 of the second application.

According to one embodiment, the electronic device 100 may obtain a set input for the first area in the translation information. For example, referring to FIG. 10B, the electronic device 100 may obtain a user input for selecting an area 1040 where a specific product is displayed within the translation information displayed on the screen 1020 of the second application.

According to one embodiment, the electronic device 100 may check the second area within the first application corresponding to the first area based on the set input. For example, referring to FIG. 10C, the electronic device 100 checks the coordinate value of the area 1040 selected on the screen 1020 of the second application and selects the area 1040 on the screen 1000 of the first application. The area 1060 having the same coordinate value as the coordinate value of can be confirmed.

According to one embodiment, the electronic device 100 may provide a page corresponding to the second area. For example, referring to FIG. 10D, the electronic device 100 runs the first application in the foreground and displays a product detail page related to a specific product displayed in the area 1060 on the screen 1000 of the first application. can do.

Figure 11 shows a flowchart of a method for providing translation information by an electronic device, according to an embodiment. The foregoing description may apply to overlapping content.

In step S1100, the electronic device may check whether information about the image corresponding to the first application has been obtained.

According to one embodiment, when checking whether information about the image corresponding to the first application has been obtained, the electronic device may check whether a new image file has been created in the set storage.

According to one embodiment, when checking whether information about the image corresponding to the first application has been obtained, the electronic device may check whether the image capture function is activated.

In step S1120, when information about the image is obtained, the electronic device may obtain translation information about the image.

According to one embodiment, when acquiring translation information about an image, the electronic device acquires translation information based on information about the image using a second application running in the background while the first application is activated. can do.

According to one embodiment, the translation information may include a second text in a second language obtained based on the first text in the first language included in the image.

According to one embodiment, when obtaining translation information about an image, the electronic device may provide a user interface (UI) to inquire whether to translate the image.

According to one embodiment, when obtaining translation information for an image, the electronic device may provide a user interface to inquire about the language to translate the image into.

According to one embodiment, when obtaining translation information for an image, the electronic device may capture the image and check whether a first type of input requesting translation of the captured image has been obtained. At this time, the first type of input may be different from the second type of input requesting capture of an image.

In step S1140, the electronic device may provide the obtained translation information.

According to one embodiment, the electronic device may obtain a translation request for a target area within an image. Accordingly, the translation information may include translation information for the target area.

According to one embodiment, an electronic device can check a time varying area in an image. Afterwards, the electronic device can check whether to translate an area that changes with time and obtain a translation request for an area that changes with time. At this time, the translation information may include translation information about an area that changes over time.

According to one embodiment, an electronic device can check a time varying area in an image. At this time, translation information for areas that change over time may be displayed in a set method.

According to one embodiment, the image may include an image captured by scrolling a scrollable page. Accordingly, when providing translation information, the electronic device may provide translation information corresponding to the area within the page displayed on the screen along with the page as the page scrolls.

According to one embodiment, the image may include an image capturing a page of the first application. Accordingly, the electronic device can check whether the page has changed as the first application is updated, and as the page has been confirmed to have changed, it can check whether to translate the image corresponding to the changed page. As it is confirmed that the electronic device translates the image corresponding to the changed page, it can obtain information about the image corresponding to the changed page. Thereafter, the electronic device may obtain translation information about the image corresponding to the changed page and provide translation information about the image corresponding to the changed page.

According to one embodiment, the obtained translation information may be displayed on the content of the first application.

According to one embodiment, the electronic device may obtain a set input for a first area in translation information and, based on the set input, check a second area in the first application corresponding to the first area. Thereafter, the electronic device may provide a page corresponding to the second area.

According to one embodiment, when the sizes of the information about the image and the translation information are different, elements set in the translation information may be displayed in a set method.

FIG. 12 shows a block diagram of an electronic device 100 according to an embodiment.

According to one embodiment, the electronic device 100 may include a communication device 1220, a memory 1240, and a controller 1260. The electronic device 100 shown in FIG. 12 shows only components related to this embodiment. Accordingly, those skilled in the art can understand that other general-purpose components may be included in addition to the components shown in FIG. 12. Since the electronic device 100 may include content related to the above-described server, description of overlapping content will be omitted. In an embodiment, the communication unit may include one or more transceivers. Additionally, in an embodiment, the control unit may include one or more processors.

The communication unit 1220 is a device for performing wired/wireless communication and can communicate with external electronic devices. The external electronic device may be a terminal or server. In addition, communication technologies used by the communication unit 1220 include Global System for Mobile communication (GSM), Code Division Multi Access (CDMA), Long Term Evolution (LTE), 5G, Wireless LAN (WLAN), and Wireless-Fidelity (Wi-Fi). ), Bluetooth, RFID (Radio Frequency Identification), Infrared Data Association (IrDA), ZigBee, NFC (Near Field Communication), etc.

The control unit 1260 can control the overall operation of the electronic device 100 and process data and signals. The control unit 1260 may be composed of at least one hardware unit. Additionally, the control unit 1260 may operate by one or more software modules generated by executing program codes stored in the memory 1240. The control unit 1260 may include a processor and a memory, and the processor may execute program codes stored in the memory to control the overall operation of the electronic device 100 and process data and signals. Additionally, in an embodiment, the control unit 1260 may include at least one processor.

The control unit 1260 may check whether information about the image corresponding to the first application has been obtained, and if information about the image has been obtained, obtain translation information about the image, and provide the obtained translation information. .

Electronic devices according to the above-described embodiments include a processor, memory for storing and executing program data, permanent storage such as a disk drive, a communication port for communicating with an external device, a touch panel, keys, buttons, etc. It may include the same user interface device, etc. Methods implemented as software modules or algorithms may be stored on a computer-readable recording medium as computer-readable codes or program instructions executable on the processor. Here, computer-readable recording media include magnetic storage media (e.g., ROM (read-only memory), RAM (random-access memory), floppy disk, hard disk, etc.) and optical read media (e.g., CD-ROM). ), DVD (Digital Versatile Disc), etc. The computer-readable recording medium is distributed among computer systems connected to a network, so that computer-readable code can be stored and executed in a distributed manner. The media may be readable by a computer, stored in memory, and executed by a processor.

This embodiment can be represented by functional block configurations and various processing steps. These functional blocks may be implemented in various numbers of hardware or/and software configurations that execute specific functions. For example, embodiments include integrated circuit configurations such as memory, processing, logic, look-up tables, etc. that can execute various functions under the control of one or more microprocessors or other control devices. can be hired. Similar to how the components can be implemented as software programming or software elements, the present embodiments include various algorithms implemented as combinations of data structures, processes, routines or other programming constructs, such as C, C++, Java ( It can be implemented in a programming or scripting language such as Java), assembler, etc. Functional aspects may be implemented as algorithms running on one or more processors. Additionally, this embodiment may employ conventional technologies for electronic environment settings, signal processing, and/or data processing. Terms such as “mechanism,” “element,” “means,” and “configuration” may be used broadly and are not limited to mechanical and physical configurations. The term may include the meaning of a series of software routines in connection with a processor, etc.

The above-described embodiments are merely examples and other embodiments may be implemented within the scope of the claims described below.

Claims

In a method of providing translation information by an electronic device,

confirming whether information about an image corresponding to the first application has been obtained;

When information about the image is obtained, obtaining translation information about the image; and

A method of providing translation information, comprising providing the obtained translation information.
The method of claim 1, wherein the step of obtaining the translation information comprises:

A method of providing translation information, comprising the step of obtaining the translation information based on information about the image using a second application running in the background while the first application is activated.
The method of claim 1, wherein the translation information is:

A method of providing translation information, including a second text in a second language obtained based on a first text in a first language included in the image.
The method of claim 1, wherein the step of checking whether information about the image has been obtained includes:

A method of providing translation information, including the step of checking whether a new image file has been created in a set storage.
The method of claim 1, wherein the step of checking whether information about the image has been obtained includes:

A method of providing translation information, comprising determining whether an image capture function is activated.
The method of claim 1, wherein the step of obtaining translation information for the image comprises:

A method of providing translation information, including providing a user interface (UI) to inquire whether to translate the image.
The method of claim 1, wherein the step of obtaining translation information for the image comprises:

A method of providing translation information, including providing a user interface to inquire about the language into which the image is to be translated.
The method of claim 1, wherein the step of obtaining translation information for the image comprises:

Capturing an image and determining whether a first type of input requesting translation of the captured image has been obtained;

The method of providing translation information, wherein the first type of input is different from the second type of input requesting capture of an image.
According to claim 1,

The method for providing translation information further includes obtaining a translation request for a target area in the image,

A method of providing translation information, wherein the translation information includes translation information for the target area.
According to claim 1,

A method of providing translation information, further comprising the step of confirming a time varying area in the image.
The method of claim 10, wherein the translation information providing method includes:

Confirming whether to translate the area that changes with time; and

Further comprising obtaining a translation request for the area that changes with time,

A method of providing translation information, wherein the translation information includes translation information for the area that changes with time.
According to claim 10,

A method of providing translation information, wherein translation information for an area that changes with time is displayed in a set method.
According to claim 1,

The image includes an image captured by scrolling a scrollable page,

The step of providing translation information of the image includes providing translation information corresponding to an area within the page displayed on the screen as the page is scrolled together with the page.
According to claim 1,

The image includes a captured image of a page of the first application,

The method of providing the above translation information is:

checking whether the page has changed as the first application is updated;

When it is confirmed that the page has been changed, checking whether to translate an image corresponding to the changed page;

As the image corresponding to the changed page is confirmed to be translated, obtaining information about the image corresponding to the changed page;

Obtaining translation information about the image corresponding to the changed page; and

A method of providing translation information, including providing translation information for an image corresponding to the changed page.
The method of claim 1, wherein the obtained translation information is:

A method of providing translation information displayed on content of the first application.
According to claim 1,

Obtaining a set input for a first area within the translation information;

Based on the set input, confirming a second area within the first application corresponding to the first area; and

A method of providing translation information, further comprising providing a page corresponding to the second area.
According to claim 1,

When the size of the information about the image and the translation information are different, a set element in the translation information is displayed in a set method.
A non-transitory computer-readable recording medium that records a program for executing the method of claim 1 on a computer.
As an electronic device,

Ministry of Communications;

Memory; and

It includes a controller, wherein the controller includes:

Check whether information about the image corresponding to the first application has been obtained,

When information about the image is obtained, obtain translation information about the image,

An electronic device that provides the obtained translation information.