US20100131997A1

US20100131997A1 - Systems, methods and apparatuses for media integration and display

Info

Publication number: US20100131997A1
Application number: US12/463,020
Authority: US
Inventors: Howard Locker; Robert D. Dickinson, III; Blake C. Ramsdell; Michael Sievert
Original assignee: Lenovo Singapore Pte Ltd
Current assignee: Lenovo Singapore Pte Ltd
Priority date: 2008-11-21
Filing date: 2009-05-08
Publication date: 2010-05-27
Also published as: US8803665B2; US8898688B2; US20100131979A1; US9355554B2; US20100127884A1; US20100131986A1; US20100131847A1; US20100131363A1; US9406217B2; US20100188249A1

Abstract

The invention, according to at least one embodiment, broadly contemplates systems, methods and apparatuses for preparing media from a plurality of media sources for display. The media sources include, but are not limited to, component media sources and related media sources. Identification of media is supported, as is the selective combined display of media from the plurality of media sources utilizing an instruction set allowing for the selective modification of the plurality of media.

Description

CLAIM FOR PRIORITY

This application claims priority from U.S. Provisional Patent Application No. 61/116,972, which is fully incorporated by reference herein.

FIELD OF THE INVENTION

The invention is directed to systems, methods and apparatuses for managing multimedia data. More specifically, the invention is directed to systems, methods and apparatuses that provide identification and selective combined display of a plurality of media data.

BACKGROUND OF THE INVENTION

An increasing number of households now have high definition televisions (HDTVs) and broadband Internet connections. Consumers watch a significant amount of television on these HDTVs. Consumers are also increasingly accessing media content via the Internet, primarily on traditional desktop and laptop computers, as well as mobile computing devices and the like. These trends are very likely to continue as more HD programming becomes available, more consumers buy HDTVs, and more consumers switch to or obtain wired or wireless broadband Internet connections.
Consumers commonly use computing and mobile devices to augment their television watching experiences (such as when using a laptop to track election results or sports scores while watching related content on the television). This trend is also likely to continue, as consumers are constantly seeking out additional information related to a television program, even while watching the programming.
However, there is currently no conventional way for allowing a user to have a truly integrated television and Internet media experience. Even when watching television programs on an Internet-connected device (e.g. laptop, desktop or mobile computing devices), a truly integrated experience is lacking, as consumers must choose between watching the program or browsing the Internet. Consequently, consumers resort to switching between media devices (e.g. between television and laptop) or applications manually, in a time consuming and disjointed fashion.
Accordingly, the inventors have recognized a need for providing consumers interested in simultaneously and flexibly experiencing the best of available media with one integrated system that provides an improved display of various media sources.

SUMMARY OF THE INVENTION

The invention, according to at least one embodiment, broadly contemplates systems, methods and apparatuses for preparing media from a plurality of media sources for display. The media sources include, but are not limited to, component media sources and related media sources.
At least one embodiment of the invention broadly contemplates an electronic device that provides the necessary processing and instructions/commands to permit an overlay pass through device (OPD) to output an appropriate media stream for display of content derived from the plurality of media sources. The display of media content can include multiple modes. The modes are selectable by the user. The media content from the component media source(s) or the media content from the related media source(s) can pass through the system and take up the entire display, or content from each of the component media source(s) and/or the related media source(s) can be combined for display together.
In summary, one aspect of the invention provides an apparatus comprising: at least one processor; and a tangible program storage device configured to store a program of instructions that when executed by the at least one processor enable the apparatus to: provide an instruction set enabling selective modification of component media content and related media content for combined display; wherein the related media content comprises media content automatically obtained based upon an identification of the component media content.
Another aspect of the invention provides a computer implemented method comprising: providing an instruction set enabling selective modification of component media content and related media content for combined output to a display device; wherein the related media content comprises media content automatically obtained based upon an identification of the component media content.
A further aspect of the invention provides a tangible program storage device, readable by machine, embodying a program of instructions that when executed by a processor of the machine enable the machine to: provide an instruction set enabling selective modification of component media content and related media content for combined output to a display device; wherein the related media content comprises media content automatically obtained based upon an identification of the component media content.
Another aspect of the invention provides an apparatus comprising: a microcontroller subsystem configured to: capture identification information from component media content; send the identification information to an electronic device; and accept an on or more instructions from an electronic device enabling combined display of component media content and related media content; wherein the related media content comprises media content automatically obtained based upon an identification of the component media content.
For a better understanding of the present invention, together with other and further features and advantages thereof, reference is made to the following description, taken in conjunction with the accompanying drawings, and the scope of the invention will be pointed out in the appended claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a block diagram of a computer system.

FIG. 2A-C illustrates a media management system.

FIG. 3 illustrates an OPD.

FIG. 4 illustrates a media management system.

FIG. 5 illustrates a combination and output of media from a plurality of media sources.

FIG. 6 illustrates a combined display of media from a plurality of media sources.

FIG. 7 illustrates a method of identifying and modifying media data for combined display.

DETAILED DESCRIPTION OF THE INVENTION

It will be readily understood that the components of the present invention, as generally described and illustrated in the figures herein, may be arranged and designed in a wide variety of different configurations in addition to the described presently preferred embodiments. Thus, the following more detailed description of the embodiments of the present invention, as represented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected presently preferred embodiments of the invention.
Reference throughout this specification to “one embodiment” or “an embodiment” (or the like) means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. Thus, appearances of the phrases “in one embodiment” or “in an embodiment” or the like in various places throughout this specification are not necessarily all referring to the same embodiment.
Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the invention. One skilled in the relevant art will recognize, however, that the invention can be practiced without one or more of the specific details, or with other methods, components, materials, etc. In other instances, well-known structures, materials, or operations are not shown or described in detail to avoid obscuring aspects of the invention.
The illustrated embodiments of the invention will be best understood by reference to the drawings. The following description is intended only by way of example, and simply illustrates certain selected presently preferred embodiments of the invention as claimed herein.
According to at least one embodiment, the invention provides a media management system that prepares media from a plurality of media sources for display. The media sources include, but are not limited to, component media sources and related media sources.
The component media sources can include but are not limited to cable and satellite television receivers, DVD players or other peripheral devices, and a system computer that plays content derived from the Internet. The component media sources provide a variety of media such as HD cable or satellite television programs, movies, music, etc.
The related media sources include but are not limited to local and remote databases providing related media content. The related media is related to the component media in some relevant way. For example, related media can be obtained utilizing an identification result of the component media. The related media can include pre-selected content stored in a local or remote database and/or related content obtained from one or more Internet sources.
Co-pending and commonly assigned U.S. patent application Ser. No. 12/415,638, incorporated by reference here, provides some useful information regarding a media management system that identifies source media content and provides targeted, related media based upon the identification.
At least one embodiment of the invention provides an electronic device that prepares the media content from a plurality of sources for appropriate display by providing the necessary processing and instructions/commands to an OPD. The display of media content can include multiple modes that are selectable by the user. The media content from the component media source(s) or the media content from the related media source(s) can pass through the system and take up the entire display, or media content from the component media source(s) and/or the related media source(s) can be combined for display together, as described further below.
Referring now to FIG. 1, there is depicted a block diagram of an illustrative embodiment of a computer system 100 (or simply “computer”). The illustrative embodiment depicted in FIG. 1 may be a notebook computer system, such as one of the ThinkPad® series of personal computers sold by Lenovo (US) Inc. of Morrisville, N.C. or a workstation computer, such as the Thinkstation®, which is also sold by Lenovo (US) Inc. of Morrisville, N.C. As is apparent from this description, however, the present invention is applicable any data processing system or other electronic device suitably configured to enable the functions described herein.
As shown in FIG. 1, computer system 100 includes at least one system processor 42, which is coupled to a Read-Only Memory (ROM) 40 and a system memory 46 by a processor bus 44. System processor 42, which may comprise one of the processors produced by Intel Corporation or a processor from AMD Corporation, is a general-purpose processor that executes boot code 41 stored within ROM 40 at power-on and thereafter processes data under the control of operating system and application software stored in system memory 46. System processor 42 is coupled via processor bus 44 and host bridge 48 to Peripheral Component Interconnect (PCI) local bus 50.
PCI local bus 50 supports the attachment of a number of devices, including adapters and bridges. Among these devices is network adapter 66, which interfaces computer system 100 to LAN 10, and graphics adapter 68, which interfaces computer system 100 to display 69. Communication on PCI local bus 50 is governed by local PCI controller 52, which is in turn coupled to non-volatile random access memory (NVRAM) 56 via memory bus 54. Local PCI controller 52 can be coupled to additional buses and devices via a second host bridge 60.
Computer system 100 further includes Industry Standard Architecture (ISA) bus 62, which is coupled to PCI local bus 50 by ISA bridge 64. Coupled to ISA bus 62 is an input/output (I/O) controller 70, which controls communication between computer system 100 and attached peripheral devices such as a keyboard, mouse, and a disk drive. In addition, I/O controller 70 supports external communication by computer system 100 via serial and parallel ports. The USB Bus and USB Controller (not shown in FIG. 1) are part of the Local PCI controller (52).
Referring now to FIG. 2A, a high level view of a media management system (203) according to one embodiment of the invention is provided. As shown, the OPD (202) is housed within a system case (200 a) along with the computer (200). The OPD (202) could be for example located on the motherboard or a daughter card. The OPD (202) receives input(s) from a least one component media source (201 a), such as a cable television box, though multiple component media sources may provide a plurality of media inputs (e.g. Media 1 . . . Media N) from a plurality of component media sources (201 a, 201 b). The OPD (202) is configured to receive media content (e.g. Media C) input from the computer (200), which can include but is not limited to content derived from an Internet source or a networked database (204) storing pre-selected content. The OPD (202) is configured to provide the media output to a display (269).
The OPD (202) is preferably configured within the system (203) to perform as little processing as possible. Rather, the computer (200) is preferably configured to handle the processing necessary to ascertain the media content input from component (201 a) and related sources (204). The computer (200) instructs the OPD (202) as to how to combine the media inputs for ultimate output to a display (269) via a suitable communication link (e.g. control bus). The OPD (202) is preferably implemented as a hardwired video overlay chip receiving the calculations (instructions) from the computer (200) on how to, e.g. overlay the media content from a plurality of media sources.
The computer (200) also provides the related media (Media C) to the OPD (202) via a suitable connection (e.g. a video bus). The connection can be any suitable one that supports video. Examples include DisplayPort, DVI, a USB bus with video encoded on USB, etc. It should be understood that an embodiment of the invention can utilized different connections, e.g., the connection from the computer (200) to the OPD (202) may be DisplayPort while the video stream into the OPD (202) from the component media source (201) may be in HDMI. It is only essential to ensure the proper transmit and receive solutions to convert the video inputs/outputs are provided. In like manner, the control bus can be any suitable control bus, such as USB bus.
Referring to FIG. 2B, an alternative implementation is illustrated according to one embodiment of the invention. As shown, the OPD (202) is located external to the computer (200) and receives inputs from the computer via suitable connections. In this example, the computer (200) sends instructions and media to the OPD (202) via cables (e.g. USB and HDMI). Thus, the OPD (202) is housed within its own docking station (200 b). Although not preferred, the computer (200) may control more than one OPD (202) in this configuration with appropriate wiring.
FIG. 2C shows another alternative implementation according to one embodiment of the invention. As shown, the OPD (202) is again located external to the computer (200), as in FIG. 2B, in a docking station (200 b). In FIG. 2C, however, communication between a single computer (200) and one or more OPD (202) (only one shown for simplicity) is accomplished utilizing a suitable wired or wireless connection, for example Ethernet or WiFi. Thus, computer (200) may instruct one or more OPD (202), as in a home network environment.
FIG. 3 illustrates in a block diagram the OPD (302) in the context of data flow within the system. As above, there may be a plurality of component media sources (301 a, 301 b) providing a plurality of media inputs (e.g. Media 1 . . . Media N), as illustrated by the parallel input Media N and subsystems (302 a, 302 b). The OPD (302) contains at least one microcontroller subsystem (302 a). The microcontroller subsystem (302 a) performs the modifications (e.g. downscale and rotate) necessary for appropriate combined display (e.g. three dimensional (3D) display, as shown and described herein). As an exemplary modification, a downscale could be accomplished for example by using discrete cosine transform (DCT)-domain downscaling algorithms or any industry standard method to ensure the best possible video quality for the scaled-down frame. As another exemplary modification, a rotation can be accomplished with any industry standard method of changing a 2D image (original input) into a 3D image with the correct Z rotation. Other modifications are of course possible and are considered to fall within the scope of the invention as claimed. Non-limiting and exemplary instructions for modification are provided below.
As illustrated in FIG. 3, Media 1 is input from a component media source (301 a) (e.g. cable TV box providing HDMI input). A receiver (e.g. HDMI REC. (304)) receives Media 1. Media 1 can be digitized (if input in analog format) and stored in a temporary buffer (e.g. Temp Frame Buffer (305)). The microcontroller subsystem (302 a) then appropriately modifies Media 1 for display in one of a number of selectable modes, as instructed by the computer (300) and further discussed below. The exemplary illustrated modifications of scaling and z-rotation would be utilized in a combined display mode, as discussed below. Ultimately, modified Media 1 is placed in an output buffer (e.g. output frame buffer (307)) for transmission via a transmitter (e.g. HDMI transmitter (309)) for display on suitable display device(s).
As above, any of a number of other media from a variety of additional sources may be chosen for combined display. As illustrated in FIG. 3, Media C (e.g. an HDMI video stream from the computer (300)) can be combined with (modified) Media 1 for display. Preferably Media C is related content automatically identified, obtained and prepared for selective display, as discussed below. Media C is preferably processed within the video graphics subsystem of the computer (300) and is combined with other media inputs (e.g. Media 1 . . . N) by the OPD (302) using methods of rendering multiple windows from different sources. Each window is placed at the correct place within the total frame utilizing an appropriate instruction from the computer (300), as discussed herein. As such, Media C will be received by an HDMI receiver (306) and stored in the output frame buffer (307) along with (modified) Media 1. Media C will undergo modification as well, such that the appropriate combined display (i.e. Media 1+Media C in this example) is obtained.
Commands to the OPD (302) are sent from the computer (300) to the OPD (302) microcontroller subsystem (302 a) over an appropriate connection, e.g. a USB bus. The OPD (302) is configured to send status and other relevant information about the input media (e.g. Media 1) from the component media source (301 a) to the computer (300) over the USB bus. This provides the computer (300), with the necessary information for identification of related media and calculation of appropriate instructions for combined display.
The media from a plurality of sources will either be displayed as full screen and/or a percent of the screen depending on the commands from the computer (300) across the USB bus (i.e. corresponding to pass through display or combined display, respectively). The media content from a particular source can be positioned to start anywhere within the total frame (i.e. the overall final output display). It can also be “rotated” in the Z-axis, giving a 3D character to the displayed media, etc. The computer (300) ascertains the total resolution of the output display, so it is capable of determining the column and row for a start position based on the desired scale down and center offset (i.e. the appropriate modification corresponding to the desired output display).
A non-limiting example of a USB command from the computer (300) to the OPD (302) is as follows:
SizeHDMI_Input1 Video(PercentTotalFrame, ColStart, RowStart, zAxisRotation)
Where:
PercentTotalFrame=Percent reduction compared to full frame
Col Start and Row Start=Exact col/row where the scaled down image starts
zAxis Rotation—Angle in the Z axis for the scaled down image (rotation from the col/row start position). Embodiments of the invention are able to support rotation from either the left edge or right edge of the video. As non-limiting examples, if the number is positive the rotation is anchored from the left edge of the video (or first column of the video); if the number is negative the rotation is anchored from the right edge of the video (or last column of the video).
It is presently preferred that the computer (300) can simply inform the OPD (302) of the four corner positions (i.e. column and row) for the particular media (e.g. Media input 1 and/or Media C) within the overall output display frame. The four corners define the size and 3D rotation of the video that is ultimately displayed. The OPD (302) will then transform the full-scale video into the correct size and rotation and place it in the correct location in the output frame buffer (307), as illustrated further in FIG. 5.
A non-limiting example of such a “four corners” command is:

- SizeHDMI_Input1 Video(upper start position, upper end position, lower start position, lower end position)
  An exemplary “four corners” instruction for Media 1 (assuming Media 1 is HDMI video input, e.g. of a popular television program) from the computer (300) to the OPD (302) would be:

SizeHDMI_input1Video=(50, 960, 300, 30)
50% reduction, Start at Col 960 and Row 300 with a 30 degree angle
The OPD (302) performs the following functions to the HDMI Input 1 video stream
1) Downscale—
2) Z-Axis rotation—
FIG. 4 illustrates an overview of the handling of a plurality of media inputs within the media management system (403). A first media input (e.g. Media 1, input from a component media source (401)) is received at an HDMI receiver (404) and stored in a temporary buffer (405), with optional digital conversion as necessary. If Media 1 is to pass through the system and thereby fill the display (469), Media 1 passes through to a multiplexer (412), where it would normally be combined with other media inputs (e.g. Media C), and is passed to an HDMI transmitter (408) where it is buffered and subsequently output to the display (469).
Alternatively, Media 1 can be passed to the microcontroller subsystem (402 a) of the OPD (402) for modification and identification. Buffered Media 1 is appropriately modified (e.g. sized and rotated) by a processor (410) utilizing appropriate formatting instructions from the computer (400). Information necessary for identification of Media 1 is extracted and sent to an appropriate location for identification, e.g. on-chip identification subsystem (409). The identification information can optionally be transmitted to either the computer (400) or on to a remote device for further identification analysis, as desired.
Upon appropriate modification, as described herein, Media 1 is passed to frame buffer (407) for combination passed to multiplexer (412) with additional media, which could be media from a variety of sources (e.g. Media C provided by computer (400) and/or from another component media source (201)). As discussed, an additional media input (e.g. Media C provided by computer (400)) is provided. This additional media, handled similar to Media 1, is either provided to the frame buffer (407) with appropriate modification instructions for combination at multiplexer (412) with Media 1, or is passed through the system to HDMI transmitter (408) to fill the display (469). It should be noted that use of multiple buffering of a frame (e.g. at 405, 407 and 408) prevents tearing and artifacts, providing the best video quality.
Embodiments of the invention can also extend these concepts to protected content such as HDCP. In this case the OPD (402) has a HDCP key which can be utilized to support encryption and decryption. All HDMI sources use the HDCP key to send protected content to the OPD (402). The OPD (402) then unprotects the content prior to placing it into a frame buffer (e.g. 405). After the modification (e.g. downscale and rotate) is complete, the OPD (402) will then re-encrypt the HDMI stream using the HDPC key and send the protected content to the display (469). The decryption of protected content can be performed at HDMI Receiver (404) and encryption can be performed at HDMI transmitter (408). Once again, used HDMI and HDCP are utilized as an example but the invention can be configured to handle any video stream with any type of content protection.
A non-limiting example of formatting and displaying a plurality of media inputs is illustrated further in FIG. 5. As shown, Media 1 (e.g. HDMI input 1 from component media source (501)) is positioned by OPD (502) as shown utilizing the “four corners” instructions provided by the computer (500) and stored in a frame buffer (505). Thus, Media 1 is located in a particular position (“A”) within the overall frame at (505). Similarly, Media C (e.g. HDMI from computer (500)) is formatted and stored within a buffer (511). Media C is located in a particular position (“B”) utilizing a similar “four corners” instruction. A video multiplexer (512) (mux) combines the outputs, with Media 1 and Media C located at appropriate positions for combined output to display (569). As illustrated, each media source (supplemented by computer instructions) determines what area of the final combined output it occupies. The rest of the area (video content) can be filled with a “known data value”. The video mux (512) combines all of the inputs by ignoring the “known data value” from each of the inputs.
Handling audio is similar to the handling of video, as described above. The computer tells the OPD which audio to pass to the combined (HDMI) output. Audio preferably will be changed on a frame-by-frame basis also using an audio USB command from the computer to the OPD, for example:
HDMIInputAudio(Input 1, Input 2, Input 3, etc.)
It should be understood that the above solutions are non-limiting examples of presently preferred embodiments and that embodiments of the invention can utilize many different commands to provide the appropriate information to the OPD (502) to determine where to start the video, how to resize it, and how to rotate it, etc, in order to format the appropriate display mode selected by the user.
FIG. 6 illustrates an exemplary combined display of media from a plurality of media inputs. As shown, the display (669) presents the output media from the OPD. The display (669) can be selected from any suitable display device, including computer monitors, flat panel displays, HDTVs, etc. It is presently preferred that the display (669) is a large (e.g. 50 inch) HDTV, providing ample room for content derived from multiple media sources to be accommodated easily on the display (669).
As shown, multiple media contents (620 a, 620 b, and 620 c) from a variety of sources (e.g. web pages from the computer and HDMI 1 media from a component media source, respectively) are presented for combined display. In this non-limiting example, HDMI 1 could be a popular 1080p HD television show from a cable TV box. The web sites could be web sites related to the popular HD television show, automatically obtained by the computer upon identification of the popular HD television show. Preferably, the resolution of media from the various sources is the same, as this simplifies the instruction set. The computer, which provides formatting instructions to the OPD for the combined display, is also configured to provide appropriate background (630) for the display. The computer is also configured to provide additional information (640 a, 640 b and 640 c) regarding the displayed media. For example, computer can provide identification information (640 c) regarding the presentation window containing HDMI 1 (620 c)—e.g. the title of the show and particular episode, air date, etc. Likewise, additional information (e.g. web page titles) can be presented at 640 a and 640 b, corresponding to areas above (a plurality of) additional related media (e.g. web sites related to the show).
FIG. 7 illustrates a non-limiting and exemplary identification procedure for gathering related media content. As shown, the system also supports identification of media content, such as Media 1 from a component media source (701). It should be understood, however, that the invention can employ identification techniques to any media source, including media provided by the computer (700).
The system can be configured to provide identification via several techniques, including identification from program guide information, closed captioning or meta data associated with the media stream or identification via digital fingerprinting, which can include video and audio digital fingerprinting. Video fingerprinting is presently preferred for identification. Video fingerprinting is a technique for automatically analyzing the video stream(s) to capture and extract key items of information from the video stream that are useful for identification of the content of the video stream(s). A fingerprint comprises a modified frame of the media (e.g. Media 1). Identification provides criteria for generating relevant content (from the worldwide web or other sources) that is related to the video stream. It should be noted that identification can be accomplished in the OPD itself, in the computer, or via a network connected device.
In this example, Media 1 (e.g. a popular HD TV show) is input to receiver (704) from component media source (701)—e.g. a cable TV box. As above, the OPD (702) will buffer Media 1 at temporary buffer (705) and, if it is not in pass-through mode, will apply modification per the instructions received from computer (700) such that Media 1 can be output to an output frame buffer (707) for combination at multiplexer (712) and thereafter passed through to display (769).
The OPD (702) can capture necessary information (710) (e.g. frame captures, closed captioning data, meta data, program guide information, etc.) to allow computer (700) to perform identification, such as video fingerprinting. Video fingerprinting could be performed locally but it is presently preferred that it is done at a remote location (e.g. via an Internet connection through computer (700)). It should also be noted that other identification techniques could be employed, and these may also be performed either locally or at a remote location.
Video fingerprinting will allow for identification (e.g. the system will recognize Media 1 is a popular TV show) by identifying key characters or other unique aspects available within the Media 1 data. The temp frame buffer (705) contents are sent over a USB bus from the OPD (702) to the computer (700) every N frames. The computer (700) or a remote networked device can then run any popular video fingerprinting techniques to complete the identification, such as that available form Digital Smiths®, www.digitalsmits.com.
Identification enables retrieval and input of related media (e.g. Media C) into a receiver (704). The related media could be acquired, for example, form a database, either local or remote, which has Internet derived content and/or pre-selected content. Thus, the computer (700) could utilize the identification result to automatically conduct an Internet query using a search engine such as Google® and return the results as related media.
The related media is optionally modified (as necessary per the selected mode of display). For example, if the user has selected a combined display mode, Media C will be modified (e.g. utilizing a “four corners” instruction) as appropriate for combined display with Media 1. Media 1 and Media C will then be stored into an output frame buffer (707). This allows for combining the two media sources at multiplexer (712) and output to a display device (769).
In brief recapitulation, a media management system provides a user with a selectable display of one or more media from a plurality of media sources, including relevant additional media automatically obtained based on an automatic identification of a first media. The user is enabled to select multiple modes of display. Accordingly, the user can select pass-through modes whereby a particular media source fills the entire display, effectively hiding the remaining media. Alternatively, the user may select any of a variety of available media for combined display.
It should be understood that many of the functional characteristics of the inventive system described in this specification may be implemented as modules. Modules may include hardware circuits such as one or more processors with memory, programmable logic, and/or discrete components. The hardware circuits may perform hardwired logic functions, execute computer readable programs stored on tangible storage devices, and/or execute programmed functions. The computer readable programs may in combination with a computer system and the other described elements perform the functions of the invention.
It will be readily understood by those having ordinary skill in the art that embodiments of the present invention may take the form of an entirely hardware embodiment or an embodiment containing both hardware and software elements. An embodiment that is implemented in software may include, but is not limited to, firmware, resident software, microcode, etc.
The computer readable programs may be stored in tangible computer/machine-readable (apparatus readable) medium. Examples of a computer/machine-readable medium include a semiconductor or solid state memory, magnetic tape, a removable computer diskette, a random access memory (RAM), a read-only memory (ROM), a rigid magnetic disk and an optical disk. Current examples of optical disks include compact disk-read only memory (CD-ROM), compact disk-read/write (CD-R/W) and DVD.
Accordingly, elements of the present invention may be implemented on at least one computer running suitable software programs. These may also be implemented on at least one Integrated Circuit or part of at least one Integrated Circuit. Thus, it is to be understood that the invention may be implemented in a combination of both hardware and software. Again, computer/machine-readable programs may in combination with a computer system perform the functions of the invention.
This disclosure has been presented for purposes of illustration and description but is not intended to be exhaustive or limiting. Many modifications and variations will be apparent to those of ordinary skill in the art. The embodiments were chosen and described in order to explain principles and practical application, and to enable others of ordinary skill in the art to understand the disclosure for various embodiments with various modifications as are suited to the particular use contemplated.
In the drawings and specification there has been set forth a preferred embodiment of the invention and, although specific terms are used, the description thus given uses terminology in a generic and descriptive sense only and not for purposes of limitation.
If not otherwise stated herein, it is to be assumed that all patents, patent applications, patent publications and other publications (including web-based publications) mentioned and cited herein are hereby fully incorporated by reference herein as if set forth in their entirety.

Claims

1. An apparatus comprising:

one or more processors; and

a tangible program storage device configured to store a program of instructions that when executed by the one or more processors enable the apparatus to:

provide one or more instructions relating to combined display of component media content and related media content;

wherein the related media content comprises media content automatically obtained based upon an identification of the component media content.

2. The apparatus according to claim 1, further comprising a microcontroller subsystem configured to selectively modify the component media content according to the one or more instructions.

3. The apparatus according to claim 1, wherein the one or more instructions comprises an instruction indicating appropriate scaling for at least one of the component media content and the related media content.

4. The apparatus according to claim 3, wherein the one or more instructions further comprises an instruction indicating appropriate rotation for at least one of the component media content and the related media content.

5. The apparatus according to claim 2, wherein the one or more instructions comprises a four corners instruction indicating an appropriate location within an output frame for at least one of the component media content and the related media content.

6. The apparatus according to claim 2, wherein the microcontroller subsystem is configured to provide identification information related to the component media content.

7. The apparatus according to claim 6, wherein the identification information comprises a portion of the component media content sufficient to enable digital fingerprinting.

8. The apparatus according to claim 6, wherein the microcontroller subsystem is configured to perform on chip identification of the component media content.

9. The apparatus according to claim 1, wherein the one or more instructions further enables selective display in a first pass through mode wherein only the component media content is displayed.

10. The apparatus according to claim 1, wherein the one or more instructions further enables selective display in a second pass through mode wherein only the related media content is displayed.

11. The apparatus according to claim 1, wherein the one or more instructions further enables selective display in a combined mode wherein both the component media content and the related media content are displayed.

12. The apparatus according to claim 1, further comprising an output frame buffer configured to store on or more of modified component media content and related media content.

13. The apparatus according to claim 1, further comprising a video multiplexer configured to combine one or more of modified component media content and related media content and disregard known data values contained in outputs relating to one or more of the modified component media content and the related media content.

14. A computer implemented method comprising:

utilizing a processor to provide one or more instructions relating to combined display of component media content and related media content;

15. The computer implemented method according to claim 14, further comprising utilizing a microcontroller subsystem to selectively modify the component media content according to the one or more instructions.

16. The computer implemented method according to claim 14, wherein the one or more instructions comprises an instruction indicating appropriate scaling for at least one of the component media content and the related media content.

17. The computer implemented method according to claim 16, wherein the one or more instructions further comprises an instruction indicating appropriate rotation for at least one of the component media content and the related media content.

18. The computer implemented method according to claim 15, wherein the one or more instructions comprises a four corners instruction indicating an appropriate location within an output frame for at least one of the component media content and the related media content.

19. The computer implemented method according to claim 15, further comprising utilizing the microcontroller subsystem to provide identification information related to the component media content.

20. The computer implemented method according to claim 19, wherein the identification information comprises a portion of the component media content sufficient to enable digital fingerprinting.

21. The computer implemented method according to claim 19, further comprising utilizing the microcontroller subsystem to identify the component media content.

22. The computer implemented method according to claim 14, wherein the one or more instructions further enables selective display in a plurality of modes; the plurality of modes comprising:

a first pass through mode wherein only the component media content is displayed;

a second pass through mode wherein only the related media content is displayed; and

a combined mode wherein both the component media content and the related media content are displayed.

23. A tangible program storage device, readable by machine, embodying a program of instructions that when executed by a processor of the machine enable the machine to:

provide one or more instructions relating to combined display of component media content and related media content on a display device;

24. An apparatus comprising:

a microcontroller subsystem configured to:

capture identification information from component media content;

send the identification information to an electronic device; and

accept an on or more instructions from an electronic device enabling combined display of component media content and related media content;

25. The apparatus according to claim 24, wherein the microcontroller subsystem is configured to selective display the component media content and the related media content according to the one or more instructions in a plurality of modes; the plurality of modes comprising: