KR101041515B1 - Methods and systems for controlling computers or linking to Internet resources from physical and electronic objects - Google Patents

Abstract

Physical or electronic objects are encoded with identifiers and served to trigger appropriate responses to the object from computer systems that encounter these objects. Since the encoding is steganographic (eg by digital watermarks), the presence of these identifiers is not apparent to the person encountering the objects. An example application is a computer system that looks at a printed magazine advertisement 20 and initiates a link to the corresponding internet page. In one such embodiment, the computer system detects an identifier encoded in the advertisement, passes the identifier to a remote database, receives a corresponding Internet address 18a, 18b, 18c from the database 17, and also a browser. Is connected to the addresses 18a, 18b, and 18c. The same configuration can be used for on-line orders from printed product catalogs. Another application is a computer system that looks at a printed spreadsheet 20 and retrieves an electronic version of the same document from disk storage for editing.

Description

Methods and systems for controlling computers or linking to internet resources from physical and electronic objects}

Methods and systems for controlling computers or linking to Internet resources from physical and electronic objects.

"Bedoop". This is a sound that people can hear and relax with a magazine ad in front of a desktop camera. Curiously, a marketing and sales web site associated with the advertisement is displayed on a computer. More info? Do you want to buy now? Will you see the entire product line? There is no problem.

"Bidoop". The same sounds when the same people put their credit cards in front of the desktop camera. On the fly, the product displayed on the web page is purchased. Behind the screen, a secure purchase link is initiated, sending all necessary information to the seller. If the credit card is turned clockwise, the buyer chooses the next day's delivery.

This is the preferred embodiment described herein. Although this example is somewhat specific, it nevertheless suggests that if an input device such as a digital camera becomes a universal user interface device with the intuition that it can be a powerful rival for a mouse and keyboard, a vast amount of applications are possible. .

One feature of certain embodiments is that the object or document so scanned includes digital information that can be quickly read and acted upon by a properly configured device, computer, or instrument. This embodiment envisions that this digital information is aesthetically hidden in the objects. These objects are pre-marked in advance using a wide range of data encoding techniques, such as digital watermarking.

Although this characteristic of the technology is focused on planar object applications where digital information is sometimes integrated into objects so that they are not perceptible, this does not imply this limitation. Objects can be information that is inherently three-dimensional and visually clear and / or pre-existing (ie, not inserted "pro-actively" or "digital" by itself). have. Such modifications are accompanied by other implementations. Similarly, while most of these descriptions are focused on objects in the form of digital messages attached to them, some features of the technique can be applied to objects that do not have them, which are further described in the prior art of pattern recognition and gesture input. It can be introduced in combination with this technology to form a wide array of applications.

As will be apparent, the technique is not limited to systems using optical inputs and encoded images. The corresponding technique may also be used with coded audio. That is, a physical or electronic "object" can utilize the principles described in detail herein.

"Bidoop". The sound, like a fridge, adds items to a cumulative delivery list with a simple camera / processor unit / net connection when a 10-year-old is in a small store with an empty milk carafe. When computer haters take skeptical first steps toward the World Wide Web, they can continue to ring in Internet cafes. The sound is heard when a regular customer comes to a fast food restaurant with a sandwich card that checks his meal, hoping for a siren to pay $ 500 to the lucky customer of the week. There are many blue scenarios.

As such, this feature of the present technology relates to a powerful modern user interface to a computer. These new user interfaces are expanding around the world every day in ways that a mouse and keyboard can never do. By allowing objects to communicate identities and functions to participating devices every day, not only does a whole new dimension be provided on the World Wide Web, but basic home and office computing can be sold with some advances.

According to one aspect, the invention includes a method of processing data in a computer system, comprising: (a) using an application program to construct an electronic version of a document; (b) printing the document on paper, the printing comprising being represented by a machine readable indicia encoding a plurality of bits of auxiliary data; And (c) storing a plurality of bits of auxiliary data associated with data identifying a location at which an electronic version of the document is stored.

According to another aspect, the present invention includes a data processing method in a computer system, comprising: (a) presenting a printed document to an optical capture device; (b) processing the image data produced by the device to decode the encoded multiple bits of data; (c) launching a software application corresponding to the printed document based on the decoded multiple bits of data; And (d) using the software application to open the electronic version of the document.

According to yet another aspect, the invention includes a method of operating a computer including an operating system with a registry database that associates a particular type of data with a particular software program corresponding thereto, the method further comprising: (a) Providing a frame of image data; (b) decoding a plurality of bits of identifier data from the image data; (c) consulting a registry database to identify a software program corresponding to the identifier data; And (d) invoking and opening the identified software program.

According to another feature, the invention comprises a greeting card having a substrate with a visually recognizable print witness, wherein the card is encoded with a plurality of bits of binary data that can be decoded by an image processing device, the card Is used to connect a computer to a web site on which an image, video, and / or audio presentation corresponding to.

According to yet another aspect, the present invention includes a method for providing a customized greeting, comprising: (a) providing a greeting card having encoded multiple bits of data; (b) customizing a website display corresponding to the card; (c) providing the card to the recipient; (d) decoding the encoded plurality of bits of data from the card; And (e) in response to the decoded multiple bits of data, presenting a website representation to the recipient.

According to another feature, a method of printing a magazine comprises: (a) processing an electronic representation of an advertisement to encode a plurality of bits of data; (b) printing the advertising page according to the electronic display to produce an encoded advertising page; And (c) enclosing the page into a magazine, wherein the multi-bit data operates to identify an entry in a database, the database entry identifying the internet address of the web page associated with the stored advertisement. Have

According to yet another aspect, the present invention includes a promotion method, comprising: (a) encoding a print advertisement to hide multiple bits of data; (b) processing the print advertisement to extract multiple bits of data; And (c) using at least some of the extracted multiple bits of data to connect an internet web browser to a web site that provides information related to the product or service promoted by the print advertisement to the consumer.

According to another feature, the invention includes a method of determining a consumer response to a print advertisement, comprising: (a) encoding a first print advertisement with first data; (b) encoding the second print advertisement with second data different from the first data; (c) the first and second data providing an identifier such that a consumer device can connect to a web page associated with the advertisement; And (d) monitoring link traffic due to each of the identifiers, thereby determining a consumer response to the advertisement.

According to another feature, the present invention includes a promotion method, comprising: (a) providing a selected object from a list consisting of a retail product, a package for the retail product, or a print advertisement within the perspective of the optical sensor device; (b) capturing optical data corresponding to the object; (c) decoding the plurality of bits of digital data from the optical data; (d) communicating at least some of the decrypted data to a remote computer; And (e) determining at the remote computer whether a prize should be awarded in response to providing the decrypted data.

According to another feature, the invention includes a method of interacting with a magazine using a computer comprising an internet web browser, comprising: (a) providing a peripheral device having a sensor; (b) placing the peripheral device in a first adjacent ad in the magazine to connect the web browser to the first internet address; And (c) placing the peripheral device in a second ad adjacent to the magazine to connect the web browser to the second internet address.

According to another feature, the invention comprises a computer peripheral and a method of use, the peripheral being used in combination with a computer system having an internet browser associated therewith, the peripheral being: (a) locked in the palm of the user and entered into the medium; A housing adapted to make; (b) an optical sensor having at least one sensing element for producing optical data; (c) a lens for imaging the medium with a sensor, the method of use comprising: (a) introducing a peripheral to a portion of the printed advertisement; (b) processing the optical data to decode the multiple bits of information encoded in the advertisement; And (c) using said multiple bits of information to connect an internet browser to an internet web page associated with said advertisement.

According to another feature, the present invention includes an electronic commerce method, which comprises: (a) providing a print catalog comprising an image of an article provided for sale by a seller when the image is encoded with multiple bits of binary data; step; (b) optically sensing the image to produce corresponding optical data; (c) decoding the encoded data from the optical data; And (d) electronically ordering articles from the seller using the decrypted data. The order may use initial stored consumer profile information (eg, clothes size data) and the encoding may be steganographic.

According to another feature, the invention comprises a wireless telephone handset comprising a microphone, a modulator, and an RF amplifier, the device operating to receive an audio and transmit an RF signal conveying the audio modulation, the handset being optical An optical sensor for making data, a lens for imaging the object with the sensor, and a decoder for decoding the multiple bits of identifier data carried by the digital watermark or barcode in the object.

According to another feature, the present invention includes an image-based network navigation method that allows a user to connect to a remote computer, comprising: (a) detecting encoded data from a print object; (b) connecting to a remote computer via a network according to the encoded data; And (c) providing the user's zip code to the remote computer.

According to yet another aspect, the present invention provides a method for detecting an object, comprising: (a) detecting an object identifier from a first object; (b) transmitting the first object identifier from a first device to a second device; (c) in response, at the second device, identifying address information corresponding to the first object identifier and transmitting it to the first device; (d) initiating a link from a first device according to the address information; (e) at the second device, identifying additional objects related to the first object, identifying additional address information corresponding to the additional object, and further transmitting the additional address information to the first device; And (f) storing the additional address information in a memory at the first device, wherein if an object included in the identified additional object is detected by the first device, the corresponding address information is associated with the second device. The memory can be recovered from the first device without communicating interference delays.

According to another feature, the present invention includes a detector of machine readable data and a software program used in association with the machine readable data, the apparatus being operable to transmit a data packet to a remote system. And the data packet comprises (a) an identifier of the software program; And (b) at least some of the detected machine readable data.

According to yet another aspect, the present invention includes a device having a detector of machine readable data and a software program used in association with the machine readable data, the apparatus being operable to transmit a data packet to a remote system; The data packet may comprise (a) a context or environment identifier; And (b) at least some of the detected machine readable data.

According to another feature, the invention comprises a networked computer system, which, in response to watermark data sent from a software program on a remote computer, initiates the transmission of advertising data to the remote computer.

In any of the above configurations, the encoding may be steganographic (eg by digital watermarking) or other machine readable data (eg barcode). More generally, the contemplated configuration has a counterpart that can be implemented with anything other than optical or image data (eg, audio data, magnetic stripe information, etc.).

Provided are improved forms of methods and systems for controlling computers or linking to Internet resources from physical and electronic objects.

1 illustrates the main processing components of a system utilizing the present technology.
FIG. 2 is a block diagram illustrating a system for executing the response processing of FIG. 1. FIG.
3 is a block diagram illustrating more specifically the creative device used in the system of FIG.
4 illustrates a particular top level data flow in the system of FIG.
FIG. 5 illustrates a specific data flow associated with the router of FIG.
FIG. 6 is a diagram illustrating a specific data flow associated with the registration process of FIG. 2. FIG.
FIG. 7 illustrates a specific data flow associated with the fabrication processor of FIG. 2. FIG.
8-10 illustrate a sequence of screen shots from a system.
11 is a block diagram depicting yet another embodiment of the present technology.
12 is a block diagram illustrating yet another embodiment of the present technology.
Fig. 13 is a block diagram showing a scanner of the prior art.
14 illustrates an object scanned along a curved path.
15 illustrates how object packaging can be detected.
FIG. 16 illustrates the use of binocular processing to determine specific object attributes.
FIG. 17 illustrates a watermark grid. FIG.
18 shows a prepackaged watermark grid.
FIG. 19 illustrates yet another prepackaged watermark grid. FIG.

Basically, the technology described herein is considered an advanced system that allows a user to interact with a computer-based device. The simple nature and adaptability to use with everyday objects (eg milk bottles) make the described technology very suitable for countless applications.

Due to the diversity and wide range of subjects described in this document, ordered submissions are difficult to achieve. For a better configuration, this specification is divided into two main parts. The first part describes various methods, applications, and systems to illustrate the variety of the present technology. The second part is more specifically focused on print-to-internet applications. In Part III a short conclusion is given.

As will be evident, many of the topical sections given hereafter all turn out to be based on other sections. For better interpretation, the sections of the first part are given in a somewhat random order. It should be recognized that the general principles and specific content from each section can also be found in other sections.

In keeping with the circumstances, this specification is described from several preferred applications filed over nearly a year. Thus, the same concept is sometimes expressed many times, each reflecting a different facet, depending on the context and date of the application that was first discovered.

The word "Bedoop" is replaced by Digimarc DediaBridge in Applicant's technical commercialization. Both terms are used herein, but refer to the same technique.

In order to prevent the length of the disclosure from being constructed out of control, various permutations and combinations of the characteristics of the other sections are not exhaustive. We intend to reliably provide such a combination / permutation, but in practice it requires that detailed synthesis be left to the person who ultimately implements the system according to these instructions.

Part I

Introduction to Digital Watermarking and Other Encoding Technologies

There are as many applications for this as technology for digital watermarking (a type of steganographic data encoding). The reader is assumed to be familiar with many different methods. Some are described below.

Previous Assignee Application No. 09 / 127,502, filed on July 31, 1998 and now published as WO0007356, allows very delicate lines to be printed on the medium to slightly alter the apparent tint of the medium while delivering digital data. Illustrate the technique. Co-owned application 09 / 074,034, filed May 6, 1998 and now published as WO 9953428, describes how the contours of a printed image can be adjusted to convey digital data. (The technique can be applied not only to printed text characters, but also to images of line technologies that are specifically considered.) Assignee's patent 5,850,481 describes that surfaces of paper or other media may be textured to convey optically detectable binary data. Explain how you can do it. Assignee's patents 5,862,260, 5,841,886, and 5,809,160 describe various techniques for encoding photos and other images using steganography.

Some watermarking techniques are based on changes made in the spatial domain; Others are based on changes made in the deformation domain (eg DCT, wavelet).

Watermarking of printed text was performed on June 12-16, 1994, at the Soc Conference, IEEE Comm. Proceedings of INFOCOM'94 Conference on Computer, IEEE Comm. Soc Conference pp. 1278-1287, "Electronic Marking and Identification Techniques to Discourage Document Copying"; 1995 Preceedings of the Twenty-Ninth Conference on Information Sciences and Systems p. "Hiding Information in Document Images" on 482-9; And the 14th Annual Joint Conference of IEEE Computer and Communications Institutes, The Conference on Computer Communications, Fourteenth Annual Joint Conference of the IEEE Computer and "Document markign and identification using both line and word shifting" in Communications Societies; Bringing Information to People (Cat. No. 95CH35759), p.853-60 vol. As shown in various papers by Brassil, including 2, 1995, it may be made by slightly changing the shape of letters, letter decorations, line spaces, and the like.

The above is just sampling of large prints in watermarking. Those skilled in the art are all assumed to be familiar with these techniques, which are generally suitable for use with the new concepts described later.

Although the above description focuses on applications using digital watermarking, certain of these applications may alternatively depend on specific application requirements, such as ID and 2D barcodes, magnetic ink character recognition (MICR), Optical character recognition (OCR), optical mark recognition (OMR), radio frequency identification (RF / ID), UV / IR identification technology, data glyphs, organic transistors Other data encoding techniques, including magnetic stripe and the like.

Basic Principles-Refrigerator and Desktop Clutter

Referring to FIG. 11, a basic embodiment 110 of the present technology includes an optical sensor 112, a computer 114, and a network connection 116 to the Internet 118. The illustrated optical sensor 112 is a digital camera with a noticeable resolution of 320 to 200 pixels (color or black and white), which captures frames of image data five times per second and stores them in one or more frame buffers. Frames of these image data are analyzed by the computer 114 in the presence of Bedoop data. (Basically, Bedoop data is a form of multi-bit data encoding that is recognized by system 110-in many embodiments, data that initiates some action.) Once detected, the system is assigned to the detected Bedoop data. (Eg, by initiating some local action, or via, for example, the Internet, through an online service such as AOL, or a point-to-point dialup such as a publishing system). up) by communicating with a remote computer using communication).

Consider the example of a milk carafe. Artwork manufacturing for the milk carafe can be adapted to convey Bedoop data. In a preferred embodiment, Bedoop data is encoded (eg, digital watermarked) using steganography on a milk carafe. Various digital watermarking techniques are known-they all convey data in a hidden form (ie, human intuition is not clear given digitally encoded data). Preferred techniques operate by slightly changing the brightness or outline of selected points in the text or plate printed on the carafe, or by scattering small ink droplets on the carafe in a random pattern. Each of these techniques has the effect of varying the brightness local to the area across the carafe-a brightness change that can be detected by the computer 114 and decoded to extract the encoded digital data. In the case of a milk carafe, the data may be operated to identify the object, for example with a half gallon milk carafe of Apenrose brand milk.

11 is a device that can be used to compile a shopping list and integrated into a refrigerator door. The milk carafe and other Bedoop-encoded packages can be held with an optical sensor. When the computer 114 detects the presence of Bedoop data and successfully decodes it, it sends a “be-doop” from the speaker or other audio transducer 122. The computer then adds data identifying the object just detected to the sundries list. This list can be maintained locally (such as in a refrigerator or disk storage or non-volatile RAM 124 in any location in the home) or remotely (for example, to a server computer located in a store of your choice or anywhere). Can be. In any case, the list is displayed on a display in the user's home (eg, LCD screen 126 installed in front of the instrument) if desired. Conventional user interface techniques may be used to allow the user to scroll through the displayed list, remove items if desired, and the like.

Periodically, the listed sundries can be purchased, and the list is cleared. In one embodiment, the list is printed (at home or in a store) and the user goes to the store and purchases it in the same way as in the conventional manner. In another embodiment, the clerk retrieves the listed items from the shelf (in response to user requests delivered to the Internet or telephone, or by gestures described later in detail). Once the list is drawn, the clerk can inform the user that he or she can pick up the merchandise (for example, again via the Internet or a store message), or the clerk can deliver the merchandise directly to the user's home. Of course, if desired, an online payment machine can be used.

Consider a Bedoop application that is not entirely related. A Microsoft Excel spreadsheet is printed on a piece of paper that is buried in a clutter pile at an office worker's desk. A few months later, the spreadsheet becomes a problem again, looking through the clutter pile. Changes must be made to the data, but the file name is long and the file name is forgotten. The employee simply needs to hold the printed page in front of the camera associated with the desktop computer. After a while, an electronic version of the file appears on the operator's computer display.

When the page is printed as it is, small droplets of ink or toner are distributed in a pattern that cannot be detected on the paper, whereby the page is sterilized using multiple bits of binary number (eg 24-128 bits). Encoded. The database (e.g. maintained by the operating system, Excel program, printer driver, etc.) is associated with the file name and path where the electronic version of the file was stored and is associated with this number (e.g., 20-bit called UID (Universal Identifier) ) Some, number of pages in the document, and other useful information (e.g., file creator, creation date, etc.).

Steganographic encoding of the document and updating of the database can be performed with a software application (eg, Excel). This option can be selected once by the user and subsequently applied to all print documents (for example, by the user's choice in the "Options" drop-down menu), or given to the user as part of the print dialog, for each print job. Can be selected (or not selected) for.

When such a printed page is later presented to the camera, the computer automatically detects the presence of encoded data in the page, decodes it, and consults the database to identify the file name / location / page corresponding to the 20-bit UID data. Also, open the identified file on the correct page (eg after starting Excel). This application is one of many "paper as portal" applications of Bedoop technology.

The above are just two of the myriad applications of the technology described herein. Many other applications are described in the following discussion. However, regardless of the length of the specification, only a few of the broad branches of this technology can be described.

More details about the basic embodiments described above may be helpful before exploring other applications.

optics

In order for the system to decode the data encoded using steganography from the object, the image of the object must be sufficiently focused on the CCD (or other CMOS) sensor of the digital camera. In low cost embodiments, the camera has a fixed nominal focal length, for example in the range of 2-24 inches (larger or smaller lengths can of course be used). As the camera continuously captures and analyzes frames of data, the user moves the object toward or away from the sensor until the decoder has successfully decoded the encoded data using steganography to transmit a confirming "bidoop" audio signal. You can move it.

In more delicate embodiments, known auto-focusing techniques can be used.

In other embodiments, one or more secondary fixed-focus lenses may be equipped that may optionally be used in a camera (or other sensor), depending on the particular application. This embodiment has a first fixed focus lens that always rests on the sensor, and one or more auxiliary lenses can be optically connected in parallel (eg by hinge or slide configurations). Such a configuration is desirable, for example, when not only a dedicated Bedoop sensor but also performs other imaging tasks. When the camera is used for Bedoop, the secondary lens is placed in a fixed position (e.g. flipped position), so that the fodder distance (unsuitably long, e.g. infinity to the Bedoop) of the first lens is appropriate. Change to imaging range (eg 1 foot).

Other lens-switching embodiments do not always use a fixed lens that rests on the sensor, but instead use two or more lenses that can be moved to a position on the sensor. By selecting another lens, focal lengths such as infinity, 6 feet, and 1 foot can be selected.

In all these configurations, it is desirable (but not required) that the portion of the object being imaged encoded using steganography fill a significant portion of the image frame. The object may be of various sizes, for example a serial box of 10 to 12 inch front panels or a proof of purchase that is exactly 1 inch squared. To meet this requirement, small objects need to be placed closer to the camera than large objects. The optics of the system can be designed by selecting the appropriate aperture size and auxiliary illumination (if necessary), for example, to properly image objects of various sizes within the range of focal length.

Some embodiments identify the intended object by preventing issues of focal length and by limiting the size and / or displacement of the object. One example is a business card reader designed to image a business card. Various devices are known.

Decryption / Encoding

Analysis of the image data can be made by various known methods. Currently, most steganographic decoding relies on general purpose microprocessors programmed by appropriate software instructions to perform the analysis required. Of course, other configurations may also be used, such as using dedicated hardware, reprogrammable gate arrays, or other techniques.

The steganographic decoding process involves three steps. The first step is to locate the object. In the second step, the directivity of the object is taken into account. In a third step, Bedoop data is extracted from the image data corresponding to the Bedoop object.

In the first step, the object location can be obtained by various clues. One is the placement of objects; Typically the center of the image field is a point on the object. Peripheral data may then be analyzed taking into account the boundaries of the objects.

Another location technique is slight migration. Although the user typically attempts to keep the object stationary, there is typically jitter of Bedoop objects within the image frame (e.g., several pixels before and after). In contrast, the background visual clutter is typically fixed. Thus, this movement is detected and used to identify Bedoop objects in the image data.

Another object-location clue is object shape. Many Bedoop objects are rectangular in shape (or trapezoidal as observed by the camera). Thus, a straight edge boundary can be used to define the area of possible Bedoop data.

Colors are another object identification clue that is useful in some contexts.

Another object location clue is spatial frequency. In an imaging system with a well defined focal region, unwanted visual clutter becomes blurred at the focal length. Bedoop objects, by contrast, are in focus and can be characterized with fine details. Analyzing image data at high frequencies associated with fine details can be used to distinguish the intended objects from one another.

Characteristic markings on an object (as will be discussed later in connection with determining object directivity) can also be sensed and used to locate an object.

Once the Bedoop object is located in the image data, masking may be applied (if desired) to remove image data that does not correspond to the intended object.

The next step in the process of decoding and determining the directivity of the Bedoop data can similarly be considered with reference to visual cues. For example, some objects include unconscious gridline data, or other measurement data encoded using Bedoop data and steganography to help determine directivity. Others may use explicit markings that are placed for one purpose (e.g., baseline or baseline) or act on other purposes (e.g., lines of text) to account for directivity. Edge-detection algorithms can also be used to derive the directivity of an object with reference to the edge.

Some embodiments filter the image data at some point in the processing to help extract the ultimate Bedoop data. The use of such filtering is to mitigate image data defects due to certain optical sensors. For example, CCD arrays have regularly spaced sensors that sample optical images at evenly spaced discrete points. This discrete sampling affects the deformation of the image data, creating specific image defects. Properly configured filters can reduce some defects.

(In some configurations, the step of determining directivity may be omitted. For example, a business card recorder may be free from defects and produce data at a known scale. Alternatively, encoding of Bedoop data may not be affected by relatively specific distortion mechanisms. For example, while the currently preferred encoding scheme in rows and columns of data points is operated on a 2D grid, the encoding can be done on another basis in another way (eg For example, the encoding form is rotationally symmetrical, so that the rotational state of the image data can be ignored.) In another embodiment, since the decoding can easily proceed without this information, the directivity-determining step can be omitted. For example, decoding that relies on the Fourier-Mellin transform produces data whose scale and rotation can be ignored.)

Once the directivity of the object is taken into account, the image data is virtually re-registered to actually map it to another perspective (eg, image plane surrounded by straight lines). This mapping may use known image processing techniques, for example, to compensate for the rotational state, scale state, differential scale state, and X-Y offset of the original Bedoop image data. The resulting data frame can then be processed more easily to extract encoded Bedoop data using steganography.

In a preferred embodiment, after the image data has been remapped in a straight line-shaped planar form, unrecognized grid data is detected which identifies the position in the image data where the binary data is encoded. Preferably, the binary data is redundantly encoded, for example, in blocks of 8-to-8 patches. Each patch contains one or more pixels. (Patches are typically square, so include 1, 4, 9, or 16 pixels, etc.) Before coding (e.g., pre-existing objects), the nominal luminance of each patch may be slightly increased or decreased. Encode binary "1" or "0". The change is generally small enough to be undetectable to the observer, but can be detected statistically from the image data-especially if such a block is available for multiple analysis. Preferably, the degree of change is adapted to the text of the underlying image, and relatively large changes are made in areas where it is difficult for a person to notice. Each block encoded in this manner may carry a plurality of bits of data (eg, 16-128 bits). The encoding of these blocks in the form of tiles across objects allows the data to be conveyed in a certain form.

Of course, many times the Bedoop sensor gazes to capture an image frame that does not have Bedoop data. Preferably, the detection process includes one or more checks to ensure that Bedoop data is not incorrectly considered from the non-Bidoop image data. Various techniques may be used to validate the decrypted data. For example, the error detection code may be included in the Bedoop payload and checked to confirm correspondence with other Bedoop payloads. Similarly, the system can confirm that the same Bedoop data in the image data is given in different tile excerpts.

Details of specific encoding and decoding techniques can be found in US Pat. No. 5,862,260 and Application 09 / 503,881. As described, data can be encoded on a tile basis, with each tile having between 64 and 256 elements on one side. Each element can be 0.01 inch squared. Bedoop payload data is redundantly represented by various error-tolerant coding techniques (e.g., convolutions coding, trellis coding, turbo coding, etc.) to fill tile blocks. Can lose. As such, each bit is coded redundantly, such that "1" is represented by an increase in one pixel and a decrease in another pixel. The increase and decrease can be scaled according to the visual masking characteristics of the image to be encoded. The measurement signal may be summed with the tile data signal and may include a signal tailored to the frequency domain to have 12-64 spectral impulses per quadrant in known patterns. During detection, the rotation or scaling of these impulses from known frequency domain coordinates allows the rotation or scaling of the image to be considered and compensated for.

Data structures, formats, protocols, and infrastructure

In a preferred system, the Bedoop data payload is 64 bits. This payload is divided into three fields CLASS (12 bits), DNS (24 bits), and UID (24 bits). (Other payload lengths, fields, and divisions are of course possible, as provided in the error-detection or error-correction bits.)

In short, the CLASS ID is the most basic part of Bedoop data and is similar to a limited number of top-level domains in friendly Internet taxonomy (eg, .com, .net, .org, .mil, .edu,. jp, .de, .uk, etc.) It is basically an indicator of an object type. DNS IDs are intermediate levels of data and are similar to Internet server addresses (eg biz.yahoo, interactive.wsj, etc.). The UID is the finest level of granularity and can be roughly similar to Internet pages on a particular server (eg, edition / current / summaries / front.htm, daily / home / default.hts, etc.).

In general, the CLASS ID and DNS ID collectively indicate to the system what kind of Bedoop data is on the object. For Bedoop systems that rely on remote servers, the CLASS and DNS IDs are used to identify server computers that respond to Bedoop data. The UID determines exactly which response should be provided.

For refrigerator Bedoop systems, what happens when you encounter an object with unfamiliar CLASS / DNS ID data? The system can be programmed to respond with no sweet voice (or other feedback) indicating no response at all or "I saw a Bedoop object, but I don't know what it is".

Most systems can respond to multiple classes of Bedoop objects. Simple software-based systems can compare the CLASS / DNS ID (optionally UID) to a fixed value and branch program execution to the corresponding subroutine. Similarly, hardware-based systems can operate other circuits depending on the detected CLASS / DNS ID.

For a computer with a Bedoop input device (for example, a Sony VAIO PictureBook laptop with a built-in camera or a desktop personal computer with a connected camera), the registry database of the operating system may have different application programs in different CLASS / It can be used to associate with a DNS ID (as .XLS and .DOC file extensions are commonly associated by existing operating system registries to launch Microsoft Excel and Word software applications, respectively). When a new Bedoop application is installed, it logs an entry in the registry database that represents the CLASS / DNS ID to process. Later, when encountering an object with a CLASS / DNS ID, the operating system automatically launches the corresponding application to service Bedoop data in an appropriate manner.

Sometimes a computer system may encounter Bedoop objects that do not have a registered application program. In such a case, a default Bedoop application may be invoked. This default application may, for example, establish an internet link to a remote server computer (or a network of such computers) and send Bedoop data (or a portion of Bedoop data) to that remote computer. The remote server may be responsible for the response itself, instructing the originating computer how to respond appropriately, or for the combination of both responses. (This configuration is further considered later.)

12 shows a design using the above configuration.

In the local Bedoop system 128 (which may be implemented using a conventional personal computer 129, for example), a camera, scanner, or other optical sensor 130 may convert the image data into a decoder 132 (operation system). (Which may be implemented with the software component of 133). Decoder 132 analyzes the image data to take into account the multiple bits of Bedoop data. The CLASS ID of this Bedoop data is applied to Bedoop registry 134. The registry is responded by identifying and starting a local Bedoop application 136 designed to provide service to the Bedoop data under consideration.

At times, system 128 may encounter Bedoop objects where several different responses may be appropriate. In the case of a printed office document, for example, one response may be as described above-providing an electronic version of the file ready for editing to the computer. However, other responses may also be desired, such as, for example, writing an email message to a writer of a document printed with the writer's email address already specified in the message address field.

These different responses may be handled by other Bedoop applications, or may be options provided by both a single Bedoop application. In the former case, when the CLASS / DNS ID is decrypted and provided to the operating system, the registry indicates that two (or more) programs can be invoked. The operating system may then present a dialog box to the user to know what type of response the user wants to be assigned. Optionally, a default selection may be made if the user does not specify within a short period (eg, 3 seconds). The operating system can then start the Bedoop application corresponding to the selected response.

Similar configurations are used where a single Bedoop application can provide both responses. In this case, the operating system launches a single Bedoop application (there is no ambiguity to be resolved), and the application presents a choice to the user. Again, the user can make a selection or a default selection can be made automatically.

In the situation described above, the user can make a selection using a keyboard or a mouse in a traditional dialog box. However, Bedoop provides another, generally easier, form of interaction. The user can make a selection through the optical sensor input. For example, moving an object to the right may cause a UI button on the right side of the dialog to be selected; If you move the object to the left, the UI button on the left side of the dialog can be selected; Moving the object towards the camera activates the selected button. As will be discussed later, many other techniques are possible.

If the registry 134 is not recognized or otherwise knows how to respond to Bedoop data of that particular CLASS / DNS, the registry starts the default Bedoop client application. This client application actually instructs the web browser 40 on the local Bedoop system 128 to communicate with the remote master registration server computer 42. The local computer passes the Bedoop data to this master server. The master server 42 examines the CLASS ID and passes the Bedoop data (either directly or via an insert server) to the corresponding CLASS server 44. (A single server can handle several classes of Bedoop data, but more typically there is a dedicated server for each CLASS.)

Each CLASS server 44 acts as a tree 46 root of the distributed DNS server. For example, in the first stage 50 of the DNS server tree, DNS server 48a handles Bedoop data with DNS IDs starting with "000". Similarly, DNS server 48b handles Bedoop data with DNS IDs starting with "001".

Each DNS server in the first stage 50 actually delivers Bedoop data to one of eight servers in the second stage of the tree according to the fourth through sixth bits of the DNS data. The tree continues in this fashion up to the terminal level of DNS leaf node server 56.

As a result, Bedoop data transmitted to this network reaches the DNS leaf node server 56. The leaf node server may handle Bedoop data or readjust the local Bedoop system to another server 58 that does so. The final server-whether it is a DNS leaf node server or another server-can contact the local Bedoop system for further information if necessary, instruct the local Bedoop system how to respond, or handle some or the response itself. You can simply reply the appropriate data back to your local Bedoop system.

In a configuration where the local Bedoop system is rebalanced to another server that actually handles the response by the DNS leaf node server, access to another server may be configured on a port 59 (e.g., specially adapted for receiving Bedoop data). URL).

In a typical implementation, most or all servers are mirrored or otherwise replicated / external, so that the failure of each computer does not impair the operation of the system.

Caching can be provided for fast response through the tree of servers. In other words, the leaf node's response to a particular commonly encountered CLASS / DNS ID may be temporarily stored in advance in the tree (s). Bedoop data propagating through the server network may prompt a response from the intermediate server if there is a cache conflict.

If desired, Bedoop traffic through the server tree described above may be monitored to collect demographic and statistical information about which system is sending what Bedoop data, and the like. One use of this information is to dynamically reconfigure the DNS network for better server load balancing to virtually relocate DNS resources closer to the area of heavy use. Another use of this information is to promote specific Bedoop features and applications for marketing, for example within a group of users that appear to use these features less (eg, the Internet domain).

Within a particular user network connected to the Internet, for example an integrated network, Bedoop data that is not handled within the originating Bedoop system may first be delivered to the Bedoop name server within the integrated network. The server recognizes certain types of Bedoop data and knows the resources within the integrated network that are appropriate to handle them. Where possible, transmissions for these resources occur within the converged network. These resources (eg, integration servers) may respond to Bedoop data in a manner tailored to the integration preferences. If the integrated Bedoop name server does not know a resource within an integrated network that can respond to Bedoop data, the integrated name server then forwards the data to the public Bedoop network described above. (These transmissions can be given to the master registration server, or the local name server can be localized to the extent that the unified name server knows the address of the appropriate server in the DNS server tree, or another server to which the DNS server points to specific Bedoop data. The system can be readjusted.)

In a typical rich Bedoop implementation, the local system may have libraries of Bedoop services, applications, or protocols. Others are common to all computers. Some are very secure and use encryption and / or hacking prevention means, or data protocols that are not generally recognized. Others are the result of shareware or open-source programming attempts.

The newly configured configuration used the 12/24 / 24-bit protocol with CLASS / DNS / UID data, but for some applications it is advantageous to match the protocol closer to what is commonly used in Internet communications. For example, IP addresses for Internet Domain Name Servers (DNS) are currently 32 bits, and are expected to expand to 64 or 128 bits in the near future. The DNS field of the Bedoop system may conform to Internet standards.

Greeting card, birthday card etc

To further illustrate some of the basic principles associated with this technique, consider greeting cards and the like encoded with Bedoop data (eg, by texture, print, etc.). When receiving such a card, the receiver keeps it in front of the image capture device on a laptop or other computer. The computer responds by displaying an Internet web page with stock or customized indications (images, videos, audio-videos, etc.) to compensate for what is given to the greeting card.

The website display may be personalized by the sender at the point of sale of the card or at any point after the card has been purchased (eg, with a text message, a recent family picture, etc.). In the latest case, for example, the cards can be serialized. After taking the card home, the buyer can visit the card seller's website and enter the card serial number in the appropriate user interface. The buyer is then provided with a variety of simple editing tools to facilitate ordering of web greetings. When the sender finishes specifying the web greeting, the terminated web page data is stored at the site corresponding to the serial number (by software at the provider website).

When the card is received by the receiver and held in front of the Bedoop sensor, the CLASS, DNS, UID data is decrypted from the card. CLASS and DNS data are used to navigate the server network described above (possibly maintained by Hallmark Congratulations Greeting Company) to the corresponding DNS leaf node server. The leaf node server indexes the UID from Bedoop data in a table, database, or other data structure, and obtains from that data structure the same address where the address ordered by the sender of the final web site is stored. The address is provided back to the local computer by the DNS leaf node server with an indication (eg, by HTML retransmission) that the web page at that address is loaded and displayed. The local computer compiles and presents the ordered web greeting to the card recipient.

In the above-described embodiment, where the pre-encoded card is purchased by the sender and the web-display is subsequently ordered, the address of the web site is typically determined by the card provider. However, this need not be the case. Similarly, cards do not need to be "purchased" in the form of a typical card store.

To illustrate this alternative method, consider accepting a greeting card online, for example by visiting a website specializing in greeting cards. With appropriate user-selection (optionally, ordering), the desired card can be printed using an inkjet or other printer in the sender's home. In this case, Bedoop data on the card can be similarly ordered. Instead of reaching the site determined by the card seller, the data may reach the sender's personal web page or another arbitrary web address.

To achieve this configuration, the sender must configure the DNS leaf node server to respond to a particular set of Bedoop data by pointing to the desired web page. When an individual typically does not own a DNS server, it is commonly owned by an Internet service provider. Just as AOL provides simple tools to allow subscribers to manage their latest web pages, Internet service providers can similarly provide simple tools to allow subscribers to use DNS leaf node servers. Each subscriber can be assigned up to 20 UIDs. The tool allows the user to define the corresponding web address for each UID. Each time the Bedoop application reaches a DNS leaf node server and provides one of these UIDs, the server instructs the originating computer to load and serve the web page at the corresponding web address.

Prior to ordering a greeting card, the sender uses a tool provided by the Internet service provider to store the address of the desired destination web address corresponding to one of the sender's available UIDs. When ordering a greeting card, the sender specifies Bedoop data to be encoded, including the UID just mentioned. The greeting card application encodes this data into the production and prints the resulting card. When this card is later presented to the Bedoop system by the recipient, the recipient system loads and displays the web page designated by the sender.

Ad on Bedoop Resources

In the above-described configuration, the Internet service provider makes a limited number of UIDs available to each subscriber at the DNS server maintained by the service. Enterprises typically need more Bedoop resources, such as their DNS ID (or their CLASS ID (s)).

While variants of the Bedoop system are basically scalable to provide a limited number of CLASS IDs and DNS IDs, these resources are limited in the described systems. Public services, non-profit, and academic applications should have relatively general access to Bedoop resources at no or only reasonable rates. In contrast, companies are expected to pay a fee that optimizes potentially infinite demand for resources. A small business can rent a block of UIDs under a given CLASS / DNS ID. Larger businesses can get full rights to the full DNS ID or the full CLASS ID (at the same size and at a higher rate).

A web-based system for specifying DNS IDs (and CLASS IDs) can be modeled after those successfully used by Internic.com and Networksolution.com for registration of Internet domains. The user fills out a web-based form with name, address, and billing information; The system makes the necessary changes to all of the hidden system infrastructure on servers around the world-updating databases, routing tables, and so on.

Control access ID

The same principle as using the ink-jet printer to fabricate the Bedoop greeting card with the above-described embodiment can be applied to an access-controlled object such as a photo-ID.

Consider a candidate who is interviewing for a new employment. A candidate's visit is expected, but it is not recognized by building guards. In this and many other applications, the following configuration can be used:

Employers may email or otherwise send the candidate access codes. (The code can be encrypted for transmission.) The code is only valid for a certain time period on a given date (eg, June 28, 1999, 9:00 a.m.-11:00 a.m.) .

Upon receiving the access code, the candidate downloads a recent copy of his driver's license picture from the website of the Department of Motor Vehicles (DMV). DMV has already encoded this picture with Bedoop code. This data reaches the state-run DNS leaf node server 56. When the server is given a decrypted UID from the picture, the server returns a text string that represents the name of the person represented in the picture to the computer that is accessing and requesting the database.

The candidate includes this photo in the access badge. Using a software application (part of an office product, for example Microsoft Office, specifically provided for this purpose), the photos are transferred to an access badge template. Access codes emailed from employers are also provided in this application. When you select "Print", the ink-jet printer associated with the candidate computer prints an access badge containing its DMV picture and name, and is also encoded using steganography according to the access code provided by the employer. .

The name printed on the badge is obtained from the DMV's DNS server in response to Bedoop data extracted from the picture (by the candidate computer). (Unlike most things in this application, photos are not scanned as part of Bedoop processing. Instead, photos are already available in digital form, so Bedoop decoding proceeds directly from the digital display.)

For security purposes, access codes are not inserted using standard Bedoop techniques. Instead, non-standard formats (typically steganographically) are used. The insertion of this access code can span the entire surface of the card, or be restricted to a particular area (eg, excluding the area occupied by the picture).

On the date promised the candidate is present in the employer's building. At the external front door, the candidate reads the inserted building access code, provides a badge to the optical sensor device that checks it for authentication, and opens the door when the candidate arrives within the allowed time.

Inside the building, candidates can meet with security guards. When you see someone unfamiliar, the guard visually compares the photograph on the badge with the candidate's face. In addition, the guard may provide a badge to one of the Bedoop devices or a number of Bedoop systems scattered throughout the building (eg, on all telephones). The Bedoop system extracts Bedoop data from the card (ie, from a DMV picture), queries the DMV's DNS server with this Bedoop data, and receives the name of the person indicated in the picture in response. (If the Bedoop system is a phone, the name can be displayed on a small LCD display that is usually provided on the phone.)

The guard checks the name printed on the badge and the name returned by the Bedoop system. If the printed name matches the Bedoop-decrypted name (and optionally checks the front door log to see that a person of that name is authorized to come in), the guard can pass the candidate.

Although the above-described configuration provides very high security, it is recognized that the candidate does not visit the employer in advance, does not know which candidate the employer is, and also uses the access badge made by the candidate himself.

Variations of this home-print badge embodiment find various applications. Consider purchasing a movie or event ticket over the web. The user can print an access ticket with an entry code inserted therein. Upon arrival at a movie theater or event site, the user provides a ticket to the optical scanning device that decrypts the entry code, checks its validity, authenticates the entry, and also marks the entry code as used (print with the same code). To prevent a large number of users from using a purchased ticket).

Another control access ID

A wide variety of access control systems can be implemented using the present technology. The above is just one example.

Another application uses ID cards, Bedoop technology, and proximity detection technology (commonly known as RFID).

The ID card may be a badge or the like having a photograph of the bearer encoded using steganography. The card also includes a proximity ID device, such as an unpowered electronic circuit that is extracted from an associated proximity detector and detected by the radiation field, assuming that the unique sign signal identifies a particular individual.

The building may be provided with an image sensor (such as a video camera, etc.), an associated Bedoop detection system, and a proximity detector. When the user with the badge approaches, the proximity detector sends a signal to the camera to capture the image data. The Bedoop detection system identifies badge photos (e.g., with or without such clues described in previous applications), captures optical data, and also extracts embedded data using steganography hidden therein. To decrypt it. The access control system then checks whether the associated badge ID from the proximity sensor corresponds to Bedoop data extracted from the photograph on the badge. If so, access is granted; If not, data is logged and an alarm sounds.

By such a configuration, home security is increased. The proximity-based access badge is no longer changed to replace the image of another person. When the picture is exchanged, the proximity system ID and the inserted picture data are not matched, flagging the unauthorized access attempt.

The same principle is not limited to RF-based proximity detection systems and can be applied to many other contexts. For example, the data decrypted from the picture can be compared to other forms of machine-sensitive personal identity associated with the badge. These include, but are not limited to, barcode IDs, mag-stripe ID cards, smart cards, and the like. Or identity measurements not associated with the badge (eg, retinal scan, voice print, or other biometric data).

Ink-jet printing

In the above discussion, mention is made of using ink-jet printing as a means to provide a substrate with stecanographically encoded representations. The following discussion expands on some of the principles of operation.

In ink-jet printers (sometimes referred to as bubble-jet printers), basic physics and very low levels of analog electronic operation are ideally suited to support very thin background digital watermarking on any type of substrate. (Watermarking through bright coloring of the substrate is discussed in published specification WO0007356, which corresponds to US application 09 / 127,502.) In general, "If you can print with an ink jet printer, you can watermark it (if "can you print it with an ink jet printer, you can watermark it)" is mostly correct even for simple text characters (perhaps especially). That is, there is some flexibility and control in the ink-jet printing area that is not commonly available in more traditional printing techniques, such as commercial offset printing and other plate-based techniques. (This is not to say that ink-jets are of better quality than plate-based technologies; they must process the ink droplet statistics more than anything.) The dark ink patterned background, the photo itself, and the like Darker colored backgrounds are also possible if the continuum ranges from very pale background coloring that the general observer sees as "white paper" in every way through everything in between.

In some embodiments, the ink-jet driver software is modified to provide lower drop angular emission control than that provided in existing printer drivers, which are inherently optimized for text and graphics. In such an embodiment, the "watermarking" print mode may be selected by the user (e.g., in addition to high quality, Econo-Fast, etc.) or to the application software that is printing the watermark data. Another option can be made automatically by

In more complex embodiments, the watermark data is applied to the printer driver software independently of other image / text data. The printer driver is configured to eject droplets at a typical print density for image / text data, and at a more delicate density that is more precisely controlled for separately applied watermark data. (The latter can consist of a slightly modulated signal for the former.) This configuration provides transparent integration by default in existing printer environments-no one needs to worry about the watermarking feature except for software applications that specifically use it. .

Consumer Marking of Web-Based Materials

Various items of video media can be sent from the web, but printed at home. Examples include movie tickets, coupons, car booklets, and the like. Bedoop data can be added or modified at the time of printing by a software application or by a printer driver. (Alternatively, Bedoop data can be ordered to correspond to the user before being downloaded to the user's system for printing.)

The advantage of local Bedoop-encoded print images as opposed to Bedoop-encoding an image file prior to downloading for local printing is that the encoding can be ordered according to certain characteristics of the local printer (e.g., By increasing certainty or reducing visibility)-properties are generally not known to remote servers.

In one particular example, the UID field of Bedoop data can be recorded along with a value that acts as an index into the database of the user profile, allowing later systems given print items to personalize their responses according to the profile data.

In another example, the UID field operates for authentication purposes, for example, to confirm that the medium was actually printed at a particular place, by a particular user, or at a particular time.

Coffee mug

In retail coffee shops, consumers generally order the same drink every day ("half-decaf, short, skinny latte"). Some consumers provide personal coffee mugs to cashiers and prefer ceramics or metals over paper to avoid waste / recycling issues.

The drinker's "regular" order may be Bedoop-encoded either in the mug itself or, more generally, in an adhesive label applied to the mug. The encoding may be in addition to another aesthetic image (eg, a work or a picture), or the marking may be pure data. Stamp-sized labels can be used.

When giving a mug to a cashier, the customer can simply say "regular." The cashier passes the mug in front of the optical scanning device of the Bedoop system associated with the cash register. The system decodes the data using steganography and writes the corresponding order ("half-decaf, short, skinny latte") literally or acoustically (e.g., Provides to cashier or cashier, by voice synthesizer). The cash register system also announces the current state of the beverage required and sets the fee accordingly.

Labels of the described type are available to cashiers in preprinted rolls, like other adhesive stickers, or can be printed on demand. (In the latter case, a small label printer is most appropriate due to space limitations in retail stores.) A customer ordering a beverage with a personal mug takes a label corresponding to the exactly ordered beverage and applies it to the mug for the next use.

In a variation of this basic theme, the mug label may be further encoded with electronic payment information, such as a consumer's credit card number or a payment count number maintained by the coffee seller for that consumer (or supplemental labels may be provided and encoded). have). When the mug is scanned for a beverage order, the system similarly detects the payment information and charges the corresponding fee for the appropriate count. (For security reasons, the system is configured so that the mug can't be used to approve more than $ 5 of a coffee drink that you purchase per day.)

In another variation of this theme, the system maintains an electronic log of coffee purchases made by the customer, according to trending marketing considerations, and the customer is rewarded with a free beverage 8 or 12 after the drinks have been purchased. .

In another variation of this theme, regular customers using Bedoop-labeled mugs can expect periodic promotional items, for example, every N such customers being rewarded with cash or purchase prizes. Bells are done when the Nth mug is watched. (N can be a fixed number, such as 500, or can be a random number within a generally known range or known average.)

Warping and Focus Issues

Coffee mugs are examples of non-two-dimensional objects. Another kind is soft drink cans. Special issues arise when encoding and decoding markings on the surface of such objects. For example, when detecting something like a camera or the like, part of the image will be out of focus by the distance difference from the camera to another part of the can surface.

Even if parts of the image obtained from non-two-dimensional objects such as cans get out of focus, they still carry useful image data. The out-of-focus part is only blurry, as if it passed through a low pass filter. However, to use this information, complexity must first be addressed: warping

Looking at the camera, the flat work surrounding the can is warped. The part closest to the camera appears at the nominal full scale, but the part leading to the curved surface of the can (as viewed from the camera) gradually appears to be more spatially compressed. Despite the use of watermarking techniques, physical warping of the can surface appears as if warping of encoded watermark data.

The way to deal with this issue is to warp the watermark pattern in advance to account for this optical distortion.

In a watermarking technique that directly deals with luminance, the grating to which watermarking is applied can be pre-distorted to overcome the continuous optical distortion of the workpiece as detected in the cylinder can. Consider a Pepsi or Coke can. The visual centerline can penetrate the center of the logo (front of the can) and serve as one centerline of the watermark grid tile. On each side of this centerline the lattice is stretched continuously. This stretching is calculated so that the watermark grid looks evenly straight when viewed with the camera, rather than being continuously compressed towards the can's edge.

An example of such an approach is shown in the figure. 17 shows an unwarped grating commonly used for watermarking flat objects. 18 shows a grating previously warped to account for optical distortion due to the curved surface of the can. The centerline of the production (ie, label) is shown in dashed lines.

Typically the grid is square rather than cylindrical. Moreover, the illustrated distortions can be expected that the depicted grating will pass at an angle of +/- 90 degrees from the canned surface. Also, the gratings are typically small (one inch on one side) so that the gratings are inclined adjacent to one another in the range of +/- 90 degrees. Further illustrated prewarping is based on infinite projections (ie, looking at the surface of the can at infinite distance will include a full +/- 90 degrees from the centerline). In general, warping will be calculated based on finite projection using a typical lens-to-object distance (2-24 inches), resulting in a field of view less than +/- 90 degrees from the centerline.

The illustrated grating is only prewarped in the horizontal direction and coincides with the surface made by geometric distortion. Other geometric distortions also occur due to other parts of the can that are away from the camera. The further away, the smaller the appearance. Thus, the grid element that will be located further away from the camera must be larger to precompensate for geometric distortion by distance. The geometric distortion by such distances appears equally horizontally and vertically. Therefore, the more precise prewarping, the more inflated the grid cells horizontally and vertically with the gradual arrangement from the centerline, thereby responding to the effect of getting smaller by distance. 19 shows the basic attributes of such prewarping.

The amount of prewarping described in the latter depends on the distance from the camera lens to the front of the can. If the distance is about 2 inches, the smaller the distance, the more obvious the distance. In the latter case, the distance to the furthest distance of the projected object may be 110% of the distance of the nearest part, while in the former case, the ratio may be 200% or more.

The illustrated prewarping is typical of such and can be used when a watermark is applied to a pixel region using a grid form. That is prewarping in the space domain. Different watermarking approaches naturally require different kinds of prewarping and will interchange the presentation of expected watermark material. For example, watermarking techniques that rely on changing image coefficients in the deformed area require other modifications. Conceptually this modification is the same (i.e. it becomes a clear image with the intended watermark information). However, the display is unlikely to be the same as the example shown in Figs. 17 and 18 (ie, warping is required in the deformation region rather than the spatial region because the deformation coefficient is changed rather than the lattice diagram).

Although the description so far has focused on prewarping the image, the problem can be treated differently. If a straight watermark that has not been prewarped is applied to the cylindrical can, then the watermark detector can be used in a non-warping operation to counteract the possible distortion. That is, the detector visually adjusts the pixel data to effectively stretch the pixels away from the centerline and restores the proper linear relationship.

As one specific example, assume that the sensed image data is planar and is first tested and decoded without nonwarping. If no watermark is detected, the same data (or the structure of subsequent image data) is experimentally dewarped to see if a readable watermark is derived. Several consecutive dewarpings of different characteristics can be tested. As another specific example, other non-warping functions (including non-warping) may be continuously used until the detector shows a non-warping function where a watermark identified from the image data is accepted.

If applicable, the user may specify the shape of the object to be applied alone or within a limited range of non-warping functions. Or the user can simply provide the sensor with an overall clue (ie, by selecting "magazines" or "grocery products", etc. in the user interface associated with the watermark). In the former case, the middle is known to be flexible and one can assume a random simple curved surface rather than a plane. In such cases, the detector may spend a lot of time deciphering the watermark, assuming that the imaged plane is flat, and sometimes one of four or eight other non-warping functions as appropriate if the magazine plane is slightly stretched in the other direction. Will try to apply. In the latter case, food products are usually relatively inflexible and therefore have the most common planar or cylindrical predictable shape. In that case, the detector spends half the time decoding and assuming that the object is a plane and the other half spends between the various cylindrical non-warping functions.

Although image watermarking has been discussed so far, this principle can be applied to audio watermarking.

Smart elevator

As another specific example, an elevator in a building is equipped with one or more optical capture devices. Each device inspects the monitoring of the contents of the elevator room and sees Bedoop encoded objects such as ID cards.

In detecting Bedoop encoded objects, the elevator determines which floor of the office the wearer uses from the data. The system will automatically assign the floor regardless of the user's button usage. (The elevator's button board must provide a new button that can be manipulated to erase the most recently selected floor. Otherwise the user will go to another floor.)

To aid identification, Bedoop objects (nameplates) may have unique colors so that they can be easily distinguished from others within the visible range of the optical capture device. Alternatively, the object may be re-reflective coated and the elevator may be equipped with one or more of a known spectral or transient level illumination source (constant infrared, single or multiple line spectral, known periodic wave light source; LED with diffuser or Semiconductor lasers or the like may be used respectively or with an image capture device). Others like auto indicators can be used similarly for object positioning. In all such cases, the optical capture device uses various visual sensors to detect automatic indication cues. The device is then physically or electrically operated or enlarged to obtain a high resolution image of the digitally encoded object suitable for decoding.

magazine

The magazine page may be steganographically encoded with Bedoop data to provide another "paper as portal." As in the case of the office document described previously, the encoded data yields an address for a computer location (web page) with the same or associated content.

As one specific example, blank magazine pages are Bedoop encoded prior to publication. Watermarking can be performed with a high speed inkjet device that sprinkles, in fine form, substantially undetectable ink droplets on each page. When each page is watermarked differently and decrypted, 21 pages of the same magazine may be watermarked differently from 22 pages (106 pages published on June 21, 1999 are 106 pages published on June 28, 1999). And can be distinguished from). If necessary, each page can be divided into sections, such as the actual boundary of an article on a later published page, or a three-column, five-row grid. Each area sends Bedoop code that directs different parts of the page to different web data.

After watermarking and publication, the produced page goes through the same process as a regular magazine book to produce a finished magazine. (Not all pages in a magazine need to be watermarked.)

Of course, watermarking can be affected by processes other than inkjet publishing. For example, lettering by pressure rollers is another option for producing large pages. The production used in the commercial can be digital watermarked using commercial watermarking software (Adobe Photoshop, Corel Image Editor, etc.).

When a magazine corresponding to the Bedoop-capable computer's optical scanner device is made, the computer detects and decrypts Bedoop data and launches a web browser to direct to the Internet address corresponding to Bedoop data. If the magazine's page is an advertisement, the Internet address provides supplemental information about the content of the advertisement. For example, if the magazine page is an advertisement for a food item, Bedoop data may identify a web page on which the advertised item is provided and available. If the magazine page contains pictures of the equatorial beach, Bedoop data can direct a web page (travel agency) that provides information about fares or rates to readers who want to go to the beaches shown in the photo. (Fare information can be ordered at the reader's residence airport with reference to user profile data stored on the user's computer and relayed to the website to allow ordering of displayed pages.)

The data led by Bedoop data can be updated weekly, daily or by some criteria. Thus, if a month's page is available on the Bedoop device, the resulting data will be updated instantly. The data to be linked may include audio or image data.

In the case of advertisements, the inclusion of Bedoop data increases the value of advertisements to advertisers, so that the publisher of the magazine can receive a large advertising fee from the advertiser. These increased advertising costs can be shared with companies that provide Bedoop technology and facilities that enable high value advertising.

Business card application

Existing business cards can be steganographically encoded with Bedoop data by text processing, watermark coloring, inkjet injection, text steganography, and the like. As with many of the specific examples described above, steganographic coding can be easily steganographically decoded by random rotation of the card, scale distortion, etc. during scanning. (Some of the techniques mentioned above appear in the applicant's previous patent applications and in the publications introduced above.) Various other techniques are well known to those skilled in the art.

When the business card recipient is holding the business card in front of the Bedoop detector, the operating system on the local system will begin applying the local Bedoop. In turn, local Bedoop will establish an external Internet connection to the remote business card server. The address of the server is already known to the local Bedoop (already stored by previous use) or the local Bedoop system can navigate the DNS network described above to connect to the business card server.

The database of the business card server maintains a large amount of business card data by storing one database for each UID. When such a server receives Bedoop data from the local Bedoop system, the server analyzes the UID and accesses related database records. These records usually contain more information than is included in existing business cards. Examples of records include name, job title, company phone number, company fax, home phone number, home fax, cell phone, email, company name, company internet address, personal homepage address, secretary's name, spouse's name, birthday, etc. have. These records are sent to the original Bedoop system.

The local Bedoop system has data and no longer needs to be instructed by the user how to handle it. Do I need to dial the phone? Should information enter the personal contact manager database on the local system? And so on.

As a specific example, the local system provides a selection available to the user by text prompt, synthesized voice, and the like. The user responds by manipulating the business card as required by the system (go to the top company phone, go to the bottom home phone, go to the right home page, go to the left personal home page, database records (filtered by template)). Rotate left to enter any element from the Personal Contact Manager database to, etc.). Respond to the local Bedoop system according to the above behavior.

Some business card senders may select additional information that may be useful to the business card recipient—known information beyond the application of contact management software. For example, one option provided by the local Bedoop system in response to the business card transfer may be to review the personal calendar of the business card provider. (The business card provider can keep a personal schedule on his web-access computer.) Such adjustments allow the business card recipient to know when the business card provider is in the company and when the appointment was made.

Normal web browsers do not allow access to the web calendar, but can be accessed in response to Bedoop data (this may be treated as a certificate or password).

Some users may have cards that have been encoded in different forms, giving different levels of access. Thus, some cards have access to the entire page without any schedule information, others have access to the same or different pages available today or only this week, and another card (spouse card) has access to the card provider's complete schedule. You can also access the same or different pages. The user may provide different cards to each individual depending on the amount of personal information available.

As a specific example of association, the database record corresponding to Bedoop business card data may include a telephone number field "now". These fields are constantly updated throughout the day to ensure the best communication with the business card provider. These data fields are constantly updated, such as when a business card provider goes from home to the office, leaves the office, moves to a vehicle, and works in a company office in another city during the week. (Wireless GPS receivers that are wirelessly connected can connect the current number among multiple phone numbers possible by an individual's immediate location.) If the database records are sorted by "current" number, then the current available information is provided.

Consider a payphone capable of Bedoop. To call, the card is placed in front of the Bedoop detector (or must pass through an optical scanner). The phone then searches the business card server's database and dials the "current" number.

To update any field stored in the database record, the card provider can use a special card that transmits written authentication privileges. This special card may be a specially encoded type of business card or may be the only other object of the card provider (card provider's driver's license).

References to business cards and personal calendars are for illustration only. Back in the century, "calling cards" have been used by those whose interests focus on social values rather than business. If the principles just mentioned apply similarly, teens carry cards with new friends to exchange personal information such as personal information, favorite music, props and video clips. The card can be decorated for purposes that are not related to steganographic coded Bedoop at all.

Gesture input

The Bedoop system can refer to embedded scale data or other techniques to determine scale states, rotation states, X-Y offsets, specific scale states, and the like. If the scan device is operated at an appropriate high frame rate (5 or 10 frames per second) the change in any or all of these four variables will be tracked over time and this will be an additional input.

In the above example, left and right movement of the object in front of the Bedoop scanner leads to selection of a button located at the left and right of the dialog box. This is the change in the X-Y offset of the scanned object. In that example, moving the object inwards towards the camera will activate the selected button.

In a similar way, twisting an object left or right allows one of two reactions to be executed immediately with a properly programmed Bedoop application. (This is a change in rotation). Similarly, tilting an object toward or away from the camera can cause either reaction in an application. (This is a change in differential scale state)

For example, in the case of the business card described above, the card may be placed in front of the Bedoop scanner of the computer. If the card is twisted to the left, the computer directs the web browser to the web page address corresponding to the card's Bedoop data. If you twist the card to the right, you open an email template that is directed to the email address preassigned by the card.

As another example, twisting the card to the right corner toward the scanner may be used as the effect of pressing the left mouse button when twisting the card to the right corner while moving away from the scanner with the effect of pressing the right mouse button.

Simultaneous conversion of two of the four position variables can be utilized in one of four ways of inputting to a computer. (1.Gap to the left while moving inward; 2.Gap to the left while moving outward; 3.Gap to the right while moving inward; 4.Gap to the right while moving outward). Simultaneous conversion of three of the four variables can be used in one of eight or sixteen ways of entering the computer in a similar manner.

Two or more simultaneous operations on an object in this mode are usually unwieldy and lose the simple and intuitive feeling that characterizes the operation in one mode. However, similar effects can be achieved by the operation of sequential cards that are not simultaneous in other modes. (Twist left and move inward). Moreover, sequential operation allows the same mode to be used twice over and over again (moving inward and then outward). Sequential manipulation of such objects sends any complex input to the Bedoop system.

It will be appreciated that the digitally encoded object does not need the above mentioned operational input. Any object that can be distinguished into image data can be manipulated by the user in the manner mentioned above and the appropriate system can recognize the movement of the object and respond accordingly. Although the provisions regarding the digital data of an object provide many dimensions of functionality (depending on the digital encoding of the object to be manipulated, allowing the same operation to have different meanings), this is not essential.

Moreover, steganographic coding is not essential within the domain of digitally coded motion objects. Any known form of optically recognizable digitally encoded (1D, 2D barcodes) can be easily operated.

In the described embodiment, business cards or photographs are used as the objects referred to herein, but the range of selectable objects is not fundamentally limited.

Dynamic behavior is not the only means of communication affected by such objects. Static locations (providing objects in different directions) may optionally be operated.

Consider a magazine ad. When the book is opened and the front of the page is provided, the first reaction is triggered. When the page is rotated 90 degrees, a second response is triggered. Similarly, a 180-degree turn (upside down) or 270-degree turn will trigger a response. The Bedoop detector senses this other form of rotation by referring to the characteristic of the watermark signal identified from the magazine page (with reference to the rotational state identified from the unconscious grid signal detailed in the applicant's previous patent).

Gesture decoding module

There are a variety of ways in which the decryption of the gesture input of the Bedoop system is affected. On some Bedoop systems, this functionality is provided as part of the Bedoop application. However, in most application areas, raw frame data must be provided to identify movement of the motion. Since this feature is typically used in many Bedoop applications, it is preferred to provide a single motion analysis software function (usually operating system level) to analyze frame data and produce motion output in a standardized form of Bedoop application.

In one such system, the motion decoding module tracks the coded object within a series of image data frames and outputs several variables that characterize the position and manipulation of the object over time. Two of these variables are the X-Y position of the object within the current frame of image data. The module identifies the reference point (or several) of the object and outputs two corresponding positional data (X, Y). The first represents the horizontal offset of the reference point from the center of the image frame, expressed as a ratio of frame widths. Two supplementary provisions or other indicia indicating positive and negative values may be used such that the variable has a positive value if the reference point is to the right of the center frame and has a negative value if the reference point is to the left of the center frame. The second variable, Y, is similarly characterized as the position of the reference point above and below the center frame (if it is above, have a positive value). Each of the two variables can be represented by a 7-bit byte. Each time a new frame of image data is processed, a pair of X and Y variables are output to the motion decoding module.

In many applications, the absolute value of an object's X and Y is not important. Rather, it is the X, Y direction of the object from frame to frame that controls any cross section of the system's response. The Bedoop application can watch for changes in the two variables mentioned above frame by frame to identify such movements. Usually, however, the motion decoding module performs this function and outputs two new variables, X 'and Y'. X 'indicates the movement in the left and right directions of the reference point after the last image frame, such as the ratio of the previous frame width. Again, this variable is provided in two complementary forms: a shift in the right direction has a positive value and a shift in the left direction has a negative value. The Y 'variable similarly indicates the movement of the reference point in the up and down direction since the last frame.

The scale, differential scale, and rotational state of the object are similarly analyzed and represented by variables output from the motion decoding module.

The scale state can be identified by referring to two or more reference points of the object (the diagonal edges of the card). The distance between two points (or the area by three or more points) is identified and expressed as a ratio (or area) of the diagonal size of the image frame. The output is a single output variable, A, which may be a 7-bit binary representation.

Similar to the X and Y data, the motion decoding module monitors the change in scale state since the last frame to produce a corresponding output variable A '. This variable is expressed in two forms: the positive value moves the object toward the detector after the last frame, and the negative value goes in the opposite direction.

The differential scale variable B can be identified by referring to the four reference points of the object (the centerline of the four corners of the card). Two points on the side of the card define a horizontal line and two points on the top and bottom edges of the card define a vertical line. The ratio of these two line lengths is a measure of the differential scale. This ratio is expressed as a percentage of the short axis relative to the long axis (this ratio is always between 0 and 1). In other words, two 7-bit displays are used when positive values are short for vertical lines and negative values when short horizontal lines. (As before, the dynamic variable B 'can also be identified and again represents the same 7-bit form of change in the differential scale variable B' since the last frame).

The rotational state variable C is identified by the angular direction of the line defined by the two reference points of the object (center line on two sides of the card). This variable is encoded as a 7-bit binary value expressed as a percentage of the rotational offset from the reference direction (horizontal) to the clockwise direction. (The two reference points should be distinguished regardless of the angular position of the object if the data is in the full range from 0 to 360 degrees. If these two points are not distinguishable, it is only possible to represent the data within 180 degrees. As before, the dynamic variable C 'can also be identified to represent a change in the rotational state variable C since the last frame, which is a two-bit 7-bit form where the positive value changes clockwise. Express

Of course, the former analytical techniques and methods of provision are just examples. Those skilled in the art will recognize many other operations to meet the needs of a particular Bedoop application.

As an example system, Bedoop application programs communicate with behavior decoding modules through standardized interface protocols such as APIs. One API may query the motion input module for some or all of the current position variables (X, Y, A, B, C). The module corresponds to the application requiring the requested variable. Other APIs may query the motion input module for some or all of the current movement data (X ', Y', A ', B', C '). Another API may require a motion decoding module to provide updated values for some or all of the position or movement data based on the current state as identified for each frame. The supplementary API stops past operations. Such operation makes all operational data available, but the Bedoop application program only acquires when required specific data is required.

In Bedoop's application, which communicates with external servers, only Bedoop data (CLASS, DNS, and optionally UID) may be sent first. If the remote server needs to consider motion data in determining the response, the remote server can register with the local Bedoop system for the necessary data. The requested operational data is delivered by the local Bedoop system in one or more separate transmissions to the remote server.

In another specific example, since the motion data is low bandwidth (approximately 56 bits per image frame), the data is sent to the remote server routinely and automatically so that the motion data is immediately available when needed. As an example, this data is assembled into an 8 byte packet such that the first byte (X variable) of the packet is fixed with the "1" sync bit and the remaining bytes fixed with the "0" sync bit. (Sink bits can be used for correct packet decoding.)

In another example, it is useful to extend the normal 64-bit Bedoop length to accommodate a combined packet of operational data. This may be affected by the use of the opposite bit (ie the UID field of the Bedoop packet). This bit normally has a value of "0". If "1" it is just 128 bits with 64 bits containing Bedoop data rather than the usual 64 bits containing the packet of operation data.

Similar extension protocols can be used to combine Bedoop data with other auxiliary data. For example, a bit stored differently in the UID field may transmit a 256-bit data field after Bedoop data-a data field to be interpreted by a remote computer that eventually supports Bedoop data in a known manner. (These bits may transmit personally identifiable data, credit card data, etc.) The sequentially appended data fields may include one or more bits indicating the appearance of further additional data.

Grannies

A common complaint is that computers are complex for most people to use. Attempts to simplify the interaction between the computer and the user so that even beginners can easily use it usually frustrate many experienced users.

According to one aspect of the present technology, complexities of a computer user are represented using steganography on an object designated by that user who interacts with the system. The computer detects steganographic coded data and assigns a mode for interacting with the user accordingly.

Consider internet browser software. Those skilled in the art are familiar with other functions that can be accessed. That is, various drop down menus, submenus, keyboard shortcuts, right-click menus, mouse scroll wheel buttons, and so on.

Although the operating interface has a great future that will simplify the interaction between the user and the computer, the dichotomy between beginners and those skilled in the art is likely to persist and a group of users will frustrate other groups of users.

To overcome these slight differences, the computer can be reacted by operation in a variety of ways leading to steps of those skilled in the art designated by the encoding of the user's object. For example, a person skilled in the art may have an operational interface activated in Internet browser software to reveal their favorite web address by tilting the left edge of the object towards the optical detector. When the address list appears, you may rotate the object to the right to scroll down the list from above. Rotate the object to the right to scroll through the list of favorites from bottom to top. The scroll speed changes with the degree of rotation of the object from the reference direction.

Conversely, manipulating objects can be more confusing for beginners. Tilting the left edge of the object towards the detector can be as common as making manipulations. Moving a cursor on the screen for such a user may provide a more satisfactory interface, such as moving the object with a simple X-Y move and moving the object toward the detector (similar to a left mouse click).

(In the example immediately cited above, a person skilled in the art calls a list of favorites. The list of other "favorites" is maintained by the computer to be operated by different objects. Provide a different "favorite" list than your wife.)

Printed photos

Steganographs with Bedoop data can be encoded to go to information (contact information, personal history information, etc.) related to the person depicted in the picture.

Such a picture may be exposed to the Bedoop detector of the phone. As a simple specific example, a telephone simply processes the Bedoop data and dials the number to obtain an associated reference telephone number. Another example is a variety of choices, such as calling home, calling to the office. For example, if you provide a picture in front of the phone and move it to the left, it is to call home and to the right to call the office.

As the performance of the telephone is further improved and multifunctional, different operations can cause different operations. For example, rotating a picture clockwise on a mixed computer / telephone device could trigger a web browser to go to a web address that obtains video material from a web camera at the home of the person in the picture. Rotating the picture clockwise can provide an E-mail form to go to a pre-determined E-mail address for the person in the picture. The movement of the picture to the right may query a database on the system for other pictures representing the same person or subject, which may be provided in response to user input or the like.

In this and other embodiments, it is useful for the Bedoop device to immediately react to the user to help manipulate the object. You can do it by voice (move the picture to the left to call home) or visually (provide arrows to the left and right).

The Bedoop data of a picture can be used to annotate the picture as described on the back of the picture or underneath the picture in the album. Bedoop data can access a remote database that allows a picture owner to enter text or voice descriptions associated with each picture's UID. After a few years, placing the picture in front of the Bedoop detector when someone's name is forgotten will make it possible for the system to provide annotations provided by the picture owner several years ago.

Driver's license and card

A driver's license, resident card or other identification document may be encoded by authenticating Bedoop data that allows access to the owner's records on the web. When providing a document to the Bedoop system, the system directs the web browser to a personal address corresponding to the data encoded in the document. At that address, the owner of the document can view government records, such as national and local tax refund data, resident registration, as well as credit checks held by the individual. User selection among the various functions can be influenced by the manipulation of documents. (Additional data, such as resident registration number and mother's name, may be required to protect your privacy if documents are lost.)

By operating the driver's license in front of the Bedoop system detector, the user can request a license renewal and pay the associated tax.

Bank cards (debit, credit, etc.) are encoded with Bedoop data so that cardholders can access the bank records for their accounts. (You may be required to enter your PIN code to protect yourself.)

Such documents can be used to access different individuals' data. One example is E-mail. Travelers may have to stop in front of the airport's Bedoop booth and provide a license. Without any manipulation, the booth may show an E-mail waiting for the traveler on the relevant screen.

Upon recognition of the license, Bedoop Booth provides access to a remote site (maintained by the Department of Transportation, other government departments, personal agencies or travelers) that authorizes operation by providing encoded Bedoop data on the driver's license. Respond to the access to obtain information previously allowed by the individual. The information may include e-mail account and password information. In using this information, the booth makes a request to the corresponding E-mail server to download for the provision of recently received mail. (In the process, for example, to ensure privacy, when querying a remote site for a sensitive E-mail password before providing a downloaded E-mail, some parts may require a PIN number entry).

Other cards in the wallet can also be encoded for various functions. A local sandwich shop that wants to give one free gift for purchases of 12 or more sandwiches can encode the main customer card into Bedoop data for sandwich delivery via the store's web. The customer may use a business card or identity document instead of the main loyalty card to signal in front of the store's Bedoop detector to earn purchase points in an account maintained by the sandwich shop's computer.

Meal tickets, health insurance cards and medical prescriptions can be encoded into digital data as well to provide new functionality.

At commercial trade shows like COMDEX, sellers do not have to publish thick and massive brochures. Instead they may only offer nicely crafted promotional cards. Later, when the card is provided to the Bedoop detector, each card is heard by the Web, which optionally includes promotional video and other multimedia elements. The user may be immediately informed of the user's specific needs. If the user wants more information, they can click the mouse (or twist the card).

Sweepstakes and Product Promotions

The product's packaging (cola cans, soda bottles, 12 Pepsi packaging boxes) can be encoded for prizes. Coding can be made on a per item basis to be recognized as a prize for 1 in 100 people who can receive cash or prizes when scanned by Bedoop. Contact information (address, phone number) so that the remote server can win prizes for items with built-in Bedoop data or issue prizes for small prizes that can be rewarded in kind or cash to local sellers. You can ask. Once the recipient of the prize is recognized by the remote server, the UID on the server indicates that the prize has been paid to prevent the item from participating in another prize.

Another such example encodes all items equally. The winner is determined randomly. For example, many people around the world may offer cola cans to Bedoop systems during the giveaway. The corresponding Bedoop on each user's computer submits the Bedoop data to the corresponding web address. As this data is connected to the server computer, every N ^th data will be considered a winner and a corresponding prize receipt or gift will be delivered to the Bedoop system with the winner.

Server computers that receive event submissions from a customer Bedoop system should take steps to prevent a single customer from involuntarily submitting multiple data for a prize. (For example, check the included E-mail address and IP address, and if the same address is in the submitted data within the last hour, then the submitted data will not be taken into account. Similar measures will be taken to prevent participation, although more complex measures can be taken.)

Non-planar product packaging, such as cylindrical beverage cans, brings some optical interest in the encoding and decoding described below.

Product Information and Order

Product packaging and advertisements can be encoded with Bedoop data, which, when provided to the Bedoop system, can link to a web page where the product was purchased or obtain a lot of information. Once connected, the user can manipulate the object in the manner described previously, ie move the object towards the camera to place an order; Spaced apart from the camera can instruct to provide information. If the user moves the object towards the camera for the order, the user can do it immediately to specify the delivery (next day delivery if rotated to the right, normal delivery if rotated to the right). If an object moves away from the camera to request product information, it can be manipulated to direct the user to the type of information they require immediately (rotate the object to the left for recipes, to the right for FDA nutrition information). Rotate to move the object up for other product information of the company, or move the object down if you want to send an e-mail to the producer).

Billings of credit or other customer cards, along with postal address information, may be stored in Bedoop system files and are initiated at the time of purchase or after the user has consented to transmit such information (consent is one of the methods described above). Are sent by manipulating the packaging or advertising) and are automatically connected to the trading website.

Computer access card

The access card considered above was used for granting access to the secured building. Associated principles may be used for computer access cards.

A driver's license, employee card or other document is provided to the Bedoop detector on the computer. The computer can then recognize the user and take different stages of the reaction.

One response is to log on to the network. Another way is to post the user profile on the computer because the computer knows how to adjust it in the way that the user prefers. By manipulating Bedoop encoded objects, users can change more environments (rotate RORC-P to the left for standard business productivity applications and software development applications; adjust lunch breaks, start inventory, and start entertainment games). In order to rotate to the left.)

Computer services in hotels are increasing. By providing a driver's license, a Bedoop-equipped computer in the hotel room can connect to the remote site indicated by Bedoop data, obtain the user's preferred data, and adjust the hotel's computer to match the user's computer workplace. Can be.

Audio / Video Discs, Software, and Books

Bedoop data can be displayed or transmitted as text on CD / DVD disc surfaces, trademarks (or certificates), inserts or productions, likewise enclosed (jewel boxes, plastic boxes, etc.), book covers, book pages, and the like. Any of the above objects can be provided to the Bedoop device for connection to the associated web site. The consumer can manipulate (or other selection) the object for different choices.

One option in the field of music may receive the same singer's music on different CDs or the same genre of other singers with an MP3 or other tool. Another option is to watch a music video with the same singer. Another option is to buy tickets for the singer's concert. At the point of sale, the consumer can listen to sample music before purchasing.

Similar choices are available for video DVDs. In the case of a video, it may include a list of other films of the same director or another film of a movie star. In the case of software, choices may include advice, bug fixes, recently released products or upgrades, and the like. Naturally, users can buy different music from the same singer, different movies from the same star, software upgrades, etc. on these sites.

Similar choices can be made for typed books using Bedoop data.

Children learn mechanics by turning the bookshelf as a child. The child learns to look at the image on the page and enjoys hearing the story associated with the image. Usually parents read books and children follow pictures. The child enjoys repeating stories. Listening to stories repeatedly while looking at images is an excellent dynamic to learn how to read and enjoy.

Built-in digital watermark data automates the above process, allowing children to view images and listen to stories independently. Such adjustments provide children with pleasure while teaching reading and make them love books, while at the same time giving children a sense of independence and automated mechanics.

More specifically, the paper surface of an image or page in a book includes a digital watermark. As the child turns the pages, the camera captures the image and the computer reads the watermark. The watermark is used to index a local or remote data store to obtain text corresponding to the page being viewed. Read text to the child using a text-to-speech device. (Unlike traditional methods, data repositories retain digitized voice rather than simple text.) So, as children turn the pages of the book, they hear the printed text on the pages. The child controls the process by turning the pages. Children learn the connection between naturally printed text and computer-readable text.

Ad tracking

Advertisers generally use different ads for the same product or service, and use a means to track which ads are more effective within a group of statistics. Bedoop can provide this functionality.

Consider a travel service website that promotes Hawaii vacations. Bedoop data from several advertisements can bring consumers to the site.

The same advertisement may be placed in several different magazines. Each is coded with a different Bedoop UID. By monitoring the UID of Bedoop queries for the site, it is possible to determine which magazine each travel service yields the best consumer response (eg, per thousand readers).

Similarly, within one magazine, two or more advertisements are encoded with Bedoop data leading to a site-again, each with a different UID. Analysis of the UID used to access the site may indicate which advertisements were more effective.

Both UIDs in the above example can lead to the same Internet destination or to different destinations.

The instantaneous nature of the Internet links allows advertisers to learn how consumer responses to print ads change over time and yield information that helps make advertising for particular products more effective.

More delicate variations and combinations of the above are possible, of course. When a consumer provides personal information in response to an advertisement (either by allowing access to pre-stored personal profile data, or by filling in a web-based form, or by manipulating the advertisement (for example, Once you have drank, please move the ad towards your Bedoop sensor if you drank coffee this morning "), and richer statistical data may be collected.

Rolodex of cards

Bedoop-encoded business cards as described above can be accumulated and kept close to a phone or computer in a rollodex-like configuration. If the refrigerator ice maker malfunctions, the home owner can find a card from a home repairman who was used a few years ago and provide it to the Bedoop sensor. A link is established to the repairman's company (eg via a web site or by phone). At the website, the repairman can provide basic information such as available time, current demand schedule, and the like. The home owner may choose an option to trigger a conference call (eg, NetMeeting) to discuss the problem (by card gesture or otherwise). Or, the home owner can choose another option to send an email. Another option allows homeowners to schedule home calls from the repairman's weekly schedule. Another option may allow homeowners to watch one or more short videos that instruct consumers on how to fix certain common home appliance problems.

Storage value card

Electronic money systems (eg, of the type described in US Patent 60 / 134,782, filed May 19, 1999) store Bedoop data on a card that leads to a stored storage of random number tokens (indicating an increase in money). Can be encoded. When the card is provided to the Bedoop system, an application is launched that reads the token, encrypts it, passes the encrypted data to the corresponding bank's clearinghouse computer, and learns the rest of the value. These tokens are decrypted and validated (though not recovered). The bank computer responds to the Bedoop system, indicating the remaining value of the token in the card.

For security reasons, storage containing random number tokens should not be much accessible. Instead, the user must provide authentication data indicative of authentication in order to gain access to that information. This authentication data may be a PIN code. Alternatively, the user can provide authentication by providing a second Bedoop-encoding object, such as a driver's license, to the Bedoop system. (Many other Bedoop systems are advantageously required to use or use two or more Bedoop objects-either one after the other or all at the same time. The Bedoop system provides a visual or audio prompt, Have the user provide another Bedoop object (s) as needed.)

Ski lift tickets

According to another embodiment, ski list tickets are Bedoop encoded to provide various functionality.

For example, instead of purchasing a lift ticket for a day, a skier may purchase a ticket product for eight lifts. This data is encoded in the ticket and can be sensed by Bedoop sensors in each lift. The sensor is networked to a common server that tracks the number of lifts actually purchased and updates that number when used. Skiers are known the number of times they remain when entering or leaving the lift. Statistical data can be collected for trail use (eg, N% of skiers ski along the lift twice a day).

Upon exiting the slope and back home, the lift ticket used may be provided to the Bedoop sensor to find out the current snow conditions and lift times, observe trail maps, or order ski vacation packages. If the ticket is encoded with an owner name, UID, or other commercial / marketing interest information, the local seller may provide a carry discount on the selected product in response to a Bedoop scan of the ticket and recovery of this information.

REI membership card

The membership card for a particular store can be Bedoop-encoded to provide additional value to that member. In an external gear shop, such as a REI, a card can be provided to the Bedoop sensor to reach a library of USGS maps, a web page containing current fishing and hunting rules, and the like. Originally, the store's online casting site is just out of twist.

Theme park tickets

Theme park tickets can be encoded with the visitor's age and gender, and additional information allows the experience to be ordered (eg, from a theme park personal roster, Indiana Jones likes). Throughout the park there is a kiosk where visitors can provide tickets to adjust the visitor to follow a specific story line. Some kiosks convey the privilege of matching the recipient's age / gender.

Car keys

The car key (or key chain) is Bedoop encoded. When the car enters the store for service, the machine provides the Bedoop sensor to the key to obtain the repair history of the car from the remote server maintained. In the home, the key can be provided to the Bedoop sensor and manipulated to navigate through various car-related websites.

In some embodiments, the Bedoop-encoded object is not used to navigate to the site, but instead is used to provide data once the user's computer is connected to the website. Web users who have finished at car value sites can provide the key to the Bedoop scanner. Bedoop data is used to access remote databases that store the automaker's manufacturer, model, and options. This data is provided to a database engine that returns the value of the evaluated car to the user.

When visiting a mechanical web site, the provision of a key or key ring can be used to schedule service reservations for a car.

Fashion coordination

Some department stores and apparel retailers provide "personal shopper" to perform various services. For example, a customer who purchases a dress can ask a personal item shopper to help them choose accessories or shoes for the dress.

Bedoop encoded garment tags on the dress can be used to obtain similar assistance. In response to such a tag, the Bedoop system can query the database to obtain a mini catalog of clothing and accessories previously identified as matching the dress identified by that tag. Each of these items may be displayed on a screen associated with the system, or the virtual model wearing the dress, together with one or more recommended accessories, may be shown composited. The store quickly observes the appearance made by the model wearing the dress along with various other pairs of shoes by repeatedly activating user interface controls that cycle through different combinations (by mouse, screen touch, or clothing tag gestures). can do.

The purchaser's credit card allows the Bedoop system of a particular store (i.e., stores pre-certified by the purchaser) to the buyer's profile (including size information, repeat purchase information, return history, style / color preferences, etc.). Can be encoded.

Credit card purchase

When a consumer wants to visit a commercial web site and purchase a displayed product, the transaction can be accelerated by simply providing a Bedoop-encoded credit card to the Bedoop sensor of the user's computer. Bedoop data on the card leads to a database entry containing the credit card number and expiration date. The Bedoop application then sends this information (optionally after encrypting it) to the website with an instruction to purchase the illustrated product.

(The impulse purchase is usually interrupted by an obstacle that lies between the impulse and the complete purchase. This Bedoop application and other applications help to reduce these obstacles.)

Product marketing

Bedoop data related to one product or service may be used to cross-sell other products and services. Consider a consumer buying a pair of golf shoes. The box is Bedoop encoded. By providing the box to the Bedoop system, the consumer is linked to a web page that provides various promotional offers. The consumer may, for example, choose to enjoy a free round of golf at one or more identified local golf courses or to print a coupon that discounts a 10% off sock order from an online sock seller. (A variety of means can be used to prevent multiple retrieval from a single box. One is a serial number that is tracked and assigned only once by a web page or cross-seller seller. The other is tracked to prevent retrieval multiple times. Identification data corresponding to the consumer.)

Product tags can similarly be Bedoop-encoded. Tags from articles of the Nike Apparel can lead to the Nike online store, where users can purchase more. Given a tag from a football shirt, a particular tag manipulation (eg, rotating to the left) leads the user to a football page of particular interest, such as the World Cup. The tag on the golf glove can lead to the website of the local golf code. Twist to the left to reserve a tee time; You can twist to the right to see course maps and statistics. In retail stores, Bedoop kiosks can be provided to allow consumers to utilize Bedoop features.

Travel planning services

After booking at the resort, the consumer typically receives various confirmation information (by email or conventional mail). If not already printed, the consumer can print this information (e.g., a confirmation card).

Bedoop-encoding in the printed object can lead to web-based information related to the reservation (eg, reservation number, customer's name, arrival / departure date, etc.). If the consumer wants to make a dinner or golf reservation, this object can be provided to the Bedoop system-at the user's home or airport kiosk, the system recognizes the object type and encoded data, and various information and schedules for the resort. Link to the remote computer providing the service. By manipulating the object (or otherwise), the consumer selects the desired dinner and golf tee times. Since the system already has a reservation number (indexed by UID), this data is prevented from being tediously provided.

In some embodiments, the remote computer is not maintained by the resort, but by an independent travel service. (Travel services can also maintain a DNS leaf node server.) The computer provides a web page (branded by travel service, etc.) that suggests schedule options that the user desires, and also provides links to other information and services. (For example, provide an entry ticket to a nearby downtown area and advertise a nearby restaurant).

Ticket tickets (or e-ticket confirmations) may be similarly encoded with Bedoop data. These items are provided to the Bedoop system-at the traveler's home or airport-to observe and change travel itineraries, booking hotels, and car rentals, ensure 1st class upgrades, check flight seating arrangements, and frequent aircraft status. Allows to observe and scan traveler information about the destination.

Movie tickets

As described above, the movie ticket may be encoded, for example, with Bedoop data identifying the movie title and data. When movie goers return home, they can provide half of the ticket to the Bedoop system. One of the options provided by the corresponding Bedoop application is to be able to start the paid movie of the movie just viewed at a discounted rate. Another is to download the movie to a recordable DVD disc at the spectator's home, possibly allowing playback only on the spectator's DVD player or playing it only a few times (again, similar to a discounted price). Another option is to provide a web-delivered video clip from the movie. Another is to offer related bargains for purchase at discounted retail prices. (This feature is only available for a limited period of time after the data has been encoded on one half of the ticket.) Another is to introduce consumers to the same genre, the same director or actor, or a new movie in the same studio. Another is to link a web browser to online ticket sales of tickets for other movies. The consumer can navigate between these options by manipulating the ticket halves or otherwise.

Equal or related options may similarly be provided in response to Bedoop data detected from a book cover given to the Bedoop system.

Video record

The video recording device may be programmed to record a broadcast program by providing the Bedoop sensor with printed promotional content (eg, an advertisement in a newspaper or TV guide) for the program. Within the printed document is data Bedoop-encoded which allows the Bedoop system (which can be installed or detached from the video recorder) to set the time, date, and channel recording.

Set top box

Many entertainment-related applications of Bedoop data can be implemented using television set top boxes. Such a box contains a processor and typically includes a return channel to the control facility. The provision of Bedoop chips and optical sensors can greatly increase the functionality these devices currently provide.

Special event ticket

Consider a ticket to play basketball. By providing a ticket to the Bedoop system, a user can access a team's website to view current scores and statistics. The user can also obtain a web-based stadium virtual tour to view seat tickets. A view of the arena can be given that can be seen from the user's coordinate position. In addition to tickets for the next game, you can order pay per game and team memorabilia. For expensive tickets, the user may be entitled to premium web features such as an online text-, audio-, or video-chat session with a team star the day before the game.

Unlike conventional tickets, Bedoop-encoded tickets do not need to restrict the user to a given seat. When the ticket is printed with a nominal seat, the user can provide the ticket to the Bedoop system and access a website where other seats can be reserved. Upon attending the event, the consumer presents the ticket to a Bedoop sensor that reads the ticket UID and finds the seat arrangement most recently selected by the consumer. It then prints a slip that the consumer may not have on the seat first selected from the transactional website.

Signet ring

The signet ring was used to identify an individual's identity or office. Personal valuables of such rings or other items may be encoded (by texturing or printing) into Bedoop data and provided to the Bedoop system as needed. The extracted Bedoop data may lead to a secure web site representing the personal name and other information (i.e., a web site with hacking prevention means to prevent illegal changes of stored identification information). Such a tension ring may be provided for Bedoop systems that require high reliability identification / certification confirmation prior to proceeding with the Bedoop function.

Tattoo

Temporary tattoos are already known and typically include ink or print films that are transferred to the skin of a person through some application processing (eg, through wet or other methods). The work of the tattoo consists of Bedoop data encoded using steganography, which facilitates machine recognition of the tattooed person (or object). Young people can compile a contact database by simply taking a digital picture of their friend using an imageable personal digital assistant. This computer device decrypts the watermark, accesses a corresponding web document of information about the person photographed, and also adds this information to the contact database.

E-pager

While it is clear that a large number of printing techniques can be used in Bedoop applications, it should be noted that e-pagers may also be used. Nicholas K of Xerox. The E-Paper, developed by Nicholas K. Sheridon and mass-produced by 3M, is a thin film of transparent plastic interspersed with millions of small beads, such as toner particles. Beads, each contained in a cavity filled with liquid, are free to rotate in these cavities. Beads are “bichromal” with hemispheres of contrasting colors (eg black and white). Under the influence of the voltage applied to the surface of the sheet, the beads rotate to be provided to the viewer on one collar or the other side. The pattern of voltages can be applied to the surface in bitwise form to create an image such as text and an image. The image remains until a new voltage pattern is applied to create a new image. The reader is assumed to be familiar with the US patent granted to Sheridon in the art.

The e-pager can basically be used to deliver digital data according to any known watermarking method, and is recognized as suitable for delivering digital information in the form of data glyphs.

Post-It Notes

Post-it note pads or other paper pads may be displayed by the manufacturer (such as texturing, watermarked coloring, ink-jet spattering, etc.) to convey steganographic data (e.g. Bedoop data). Can be. When such a note is given to the Bedoop system, the system can launch an application that stores a snapshot of the note. More specifically, an application can mask the note portion of image data from other image data, virtually remap it to a standardized pixel-dimensional square format, JPEG-compress the resulting image, and also color it in the note. And / or a name indicating the date of image capture, together with the size, in a particular computer subdirectory. (These two data can be represented as data contained in the Bedoop payload.) If the color of the note is represented as digital data (eg in a file name), the image itself can be stored in grayscale. . When later called for display, the white image background can be colored in color according to the digital color data.

The Bedoop system can buffer several past frames of image data. When an object is recognized as a post-it note where the image should be stored, the system analyzes these frames to identify the one that is most suitable for storage (e.g., the image of the note imaged in each frame to identify one with the finest content). Check the spatial frequency content) and store it.

When a post-it note is recognized by the Bedoop system, the system emits a confirmation tone (or other response) to indicate that the object has been recognized, but may not immediately perform a snapshot operation. Instead, the system may wait for another instruction (eg, a gesture) to indicate which action you want.

For example, by moving the note towards the sensor, the user can signal that the snapshot operation is to be performed. (Providing a closer note also allows the system to capture more detailed frames of image data.)

By moving the notes away, the system can read, decompress, and display the six recently stored post-it note images in tile form on the computer screen to respond. Each note can be displayed in its original dimensions or resized to fill the entire height or width of the tile. User interface controls allow the user to scroll back to the desired date (in response to gestures, mouse movements, keyboard scroll arrows, etc.).

The full 64-bit Bedoop payload of other embodiments may not be needed for payload notes. In the example just given, for example, the Bedoop system responds to all post-it notes in the same form. Thus, the short Bedoop format, which simply indicates' Post-it note, yellow, size 3 "x 3", can be supplemented. The 12 bit CLASS ID is sufficient with another 8 bits representing the color / size combination. Reducing the payload allows for more reliable coding on small objects. (As described later, the Bedoop decoding system may find several different data formats / protocols when attempting to extract Bedoop data from an object.)

Arranging Documents for Other Purposes

While triggering the Bedoop response when the just-labeled paper just described is given to the Bedoop sensor (ie, taking a snapshot of the paper), the marking can be used for purposes other than triggering the Bedoop response. have.

Regardless of the specific data on which paper is encoded, registration data encoded using embedded grid lines or other steganography may be used by other applications to correct misalignment of scan data. For example, in a photocopier, the document does not need to be exactly squared on flat glass to yield a properly aligned photocopy. The scanner scans the oblique document and then detects steganographic registration markings in the resulting scan data. This data is then processed to virtually re-register it so that the registration markings are in the desired alignment. The processed scan data is then provided to a xerographic reproduction unit to produce a photocopy with the tilted effect removed.

The same technique can be similarly applied to video recorders, digital cameras, and the like. If such a device images an object (eg, a photo) with steganographic registration markings, these markings can be used as a guide when re-registering the resulting data to remove the misalignment effect.

Mailing information

Many contexts arise that are valuable only when the data given to the consumer is appropriate. Postal service mail is not suitable for some of this information because of the potential between printing the document and finally delivering it to the recipient. However, the Bedoop principle allows the recipient to have a mail object that was printed prior to delivery and use it to receive up-to-date information upon receipt (ie, when providing it to the Bedoop system). In these and other embodiments, Bedoop data also uniquely identifies an address / recipient / user, so that a web site can provide data tailored to that user.

Distributors of print advertisements can reward Bedoop-driven consumers who visit a website by issuing digital tokens or coupons that can be redeemed for premiums, cash-backs, and the like. Every millionth visitor will receive $ 10,000 (in proper defense, for example, by preventing more than one entry per hour).

Classes of Bedoop Encoding

The embodiment described above focuses on the use of Bedoop data after decoding. Additional insight can be obtained by examining the initial part of the process-encoding.

Encoding can be formed in many contexts and conceptualized as falling into three broad classes. The first class is static marking, where document designers, pre-presser service bureaus, advertising agencies, etc. insert Bedoop data into it. The second class is dynamic marking, where an automated system encodes or changes Bedoop data "on the fly." Such a system may specifically tailor Bedoop data to fit the context, for example, the moment, place, user, and so forth. The third class is consumer marking, which is added to a document when Bedoop data is printed.

Encoding of the second class enables features that are not available from the first class. Consider an American Express travel web page with information about traveling to Hawaii. The DNS leaf node server directs this page in response to specific Bedoop data-for example, data encoded in a magazine photo of a Hawaiian beach scene.

In fact, all Bedoop data with a particular CLASS and DNS ID can reach this web page, regardless of the UID data. If a magazine photo is encoded with a particular "don't care" UID field (e.g., 11111111111111111111), this may be applied to an outgoing Bedoop system where any data can be inserted into the UID packet of that Bedoop packet. Alternatively, the signal can be passed to the system between Bedoop data passes. The originating Bedoop system may, for example, insert a dynamically configured series of bits into this field. Some of these bits provide a remote server in the user's profile so that Bedoop responses can be customized to the user. (Users will, of course, pre-approve information for use in mitigating prior interest.)

As an example, the local Bedoop system may set the least significant bit of the UID field to "0" when the user is male and set to "1" when the user is female. The next four bits may indicate the age of the user in one of sixteen age ranges (eg, 3 years old or younger, 4-5, 6-7, 8-9, 10-11, 12-13, 14-15). , 16-17, 18-20, 21-24, etc.).

Alternatively, or in addition, the local Bedoop system may be based on signature data that tends to uniquely identify the local Bedoop system (e.g., based on the system serial number, unchanged data unique to that system). You can fill in the don't care UID field (all or part of it). With this data, the remote server can identify repeat visits by the same user and tailor the response accordingly (e.g., to retrieve a profile of information previously entered by the user and stored on the remote server, By eliminating the need for data re-entry).

Learn more about optical input devices

Image input devices are expected to be common soon. Providing a digital camera as a built-in component of a particular computer (eg Sony Vaio laptop) is one indication of this trend. Another is typical of US Pat. No. 5,841,126 and described in Nixon, “256x256 CMOS Active Pixel Sensor Camera-on-a-Chip,” IEEE J. Solid-State Cricuit, Vol. 31 (12), pp. 2046-2051 (1996) and Fossum, "CMOS Image Sensors; Electronic Camera-on-a-Chip", IEEE Transactions of Electronic Devices (IEEE Transactions) of Electron Device), vol. 44, No. 10, Oct. 1997, is a camera-on-a-chip system. Another is a head mounted camera (such as currently used in some computer-enhanced vision systems). These image input devices and others are all suitable for use in Bedoop systems.

On-chip camera systems can be equipped with Bedoop detector hardware integrated on the same chip substrate. This hardware can be configured to find and decode Bedoop data—even scale, rotation, differential scaling, and the like—from image data. Gesture decoding can also be provided in hardware, and the resulting data is output in packet form on the serial output bus. Thus, such a chip can provide multiple outputs-image data (in raw pixel form or as a data stream representing an image in one of various image formats), 64-bit Bedoop data (in parallel or in parallel), And decryption gesture data.

In another embodiment, the Bedoop detector (and / or gesture decoder) may be on a substrate separate from the camera system.

To accommodate other Bedoop data formats and protocols, the hardware may include RAM or ROM in which other format / protocol information is stored. (These different formats / protocols may relate to Bedoop systems using, for example, different data payload lengths, different constraint grids, different encoding techniques, etc.) When the Bedoop system captures / analyzes a frame, each Frames may be analyzed according to several different formats / protocols in order to find and attempt to format / protocol that yields valid Bedoop output data.

Movable Bedoop Sensor

Although the Bedoop systems described are generally static, they do not have to be. It can be portable. Such a system uses, for example, a palmtop computer with an array of optical sensors. Once a palmtop is provided with a live network connection (eg, wirelessly), Bedoop applications that depend on a remote computer can be implemented as described above. If the palmtop does not have live network connectivity, Bedoop applications that rely on remote computers can simply queue up communication, and when the next palmtop has remote access (for example, the palmtop is placed on the next recharger and Internet access is lost). When connected to a modem that can be made).

Another variant is a Bedoop sensor with an array of 1D or 2D photosensors (eg CCD or CMOS) that are movable around a desk or other work surface, such as a mouse. Such a sensor can be connected to a computer associated with a cable or a wireless interface can be used. The peripheral is configured such that the object is placed on top of the item to read the marked digital data. (Because devices can shield encoding, built-in illumination may be required.) Some types of peripherals are adapted to operate as both general-purpose digital cameras and Bedoop sensors.

These peripherals find many applications. For example, when "reading" a magazine or book, it may be more intuitive to place a Bedoop reader "on" the object being read, rather than keeping the object in the air in front of the Bedoop sensor. This is particularly useful when the user wants to confirm that a magazine page or the like has some differently coded Bedoop sections (corresponding to other articles, advertisements, etc.) and that the desired Bedoop-encoded sections are read.

The "bookmark" archetype of an internet browser may be supplemented with Bedoop data encoded on a paper bookmark, eg, one or more paper pages. In order to connect the browser to a particular bookmarked destination, the peripheral is simply placed on top of (or in part of) a page marked with corresponding Bedoop data. The user prints a "map" consisting of stamp sized tile areas, each area representing a preferred web destination.

This map can be printed on the mouse pad. That is, a mouse pad with a specific map pre-encoded is suitable as a promotional material. The company can provide for printing family photos on these pads. Within the photo or pad text, the address of the web site that paid for this method of access from the user's desk is encoded.

In this and other contexts, it is recognized that the gesture input concept described above involves relative movement between the sensor and the encoded object. In most of the examples, the sensor is static, so the gesture is made by moving the object. Of course, if the sensor is movable (such as a mouse or a portable phone), gesture movement can be accomplished by moving the sensor instead.

In the above, the configuration of a particular embodiment has described in detail a modified version of Microsoft IntelliMouse with IntelliEye technology. The device includes multiple element optical sensors integrated on the IC with various detectors and processing circuits, and operates in conjunction with short focal length imaging lenses and LED illumination sources (all available from Agilent as will be described later). . The circuit tracks the movement of the pattern through the field of view of the sensor, whereby the movement of the mouse can be deduced. Microsoft products collect 1500 data sets per second-this is a much higher frame rate than is normally required for this application. This embodiment combines the functionality of the mouse with the Bedoop image sensor.

Such peripherals, such as a mouse, can omit the button and position-sensing features typically found in traditional mice, making a simple desk palm camera that generates a data frame corresponding to a small area under the sensor portion of the mouse. However, more typically, peripherals are buttons, roller wheel, and / or X- / Y-position sensing configurations of a traditional mouse so that the button and position forms of data input can be utilized when interacting with Bedoop applications. It includes.

Optical data collected by the sensor can be processed within the peripheral processor to extract binary Bedoop data encoded therein using steganography. Alternatively, this processing burden can be handled by the associated computer system, and the peripheral simply processes the raw sensor data and formats it into a sequential frame of image data and outputs it to that system.

Scan peripherals of the type described above are typically wired to an associated host system, but wireless links (e.g., wireless, infrared, ultrasonic, etc.) may of course be tolerated, allowing the user to Can be free.

Hand scanner-more details

To explain the above concepts in detail, the following paragraphs detail the hand scanner, which is a model particularly after the Hewlett-Packard CapShare 920. The scanner is configured to be used with any kind of identifier, for example watermarks, bar codes, OCRs and the like.

The reader is assumed to be familiar with the operation of the HP CapShare scanner. This information is provided in a technical document, for example Allen Allen Ross R. "Processes for Freehand Image Capture: HP CapShare Technology". The issue was published on 26 March, at the Society for Imaging Science & Technology PICS Conference, pp.43-46. CapShare scanners are available in part number HDNS2000 (IC sensor chip), HDNS-2100 (lens with light pipe), HDNS, described in Agilent Application Note 1179, available from Agilent Technologies, Palo Alto, CA. An optical sensor tracking engine assembly consisting of -2200 (LED clip), and HLMP-ED80 (639 nm LED) is used.

As shown in FIG. 13, the CapShare scanner 210 functionally includes a CPU 212, a memory 214, a linear CCD array 216, first and second 2D CCD arrays 218a, 218b, a user interface ( 220, including an LCD screen and associated buttons, and an interface 222 (eg, serial port and IRDA port / transceiver) to the secondary device 224. Although not specifically shown, each CCD sensor has an associated LED system to illuminate the object being imaged.

Another enhancement to the CapShare scanner is to provide a wireless Internet interface to interface 222, allowing direct communication between device 210 and the Internet.

The linear sensor 216 operates in a conventional manner, accepting continuous abnormal pre-scans under the scanner at a pixel data rate of about 18 Mbits / second (allowing full accurate scan even at instant scanner movement speeds up to 16 inches per second). ).

The two 2D CCDs 218a and 218b are spaced apart adjacent to the linear sensor and are used to track the movement of the scanner. The area scanned by these CCDs is irradiated obliquely by IR light, emphasizing the microscopic media surface properties. The CPU identifies the pattern in the surface characteristics detected by each of these CCDs and tracks the movement from one frame to the next to distinguish the movement of the two CCDs. Knowing the movement of the two CCDs, the movement of the scanner itself can be determined. This scanner movement information is used to remap the wide scan data received from the linear scanner array into composite pixel data with uniform scanline spacing. This remapping data is typically the final scan data provided to the terminal user or application.

Determination of linear and 2D CCD arrays is left to the designer. If 600 dpi scan resolution is desired over a 4 inch scan width, a 2400 CCD linear CCD is used. 2D CCDs are typically smaller in area. For example, it has an opening of 0.25 inches by 0.25 inches. The space of the CCD element is selected depending on the size of the surface characteristic to be tracked.

Memory 214 includes both RAM and ROM. RAM is used for both raw data storage and final result storage. The operating system of the scanner, pattern matching and data processing algorithms, and other program code are stored in ROM.

In accordance with the described embodiment, the ROM code also includes instructions for examining the final-result scan data (ie, after remapping) for the watermark data. In one particular embodiment this is a two step process. The final result data is first screened by the CPU to determine whether it has a particular hallmark associated with the watermarked data (eg, the presence of a measurement signal as described in US Pat. No. 5,862,260). If such a hallmark is found, a more computational and stringent watermark decoding algorithm is executed. By checking the watermark information in this two-step process, CPU time is unnecessarily consumed and attempts to extract a watermark from image data that does not have a watermark clearly.

In some embodiments, the watermark processing operation (s) occurs without user intervention when raw scan data is processed and remapped to the final result form. In another embodiment, the watermark processing is started in response to a user command entered through the user interface 220. In another embodiment, watermark processing is initiated in response to instructions provided to the scanner from the associated secondary device 224 (eg, local personal computer, remote server computer, special Internet appliance, etc.).

In some embodiments, the decrypted watermark payload data is sent to the associated secondary device each time such data is detected. In another embodiment, no data is transmitted unless the secondary device 224 first asks for it. Such an auxiliary device request may be made by a software program running on the auxiliary device, for example, via an API call.

The watermark decrypted by the scanner may be provided to the user on the LCD display 220 of the scanner. Alternatively, the scanner may convey the decrypted watermark to the remote device 224, which may respond with supplemental data to be provided to the user, as described herein.

In some embodiments, the information obtained by the 2D CCDs 218a and 218b may be used to augment the information provided to the CPU to aid in the watermark detection / decoding process. For example, the 2D sensor provides 2D information immediately-without the time delay associated with remapping the 1D data from the linear CCD array to the final form. This immediate 2D information is analyzed for the presence of the measurement signal-perhaps shortening or eliminating the analysis of the measurement signal in the final data.

2D data can also be used to characterize the texture of the imaged substrate. Knowing the substrate texture, the CPU is allowed to apply filtering or other watermark-enhancement / image enhancement steps to best suit the particular environment.

2D data also allows the CPU to access the relative quality of the pre-scan data of other areas in the final scan data. In FIG. 14, the object 230 is scanned by the scanner 210 traveling along a curved path, yielding row scan data 228 of continuous lines. The curve's geometry is revealed by the data collected by two 2D CCDs (which act as mobile encoders). If consecutive lines are spaced closer together (like region A), the raw scan data has a higher resolution (relatively "oversampling"). In contrast, when consecutive lines are spaced apart (such as region B), the raw scan data has a lower resolution (relatively "undersampling"). In the analysis and screen processing for the sensitive watermark data, the CPU may be instructed to look at area A first because the data in the higher query in area A is more likely to contain available watermark information. (In some embodiments, watermark stream processing and / or detection may proceed with reference to the scanline data rather than waiting for the data to be remapped to its final form.)

The two CCDs 218a and 218b also allow various two-color processing to be performed to enhance or allow more intelligent processing of the linear scan data. For example, consider the case where the image being scanned does not carry a completely planar (eg straight) signal. The substrate may be warped, the image may be printed in an oblique manner, or some other transformation may occur that produces an image that changes orientation / scale over its range. One such example is shown in FIG. 15, which shows a watermark measurement signal (given here by an open grid to simplify the description). The two 2D CCDs 218a and 218b each sense a measurement signal directed in a different direction. Based on the differences sensed by the two CCDs spaced apart, the CPU infers scale, rotation, or other image transformation at the midpoint and applies compensation to the scan data to respond to this distortion.

This compensation need not be based on the watermark measurement signal. At two points apart in space, different optical sensing properties can be used to infer these or related properties at intermediate locations, allowing the appropriate compensation (s) to be applied.

Another application of sensors 218 spaced apart is to optically determine the distance from the scanner to the object being imaged. A scanner 10 having two CCD sensors 218a and 218b spaced apart and imaging warped objects 232 (e.g., warped paper substrates or curved objects such as beverage cans) Consider FIG. 16, which is shown. In the example shown, the first CCD 218a detects a very active image of an imagery or surface texture, while the second CCD 218b detects an image similar to the first CCD, as though low pass filtered. Not cloudy The software instructions for the CPU 212 indicate that the scanner recognizes that some of the objects 232 under the CCD 218a are directly adjacent to the scanner, while some of the objects under the CCD 218b are somewhat spaced from the scanner. Let's do it.

Having information about the space of the object 232 from other parts of the linear sensor array allows certain compensation to be made in the data collected by the linear scanner. In the case of watermark detection, the CPU may recognize that the data from the terminal portion of the linear scanner 216 remote from the object 232 does not have meaningful high frequency information. In many watermark detection algorithms, the CPU can ignore the data from this portion of the linear scanner-focusing on some of the image data with the necessary high frequency components, allowing faster and more reliable results.

In other watermarking systems, two watermarks may be given-one delivered in dependence on the high frequency image components, and the other mostly encoded into the low frequency image components (i.e. expected when the object is imaged slightly out of focus). ). Recognizing that the linear scan data is weak in high frequency components, the CPU may be programmed to look for only low frequency watermark data.

As scanner 210 can perform watermark decryption, it can similarly perform barcode decryption. Conventional pattern-recognition algorithms may be applied to raw scan data or final scan data by the CPU 212 to identify barcode patterns. Once the pattern is identified, decryption is facilitated by applying a known barcode alphabet. As in the watermark case, barcode decoding can be performed autonomously or in response to user / secondary device commands. Similarly, decoded bar code data may be provided to the secondary device whenever detected or in response to the secondary device query. The 2D data is similarly used to augment the information provided to the CPU to aid in the barcode detection / decoding process.

In a similar form, the above-described advantages associated with two 2D CCDs can similarly be applied in the barcode context.

As described above, once the identifier information is extracted from the image data, the internet link may be based thereon to provide supplemental information, e-advertising opportunities, and the like. In many implementations, scanner UI 220 is used to provide supplemental information to a user, for example, by software instructions where HTML instructions are made to be presented on a UI display screen. UI controls (eg, buttons) can similarly be used to receive user instructions and commands to reconnect to the Internet.

A portable telephone that is a Bedoop device; GPS receiver

Bedoop technology can be integrated into portable communication terminals such as portable telephones (eg, manufactured by Motorola, Nokia, Qualcomm, and others). Such a telephone is equipped with a 1D or 2D image sensor and its output is applied to the Bedoop decoding circuit within the telephone. This decryption circuit may be the main CPU of the telephone, or may be a processing circuit dedicated to the Bedoop function. (As in other embodiments, the decryption may be performed here by dedicated hardware, decryption software implemented on a general purpose CPU, etc.)

Portable phones already have a variety of features that make them very suitable for Bedoop operation. One is that the portable phone typically includes an LCD or similar screen for the display of texture or graphical information, and additionally includes buttons or other controls to select from menu options given on the screen (e.g., by moving the cursor ). In addition, portable telephones inherently include both audio input and output devices (ie, microphones and speakers). In addition, since the protocol by which the portable telephone transmits data includes data identifying the telephone, such data need not be separately encoded. Also, finally, the portable telephone certainly provides a ready link to the remote computer system. In aggregate, these functions are comparable to those of the best equipped desktop computer system. So basically, all the applications detailed in this specification can be implemented using a portable telephone Bedoop system.

For other Bedoop systems, when Bedoop data is detected, the phone can either locally respond to that data or communicate it to the handling remote system (or computer network) via the cellular network.

One application that can be invoked locally (ie within a telephone) is dialing of the corresponding telephone number. In some embodiments, the telephone number is encoded as part of the Bedoop data payload. In another embodiment, phone numbers are stored in memory in the phone and indexed according to identifiers decrypted from Bedoop data.

The various actions that can be handled remotely are practically unlimited. Some involve interaction with the user. For example, a remote system may initially display a menu of options on the display screen (e.g. Purchase, Add to Shopping List, Request Sample, Add to Notepad). ), Etc.) to the user. The user then responds by providing another input (eg, by operating one or more buttons on the phone, by providing a generation indication to the voice recognition sub-system in the phone, and so on). This further data is dispatched from the phone to take the required action. Other actions do not require another interaction with the user, but immediately trigger a corresponding action.

The above-described configuration includes Bedoop decryption functionality in the telephone, but in other embodiments image data may be transmitted from the telephone and decrypted at a remote location.

GPS receivers are consulted when a person's location is tracked and thus allows for contact with updated information. The GPS receiver finds many other applications in the Bedoop context. For example, the Bedoop system's response may be tailored or vary depending on the location of the person invoking the action. For illustrative purposes, if a user provides a dopedop sensor on a GPS-equipped mobile phone with coupons or coupons for domino pepperoni pizza food, the GPS data will be encoded into Bedoop data dispatched to Domino's Bedoop server. Can be. The server can consult the GPS data to identify the location of the nearest domino shop and send the order to the shop. The store actually calls the user (with reference to the telephone number data from the mobile phone) to confirm the order, asks if they want additional items, informs the user of the final price, and also accepts the delivery order. (Later steps may be omitted: the store may use GPS data to obtain a corresponding street address from map data licensed through, for example, NavTech in Chicago.)

Protocols in which Bedoop data, GPS data, and portable telephone identification data are communicated from a telephone to a cellular network can take various forms; The design of such a system is similar to the related art. In one embodiment, the protocol in which some portable telephones are provided with email or Internet access also applies to delivering Bedoop and GPS data. Protocols for carrying phone identification data are already established. Once received by the cellular network, Bedoop data can be formatted into packets by including GPS data and phone data in the same packet structure or by formatting such data into separate packets accompanying the Bedoop packets.

Providing an image sensor in the portable telephone enables other functions. One is to capture still or video images. Such image data can be compressed (typically by loss processing such as MPEG, JPEG, etc., implemented with a dedicated hardware CODEC) and transmitted with the audio data. The screen on such a phone can be used to display similarly incoming image or video data.

Another function enabled by the image sensor in the portable telephone is to verify the user by retinal scan or other optically sensed biometric measurement before the telephone allows to call.

The cellular telephone Bedoop sensor does not always communicate with the cellular telephone network. The phone may be out of range of the cell site or in an operating mode in which no RF link is established. In this case, Bedoop data sensed by the remotely handled telephone is preferably stored locally within the telephone and waited for transmission to the cellular network where the next time communication takes place (“store and forward”). "Also called a motion of form".

Catalog advertising

The form of the handheld scanner—whether just described or otherwise known in the art—provides a convenient way to interact with the catalog advertisement. For example, L.L. Imagine a traditional paper catalog from Bean or Lands End. Each image in the catalog is Bedoop-encoded with a code that identifies the depicted product (if necessary, the producer, etc.). A user browsing the catalog sees the product of interest and places the scanner on the image (optionally pressing a button or requesting another signal to initiate another process). The scanner detects Bedoop data and relays it to an associated computer (optionally with data identifying the consumer). The computer elects a remote subcomputer maintained by the vendor and responds with data corresponding to the item shown in the scanned image. This return data may include data indicating available sizes, data indicating available colors, data indicating available styles of change, flag bits indicating whether each item is in storage, and the like. This return data can be given to the consumer-typically in a display device, in a form that can be heard in other ways.

Preferably, the consumer's body measurements (distance size, inner seam length, neck size, etc.) are stored in the user profile on the local computer, the seller's server computer, or the third party service provider's computer. This allows the system to tailor the data provided to the user-for example, only the color options and the availability for dedicated shirts with 16 inch neck and 34 inch sleeve length.

If desired, the user can select between color or style options using a portable input device (such as a button or gesture) or other input device. Or it may be an item that does not need more designation. In any case, once enough desired products are specified, the user sends a signal to the system to place an order. Details of payment and delivery may be determined through various techniques known in the art, for example, by paying by credit card number and shipping to an address on the merchant file.

Revenue sharing

When a computer provides a Bedoop-encoded object to a sensor and, as a result of the link established by it, purchases a product or service, the revenue from the transaction can be shared with the participants who made it possible. In the case of Bedoop-encoded magazine advertising, some participants: (1) the photographer or graphic designer who created the product used in the advertisement; (2) the advertising agency that led to the creative attainment; and (3) the magazine in which the consumer created the advertisement. (4) service provider (s) that provided the transport channel (s) between the consumer and the supplier; And (5) a service provider that eventually maintains a server that connects Bedoop data to the provider's web site.

Gambling applications

Casinos around the world deploy significant resources every year to combat cheating. To this end, game cards and chips used for games and bets may be encoded with digital watermarks and used to enhance security. In order for the application to work, game cards and chips are encoded with unique data for each casino, game, card deck, and / or face value. Then, a camera placed on the game table is used to read the information of the cards and chips. Information from the card can be used to record or warn of suspicious activity in the following ways.

A camera placed on a card game table (such as Blackjack) can read the encoded data behind the card to determine its authenticity and face value. This can be done by flagging a card that is not part of the deck currently in use, or by flagging a card that is a different value than the one distributed to the player, so that the camera can find someone trying to replace the card after it has been dealt. Means. If the player exchanges cards after they have been distributed, the watermark reader in the camera may warn the security of the cheating.

In addition, the data encoded in the card can be used to measure the winner in the table. If the dealer and the player collide, the player may be assigned a winning card on a regular, predetermined basis (based on the dealer tricking and mixing). By observing the pattern and facet values of the cards being dealt, it can be determined whether the player wins much higher than the average rate. Again, guards can warn of suspicious activity.

A camera placed on a game table where chips are used (not specifically limited to card games) may be programmed to look for a watermark on the chip. A chip that does not contain an appropriate watermark may indicate that the chip may be forged. In another situation, the face value of the chip may be determined to be placed on the table so that the exchange of chips is noted after the game begins.

The above principle can be applied to a monitoring and viewing system that uses digital watermark processing to more generally identify the monitored object.

Part II

As described, the description focuses on a particular application-a system for linking printed media to electronic content. However, it should be noted that the technology is not so limited, and more generally observed as a system for linking objects (physical or electronic) to the corresponding network or local resource.

According to an exemplary application of the technologies described in detail below, digital watermarking is used to convey multiple bits of identifiers in printed media such as magazine advertisements or articles, direct mail coupons or catalogs, cash cards or credit cards, and business cards. Is used. This identifier is read by software at the user's computing device and communicated to the remote device. The remote database identifies the URL corresponding to the identifier and provides the URL back to the user's computer-allowing the browser on the user's computer to display a URL-identified web page. The web page may provide additional information or service-it may be provided in a timely and / or more extensive manner than that provided by the print. With this configuration, more effective internet navigation and access is provided to the consumer, and more effective means for linking the reader to the e-advertisement point of sale is provided to the advertiser.

Before beginning the description, it is helpful for the present invention to provide an overview of the larger system that forms part. As shown in FIG. 1, a larger system includes four basic processes-registration, insertion, detection, and response.

Registration refers to the process of assigning an ID to an object and associating that ID with the corresponding action or response. Additional steps may be included, such as logging the name and / or organization of the registrant, the name of the product, the description of the object, and the context in which it is found (magazine, book, audio track, etc.).

Insertion refers to the process of encoding an object with a digital identifier (e.g., a watermark that carries a serial number in the payload).

Detection is a supplementary action on insertion, i.e. the process of finding a digital identifier from an object.

The response refers to the action taken based on the identifier found.

Two intermediate steps-insertion and detection-include 1D and 2D barcodes, magnetic ink character recognition (MICR), optical character recognition (OCR), optical mark recognition (OMR), Radio frequency identification (RF / ID), data glyphs, organic transistors, magnetic stripes, metadata, file header information, UV / IR identifiers, and multiple bits of electronic or physical objects Any of a number of known techniques can be used, including other machine readable indicators and techniques for associating digital data. The detailed embodiment uses a watermarking embodiment, which is only described.

With reference to FIG. 2, system 10 according to a preferred embodiment includes an originating device 12, a router / server 14, a product handler 16, a registration database 17, and one or more remote resources ( 18).

The originating device 12 may take many different forms, such as a portable telephone, a personal digital assistant (eg, Palm Pilot), a personal computer, a barcode scanning system, and the like. For convenience, embodiments are described with reference to a personal computer for device 12.

Device 12 interacts with object 20. The object may or may not be electronic. The electronic object 20 may include display of a computer file, audio, video, or still image (eg, in the form of a file or a stream), and the like. Non-electronic objects may include physical objects such as newspapers, magazine pages, posters, product packaging, event tickets, credit cards, bills, and the like. Non-electronic objects may include sound provided by the speaker.

When used with non-electronic objects, device 12 (FIG. 2) typically includes some form of sensor or transducer 22 to produce electronic signals or data corresponding to the object. Examples include CCD or CMOS-based optical sensors (still or video cameras, flatbed scanners, mice, or parts of others), microphones, bar code scanners, RF ID sensors, mag stripe readers, and the like. Included. In this case, the sensor 22 may be connected with an associated interface electronic circuit 24, which is in fact connected to device driver software 26 connected to one or more application programs 28. The device driver software 26 acts as a software interface to communicate with the application program 28 at a relatively high level (e.g., via API instructions in which content and format are standardized to facilitate application programming) and Communicates with the interface electronics 24 at a low level.

The detailed embodiment contemplates that the object 20 is a magazine advertisement encoded with a steganographic watermark that carries a multi-bit object identifier. The watermark is hidden in the advertisement image in a way that is indistinguishable to the observer but can be detected by computer analysis. The analysis is performed by the watermark detector 30.

The watermark detector 30 may be implemented at various different locations in the system of FIG. Typically, the detector is implemented in the originating device 12, for example driver software, or application software 28c operative to connect to an external resource based on the detected watermark. However, it may be implemented elsewhere, for example in the hardware of the interface electronics 24, in an operating system associated with the device, or external to the device 12. Some systems have multiple watermark detectors and may be implemented at other locations throughout the system.

In the system shown, a watermark detector is implemented in the device driver 26. The function of the detector is made available to the application program 28c via one or more APIs specific to the function related to the watermark. One function is to read the watermark data payload from the object 20.

The illustrated application 28c is a software program that operates to communicate watermark data from the device 12 to the router / server 14 via one or more communication links 32 (eg, the Internet). Application 28c also receives information from communication link 32 and provides it to the user (or otherwise uses it).

The router / server 14 is a high performance computer that includes one or more CPU, memory, disk, and I / O ports. As is familiar to those skilled in the art, a disk stores operating system software and application programs along with data transferred to memory as needed by the CPU. The router basically operates between the application 28c and the product handler 16. As will be described in detail later, the router receives requests from the application, places them in the transaction log 15 and sends them to the appropriate product handler.

As will be described in more detail below, the handler 16 provides a response according to a particular watermark payload. The response may be sent directly to the device 12 by the product handler, or the handler may respond by communicating with a remote source 18 (which may be a data store or service provider, for example).

In the former case, the handler 16 identifies the URL corresponding to the watermark (using the database 17) and returns the URL to the application 28c. The application 28c sends the URL to the web browser 28b in the device 12 and initiates a link to the internet site identified by the URL. Alternatively, the handler may store some local data (eg, audio or video, or software updates) and send it to device 12 according to the watermark.

In the latter case, the handler 16 does not respond directly to the device 12. Instead, the handler responds by communicating with the remote resource 18. The communication can be as simple as logging the receipt of the watermark message at the remote repository. Alternatively, the device 12 (or its user) can be authenticated to the remote resource if it expects more transactions (eg, the communication may form part of an online authorization or digital rights management transaction). Alternatively, communication may require a remote resource to provide data or service back to device 12 or another destination (e.g., initiate an FTP file transfer, or select a song identified by a watermark). To download to a personal music library, or to update the software installed on device 12).

In another case, the hybrid of the two cases may be used. For example, handler 16 may communicate some data back to device 12 while communicating with remote resource 18.

In some cases, the response returned to device 12 by handler 16 (or remote resource 18) may operate to trigger some action by device 12. For example, the response returned to device 12 may include a window media audio file with a request that device 12 attempts to start a window media player installed on the device. (Starting the browser pointed to at the URL is another example of such a trigger.)

The illustrated product handler 16 basically includes the same hardware elements as the router 14, such as a CPU, memory, and the like. Although FIG. 2 shows only one product handler, multiple product handlers may be included in the system—either geographically distributed or co-located. Other handlers may be dedicated to other functions (e.g., URLs providing services, music providing services, etc.) or other watermark sources (e.g., one responds to watermarks found in audio, Another responds to watermarks found in print advertisements, etc.). It is also desirable to further specify (e.g. one handler responds to an advertisement by Ford and another responds to an advertisement by Chevrolet; one handler responds to Wired magazine). One may respond to an ad shown, and another may respond to an ad appearing in a time magazine or the like). In one particular embodiment, the router 14 receives data coming into one of several handlers according to (1) the vendor of the originating application 28c and (2) the specific identity identifying the application 28c. Dispatch.

The following discussion focuses on the data exchanged between the application 28c, router / server 14, product handler 16, and associated protocols in one embodiment.

Action concept

When showing a watermarked image, application 28c analyzes the image and extracts the embedded watermark payload (which will be described in more detail later) from the image. The application sends all or part of this information to the router 14 in a message format.

The router 14 decrypts the received message to find supplier and product information. Based on this information, it sends a message to the corresponding product handler 16.

The product handler receives the message and attempts to match the detected watermark serial number with the registered watermark serial number previously stored in the database 17. If a match is found, the product handler performs the desired action. As described, typical actions include the return of a URL for web retransmission, an HTML page service for initial user navigation, software download initialization, and the like. If no match is found, the product handler returns an error code and message to application 28c. If a match is found, but the corresponding action is not available, incomplete, inactive, or invalid, the product handler returns an error code and message to the calling application.

The contents generalizing the above are shown in FIG.

(Note that although the system is focused on a particular kind of object 20 and a particular provider's application 28c, the design is designed to support accessing product handlers from other suppliers in response to other objects. Make the system appropriately an information center for handling all machine-readable markings on the enable device.)

Below are the preferred detection and response cycles.

The system is intended to support a product from a supplier that can transmit a message over the Internet in accordance with the expected request format (e.g., product code, message type, and identifier) and receive the message in a corresponding response format. Generalize the example. One set of message formats suitable for use in such a system is described in detail later.

Watermark Registration-First Steps in Processing

In order for the system to identify a response (eg, a URL) corresponding to an object identifier (eg, a watermark), this data must first be associated in the database 17 in association with the corresponding watermark. The watermark registration process captures some basic identification information that is used later to validate incoming messages and identifies associated information / actions. In the example shown, the identification information includes:

* Consumer account

* Object and associated properties (name, description, expiration, etc.)

* Action, and

* Registered serial number (for registration update)

The consumer account identifies the watermark registrant. In most cases, this is also where service fees are charged. For validity and security reasons, the consumer account is required to be a known existing account. Account information, including the account password, is maintained by the account management system.

The object and associated properties identify the watermarked object. Object properties typically include a name and description of the object and a list of accounts authorized to access the object registration. An authenticated "supporting" account is typically an advertising agency, pre-press house, or the like, involved in the watermark embedding process of a print advertisement considered here as an example.

The action defines the response the consumer wants when the watermark is detected. This varies by product, but in the illustrated embodiment involves the return of some additional information for the watermarked object. In the system shown, the action is the return of the URL or HTML used to display the web page associated with the watermarked object. For other products, the desired response is the display of object & owner information of the object, software / data download, transmission of stream audio or video, advertisement display, initialization of object-based actions, and the like.

The registered serial number forms the final component of the registration. This is the designated supplier and product-unique identifier that the system allows to obtain information / actions specific to the object in question.

Some key product registration concepts-

Watermark registration is product-specific processing-

The registration process is product specific in order to allow each product to freely update its function without affecting other product functions or schedules.

Watermark Registration is Web-Enabled-

Preferred registration is a web-enabled process that requires basic identification information from the object owner (publisher, advertising agency, studio, etc.) and returns to register the packet with a unique identifier inserted into the object. The watermark embedding application (i.e. software) uses this packet to insert the watermark type and serial number into the client's object. In the described system, only one watermark can be inserted into a single object. In another embodiment, multiple watermarks may be inserted into a single object.

When a consumer registers a watermark, the system associates the watermark serial number with the information provided by the consumer during the registration process. Associated information may change in other products. One association set for the preferred magazine advertisement object is shown in the following diagram:

Table 1. Registration database elements

Watermark Registration Expired-

In some products, watermarks are only awarded for a limited time period. For these watermarks, the registration process uses the expiration date for the designated serial number. When the system receives a message request action for an expired serial number, an error is returned. Registrants can extend the watermark serial number expiration by updating the expiration date. The extension of expiration will result in a fee for the consumer.

Watermark registration can be completed in one or more web sessions −

Registration can be single or multi-step processing. If the media owner has all of the information required at the start of the process, the system can provide a simple web-enabled method to request the watermark serial number (s) online. With all the information provided, the registration is considered to be "active." That is, it is available for intermediate use by the consumer. If the registration does not have all of the required information available in the initial session, by providing only a minimum set of information (eg, name and / or organization name + product), the product watermark serial number will continue to be registered for use in the insertion process. Can be electrolyte. The most typical use of this partial registration occurs when the action (eg, URL, etc.) associated with the watermarked medium is not yet known. Partially registered serial numbers are considered to be "inactive" until all required registration information is completed. The system sends an error message if required to handle an "inactive" serial number. Whether active or inactive, these registrations can be considered items that can charge for the terms and conditions of the applicable contract.

The registration may be updated by the consumer to reflect the new information and / or complete the previous registration session. For example, a registered consumer may request a watermark serial number without specifying the URL used to resend to the consumer. The system assigns a serial number so that the consumer can continue the insertion process, but the registration is not thought to be completed until the consumer updates the registration with other essential information and URLs for this serial number.

Watermark registration is secured-

Only accounts that are registered and registered have access to a particular watermark registration.

In the described system, a consumer account that registers a watermark may grant permission to a particular advertising agency and / or pre-press house to change certain fields in the registration to a normal portion of the job. Each consumer, agency, and pre-press house needs an account in the file to grant access to the watermark registration. Consumer accounts are established as part of contract processing. In advertising agencies and pre-press houses, accounts are established based on the need through a control website accessible to the consumer.

For all products, the same basic precautions are maintained-access to the registration information is limited only to accounts that are authentically authenticated. The account is a password that is protected. In advertising agencies and pre-press houses a single password may be shared. In other embodiments, a unique password may be assigned to each part.

Watermark registration changes are logged-

All registration actions-creation, modification, and deletion-are logged in the audit log. The authenticated username, date / time of action, and the action itself are all stored to provide a complete trace of the audit.

The processing and data flow associated with registration is shown in FIG. 6.

Enter data into the registration database

Although the client application, router, and product handler were first described in association with the response to the watermark information detected from the media object, the same infrastructure can be used earlier in the process to enter data into the registration database 17. That is, a variant of a suitably configured application 28c may, by publisher, advertising agency, pre-press house, etc. (a) provide initial data to the database; (b) update this data; And (c) query the database for the current value. Alternatively, a dedicated registration server 19 (FIG. 2) can be used.

Including multiple populations in the registration process is easy by summarizing the database record content for a given watermark in a file that is continuously added with information by another entity and used to pass data between the database 17 and the entity being cited. Can be done.

Consider a case where Nike is advertised in Wired Magazine. Wired's advertising department agrees to sell space in response to requests from Nike's media buyers. Wired may begin the relevant watermarking work by ensuring a specific watermark identifier from the operator of system 10. (This is done by a computer speaking to the computer according to the instructions provided by the appropriate software, which is mostly used by various participants, etc. in the following process. In the following discussion, the software is a registration server 19, but as described, the product handler (16) can be configured to perform these functions.) Wired provides the operator with a publication identifier (e.g., the San Francisco edition of July, 200 publication) and internal tracking information used by the magazine. The registration server 19 responds by sending a confirmation file by email to Wired that summarizes the information so far (ie, the watermark identifier, issue ID, and magazine tracking information). The server 19 creates a new database record and interprets the received information as a record of the corresponding field.

Wired forwards the file received from the registration server to Nike's media buyer. Nike supplements the information with additional information, including the ad name and internal tracking information. Subsequently, the updated file is transmitted to the server 19. Again, the server processes the file and updates the database record with the new record. This emails the confirmation data file to both Nike and Wired, with each recent set of information.

Processing continues in this form. Each entity provides new data to the registration server 19 via an emailed summary file. The server updates the corresponding database record and dispatches the updated version of the summary file to the identified participants so that each has the latest information.

Once Nike enters data through this process, it can pass the summary file to an external advertising agency. The advertising agency similarly uses the file to add specific information and deliver the file to the server. The server updates the database record accordingly, adds the advertising agency to the email distribution list for the summary file, and also dispatches the latest version of the file to Wired, Nike, and the advertising agency.

The pre-press house can be the next group to be included.

The identification of the URL to which the watermark ID corresponds, and hence the updating of the database record, does not occur until the process is almost finished.

At any time, any group may provide additional information to the database and share this information with other groups through the same processing. (Some information is not appropriate to be distributed across all populations involved and may be flagged accordingly.)

The server 19 does not always need to be a hub where all communication takes place. For example, a file updated by Nike can be delivered directly by Nike to an advertising agency. The advertising agency may add the information and then provide the updated file twice to the server or the like.

By using a distribution file that is close to the actual database record, a number of benefits arise. One is that recent information is locally available to all groups without the need for an internet connection. So, if the creative director wants to work by the sea, or otherwise separates from the net, the necessary information is still available. Another is that it's easier to integrate a file of local data assigned to a specific advertisement and software tools in each group, rather than requiring design confusion to interface with remote devices and navigate participation authentication and security barriers.

The above discussion is made with reference to an email sending file, but a typical email program is not commonly used. Instead, specialized file management / mail programs are used by each group to better manage the participation logic. This program tracks the latest file for each advertisement, making it readily available for updating as desired, and indexing the file with various content fields. Thus, the user interface provides a list of files grouped or categorized by database fields, allowing editing or addition of information by simply clicking on a given field or tab.

Of course, the file-distribution system described above is not essential to the system. A wide variety of other configurations may be used in nature. One is that each group is logged on to server 19 as needed to examine or update database fields with appropriate permissions.

Numbering structure

Payload information encoded by an object (eg, by watermarking) may have a plurality of shapes and sizes. In the following, four preferred classes are discussed:

a) domain-based payload splitting;

b) consumer / use-based payload segmentation;

c) undivided payload; And

d) unique ID

Domain-based Payload Segmentation

The domain-based payload splitting approach divides the payload into fields that each have a clear meaning. The CLASS / DNS / UID configuration described above is an example of this kind of approach.

Consider a 60-bit payload. The 12 bits may form a class ID. These bits act as identifiers for the top level domain. The other 24 bits can form a DNS ID. These bits identify the mid level domain. Together, the class and DNS IDs collectively identify the class, consumer, and server that should respond to the object from which the data originated. (Some responses can be handled by the client computer rather than being rapid to the remote server.)

The remaining 24 bits are the user ID, which acts as a particle identifier that represents the particular source of the payload. Based on this ID, the response server knows exactly which response should be provided.

This payload is inserted into the consumer's object as a whole. When detected by the client computer, the application 28c disassembles (decodes) the payload into the CLASS ID, DNS ID, and User ID fields. The class ID is used to trigger one or more client or server side programs. Once "launched" these productions, they use the class ID in association with the DNS ID and user ID to complete the desired action.

One of the class IDs may mean that the object is a magazine page. Based on this, application 28c delivers the payload in response to the router / handler described above. Another class ID may mean that the object is music. Again, the application delivers the payload to the same router. Alternatively, the application may in turn send a payload to a service maintained by the music industry association. Another class ID can mean that the object is a store package, and the payload must be delivered to the online store in response. Another class ID may mean that the object is a business card, and the payload must be processed locally at the client machine. The mapping between the class ID and the corresponding response mechanism for which the application 28c should carry the payload may be maintained by a database associated with the client computer's operating system, as described above (eg, the Windows registry). (Windows Registry).

Once the payload is dispatched to the appropriate response destination, the entity looks up the DNS ID to further classify the correct response entity. For example, different IDs may correspond to different classes of servers in the server's tree.

Once the payload is delivered to the server of the correct class, the user ID finally defines a terminal "leaf" in the tree that defines the response (e.g., a database record).

Consumer / Use-Based Payload Segmentation

The second approach uses a repartitioned payload technique. However, in this configuration, the first field defines the interpretation of the next bit (for example, division into other fields).

Consider again the example for a 60-bit payload. The 12 bits can be a version ID. These bits indicate how subsequent bits are dissected and interpreted, and may indicate the particular application program 28c that should be used (same class ID in the above approach). Thus, the version ID bit operates to indicate the payload type. In the described embodiment, one of these kinds means that the payload comes from the magazine page and should be treated accordingly. In this case, the remaining 46 bits may be dissected into three fields: owner ID (15 bits), issue ID (15 bits), and media ID (18 bits).

The owner ID identifies the consumer for which the watermark is registered (eg Nike). This is used for advertising efficiency analysis and billing purposes. The publication ID identifies a particular publication (eg, the San Francisco edition of Wired Magazine, July 2000). The media ID identifies a particular page location within that publication.

As above, the payload is entirely inserted into the consumer's object. The payload is first dissected to determine the version ID. When the user device 12 is programmed to locally process such an object, another dissection is executed according to the data corresponding to the version ID, and the associative processing of the disassembled data is executed. If the device is instructed to dispatch this payload to a remote location for service, then the complete payload can be delivered with this dissection only as required to correctly identify the corresponding remote service providing entity.

Undivided Payload

The unpartitioned payload consists of only two parts: the version ID (as described above) and the object ID. In the case described, the 60-bit payload is used again, with 12 bits operating as version IDs and the remaining 48 bits operating as object IDs.

In this approach, the relationship of owner / consumer, publication, publication, and media is all maintained in the database 17 rather than in some form within the object identifier.

Unique ID

This case is similar to the non-partitioned payload, but consists of only one field-a unique identifier. The same application 28c is always used and always handles payload data consistently (eg, locally processed or dispatched to a given destination) regardless of payload content.

Combinations and hybrids of the above approaches can of course be used. Also, only the 60 bit payload length is described. Longer (eg, up to 1024 bits) or shorter (eg, 8 bits below) payloads can of course be used.

In a particular embodiment a 32-bit undivided payload is used, which consists of a 10 bit payload type and a 22 bit watermark serial number. Some content (eg, advertisements containing composite graphics) may be encoded with several serial numbers. The mapping between this payload and consumers / issues / etc is maintained in the database 17.

(As will be described later, the data sent from the application 28c typically includes information other than the identifier payload, such as the type and version number of the application 28c, the electronic address of the expedited application, etc.)

router

Router 14 allows any number of other products to be used as an indication detection and response model. By separating and generalizing this functionality, new products can be added to existing products or product handlers 16 without changing the design. There are two factors that make this approach successful: speed and elasticity. By using a standardized open interface, routers can facilitate both of these purposes.

The assumption of the preferred interface is the envelope technique that allows the router to "open" the external transaction envelope and extract the supplier and application IDs without decrypting the rest of the transaction (message). Given these two pieces of information, the router uses a simple lookup table to determine the appropriate product handler to complete the transaction. The router then passes the supplier, the application, the rest of the transaction, and the Internet "reply to" address to its appropriate product handler. This simplicity of handling delays the actual response processing to the supplier / product-specific handler, while keeping the routing delay to a minimum. By including a "response" address in the data sent to the product handler, the router is free from routing return responsibility for the product response.

Looking back, the router is:

1. Decode the request packet received from the client product into the basic components of the packet: supplier ID, application ID, and message;

2. validate the request packet base component against a list of known good values;

3. If the request packet component is found to be invalid, convey an error message indicating the invalid component and returning it to the calling session (eg, product);

4. send the decrypted request packet content and request identification of the calling session to an appropriate product handler; Also

5. Report any errors encountered, including invalid received packets, to the system monitor.

The specific data flow associated with the router is shown in FIG.

Product handler

The primary function of the product handler 16 described is to process the request received from the application 28c via the Internet and the router 14 and return the required information / action to the originating device 12. In the described embodiment, the required information is a URL associated with the watermark payload sent by the application. In other embodiments, other actions and / or information may be required.

Each received watermark payload is validated using information from the database 17. If the watermark payload ID is found and activated, the required action is performed. If the watermark payload ID is not found or in an inactive state, an error message is returned to the requesting application.

All requests are logged in the transaction log for tracking and billing purposes. This includes secondary payload information (zip code, demographic household ID, etc.) sent by the application 28c. The log may be maintained by the product handler 16 or anywhere.

To speed up the system response, the product handler 16 can send a URL to an application that anticipates and corresponds to a watermark payload that the handler expectation can convey. These URLs can be cached in the memory associated with the application 28c and recalled quickly if needed by the application.

For example, consider a magazine containing a watermarked advertisement. When the user provides the first advertisement to the device 12, the watermark is decrypted and passed to the product handler 16, which responds with a URL corresponding to that advertisement. Application 28c then sends the received URL to web browser 28b on device 12 and initiates a link to that Internet address. However, the handler now knows what magazine the user is reading. With reference to the watermark received first, the handler may know, for example, that the user is reading the San Francisco edition, Time magazine of March 14, 2000, and has just seen page 85. Based on these information, the handler can query the database 17 for a URL associated with another publication of the publication. (Database indexes are configured to allow fast queries to identify all advertisements in a given magazine publication and other aggregate data sources.) These URLs are sent back to the application 28c and cached. When the user next delivers an advertisement from page 110 to device 12, application 28c finds that it already has a corresponding URL that is locally cached. The application then sends the corresponding URL to the web browser. The web browser immediately initiates the link and prevents data round trips between the application and the remote system.

Cache processing can be optimized in various ways. One is to send the URL corresponding to the page that we expect to see next. For example, if a user provides 85 pages to the sensor 22 after sending the URL for that page, the handler 16 next sends a URL associated with the pages 86, 87, and the like. By sending the URL for the magazine's last page (typically the back cover), the handler can send more URLs starting at the beginning (typically the front cover) and up to 84 pages. Another optimization first caches the URL for the most obvious ad. For example, the URL for a two-page spreading ad is first sent, then for each previous page ad, and subsequently for each successive smaller partial page ad. Another approach is for the handler 16 to dispatch the URL to the device 12 for caching according to the contractually agreed priority. One advertiser may, for example, pay the premium advertising rate when exchanged to be cached before another advertiser who has not paid the premium. Other cache processing priorities and combinations of these priorities can of course be used.

In some systems, the advertiser or publisher may be charged for using the system based on the number of URLs provided by the system for the link. If local cache processing of URLs (eg, at device 12) is used, device 12 reports URLs that are actually recovered from the local cache and used for links to router 14 (or handler 16). Since it is desirable to do so, the remote system can log it. Thus, whenever a user provides an object to a sensor 22 whose corresponding URL has already been cached, the application 28c dispatches a message to the router 14 reporting the event (typically, the specific URL contained therein). ). This event is then logged in the transaction log.

The expected dispatch of this URL is one other function that can be executed by the product handler. Another is when the application 28c consults the product handler, determining if the most recent version of the application is available for download. In that case, the application-through interaction with the user-may require the product handler to respond with a software download.

In more detail, application 28c may periodically query the product handler for the identity of the latest version of application 28c (eg, when the application is first used each day). Device 12 has version 3.04 and the remote system can respond that version 3.07 is current. In such a case, the application 38c may warn the user—by appropriate text, graphics, or other means—that a more recent version of the program is available and ask whether an updated version should be obtained. If the user so instructs, the handler 16 can provide the device 12 with the latest version of the application (or a patch that allows the currently installed version to be updated).

Sometimes there is no need to update the application version. Instead, data from a remote system may indicate that it is desirable, necessary, or changing one or more parameters in the application 28c. For example, new security elements may be dispatched to device 12 by handler 16 periodically and used to change the security configuration of the application. Or, the application 28c may be instructed to further forward external watermark traffic-for another time, date, or until otherwise indicated. This indication can be used to optimize system performance, for example, to avoid slow internet routes, for router load balancing purposes.

In summary, the handlers described in detail are:

1. Validate the received identifier against a list of activation identifiers; If no serial number is found, return an error message to the calling session and log the error to an error handling routine;

2. For each received valid watermark serial number, find the corresponding active primary action from the database;

3. For each received valid watermark serial number, if the handler finds that the corresponding primary action is not currently active, it performs the other method's "default" action instead;

4. If the handler finds an activation primary action associated with the received valid watermark serial number, it either returns the URL for application use (return approach) on retransmission or passes the found HTML page to the calling session;

5. If the handler does not find an activation primary action associated with the received valid watermark serial number, but finds an associated default action, it either returns the URL for application use on a retransmission (round trip approach) or the HTML page found. To the calling session;

6. If the handler does not find a valid active primary or default action associated with the watermark serial number, it returns an error message to the calling session and logs the error to an error handling routine;

7. Record each transaction, including being given an error message for billing and analysis (in other embodiments this function may be executed by the router instead);

8. Respond to the "software version request" by returning the most recent available application software version number to the calling session;

9. Respond to the "software download request" by initiating a file transfer to the calling session of the most recent available application software;

10. Respond to a valid request for registration packet unloading (appropriate format, existing serial number, account ID, and corresponding valid account password) by returning the current registration packet for the provided watermark serial number;

11. Respond to an invalid request for a registration packet by returning an error message to the calling session that notes the failure;

12. Respond to local transaction cache flush requests by writing locally cached transactions to the transaction log; Also

13. Respond to the plurality of URL requests by first returning the URL associated with the provided serial number, and then returning all other active serial numbers and URLs for the publication, publication, and optionally provided area codes.

Certain of the processes described above in connection with the product handler are shown in FIG. 6.

URL performance monitor

Returning to the operation of the system, the URLs identified in the database 17 do not work well or become corrupted due to equipment problems, or other problems, at the remote shock site over time. If desired, handler 16 (another component of the system) periodically tests each link registered as active in the database (e.g., once a day) and measures how long the associated web page loads. Can be programmed to do so. If a web page fails to load or takes much longer to load than usual (if the condition is found to be exceptional in the retest), these conditions can be flagged in the corresponding database record. If a handler is required to provide such a URL to device 12, the handler may send a message-with or without the URL-indicating that the URL is behaving incorrectly.

If the URL is working, but excessively slow to load (compared to the previous performance or compared to other URLs), handler 16 may provide an intermediate switch to device 12. For example, the device can be instructed to invoke a second browser window, which informs the browser of another destination and entertains the user while waiting for the intended page to load. When the intended page is finally loaded, the first browser window can be displayed-either by closing the second transition window or by bringing the first window to the front while keeping the second window in the background.

This other destination is preferably a low bandwidth page, so it is not slower to accommodate the load of the desired URL. This other page can be selected by the handler such that the URL is sent after the desired URL. Alternatively, instead of providing a URL from a handler, the handler can provide HTML or another page directly to device 12. Alternatively, another URL may be stored in device 12 and used to invoke a second browser window when receiving data from handler 16 indicating that desired content is displayed slowly. In some embodiments, a user may identify several different URLs (eg, weather, stock information, humor) and the handler or application 28c may choose between them randomly or in other ways. Alternatively, an HTML page or other application may be loaded locally on device 12 in response to a "get ready to wait" indication from handler 16.

If the URL is marked as slow or inoperative in the database 17, the scan operation periodically checks the URL again to see if the state in the database should change (eg, change from inactive to active). do. Inactive URLs are reported by email to the registrar and are manually flagged unless they are resaved to work within a predetermined period.

Descriptive response by product handler

Part I above provided a sampling of the various applications that are enabled by the system 10 described. Some are described in more detail below.

Consider using the system 10 to enable a personal greeting card. The greeting card company prepares a watermarked printed sticker for use with the card or other counterpart. A consumer shows a sticker on a computer equipped with a camera (in a retail store, at home, etc.). The computer decrypts the watermark and sends it via the router 14 to the corresponding product handler 16. The handler-recognizes the watermark with an unregistered greeting card sticker-to direct the consumer to the destination URL, such as the consumer's personal web page. This information is entered by the consumer and given to the remote system as an entry in the registration database 17. Thereafter, whenever a sticker is shown on a properly enabled system (eg by a card recipient), the browser window is automatically launched and leads to a web page designated by the purchasing consumer. (The same result can, of course, be achieved by coding the greeting card itself, without using a sticker.)

In some applications, the product handler may have a library of other responses that may be provided to the user in a particular context, depending on another choice of the user. Consider a college student with a properly watermarked college ID card. When a card is given to the device 12, the product handler is responded with an HTML instruction so that an option menu is displayed on the device screen. For example,:

1. Observe the schedule of upcoming university academic events

2. Observe the schedule of upcoming college sporting events

3. Observe the current class schedule

4. Choose a course for the next semester

5. Observe credit

When the student makes a selection (for example, by moving the ID card with a mouse or in a particular manner), the application 28c dispatches the data corresponding to the selected option to the product handler, which responds to the required data. .

In some cases (eg, observing current class schedules, choosing courses for the next semester, observing grades), you should be around to protect this information from anyone trying to access it using a lost or stolen ID. Thus, when any of these options are selected, handler 16 may respond to device 12 by first inquiring about a password or PIN. The required action is only performed after the correct password / PIN has been entered. (For security reasons, colleges prefer that password authentication processing be performed by dedicated on-campus servers rather than product handlers 16. Of course, this and other tasks may be delegated to processors other than handler 16 as appropriate. Can be.)

In other cases, an option menu need not be given-the correct response is inferred from the context or environment. Consider a watermarked driver's license with the owner's identity. Given to an email kiosk 12 at the airport, the decrypted watermark can be used to look up an email account corresponding to that individual and download a new mail. If the same driver's license is given to the check-in kiosk, the decrypted watermark can be used to look up the person's flight reservation and issue a seat assignment. In both cases, the kiosks can be basically the same. However, one identifies itself to the router / product handler with an email kiosk and the other identifies itself with a check-in kiosk. The response handled by the router / product handler is different accordingly.

In college, for example, there may be cases where a student wants to change a picture on a student's ID in order to have a scammer take a graduate qualification test instead of a disqualified student. In normal cases such photo exchange is difficult to detect. This problem can be encountered with a test check-in process that includes each student having the device 12 provide their ID. The application 28c specialized in this task may pass the watermark decrypted from the ID picture to the handler 16, which responds by causing the image of the identified student to be displayed on the device. (Universities can compile the required database of student images when issuing an ID card.) If the examiner sees an image on a device that does not match the image on the IC card, appropriate action may be taken. (This configuration can be applied anywhere a Photo ID document is used, including airport check-in, customers, immigration, etc.)

Another application of the described system is to browse or act on metadata associated with the displayed object. Consider an image, video, or audio file that the user downloads from the Internet. Friendly applications, such as Microsoft's Windows Explorer (including Internet Explorer), are accessed by right-clicking on a property panel (eg, by clicking on the file icon or name and selecting the "Properties" option). ) May be configured as an activated watermark decoder. When a watermark is detected in the file, the explorer application can send the corresponding packet to a remote system (eg, illustrated router / product handler / database). The remote system recognizes the outgoing packet via the properties panel of Windows Explorer and finds the watermark ID in the database 17. The metadata corresponding to the file (e.g., owner, creation date, authorization item, exposure data, subject, etc.) is stored in the database 17 (or from a router, handler, or another database identified by the database). 28c), and is displayed in the properties panel (optionally under the appropriate "tab").

(This assignee has long provided "MarcCentre", which acts as a clearinghouse where watermark identifiers found in photographs, etc. can be used to identify the owner and associated information corresponding to an object.) A given implementation using this service In an example, router 14 sends a request to a MarcCentre server (product handler in this example), which provides the recommended information back to the originating application. The transferee's MarcSpider service complements the service provided by the Media Commerce product. The MarcSpider service continues to scan an Internet site that evaluates each graphic it encounters to determine whether it contains a watermark. (Audio and video can be analyzed similarly.) With each watermark detected, the MarcSpider service records the graphic file name, size, format, date / time, and URL where the graphic was found. This information is then made available to MarcSpider customers in report form.

Instead of simply displaying metadata, an application and / or a remote system can use it. For example, if the metadata indicates that the owner of the watermarked image is Corbis, and the image can be licensed for a particular use under a particular item, then the remote system can be used as a licensing server-from the user. Receive payment information, grant permission, and pass transaction details to Corbis.

Another application is the sale or promotion of music or video over the Internet. In the case of music, the musician is free to distribute a low fidelity (or otherwise modified or summarized) version of the song. Low fidelity may be due to bandwidth limitations (eg 500 Hz-2.5 KHz), mono (as opposed to stereo), or for other reasons. The musician can distribute the low fidelity version as widely as possible to act as a marketing agent for the musician's other works. (The free distribution of lower bandwidth audio can work to alleviate some of the network bandwidth problems faced by universities where students actively participate in transmitting free music over the Internet.)

Each low-fidelity version may be processed to extract an identifier (e.g. steganographic in-band watermark; numeric ID or song / artist name field in file header; hashing algorithm with music data, music 128-bit hash value obtained by applying field header data, part thereof, etc.). If the listener is interested in obtaining a full fidelity version of the task, the listener can operate a suitably programmed computer or acoustic appliance that extracts the identifier from the task and passes it to a remote system. The remote system may, in various ways, for example, provide the user with the same full fidelity version of the same work again (such as an MP3 download) and charge the user's credit card (eg, $ 0.99); Or by connecting a web browser on the user's computer to an e-store / fan website associated with the music. Such functionality may be provided to a general purpose program such as Microsoft's Internet Explorer, for example, by right-clicking a file to obtain a menu containing this and related functions.

8 to 10 show the sequence of screen shots from this embodiment. In FIG. 8, the user right clicks on the MPE file icon in directory list 200. A property menu 202 is displayed that includes "MP3Bridge" as a second option.

9 is what happens when the user selects the MP3Bridge option. MP3 player 204 is invoked and dialog box 206 appears. The dialog asks the user "More Information About the Artist? Yes No".

10 illustrates what happens when a user selects "Yes". The software sends the identifier-extracted from the MP3 file-to the remote system. The remote system responds with the address of the associated web page and instructs the user's computer to launch a new browser window that links to that page.

The new functionality can be invoked through the user interface of the original MP3 player (or Windows Media Player rather than Internet Explorer). The music application may create a separate window or provide options and associated data within an existing window.

Another application of the remote system is a "net nanny" filter. Links required through the system are checked against keywords, adult-content flags, content classes, or other indications of age suitability and can only be provided to the requesting computer 10 if they meet certain initial selection criteria. have.

Again, it is evident that the above examples are numerous applications that are enabled by the described system.

report

The system software may enable the provision of consumer-accessible reports (accessible via the Internet) showing detailed information and summary usage information by date, consumer, publication, date of publication, area, product / version, and the like. These reports can also be regularly and specifically scheduled. Details, relationships, and timing of reporting on content may be defined online by the customer.

The report described details the following:

a) Transactions per consumer per hit ratio / ad

b) hits per transaction per consumer per publication per ad

c) hit ratio / transaction per consumer per publication per publication per ad

d) Transactions per consumer per publication per publication per hit ratio / per area per ad

e) transaction by hit ratio / originating application 28c

f) hit ratio / transaction by originating application provider

g) transactions by hit rate / originating web domain (e.g. aol.com)

h) Transactions by hit rate / post / region code

i) Transactions by Hit Rate / Country

Additional marketing / market reports may also be made for internal analysis by the service provider and also for sale to external entities. These reports typically provide more general impact and use of the system. Using the information stored in the statistical database in combination with these usage patterns, the system can provide more detailed demographic / statistical data on the use and efficiency of the system to consumers and research agencies.

In the described system, certain statistics in the statistics database are compiled using statistics from a user's sample that allows the action to be tracked in detail, taking into account certain extra income (e.g., giveaway cameras, barcode scan pens, or other devices). do. These users are referred to as "Demographic Household." The software program included in the system requests the information described in the following table from these users via the Internet with a web-enabled interface. The related program allows the user to update / edit the previously entered user / furniture information. Each of these sessions is password authenticated for security.

Audio and video

Like paper advertising, the described system provides a user of a web-connected PC or other organization a method of obtaining information, content, related products, or related services using the same principles as described above.

For example, application 28 may use a recording device (a microphone connected to a note recorder, PC, MPC, etc.) to “capture” music or other audio, and capture captured audio to detect embedded watermarks. Can be analyzed. Once detected, the application sends some or all of the watermark payload information to the router with the identification of the application and the provider. The router passes the payload information to the handler corresponding to the application. The response of the product handler changes depending on the context and nature of the data. For example, the handler can return the artist, title, track, album, web URL, and purchase information to the user. The recorded news and entertainment segments may include copies of the segments (audio, video, and / or text) along with other relevant web site information. The handler may simply cause the device 12 to launch a browser window connected to a music commerce web site where music may be purchased.

security

The basic security concept of the system is to allow each consumer's information to be accessed only by the user authenticated by the consumer. For this purpose, the system is preferably:

1. Create and maintain a list (account) of authorized users.

2. Security measures should be used to deny access to unauthorized users.

3. Restrict users to access only those objects authorized for access (typically, objects belonging to that consumer).

4. Report and record all unauthorized access attempts.

5. Keep a log of all authorized user logins (sessions).

6. Watermark registration should provide the ability to grant access to other accounts (such as agencies and pre-press houses).

7. An initial password must be established for each account.

8. Each authenticated user / account shall provide the ability to change their password.

9. Provide a function to reset the password of an authorized user / account if the current password is lost.

10. All passwords should be stored in encrypted values (to prevent theft of passwords).

11. Provide authorized users with the ability to create, modify, delete, and limit listing / observation of account information.

Thanks clue

Because of the financial relationship of system behavior, all changes to registration or consumer data need to be recorded. This audit trail provides operators and consumers with accurate accounting of the current and previous state of the data.

The clue software preferably records the creation, modification and deletion of all registration and consumer data. The clue software also records the user name, the date / time for the creation / modification / deletion of the record, and the before and after image of the changed data.

Application-to-product handler interface definition

The interface basis between the application 28c and the handler 16 is (a) a flexible request and response package structure, and (b) a connection method defined based on industry standards. The message transmission described utilizes http and / or https protocols for sending and receiving messages among system components. An overview is provided in FIG. 4.

Message format

The message format conforms to XML and is defined by the following XML DTD −

Application 28c attaches data to this header for transmission to product handler 16. Preferred messages and product handler responses are described in detail in the sections that follow.

Application message definition

The application message definition may be divided into request code, primary, and secondary information.

The request code instructs the product handler 16 to take the specified action.

The primary information portion contains the data required to provide services appropriately for the needs of the application. The primary information changes based on the request code.

Secondary information is intended to be used by analyzing and reporting the tool and is not directed to assist the product handler in servicing the user's needs. The secondary information content changes based on the request code, and not all request codes are required to have associated secondary information. In addition, most secondary information is required by the consumer to give prompt consent to his collection. If no consent is given, the application does not send secondary information. There is a special case where the selected consent consumer is part of the statistics database.

The primary and secondary information may vary depending on the type of request, but generally follow the definition above. The general format for the product handler is also defined below.

The primary information includes application version, watermark type, watermark serial number, context, and environment.

Application version: typically used to modify the behavior by the product handler for backwards compatibility.

* Watermark Type: The top 9 bits of the described watermark payload. It is used by the product handler when processing the watermark serial number.

* Watermark Serial Number: The rest of the watermark payload. Provide an index used by the product handler to access the watermark in the registration database.

Context: Instructs the product handler to modify / purify the action based on the context of the consumer request.

Environment: Instructs product handlers to modify / purify actions based on the environment of the consumer's needs. (The environment may be designated, for example, at home, in an office, in a car, or in a portable device.)

Of course, other request codes may be used. Each has its own list of mandatory and optional primary information fields. Optional fields are excluded from the primary information when there is no associated value.

Secondary Information:

Statistics Household ID: The identifier for the selected statistics group. This is used as an index in real statistics.

Input device: The make, model, and version of the device (eg, TWAIN driver string) used to detect the watermark.

* Operating system: The operating system used on consumer PCs.

* Processor: Processor type / class on the consumer PC.

* Processor Speed: Displays the processor clock speed in MHz on the consumer PC. (Can be entered or auto-detected by the user.)

* Language: The preferred consumer language.

* Country: The country where the consumer PC is located.

* Zip code: The consumer's postal code (used with the country to accurately indicate the consumer's location).

(In addition to these explicit data, the packet 12 sent from the device 12 also carries an IP address (when using the http protocol), so that the remote device (e.g. router / handler) can respond. Has an address.)

Response from product handler

Request for URL

Input required

Header (XML Format)

data

Information Required-

Optional information-

Yes:

Response from product handler

Reason for error:

Type 1 and serial number are good, but the URL is not in the database (both primary and default URLs are lost).

-The two types and serial numbers are good, but the URL is marked as inactive (primary and default also inactive).

There are no records in the database that match the 3 types and serial numbers.

-4 Request format error-Incomplete data.

Demand for Configuration

Input required

Header (XML Format)

data

Information Required-

Yes:

Response from product handler

Reason for error:

-5 unknown operating systems.

-4 Request format error-Incomplete data.

Request for Associated URL

Input required

Header (XML Format)

data

Information Required-

Yes:

Response from product handler

Reason for error:

8 types and serial numbers are good, but the associated watermark or URL in the database is not.

-9 types and serial numbers are good, but all associated URLs are marked inactive.

There are no records in the database that match the 3 types and serial numbers.

-4 Request format error-Incomplete data.

Demand for transaction download

(This is necessary to think of retransmissions that are cached locally. One request per local retransmission.)

Input required

Header (XML Format)

data

Information Required-

Optional information

Yes:

Response from product handler

Reason for error:

-4 Request format error-Incomplete data.

In order to provide the fastest possible system response, the shortest possible data exchange between the originating device 12 and the remote system-preferably the size that can be transported in a single Internet data packet (i.e. about 536 bits or less)- desirable. This configuration prevents the overhead associated with data segmentation at transmission and data reassembly at reception.

Generally speaking, the combined elapsed time of system services for a single request (i.e., watermark recognition by application 28c, packet transmission to router, decryption by router, handling by product handler, and Return of Response) The measurement from the receipt of the request to the first byte sent in response to the request should be averaged to 3 seconds or less. Typical rates are less than 2 seconds and many responses are provided in less than 1 second.

The following discussion observes much of this but adds details from other priority cases.

MediaBridge digital messages consist of two codes, all of which are embedded in a MediaBridge-enhanced image. The media owner code is designated by a system administrator (e.g., Digimarc) and identifies the entities authorized to add MediaBridge enhancements to the image. The routing code is specified by the media owner (advertiser, publisher, producer, etc.) and the end-user determines the location of the electrolyte when the client application reads the MediaBridge code. Two different advertisements inserted with the same code can direct end-users to the same web page, and the same advertisement from different publications using different codes can go to a different web page, so that either advertisement or magazine is the media owner's web. It can be used to track whether it is most effective at bringing the end-user to the site.

There are three main components of the MediaBridge system. Client applications are typically used for home and business purposes by consumers (MediaBridge end-users) to automatically navigate to additional information from images or objects on the Internet. The MediaBridge Router provides the client application with the appropriate Internet address for the given image or purpose. The insertion system is used by the media owner to insert the MediaBridge code into the image prior to printing.

MediaBridge Client Application

Client applications can be distributed in OEM relationships with limited digital camera manufacturers such as Logitech, 3Com, Creative Labs, and others. It is installed by the end-user with the camera driver and a support program from the camera manufacturer's installation CD.

The MediaBridge Client Application can run on a 200MHz or faster Pentium or PowerPC computer configured with a limited video camera under the Windows 95/98 / NT 4.0 and Macintosh OS 8.6 operating systems.

MediaBridge client applications are first concentrated using MediaBridge-enhanced images to be directly browsed for additional information on Internet Web sites. Therefore, it requires a connection to the Internet via a dial-up modem or permanent connection. However, the client application preferably has a scalable design that supports browsing to local data and connecting to other applications, for example on a CD.

The MediaBridge system can be used anywhere a video camera can present a high quality image that is clearly in focus. It operates under varying lighting conditions from dark (about 40 lux) to bright (3900 lux). In very dark light it works with the aid of an auxiliary light source, in bright places it works when the image or object is shielded from the light source.

Operation of the End-User Client Application

When the first client application is launched, it provides the user with a wizard that tells them the best technology to deliver MediaBridge-enabled images to the client application via a video camera. The wizard is switched for each camera and can use sample images or live playback images.

Informing the user of the best way to use the client application-for example, focusing the camera, positioning the coded object in the camera's focus area, keeping the object still for a few seconds necessary for decoding, Allowable lighting conditions, etc.-One or more games may be shipped on an installation disc (eg, CD). The camera box inserted into the box can be MediaBridge encoded and connected via a browser to a corresponding introduction page managed by the camera supplier or system administrator. T-shirts or other merchandise can be awarded randomly or through a website if they demonstrate proficiency in some characteristics of MediaBridge operation.

In the described configuration, the MediaBridge Client Application is always running, activated or deactivated. When activated, the MediaBridge video camera window is always placed on top of other windows as a whole, and the client application is constantly checking video for MediaBridge-enhanced images. When a MediaBridge-enhanced image is found, the appropriate information is displayed-as often as a web page-and the client application is selectively disabled. When the client application is deactivated, it hides the video window, places the camera to be available for other applications, and also uses little memory or computer resources.

In one implementation, the client treats the camera as a line of reusable devices. That is, when the client application is active and checking video, no other application can access the video camera. Similarly, the client application cannot access the video when another application is using the camera.

The MediaBridge Client Application may include the following features:

1) Start browser / application. When a MediaBridge-enhanced image is found, the client launches the user's web browser or another application. When browsing on an Internet web site, the client provides the site with the identity of the image and the postal code of the end-user, if available. This information allows the media owner's web site to display localized web pages to the user. When launching another application, the client provides the image owner code and routing code to the application.

2) Destination menu. If multiple URLs are specified in the routing code, the client application allows the end-user to select a page to display rather than navigate directly to the web page.

3) Branding. When recovering information to display a web page, the client application displays a local web page with a pre-stored brand (eg, Digimarc) and explains that the desired information is being recovered. This branding page is replaced with the media owner's page without causing any delay to the user. (The content of the branding page can be updated during the automatic software update.)

4) elastic activation. When the client application is inactive, the user can activate it by:

a) Click the appropriate icon.

b) Press the hot-key combination.

c) Click the tray icon (Windows).

d) Click the button in the browser toolbar.

e) Restore the minimized MediaBridge Client video camera window. (window).

5) Status display. When activated, the client application provides feedback to the user via the status display panel, which is displayed in the video camera window following the scene viewed from the camera. The status includes the ambient lighting conditions, the distance from the camera focus (if available), and the current status (preparing to display image-related information, or attempting to read an image, or a MediaBridge-enhanced image displayed on the camera). Feedback).

6) Support multiple cameras. The MediaBridge Client application described uses one camera at a time, but can use an installed MediaBridge-enabled camera. If the computer has only one camera, the camera is automatically selected. If there is more than one camera, the user can select which camera will be used and change the selection while the client is running.

7) Automatic software update. The user can automatically install updates to the client application by making a simple menu selection. The client is connected to the Internet to download and install available updates. In addition, the client proposes to check for updates when it finds a MediaBridge-enhanced image using a newer protocol (e.g., referring to the version indication encoded with MediaBridge data).

8) Configure consumer options. Almost everything in the client application can be configured by the user.

a) Automatically start the client each time the computer starts (default).

b) Activation by hotkey.

c) Display an activation icon in the system tray (add a button to the Windows Windows Web browser (after Internet Explorer 4 and after Netscape 4).

e) Selection of the latest camera used at startup when multiple cameras are installed.

f) reminding to periodically check for software updates.

g) Run the wizard at start up (default).

h) Automatically disable when MediaBridge-enhanced images are read.

i) Automatically disable if nothing is given to the camera during the specified period.

j) Read MediaBridge-enhanced image to play or play a sound file.

k) Blocking of sites based on RSACi rates. Each media owner is expected to use the RSACi scale for categories of language, nudity, sex, and violence to determine their own proportions of the site. The router blocks the end-user from the site if the RSACi ratio for any category exceeds that allowed for the end-user. (Routers do not actually check web pages for RSACi code, relying solely on the media owner's own ratio.) If this option is selected, RSACi (http: //www.rsac. org / ratingsv01.html) will be displayed.

l) Automatic connection to the Internet through an existing dial-up (modem) connection.

m) Password protection at start up.

n) User Registration Information. (Inform the user that all information is optional and that specific items will be provided to third parties if provided.)

o) A terminal-user postal code that is not required to provide to the user is used as the basic part of the statistical information sent to the media owner when voluntarily provided by the terminal-user.

9) Wizard. This wizard is your gateway to more information and provides the best technology for using MediaBridge and guides you in setting up specific cameras for the best results with MediaBridge. By default, the wizard runs every time the client application starts. It also runs for the first time when a new camera is selected by the user.

10) Camera Verification. When a new camera is selected as the MediaBridge Client input, the Client confirms that the camera is MediaBridge-enabled. Otherwise, the user is warned that the camera is not supported and may not work correctly with the client application.

11) Scalable design. MediaBridge clients typically deal with reading MediaBridge-enhanced images and linking to the desired information by displaying Internet or local web pages in a browser window. Its functionality can be extended by a MediaBridge-enabled application registered for the handling of specific data. For example, given a MediaBridge-enhanced business card, a MediaBridge client can run a MediaBridge-enabled application that downloads business card information from a web site and updates the user's contact list with new information. Another example is a page from a MediaBridge-enhanced children's book that allows an audio file to be played for a page.

12) Insertion support. If the client application is being used by the media owner to verify that the image has the correct MediaBridge code embedded in it, the client displays the media owner's instructions, routing information, and the relative strength of the MediaBridge watermark when the MediaBridge image is read. do. To protect the privacy of each media owner, this information is provided only if the user can provide a valid embedded username and password for the media owner.

MediaBridge Router

The router is basically transparent to the end-user. When the client application detects a MediaBridge-enhanced image linked to information on the Internet, it communicates with the router to obtain Internet address information of the displayed web page.

The router also includes the following functionality:

1) Maintain routing information. Within the router contains information that links each unique MediaBridge code to one or more related Internet addresses. In one implementation, a system administrator (eg, Digimarc) maintains routing information using information supplied by the advertising consumer. In another implementation, the media owner updates itself by using a secure internet connection with the router.

2) Problem Handling. If the routing request cannot be satisfied, the router responds in a way that minimizes the impact on the end-user. For example, if the MediaBridge code is not known, the client application is provided with a URL to the media owner's home page. In addition, the day the error occurs, the router notifies the media owner that the error has occurred by e-mail.

3) Content class rate. If the media owner has provided RSACi class ratio information for an image or site, and is assigned a RASCi class ratio that specifies that the end-user is blocked at that site, the router will not connect the site to the media owner site with inappropriate content. Return the web page to the user.

4) Validity. Periodically (eg daily), the router validates all activation information in the database. If an error is found, the media owner is notified by email. If the error is not fixed to one day, Digimarc is notified. The conditions checked are:

a) Lost URL in Activation Link.

b) URL referencing a page that does not exist.

c) Pages beyond the acceptable download time required by Digimarc.

5) Tracking. The router may log client application requests to develop marketing information. Such tracking logs may include:

a) Request date and time.

b) Media owners and images.

c) the end-user's postal code, if provided.

d) IP address that issues the request.

6) reporting. Using tracking information, the router provides the following reports used by Digimarc:

a) Number of URL requests for a particular data range.

b) Number of URL requests by the media owner for a particular data range.

c) Number of URL requests by MediaBridge-enhanced image for a particular data range.

(Tracking and reporting are preferably configured to allow the media owner to obtain traffic and marketing reports online from the router.)

MediaBridge Insertion System

The insertion system includes:

* Photoshop-compatible embed plug-in for inserting MediaBridge code,

* MediaBridge-Photoshop-compatible reader plug-in for checking digital images,

* Client application to verify MediaBridge-proof evidence and press print, and

* Internet-based router maintenance application for obtaining MediaBridge codes and assigning URLs to MediaBridge codes.

Plug-ins specify a new MediaBridge routing code and require a connection to the Internet for authentication only. Router maintenance applications always require an Internet connection.

Router maintenance application

The main purpose of the router maintenance application is to specify routing information in the MediaBridge code so that the end-user is given the appropriate web page when the client application reads the image.

Router maintenance applications include the following functionality:

1) multiple locations. A single media owner account can be accessed by different people in different locations. It may also be accessed by people of other organizations, such as, for example, advertisers and advertising agencies.

2) Restricted Access. Each media owner has the ability to restrict who can access router information, specify who can generate new MediaBridge codes, use existing MediaBridge codes, or change information about routing codes. The media owner can add, change, or revoke access at any time.

3) Secure Access. All Internet access to the router is through a secure connection.

4) Writing code. Assign new routing codes, change existing routing, and delete or reuse old routing codes.

5) time-based routing. Each routing code may be assigned a number of URLs, each of which may optionally have a valid date and an expiration date that determines when it should be used in routing. Expired URLs are automatically deleted by the router after three days.

6) Multi Routing. If multiple URLs are assigned to one routing code, the router returns to the end-user a browser-based menu that allows the end-user to select a page to display rather than navigate directly to the web page. In one embodiment, a single routing code may have up to four non-expired URLs. Each URL may have a short description that is a browser link to the URL, a long description of up to 500 characters, and a URL for the icon displayed in the menu. An icon cannot be larger than 50 pixels by 300 pixels wide.

7) Logging. All changes to the routing information can be logged and observed by the media owner.

Plug-In Overview

When the insertion is made for the first time on the media owner's computer, the embedder requires an account number for the media owner, an effective username and password defined by the media owner. The embedder connects to the router via the Internet, verifies that the user is authenticated, and downloads information about the media owner. Another insert into the media owner can be performed without being connected to the router. However, whenever a user is connected to a router to get routing information for insertion, the router checks the username and password against the currently valid name and password. If the previously used name and password are no longer valid, the user must provide a valid name and password to continue inserting to the media owner.

The media owner account number is only needed for initial authentication, since future verification is done using the name of the media owner as maintained by the embedder. Since a graphic designer in an advertising agency can work on a project for more than one owner, one graphic designer can have a different username and password for each media owner.

Reader plug-ins are often used for identification of images after insertion. This restricts access in the same way as an embedding plug-in with the additional restriction of providing information about the image only if the user can provide a valid username and password for the media owner. If the media owner information is not currently available on the computer, the reader is connected to the router for verification.

Plug-ins share this functionality:

1) Multimedia owner. The user can perform an insert or read on the multimedia owner. The user must have a valid username and password for each media owner.

2) Multiuser. A single computer can be used by several people. If they do not use the same username and password, they do not share information.

3) Automatic software update. The user can automatically install updates to the plug-in by selecting a single menu. Plug-ins connect to the Internet and download and install available updates.

4) Secure Access. All internet access is through a secure connection.

Embed plug-in

The embedder plug-in includes the following functionality:

1. Specify routing code. The user selects a routing code to be inserted from the list of existing codes provided by the router via an internet connection. Each routing code is identified by a unique routing number and description. Once selected through the router, the information about the routing code remains on the local computer until the user removes it.

2. New routing code. Once authenticated, the user can connect to the router via the embedder to generate a new routing code. The routing code can be generated with or without URL information that can be added later.

3. Update routing code. Each time the embedder is connected to the router, it downloads the updates made to the locally cached routing code.

4. Masked inserts. The user can mask off part of the image so that the MediaBridge code is inserted only in the masked area. In one embodiment, different codes are applied to different mask areas. Typically, the user can insert different MediaBridge codes in the same portion of the image.

5. Variable strength. The user can overall change the intensity (and thus visibility) of the MediaBridge watermark from light to very heavy, and also locally change the intensity in other areas of the image (ie, the intensity is adapted to the local image characteristics). ).

6. Logging. Each time the user inserts into the image, the embedder records a log of the date, time, embedded information, insertion settings (e.g., intensity), user name, computer name, and name of the input image file. The log is a text file on each user's computer. This can be observed in any text editor.

Reader plug-in

Reader plug-ins also include the following functionality:

1) Reading. Read the digital image that can be scanned and display the media owner, routing information, and strength measurements of the MediaBridge watermark if the user is authenticated.

2) Masked Reading. The user can remove the mask of part of the image and read MediaBridge information only from that part.

Part III

Inventive Combination

In addition to the inventive features detailed above, some of the inventive combinations claimed as priorities include:

A1. Providing a business card of the individual to the optical sensor, the optical sensor producing output data;

Decoding the multi-bit encoded data using steganography from the sensor output data; And

Using the multi-bit data to establish a link to an internet address having data related to the owner of the business card.

A2. The method of claim A1 comprising obtaining calendar data detailing the specific activity of the individual from the Internet site.

A3. The method of claim A2 in which the amount of calendar data obtained depends on the authentication level.

A4. The method of claim A3 in which the authentication level is reflected in the multi-bit data encoded in the individual's business card, and the individual can distribute a differently encoded card to another recipient to give the recipient different access rights to the calendar data.

A5. The method of claim A1 wherein the optical sensor is also a business card reader that also operates to input text information from the business card to the personal information manager.

A6. The method of claim A5 comprising storing in the personal information manager an internet address distinct from steepographically encoded multiple bits of data.

A7. The method of claim A1 in which the optical sensor is a digital camera.

A8. The method of claim A7 wherein the digital camera is installed in a computer display device.

A9. The method of claim A7 in which a digital camera is installed on the user's head.

A10. Providing a printed promotion to an optical sensor at a first site, the optical sensor producing output data;

Decoding the multi-bit encoded data using steganography from the sensor output data;

Using the data to establish a link to an internet site related to a company, product, or service promoted by the printed promotion; And

Transmitting additional information relating to the company, product, or service from an internet site to a first site.

A11. Providing an identification badge printed on the optical sensor, the optical sensor producing output data corresponding to the optical characteristics of the badge;

Decoding the multi-bit encoded data using steganography from the sensor output data;

Checking a plurality of bits of data to determine whether it corresponds to a valid access card; And

Releasing a lock depending on a result of the check operation.

B1. Providing a digitally encoded object to the optical sensor, the optical sensor producing output data;

Decoding a plurality of bits of data from the sensor output data; And

Using the multiple bits of data to establish a link to an internet address having data associated with the object.

B2. The method of claim B1 in which an object is encoded using steganography with the plurality of bits of data.

B5. Refrigerator;

An optical sensor associated with the refrigerator and adapted to provide image data corresponding to an object held in front of the sensor; And

An image processor for processing image data and extracting multiple bits of binary data,

A useful device for compiling a list of shops.

B6. In a method of operating a computer, the computer includes an operating system having a registry database, the registry database associating a particular type of data with a particular software program corresponding thereto,

Providing a frame of image data;

Decoding a plurality of bits of identifier data from the image data;

Consulting with a registry database to identify a software program corresponding to the identifier data; And

Launching the identified software program.

B7. Decoding multiple fields of data from the image data using steganography, one of the fields comprising identifier data; And

The method of claim B6 comprising providing data of another said field to a software program identified for use by it.

B8. A system for responding to an encoded object to carry multiple bits of auxiliary data.

An optical capture device that produces image data corresponding to a field of view, wherein the field of view includes objects as well as visual clutter;

A visual cue-based subsystem for identifying objects from visual clutters in image data, and

And a processor for extracting a plurality of bits of auxiliary data from an object identified by the subsystem and applying the extracted auxiliary data to a response system.

B9. The system of claim B8 in which the visual cue-based subsystem identifies the object from the visual clutter with reference to spectral attributes.

B10. The system of claim B8 in which the object comprises a retroreflective member and the system comprises an illuminance source of spectral characters reflected by the retroreflective member and detected by the optical capture device.

B11. The system of claim B8 in which the visual cue-based subsystem identifies the object from the visual clutter with reference to temporal attributes.

B14. By creating an ordered object to authenticate a personal entry to a first location,

Providing an access code to the individual;

The access code generating printable data encoded using steganography; And

Using the printer at a second location away from a first location to print the printable data.

B15. The method of claim B14 wherein the access code is provided to the individual via a networked computer and the first location is a theater.

B16. A printer that responds to commands from printer driver software, characterized by the printer driver software responding to two kinds of input data, one of which includes text data and the other of which includes watermark data And the two types are applied separately to the printer driver software rather than integrated together prior to application to the printer driver software.

B17. In a method of creating a printed output using a computer, the printed output includes text, the computer includes software, the software provides a print dialog in response to user control, and the print dialog status provides a graphical user interface. Improvements that allow the user to select steganographic encoding of the print output.

B18. In how to print a magazine,

Processing the blank page to encode the majority bit data using steganography;

Then printing text and / or images on the processed page to produce a print magazine page encoded using steganography; And

Grouping at least one of the printed magazine pages encoded using steganography into each of a plurality of magazines.

B19. In the travel promotion method,

Encoding the travel photograph using steganography to conceal a plurality of bits of data;

Processing the travel photograph to extract a plurality of bits of data;

Using at least some of the extracted multi-bit data to connect an internet web browser to a web site that provides travel information useful to a consumer who wishes to visit the location depicted in the photograph.

B21. In the gesture method of providing input to the system,

Maintaining the object close to the user, positioning the object to be at least partially within the field of view of the optical scanning device, and also manually moving the object;

Capturing a frame of image data several times per second with a scanning device;

Processing a frame of image data to identify the displayed object;

Analyzing the representation of the object in a frame of image data to detect a change over time in at least one parameter selected from a list consisting of rotational state, scale state, differential scale state, and X-Y offset; And

Controlling some characteristic of the system in accordance with the detected change.

B22. The object is digitally encoded;

The method includes decoding a plurality of bits of digital data from an object; Also

Responsive to at least some of said decoded digital data in a system.

B23. The method of claim B21 in which a system responds to at least some of the decoded digital data by setting a level of complexity of system-user interaction accordingly.

B24. The method of claim B21 comprising analyzing a representation of an object in a frame of image data to detect a change over time in a rotational state, and controlling some characteristic of the system in accordance with the detected change.

B25. The method of claim B21 comprising analyzing a representation of an object in a frame of image data to detect a change over time in a scaled state and controlling some characteristic of the system in accordance with the detected change.

B26. The method of claim B21 comprising analyzing a representation of an object in a frame of image data to detect a change over time in a differential scale state and controlling some characteristic of the system in accordance with the detected change.

B27. The method of claim B21 comprising analyzing a representation of an object in a frame of image data to detect a change over time at an X-Y offset, and controlling some characteristic of the system in accordance with the detected change.

B28. Analyze the representation of the object in a frame of image data to detect a first change over time in one of the parameters and then detect a second change over time in one of the parameters, and the detected change The method of claim B21 comprising controlling some characteristics of the system in accordance with the invention.

B29. The method of claim B21 in which the first and second changes are all related to the same parameter.

B30. In a method of operating a device, such as a telephone, which constitutes a voice circuit between a caller and a receiver, a substrate having an indication is provided to an optical sensor, an image of at least part of the substrate is imaged to produce corresponding image data, and an image Decoding the plural-bit digital data from the data using steganography and establishing a speech circuit in accordance with at least some of the plural-bit digital data.

B31. The method of claim B30 in which the indication comprises a photographic image showing the recipient.

B32. Positioning the identity object at least partially within the field of view of the optical scanning device, the identity object being selected from a list consisting of a driver's license, a government identity document, a bank card, and a stamped item of jewelry;

Capturing a frame of image data with a scan device;

Processing image data therefrom to decode multiple bits of data using steganography; And

Providing at least a portion of the data decoded using steganography to a computer system.

B33. The method of claim B32, further comprising obtaining email data from an email count corresponding to the individual identified by the identity object.

B34. An email data terminal for use in a public place, the terminal including input and output, the output providing an email data to a user, the input including an optical scanning device for generating scan data corresponding to an identity object; Email data terminal, wherein the identity object is selected from the list consisting of driver's license, government identity document, and bank card.

B35. Providing an object within the field of view of the optical sensor device, wherein the object is selected from a list consisting of a retail product or a package of retail products;

Receiving optical data corresponding to the object;

Decoding a plurality of bits of digital data from optical data;

Transmitting at least some of the decrypted data to a remote computer; And

Determining at the remote computer whether a prize should be awarded in response to the transmission of the decrypted data.

B36. In the method of ordering the desktop of the first computer to meet the preferences of the user,

Providing an identity object to an optical sensor associated with the first computer;

Analyzing image data given from the optical sensor to decode a plurality of bits of data; And

Ordering a computer desktop according to the decrypted data.

B37. Using at least some of the decrypted data when communicating from the first computer to a remote computer, the remote computer having profile data stored in association with the user's preferred desktop configuration;

Transmitting the profile data from a remote computer to a first computer; And

The method of claim B36 further comprising configuring a desktop in a first computer in accordance with the profile data.

B38. Purchasing a product or service from a first supplier;

Receiving proof of purchase;

Providing evidence to the optical scanning device to produce image data;

Decoding a plurality of bits of data from the image data;

Using the decoded multi-bit data to derive a value from a second provider that is not related to a first provider.

B39. The method of claim B38 in which the value provides a free or discounted product or service from a second supplier.

B40. Providing a travel document to the optical scanning device, the travel document having encoded multiple bits of data, and the optical scanning device providing corresponding image data;

Distinguishing a plurality of bits of data from the image data; And

Based on the multi-bit data, establishing a link to a remote computer and using the computer to change the travel plan.

B41. Providing a business card to the optical sensor, the optical sensor creating image data, and the business card corresponding to the individual;

Distinguishing multiple bits of digital information from the image data;

Using the distinguished digital information to access a remote data store;

Obtaining update contact information for the individual from the data store; And

Communicating with an individual using the update contact information;

A method of communication in which contact information in a remote data store is updated during a date when personal activities make different contact information appropriate at different times.

B42. The method of claim B41 in which a plurality of bits of digital information are encoded using steganography on a business card.

B44. Providing a physical object within the field of view of the optical sensor associated with the first computer, the optical sensor providing image data;

Decoding a plurality of bits of digital data from the representation of a physical object in the image data;

Using a first portion of said multiple bits of digital data to establish a link between a first computer and a remote computer;

Providing a second portion of said multiple bits of digital data to a remote computer; And

Providing information from the remote computer to the first computer, the information corresponding to the physical object.

B45. The method of claim B44 comprising using a first portion of the plurality of bits of digital data to traverse a network of multiple server computers and thereby identify a remote computer.

B46. The method of claim B44 in which the object is a business card associated with a person and the information provided from the remote computer includes contact information relating to the individual.

B47. Steganographically encoding the print advertisement with assistance data;

Issuing the advertisement;

Billing an advertiser for the publication;

Providing a technique in which steganographic encoding can be used to provide improved value to an advertiser; And

Sharing a portion of the fee with a provider of the technology.

B48. Inserting a multi-bit digital identifier into the picture using steganography;

Storing the annotation in a database associated with the identifier;

Providing a photograph to the optical sensor, the optical sensor providing image data to decode an identifier from the image data;

Using the decrypted identifier to access the annotation stored in the database; And

Providing the accessed annotation.

B49. In how to navigate the internet,

Connecting to an internet address using a computer and providing information from the address to a user of the computer;

Providing an object to an optical sensor associated with the computer, the sensor creating image data corresponding to the field of view;

Physically manipulating an object within the field of view;

Distinguishing the manipulation of the object from the image data and producing output data corresponding thereto; And

Changing the internet address in accordance with the output data, thereby changing information provided to the user.

B50. A note pad comprising several sheets removably joined to each other at one edge by an adhesive, wherein each sheet is encoded with a plurality of bits of digital data.

B51. The note pad of claim B50 in which each sheet is encoded using steganography with a plurality of bits of digital data.

B52. The note pad of claim B50 in which each sheet is encoded with the same multiple bits of digital data as the other sheets in the pad.

B53. Providing an object to the optical sensor, the optical sensor creating image data, and the object being encoded using steganography into a plurality of bits of digital data;

Decoding the digital data from image data; And

In response to the digital data, storing image data from the sensor showing an object.

B54. The object is a sheet of paper,

Capturing several frames of image data from the optical sensor;

Identifying one of the frames having an appropriate object representation;

Masking image data corresponding to the sheet from other image data in the identified frame;

Processing image data corresponding to the sheet to change geometric attributes;

Distinguishing a size and a color of a sheet from a particular one of the digital data decoded therefrom; And

The method of claim B53 comprising storing masked image data corresponding to the sheet in association with data representing the date and time the image was stored, together with the size and color of the sheet.

B55. Providing an object to the optical sensor, the optical sensor creating image data, and the object being encoded using steganography into multiple bits of digital data;

Decoding the digital data from image data; And

In response to the digital data, displaying an image of a similar object.

B56. The method of claim B55 comprising the step of displaying an image of the plurality of paper sheets previously stored in the display order corresponding to the time sequence in which the object is a paper sheet.

B57. In a computer operating system, gesture data is extracted therefrom in response to two-dimensional image data of consecutive frames, the gesture data comprising at least two data in a list consisting of XY position data, rotational state, scale state, and differential scale state. Software module.

C1. Providing a business card to the optical sensor, the optical sensor creating image data, and the business card corresponding to the individual;

Distinguishing multiple bits of digital information from the image data;

Using the distinguished digital information to access a remote data store;

Obtaining update contact information for the individual from the data store; And

Communicating with an individual using the update contact information;

A method of communication in which contact information in a remote data store is updated during a date when personal activities appropriate other contact information at different times.

C2. The method of claim C1 in which a plurality of bits of digital information are encoded using steganography in a business card.

C3. Providing a physical object within the field of view of the optical sensor associated with the first computer, the optical sensor providing image data;

Decoding a plurality of bits of digital data from the representation of a physical object in the image data;

Using a first portion of said multiple bits of digital data to establish a link between a first computer and a remote computer;

Providing a second portion of said multiple bits of digital data to a remote computer; And

Providing information from a remote computer to the first computer, the information corresponding to the physical object.

C4. The method of claim C3 comprising using a first portion of said multi-bit digital data to thereby identify a remote computer across a network of multiple server computers.

C5. The method of claim C3 in which the object is a business card relating to an individual and the information provided from the remote computer includes contact information relating to the individual.

D1. Providing a digitally encoded object to the optical sensor, the optical sensor producing output data;

Decoding a plurality of bits of data from the sensor output data; And

Using the multiple bits of data to link to an internet address having data associated with the object.

D2. The method of claim D1 in which an object is encoded using steganography with the plurality of bits of data.

E1. Encoding the print advertisement using steganography to hide a plurality of bits of data;

Processing the print advertisement to extract a plurality of bits of data;

Using at least some of the extracted multi-bit data to connect an internet web browser to a web site that provides consumer information related to a product or service promoted by a print advertisement.

E2. In how to determine consumer response to print ads,

Encoding steganography with the first data using the first print advertisement;

Encoding the second print advertisement using steganography with second data;

Decrypting the first and second data when the consumer provides the first and second advertisements to the optical sensor; And

Calculating the number of decrypted first and second data, respectively, to determine a consumer response to the advertisement.

E4. In the way of travel promotion,

Encoding the travel photograph using steganography to conceal a plurality of bits of data;

Processing the travel photograph to extract a plurality of bits of data; And

Using at least some of the extracted multi-bit data to connect an internet web browser to a web site that provides travel information useful to a consumer who wants to visit the location shown in the photograph.

F1. In a mouse having X- and Y-position encoders and circuitry that generates and responds to generating and providing X- and Y-movement data to an associated computer, an optical sensor disposed in the mouse, and gray scale optical image data are provided. And circuitry responsive to provide it to an associated computer, wherein the mouse operates as both a positioning device and an optical input device.

F2. A substrate formed thereon;

A two-dimensional image sensor producing a sensor signal;

A first circuit responsive to the sensor signal to produce image data corresponding thereto and to provide the image data to an image data output of the semiconductor; And

And a second circuit responsive to the sensor signal or the image data therefrom for decode plural-bit digital data using steganography and provide it to the steganographic decoder output of the semiconductor.

F3. In a method of operating a computer system comprising an internet web browser,

Providing a peripheral device having a sensor;

Positioning a peripheral device adjacent to the first object to connect the web browser to the first address; And

Positioning the peripheral device adjacent the second object to connect the web browser to the second address.

G1. In a peripheral device for a computer system,

A housing adapted to fit in the palm of the user and to slide over the media;

An optical sensor having a plurality of sensing elements to produce an image signal;

A lens for imaging the medium to the sensor;

Circuitry disposed within the housing and coupled to the sensor for processing a signal from the sensor and formatting it into a frame of output data; And

Peripheral means for transmitting output data from the peripheral device to the computer system.

G2. The device of claim G1 wherein the transmission means is a cable.

G3. The device of claim G1 in which the transmission means is a wireless link.

G4. The device of claim G1, wherein the circuit decodes the multi-bit binary data encoded using steganography in the image sensed by the sensor, and the transmission means relays the decoded data to the computer system.

G5. Providing a print catalog comprising an image of an article provided for sale by a merchant, wherein the image is encoded using steganography with multiple bits of binary data;

Optically sensing the image to produce image data corresponding thereto;

Decoding the encoded data using steganography from the image data; And

Electronically ordering an article from a merchant using the decrypted data, wherein the order first uses stored consumer profile information.

G6. The method of claim G5 in which the consumer profile information includes garment size data.

G7. Processing the decrypted data for transmission to a remote merchant computer, wherein the processing includes supplementing the decrypted data with supplemental data corresponding to the consumer;

Transmitting the processed data to a remote merchant computer;

In response to the transmitted processing data, receiving first order data from a remote merchant computer;

Providing first order data to a consumer;

Receiving from the user another input selected from the options included in the first order data; And

The method of claim G6 further comprising sending the another input to a remote merchant computer.

G8. The method of claim G7 in which the supplemental data includes consumer profile information.

G9. The method of claim G7 in which the supplemental data includes data identifying the consumer.

G10. badge;

Including a badge photo,

The picture has a plurality of bits of auxiliary data hidden therein, the auxiliary data corresponding to an authenticated bearer of the badge;

The badge comprises another bearer-identifying information, the further information being machine-detectable.

G11. The device of claim G10, wherein further information is encoded by an RF proximity circuit formed of a badge.

G12. The device of claim G10, wherein further information is encoded in a barcode on the badge.

G13. The device of claim G10, wherein further information is encoded in a magnetic stripe on the badge.

G14. In how to access a stable home,

Providing a badge to the sensor station;

Generating optical data corresponding to the photograph on the badge;

Processing optical data to extract a plurality of bits of auxiliary data encoded using steganography;

Machine-detecting the ID of the badge from data not encoded in the photograph; And

Checking the correspondence between the auxiliary data extracted from the photograph and the machine-detected ID to determine whether access to a stable home is authenticated.

I1. Optical input device;

A client application responsive to digital data encoded in the image; And

An image-based navigation system comprising a router responsive to at least some of the digital data.

I2. A computer storage medium having instructions for causing a computer system equipped with a camera to participate in training exercises with the user so that the user is familiar with the operation of the camera-based internet navigation system.

I3. A substrate having a printed image, the image having first and second data encoded, the first data comprising an owner code, the second data comprising a router code, and the device being a camera-mounted computer An internet navigation device useful for connecting a system to an internet address.

I4. The device of claim 13, wherein the owner and router code is encoded using steganography in the image.

I6. In an image-based network navigation method that allows a user to link to a remote computer,

Detecting encoded data in the printed image;

Linking to a remote computer via a network according to the encoded data; And

Providing an identity of an image to a remote computer.

I7. In an internet navigation system comprising a client application in communication with a remote user,

Providing first data to the router from the client application;

Providing a plurality of data to the client application at the router;

At the client application, providing a menu corresponding to the plurality of data; And

Accepting user input corresponding to one of the plurality of data.

I8. The method of claim I7 comprising deriving first data from image data provided from an optical input device.

I9. The method of claim I8 comprising linking the client application to a web page corresponding to the accepted user input.

I10. The method of claim I9 comprising waiting for the linked web page to be displayed, while providing the user with a branded display stored locally in the client application.

I11. In a router accepting a routing code and responding with a corresponding URL, each router comprising time information corresponding to the plurality of URLs and describing when the URL is used.

J2. Generating a database record comprising a plurality of data fields;

Generating a file corresponding to the database record and including data from at least a specific portion of the fields;

Electronically distributing a copy of the file to each of the plurality of recipients;

One of the recipients appending data to the copy of the file or changing the data in the copy of the file and transferring the file to a database;

Updating a database record according to the changed file;

Generating a new file corresponding to the updated database record and including data from at least a specific portion of the file; And

Electronically distributing a copy of a new file to each of said plurality of recipients.

J3. In a system that is linked from a physical or digital object to a corresponding digital resource,

Registration means for receiving data related to the object, including identity and owner, and associating it with data related to the corresponding response in the database;

Originating device means for sensing data from an input object, processing it, and passing it to routing means;

Routing means for processing the processed data from the originating device means, logging the information therefrom, and conveying at least a specific portion of the processed data to a product handler means; And

Product handler means for providing a response to the originating device means in accordance with the information provided by the routing means.

J4. The system of claim J3 in which the routing means comprises means for checking information in a database.

J5. The system of claim J3 in which the registration means comprises means for generating an encapsulated file and means for distributing the file to a given population.

J13. A networked computer system responsive to watermark data sent from a remote client application to initiate transmission of audio or video data.

J14. The networked computer system of claim J13 responsive to watermark data sent from a software program at a remote computer to initiate transmission of audio or video data to the remote computer.

J16. Networked computer system responsive to watermark data sent from a software program at a remote computer to initiate transmission of updated software to the remote computer.

K1. In a scanner comprising a CPU, a memory, a linear sensor array, and a spaced apart first and second 2D sensor array acting as a motion encoder, the CPU is linear from an object imaged according to the scanner movement data provided from the 2D sensor. A scanner comprising software instructions in memory operative to process the raw scan data collected by the sensor array into final scan data and causing the scanner to distinguish machine-readable identifiers from the scan data obtained from the object. .

K2. The scanner of claim K1 further comprising a wireless interface and a display, wherein the software instructions cause the scanner to relay an identifier by a wireless interface to a remote server and to process information returned via the wireless interface to be displayed on the display.

K3. The scanner of claim K1 in which the software instructions process data from a 2D sensor array for the purpose of adding to the CPU detecting the scanner movement.

K4. The scanner of claim K3 in which the purpose includes starting a watermark detection process before the data from the linear sensor array is finally processed.

K5. The scanner of claim K4 in which the purpose comprises starting to detect a watermark measurement signal.

K6. The scanner of claim K3 in which the purpose includes identifying a portion of data selected by a linear sensor array that is relatively prone to contain detectable identifier data.

K7. The scanner of claim K3 in which the purpose is to quantify object surface properties, and a filter can be applied accordingly to the scan data.

K8. The scanner of claim K3 in which the purpose is to assess the relative distance to the object from another portion of the scanner.

K9. The scanner of claim K3 in which the purpose is to determine the amount of fine distortion in the scan data, so that a compensation can be applied thereto.

K10. The scanner of claim K1 in which the identifier is encoded using steganography as a digital watermark.

K11. The scanner of claim K1 in which the identifier is encoded by a barcode.

K12. A linear sensor array, a CPU, an interface for connecting to the Internet, and a user interface including a display screen and user control, adapted to transmit machine-readable data detected from the scanned object to the Internet, and from the Internet Scanner that provides the user with the HTML information received back.

L1. In an image watermark processing technique comprising changing an optically detectable pattern in an object, the object has a non-planar surface and the watermark applied to the surface is prewarped to compensate for the nonplanarity in advance. Mark processing technology.

L2. The method of claim L1 in which the watermark is geometrically prewarped.

L3. The method of claim L1 in which the object has a cylindrical surface.

L4. The method of claim L3 in which the object is a container for consumer goods.

L5. The method of claim L4 in which the object is a beverage can.

L6. The method of claim L1 in which the watermark is prewarped in the transform domain.

L7. The method of claim L6 in which the watermark is prewarped in the wavelet domain.

L8. The method of claim L6 in which the watermark is prewarped in the DCT domain.

L9. The method of claim L1 in which the watermark has been warped in advance taking into account the curvature-derived clear geometric distortion.

L10. The method of claim L1 in which the surface is prewarped in view of the distance-induced clear geometric distortion.

L11. The method of claim L10 in which the watermark has been warped in advance taking into account the curvature-derived clear geometric distortion.

L12. A watermark decoding technique comprising analyzing a frame of image data to decode an encoded watermark, wherein the watermark decoding technique comprises applying a distortion prevention function to the image data before decoding the watermark. .

L13. The method of claim L12 comprising successively applying another warp protection function to the image data to attempt to decrypt valid watermark data.

L14. The method of claim L12 comprising selecting one or more warping features to apply based on information provided by the user.

L15. The method of claim L14 in which the information identifies a general class of object to which the object belongs, without specifically identifying a particular object shape.

conclusion

While the principles of the present technology have been described with reference to the described embodiments, it should be appreciated that the invention is not so limited.

For example, although certain embodiments have been described with reference to Internet-based systems, the same techniques may similarly be applied to other computer-based systems. These include non-Internet based services such as America Online and Compuserve, dialup bulletin board services, and the like. Similarly, in an internet-based embodiment, the use of a web browser and web page is not essential; Other digital navigation devices and other online data stores can be similarly accessed.

Similarly, although the details of the described system are specifically given, the basic principles may be used in a variety of other forms.

For example, one other form is to steganographically encode a physical object into a Digital Object Identifier (DOI). The National Research Initiatives and the Digital Object Identifier Foundation (www.doi.org) establishes an infrastructure for distributing, tracking, and managing digital objects. Execute expanded work at the time of establishment. Some of these same infrastructures and techniques can be adapted in accordance with the instructions provided above to associate new functionality with physical objects.

Another form is to refer to the remote data store by the data inserted into the object, instead encoding the final data directly into the object. For example, a picture can be encoded into a phone number and encoded. By providing a photo to the optical sensor on the phone, the phone can analyze the optical information to extract the phone number and link the number without requiring external data. Similarly, the printed office document (e.g., spreadsheet) is encoded with the file name and path of the corresponding electronic file, so as to correlate the indirect link (e.g. to the remote database to correlate the UID to the computer address). ) Eliminate the need. Personal business cards can encode the person's email or web address directly. Most of the embodiments described above are suitable for direct encoding of related data.

In the example of the business card given above, the detailed description can supplement the existing optical character recognition technology. That is, image data from the optical sensor can be applied to both Bedoop decoders and OCR systems. Text characters distinguished by the OCR system can be entered directly into the contact manager personal database. Techniques used in Bedoop systems to locate coded objects and to handle visual distortions (e.g., visual defects due to scale, rotation, etc.) are advantageously used for OCR detection, so that OCR without careful placement of the card Allow extraction of information.

Although this particular embodiment refers to ink-jet printing, similar advantages may be obtained with other printing techniques, such as laser / xerographic printing, offset printing, and the like.

In this embodiment, Bedoop decoding generally proceeds from image data obtained from a physical object. However, in some contexts it is advantageous to Bedoop-decrypt image data provided electronically, for example, via the Internet.

Likewise, the embodiment generally relies on a Bedoop image sensor staring at the object at the expected point, but in other methods embodiments a sensor may be used to search rather than stare (described above in connection with the example of an elevator). As).

Similarly, the described embodiment generally used sensors that repeatedly capture frames of image data, but need not be the case. A single frame system, such as a flatbed scanner, and a video system configured to capture a single frame can be used in other ways-with or without the TWAIN interface.

As indicated above, in the preferred embodiment steganographic encoding of digital data is used, but in view of aesthetics, a visible form of digital encoding-such as a barcode-can be used naturally.

In certain embodiments, digital data transmitted by means other than optical means may be used. As in the above embodiment, it is allowed to "at-a-distance" the reading of data from the physical object and use it in electromagnetic detection (e.g., in proximity-based card-access systems to decrypt digital data). Being of a kind).

Since Bedoop image sensors typically accept multiple frames of data, the extraction of digital data may be based on one or more image frames. A greater certainty in the results is to accumulate the decoded data over several frames. In addition, object movement within the field of view of the sensor allows the system to accept information from another perspective, thereby enhancing system operation.

Preferred embodiments use a 2-D image sensor (eg CCD), but other optical sensing techniques can be used in other ways. For example, a supermarket laser scanner can read bar-code data. Raster scans of such systems may allow for the capture of 2-D data (in bit-mapped form or in gray-scale).

Some embodiments may advantageously use texture-based Bedoop encoding of an object. Bedoop textures can be implemented by a variety of means including pressure rollers, chemical or laser etching, and the like.

It should be noted that the response triggered by the watermark may change over time. This extends the useful life of the encoded object. For example, the encoding link in a magazine advertisement for a 1999 Ford Explorer pointing to a Ford URL related to the 1999 Ford Explorer may be updated to point to the URL for a 2000 model New Year's version when marketing of a new model New Year's vehicle begins. .

Of course, in other embodiments the character URL may be encoded and used to link a browser or other information instrument to that address. In addition, the character URL can be encoded, but need not necessarily be used. Instead, the encoded URL can be mapped to a real URL (ie, a URL to which a browser is connected or a response to a MediaBridge object is keyed) through a database. In this embodiment, the URL is encoded in the object with a future date. When "reading" an object, the local (client) computer checks the associated date. If the date has not passed, the text URL is used as the actual URL. If the date has passed, the client computer consults the code for the remote database (eg on the router) to get the actual URL (which may be the same if an update is required or new). The actual URL is then used to provide a response to the MediaBridge object.

Implementation of the watermark encoding and decoding system is simple for those skilled in the art, and is not overly discussed here. Such techniques are conventionally implemented by appropriate software stored in long-term memory (e.g. disks, ROMs, etc.) and transferred to temporary memory (e.g. RAM) to be implemented in the associated CPU. In other implementations, functionality may be accomplished by dedicated hardware or by a combination of hardware and software. Reprogrammable logic, including the FPGA, can advantageously be used in certain implementations.

The above embodiment generally used planar objects to convey digital encoding, but the case need not necessarily be used. Other shaped objects can be used similarly. Some shapes provide a relatively simple image processing operation. As described above, data imaged from beverage cans or other cylindrical surfaces can basically be remapped using known geometric transformations to "unwrap" printing from the can. Other geometries can provide more complex remappings, but they are generally similarly within the skill of those skilled in the art (this remapping is facilitated by encoding certain reference marks, such as grid lines, in the data. Unknown 3D The imaging of the shape object can be inferred from the apparent distortion of the reference marks in the 2D image data typically generated by the scanner Once the distortion is characterized, it is generally to avoid distortion to prepare the image data for decoding. Simpler)

At one time, it was common to predict that paper documents would be replaced by electronic media. In the past, electronic media can be perceived as poor agents for paper. Electronic media deliver information without flaw, but lack empirical attributes. Paper may be retained, stacked, owned, erased, provided, and protected. This provides an opportunity for lack of physical domination in electronic media.

From the discussion above, it can be seen that rather than replacing paper with electronic media, it will probably provide digital attributes to the paper-by blending the physical experience of the paper with the technical advantages of digital media. This configuration makes many new functionality available and accessible through friendly paper items rather than "computer input peripherals."

In order to provide an understandable description without unduly extending the present specification, applicants refer to the patents, applications, and publications identified above by reference.

In view of the many embodiments to which the principles described above can be applied, the detailed embodiments should be recognized as merely illustrative and should not be taken as limiting the scope of the invention. Rather, all such embodiments falling within the scope and spirit of the following claims and equivalents thereof are claimed in the present invention.

Claims (34)

Receiving, at a remote computer, first data indicating a location of a first user, the first data being based on wireless signals received by the portable device of the first user; Displaying, the receiving step;
At the remote computer, transmitting information to a portable device of a second user, wherein the second user device is associated with a second user different from the first user, the information indicating the location of the first user. And transmitting, dependent on the first data. The method of claim 1,
The first data is transmitted from a portable device of the first user. The method of claim 1,
And the first data comprises data from a GPS receiver. The method of claim 1,
Wherein the information transmitted to the portable device of the second user includes data obtained from a database record. The method of claim 1,
The information transmitted to the portable device of the second user includes a telephone number to which the first user can be contacted. The method of claim 1,
And the information transmitted to the portable device of the second user includes object data. The method of claim 1,
The information transmitted to the portable device of the second user includes object data derived by processing object information captured by the portable device of the second user. The method of claim 1,
The information transmitted to the portable device of the second user includes image data derived by processing the image information captured by an image sensor at the portable device of the second user. Processing the signal captured by the sensor at the portable device of the first user,
Transmitting information corresponding to the captured signal from the portable device of the first user to a remote computer server;
At the portable device of the first user, receiving information in response to the transmission; And
Presenting, on the screen of the portable device of the first user, a display corresponding to the received information;
And the received information is dependent on location data indicating the location of the portable device of the second user based on wireless signals received by the portable device of the second user. delete The method of claim 9,
Receiving at the remote computer server location data transmitted from the portable device of the second user. At a sensor in the portable device, capturing data corresponding to the object;
At the processor in the portable device, processing the captured data to generate processed object data;
Determining, at a processor in the portable device, location information indicating a location of the portable device based on wireless signals received by the portable device;
At the portable device, wirelessly transmitting the processed object data and the location information to a remote computer; And
At the portable device, in response to the transmission, receiving response information associated with the transmitted processed object data and the transmitted location information. The method of claim 12,
The processed object data includes data to identify the object. The method of claim 12,
The object comprises a physical object. The method of claim 12,
Capturing optical data from the object using the image sensor portion of the portable device. The method of claim 12,
Capturing RFID data from the object. The method of claim 12,
Generating identification data corresponding to the object using the processor at the portable device. The method of claim 12,
Using a processor at a computer remote from the portable device to derive identification data corresponding to the object based on the processed object data sent by the portable device. Capturing image data from a machine readable indicia on a physical object using an optical sensor in the user's portable telephone device;
Processing the captured image data using a processor in the portable telephone device to generate an identifier;
Determining, at a processor in the portable telephone device, location data indicating a location of the user based on wireless signals received by the portable telephone device of the user;
Wirelessly transmitting, at the portable telephone device, both the identifier and the location data to a remote computer; And
At the portable telephone device, providing the user with a response associated with the transmitted identifier and location data. The method of claim 19,
The indication comprises a barcode. The method of claim 19,
The indication comprises a digital watermark. The method of claim 19,
At the remote computer, determining a physical item to be delivered to the user. The method of claim 19,
At the remote computer, determining a food item to be delivered to the user. Receiving information from a user's portable device, wherein the received information includes identification information derived by processing image data captured by the user's device, the user's portable based on received wireless signals Receiving the information, the location information also determined by the device;
At a remote computer, referring to the received identification information and the received location information, among the plurality of businesses, due to the location of the particular business, determining whether or not the particular business will deliver a physical item to the user. step; And
At the remote computer, transmitting at least a portion of the received information to the computer of the particular business to enable the business to provide delivery of the physical item to the user. 25. The method of claim 24 including determining, at the remote computer, the physical item desired by the user with reference to the identification information. 25. The method of claim 24, comprising determining, at the remote computer, a street address for delivery of the physical item with reference to the location information. delete delete delete delete delete delete delete delete

Images