ES2720661T3 - Method to provide images of a tissue cut - Google Patents

Abstract

A method for differentiating areas in a series of N primary digital images of a tissue cut, where N is an integer> 1, thus creating a new image, said method comprising the steps of: a) providing a series of N images (221) primary digitals comprising first marker areas, wherein an In + 1 image comprises at least the same first marker areas as a primary In digital image at for 1 <= n <N where n is an integer; b) evaluate (211, 222, 310) each digital image In primary for 1 <= n <= N according to the predetermined selection criteria and identify as image marker areas those areas of the first marker that meet the criteria of predetermined selection, and storing information (311) on any of those image marker areas in or in connection / association with a corresponding resulting secondary digital image, thereby obtaining a series of N secondary digital images; c) provide (223) a new new image of tissue cutting without marker areas; d) inserting (212, 224) new image marker areas into the new New image, said new image marker areas having the same shape and position as the image marker areas present in image I1, and said areas being of marker of identifiable image in Inueva, by a unique characteristic; e) for each n for 2 <= n <= N of the series of secondary digital images obtained in step b), insert new image marker areas in the new New image, said new image marker areas having the same shape and position of the image marker areas present in the In image but not in the In-1 image, and said new image marker areas being identifiable in Inueva by a unique feature.

Description

DESCRIPTION

Method to provide images of a tissue cut

Field of the Invention

The present application refers to the field of immunohistochemistry as well as computer-based image analysis. More specifically, the application provides a method to differentiate areas in a series of images.

Background of the invention

The analysis of a histological tissue sample is commonly used for diagnostic purposes, for example, analysis of a breast tissue sample to diagnose breast cancer or for research purposes, for example to study inflammatory cell types in inflammatory conditions such as asthma, arteriosclerosis or inflammatory bowel diseases.

Immunohistochemistry (IHC), whereby a marker (that is, an antigen) is detected by an antigen specific antibody, is commonly used to identify cells in histological sections. Ideally, the identification of a cell type can be obtained with the detection of a cell-specific cell antigen. However, for several cell types combinations of several antigens must be analyzed for proper identification.

Any diseased tissue is normally associated with an altered cell composition. For example, in the inflamed airways in asthma there is an altered composition of the structural cells that form the airways, such as epithelial cells, gland cells, blood vessel cells, nerves, etc. In addition, several types of immune cells (i.e., leukocytes) infiltrate the inflamed airways.

In many diseases, the pathology (that is, destructive alterations) in the tissue is not caused by one type of cell, but by a complex interaction between several types of cells. Therefore, when exploring a sample of diseased tissue, it is often convenient to study various populations of tissue cells and structures. Information on cellular content can be obtained by staining one type of cell each time in serial sectioned sections. Although this treatment provides a good estimate of the content of various cell types in a tissue sample, it does not provide detailed information on the spatial relationship (i.e., the physical relationship) between the cell types analyzed.

To better explore how the composition of cells can define certain disease conditions, or study how cells interact and relate to each other within a diseased tissue, it is desirable to develop means to visualize multiple types of cells within the same three-dimensional space, for example within a single tissue cut.

With the IHC techniques currently available it is possible to stain up to 4 cell types in a cut using multiple chromogen or multiple immunofluorescence techniques. However, in usual practice, often only 2 types of cells can be detected simultaneously due to the lack of appropriate combinations of primary detection antibodies.

US 2011/0091081 A1 describes a technique for obtaining and storing data on the expression of multiple biomarkers in individual cells or individual cell compartments in a tissue sample. Methods for using the data to create groups are also described, whose elements share a certain degree of similarity greater than the general population from which the data is extracted, by analyzing digital images of a part of the tissue sample that has been stained iteratively to generate optical signals, usually fluorescent, that reflect the amount of each of the biomarkers examined. The analysis of the images implies a segmentation routine in which each pixel of the examined images is assigned to an individual cell or to a compartment of an individual cell, the intensity of the representative signal of each biomarker is measured for each pixel, it is created a data set in which each cell or compartments of each cell is associated with a signal strength for each biomarker examined, and the data set is consulted with appropriate numerical tools to create groups. It also describes the visualization of such groups in images of the examined tissue, so that the individual cells belonging to a particular group are marked or indicated in one of the examined images.

To increase the number of markers in a tissue cut, new methodological approaches have been developed, such as the SIMPLE technique described in WO 2010/115089 and the MELC technique (Schubert et al., Nature Biotechnology vol. 24, p. 1270-1278). Although powerful, these new types of techniques have been developed primarily for colocalization studies and include either destructive tissue procedures, procedures that involve the destruction of detection groups or the dependence of primary antibodies labeled with detection molecules, characteristics that limit the number of cell markers that can be stained.

Since the techniques mentioned above were mainly developed for colocalization studies, they do not address the fact that many identification markers may also occasionally be present in cell types to which they are not intended.

Therefore, there is a need for new technical treatments by which a large number of cell types can be properly identified simultaneously within the same physical space, such as a tissue cut. Ideally, any technique of this type should be able to analyze a large complete section of samples and provide detailed information on all marked individual cells, such as their spatial coordinates in the tissue, their size and shape parameters, etc.

Summary of the Invention

In a first aspect, the present invention provides a method for differentiating areas in a series of N primary digital images of a tissue section where N is an integer> 1, thus creating a new image, said method comprising the steps of:

a) provide a series of N primary digital images comprising indeterminate marker areas, where an In ⁺¹ image contains at least the same indeterminate marker areas as a primary digital image In for 1 <n <N, where n is an integer

b) evaluate each primary digital image In for 1 <n <N according to the predetermined selection criteria and define image marker areas as indeterminate marker areas that meet the predetermined selection criteria, and store information on any of said image marker areas in or in connection / association with a corresponding resulting secondary digital image, thereby obtaining a series of N secondary digital images;

c) provide a new new image of tissue cutting without marker areas;

d) inserting new image marker areas in the new Inueva image, said new image marker areas having the same shape and position as the image marker areas present in the image h, and said image marker areas being identifiable in Inueva, for a unique feature;

e) for each n for 2 <n <N of the series of secondary digital images obtained in step b), insert new image marker areas in the new New image, said new image marker areas having the same shape and position that the image marker areas present in the In image but not in the In ^-1 image, and said new image marker areas being identifiable in Inueva by a unique feature.

Preferably, the new Inueva image is a copy of one of the images in said series of images.

Preferably, said unique feature in steps d) and e) is a feature that has a unique value for each n, 1 <n <N.

Preferably, said unique feature is a general color and said unique value of said unique feature is a specific color associated with a particular cell marker.

Preferably, the predetermined selection criteria comprise a threshold for a visual property of an undetermined marker area.

In a second aspect, the invention provides a method for visualizing cell populations within a histological section of tissue, said method comprising the steps of:

a) providing a tissue cut that has been prepared for molecular staining in a previously known manner;

b) providing a series of K particular molecular detection means to specifically bind and detect elements of a predetermined series of markers of K cells that may be present in the tissue section of step a), said molecular detection means being able to generate the formation of an initiable and detectable response, where K is an integer> 2;

c) for each particular molecular detection means k = 1,2, ..., K of step b) carry out the following procedure:

1) contacting said tissue section of step a) with said particular molecular detection means resulting in a specific binding to a particular element of said predetermined series of cell markers;

2) washing said tissue cut to remove molecular detection means that have not bound to any cell marker;

3) initiate the response of the molecular detection means that may have been attached to cellular markers of the tissue cut thus allowing the detection of said molecular detection means; Y

4) when said molecular detection means can be detected, scan / obtain tissue cut images to generate a primary digital image Ik that may contain one or more undetermined marker areas associated with the generation of a detectable polymer;

whereby a series of K primary digital images Ik is obtained for k = 1, ..., K containing an increasing amount of indeterminate marker areas;

d) carry out the method of the first aspect in the series of K primary digital images Ik for k = 1, ..., K obtained in step c), thus generating a New image that makes said cellular structures visible.

In a preferred embodiment of the method of the second aspect, said molecular detection means are a set of antibodies, preferably monoclonal antibodies or antibody fragments, wherein each antibody binds to a specific cell marker and where an enzyme has been conjugated to each antibody, said enzyme being capable of generating the formation of a detectable polymer in the presence of one or more suitable substrates,

where points 1) and 2) of step c) are carried out in such a way that:

i) the tissue cut of step a) is contacted with an antibody that specifically binds to a particular element of said predetermined series of cell markers; said antibody being conjugated to an enzyme, said enzyme being capable of generating, in the presence of one or more suitable substrates, the formation of a detectable polymer;

ii) after step i) above, the tissue cut is washed to remove unbound antibodies; Y

wherein point 3) of step c) is carried out in such a way that:

iii) after point 2) tissue cutting is exposed to one or more suitable substrates for said enzyme, which leads to the formation of detectable polymers in case said particular element of said predetermined series of cell markers is present in said tissue cutting

In another preferred embodiment of the method of the second aspect, said molecular detection means are a set of molecular complexes, wherein each complex comprises a first antibody, preferably a monoclonal antibody, that binds to a specific cell marker, a second antibody or a fragment of antibody, preferably a monoclonal antibody specifically bound to said first antibody, and an enzyme conjugated to said second antibody, said enzyme being capable of generating the formation of a detectable polymer in the presence of one or more suitable substrates,

where points 1) and 2) of step c) are carried out in such a way that:

i) the tissue cut of step a) is contacted with a first antibody that specifically binds to a particular element of said predetermined series of cell markers;

ii) after step i) above, the tissue cut is washed to remove unbound antibodies;

iii) after step ii) above, the tissue cut is contacted with a second antibody that specifically binds to said first antibody, said second antibody being conjugated to an enzyme, said enzyme being capable of generating the formation of a polymer detectable in the presence of one or more suitable substrates; Y

iv) after step iii) above, the tissue cut is washed to remove unbound antibodies; Y

wherein point 3) of step c) is carried out in such a way that:

v) after point 2) tissue cutting is exposed to one or more suitable substrates for said enzyme, which leads to the formation of detectable polymers in case said particular element of said predetermined series of cell markers is present in said tissue cutting

Preferably, said enzyme is chosen from the group of alkaline phosphatase and peroxidase, such as horseradish peroxidase.

Preferably, said substrate is selected from the group of 3, 3'-diaminobenzidine, Ferangi Blue, Vulcan Fast Red, Vina Green and aminoethyl carbazole (AEC). Vina Green is an example of a substrate that generates a polymer that is at least partially soluble in water under certain conditions. Aminoethyl carbazole (AEC) is an example of substrate generating polymers that are at least partially soluble in lower alcohols such as ethanol, under certain conditions. 3, 3'-Diaminobenzidine, Ferangi Blue and Vulcan Fast Red are examples of substrates that generate insoluble polymers.

It is understood that different substrates and / or enzymes can be used during the execution of the method. For example, diaminobenzidine can be used as a substrate in the first run of item 2), and Vulcan Fast Red can be used as a substrate in a later run of item 2).

In another preferred embodiment of the method of the second aspect, said molecular detection means is a set of molecular conjugates comprising a recognition part attached to a detection part, wherein said recognition part is capable of specifically binding to a particular element of said predetermined series of cell markers, said recognition portion being selected from the group of: an antibody, such as a polyclonal antibody, a monoclonal antibody, or fragments thereof, and a nucleic acid molecule, such as an RNA molecule or a DNA molecule, said detection part being a <t0 /> fluorochrome, said fluorochrome being capable of emitting radiation of a particular wavelength after exposure to an initiation radiation other than said emitted radiation

wherein point 3) of step c) is carried out in such a way that the tissue cutting and any molecular detection means that has been attached thereto are exposed to the initiation radiation that leads to the emission of radiation from a particular wavelength in case said particular element of said predetermined series of cell markers is present in said tissue cut; and wherein point 4) of step c) is carried out when said radiation of a particular wavelength is emitted.

In yet another preferred embodiment of the method of the second aspect, a substrate is used that generates at least partially soluble polymers as detectable polymers, such as Vina Green or aminoethyl carbazole (AEC). This embodiment further comprises the steps of

e) washing said tissue cut to remove detectable soluble polymers; Y

f) repeat steps b - d with a new series of molecular detection means.

This embodiment is useful when assessing the presence of a large number of cell markers.

In a third aspect, the present invention provides a method for visualizing the three-dimensional distribution of multiple cell populations and cell structures within the same three-dimensional space in a histological sample, comprising the steps of:

A) providing a tissue sample, and cutting said sample into a plurality of originally superimposed tissue cuts in a previously known manner;

B) carry out the method according to the second aspect for all the tissue sections obtained in step A) and C) superimposing the images obtained in step B) according to known principles, thus obtaining a three-dimensional visualization of the three-dimensional distribution of multiple cell populations and cell structures within the same three-dimensional space in a histological sample.

Definitions

In describing and claiming the invention, the following terminology will be used in accordance with the definitions set forth below. Unless otherwise defined, all technical and scientific terms used in this document have the same meaning commonly understood by one skilled in the art to which this invention pertains. Although methods and materials similar or equivalent to those described herein may be used in the practice of the present invention, the preferred methods and materials are described herein. As used in this document, each of the following terms has the meaning associated with it in this section. The specific and preferred values listed below for radicals, substituents and ranges are for illustration only; they do not exclude other defined values within defined ranges for radicals and substituents.

A cell marker means a specific structure that, depending on the type of cell, may appear more or less specifically, often on the surface of a cell of the cell type, but sometimes also within the cell. Normally, a cell marker is a receptor capable of specifically binding to a particular target molecule. The cell marker can be a protein, a glycoprotein or a carbohydrate, a nucleic acid, a lipid or other type of naturally occurring biological molecule. The person skilled in the art knows such cell markers and cell types in which they appear. The presence of one or more of these cell markers may indicate that the cell belongs to a certain class or type of cell.

Molecular detection means means a bifunctional aggregate or conjugate comprising a first part capable of specifically binding to a particular cell marker, and a second part capable of generating a detectable response. Many different types of molecular detection media could be used in the present invention.

Normally, said first part capable of specifically binding to a particular cell marker could be an antibody or a fragment thereof, such as a Fab fragment, or a nanoantibody, or a nucleic acid molecule, such as a DNA molecule or a molecule of RNA or a derived nucleic acid such as PNA. Normally, said second part capable of generating a detectable response could be an enzyme, or a chemical compound capable of generating some type of detectable signal, such as a fluorochrome, when induced by a specific action.

A molecular detection means could, in its simplest embodiment, comprise a first part, such as an antibody or a nucleic acid molecule, to which the second part has been bound, such as an enzyme or a fluorochrome. Alternatively, a suitable molecular detection means could be a complex comprising a first entity, usually a monoclonal or polyclonal antibody that specifically binds to a particular cell marker, a second entity, usually a monoclonal or polyclonal antibody, that specifically binds to the first entity, and a third entity linked to the second entity, where said third entity could be an enzyme or a chemical compound capable of generating some type of detectable signal, such as a fluorochrome, when induced by a specific action.

A detectable response is understood as a response that could be detected in a scanning / imaging step in such a way that the response could be found within the image produced by said scanning / imaging step.

In one embodiment, the detectable response is the formation of an opaque and / or colored polymer. Such polymers could be formed by contacting certain enzymes that are part of a molecular detection medium with specific substrates under the appropriate conditions. Examples of suitable enzymes are alkaline phosphatase and peroxidase, such as horseradish peroxidase. Examples of suitable substrates for such enzymes are 3, 3'-diaminobenzidine, Ferangi Blue, Vulcan Fast Red, aminoethyl carbazole (AEC) and Vina Green.

In another embodiment, the detectable response is the emission of radiation of a certain wavelength, such as the radiation emitted by a fluorophore after excitation.

In another embodiment, different forms of detectable responses are used within a single method execution.

Primary digital image is a digital image that has been obtained by direct scanning (for example, slide scanning or photomicrograph) of a tissue cut. No additional adaptation, editing or evaluation of the image has been made. This image should be considered as raw source data. Secondary digital image means an image obtained by some type of digital editing or evaluation of a primary digital image. A secondary digital image can be obtained, for example, by editing and / or evaluating a primary digital image. The primary digital image is redefined as well as a secondary digital image.

An indeterminate marker area means an element or structure detectable in a primary digital image of a tissue cut. An indeterminate marker area may indicate the presence of opaque structures and elements of natural origin, such as blood vessels and cellular organelles, in the cutting of tissue or endogenous pigment in tissue elements. It can also indicate the presence of detectable marker media, such as a detectable polymer or a fluorochrome, which in turn indicates the presence of a cellular marker that has been detected by a molecular detection means that generates the production of said polymer or a radiation emission detectable after excitation.

An image marker area means an area in a secondary digital image of a tissue cut. An image marker area corresponds to all or a portion of an undetermined marker area in a primary digital image. An image marker area is obtained by evaluating a primary digital image, and in particular undetermined marker areas of a primary digital image, and defining marker areas of image according to the specified selection criteria. A secondary digital image that contains image marker areas also contains or is connected / associated with information about each image marker area.

The shape of a marker area means the shape of the perimeter of the area. There are several known methods for determining the shape of areas in a digital image, included in software such as ImageJ provided by the National Institute of Health (NIH) and Photoshop® supplied by Adobe®.

The position of an area in an image means what position the area in the image has. By the same position for an area in different images is understood

- the same position in relation to a coordinate system also built for images; or - a corresponding position in terms of the reason represented, if the two images represent the same reason.

Selection criteria means selection criteria that can be used to evaluate a primary digital image to define image marker areas. The criteria may, for example, comprise color and / or intensity thresholds for a pixel or a group of adjacent pixels in an image. The selection criteria could include form criteria, color criteria, size criteria or other types of criteria that will be set forth in the detailed description and that will be appreciated by the person skilled in the art. In addition, it is natural for the skilled person who performs the present method to optimize parameters and selection criteria for particular circumstances.

A single characteristic is understood as a characteristic of one or more image marker areas identified as a particular type differentiated from other image marker areas. The unique feature may comprise a visual feature such as a color, a symbol, a shape, a mark or be a digital association between the image marker areas and their particular type (for example, a primary cell type). The association is stored, for example, in a database. The unique feature can be any other characteristic suitable for distinguishing a marker area of a certain type in a digital image from marker areas of other types.

This characteristic can also be further subdivided into unique values. For example, molecular detection means for detecting similar but different cell markers could be identified by a unique feature (such as a color) and each individual cell marker could be identified by a unique value (such as a hue of that color).

Brief description of the drawings

The present invention will now be described with reference to the attached figures, in which:

Figure 1 illustrates an apparatus that executes the method according to the present invention.

Figures 2a-b illustrate a method for differentiating marker areas in a series of images according to the present invention.

Figure 3 illustrates an evaluation of a primary digital image.

Figure 4 illustrates a series of images of a tissue cut.

Figures 5a-b illustrate images created by the method according to the present invention.

Figures 6a-c illustrate different unique visual characteristics of areas of image markers in images created by the method according to the present invention.

Figure 7 illustrates a graphical interface according to an embodiment of the present invention.

Figure 8 illustrates a detection process according to the present invention.

Detailed description of the present invention

The present invention relates to a technique called multiple staining based on exclusion and subtraction, abbreviated ESMS. The technique makes it possible to provide a high resolution image of a histological section of tissue in which a plurality of cell types and tissue structures could be identified. Spatial analysis methods for the plurality of cells and cell types can be applied to provide useful spatial information for a tissue cut that cannot be provided with known techniques. The present invention It is based on the understanding that such information can be of great value when analyzing tissue cuts. The inventor of the present invention has devised a technology to provide images of tissue cutting comprising spatial information in a simple and efficient manner. The technology also makes it possible to provide high-resolution image data in three dimensions, which facilitates studies of complex interactions of cells and tissue structures within the same three-dimensional space in a tissue sample.

The method provides composite images of tissue sections that, in addition to visualizing the distribution of multiple markers within the same cut, provide the basis for extracting information that can be used for advanced mathematical analysis of the patterns of structure and element distribution in the tissue cutting Examples of such information are the relationship between elements, such as cells, or information on structures in the sections revealed by conventional blue-background staining, such as histological structures or focal areas of tissue damage, hypotrophy, tissue remodeling, etc.

The present invention could be carried out from any tissue sample that can be used to detect molecules or structures in histological studies. The examples of molecular detection in the present application refer to immunohistological methods, but other means could also be used to stain molecules; for example, in situ hybridization, binding techniques to non-antibody dependent ligands or enzyme histochemistry. Normally, before molecular detection, a tissue sample is immersed in fixative (for example, 4% buffered formaldehyde, pH 7.6) overnight followed by dehydration in a series of solutions with increasing concentration of alcohol (EtOH) and final immersion in xylene. After the dehydration step, the dehydrated sample is embedded in paraffin and the paraffin cuts are generated with a routine paraffin cut microtome. The paraffin sections are mounted on standard microscope glass slides and stored at 4 ° C until use. The person skilled in the art is well aware of different fixators and fixation processes suitable for different types of tissue cuts and, therefore, can use other methods similar to those mentioned above, including cryogenic cutting techniques.

Prior to the immunohistochemistry itself (abbreviated IHC), paraffin sections are deparaffinized and normally undergo recovery of heat-induced or enzymatic antigen. Such processes are also well known to experts.

The tissue cut obtained after dewaxing and recovery of the heat-induced antigen is then subjected to an additional specific detection. It is essential to be able to determine the types of cells and types of tissues included in said tissue section. In order to do that, the appearance of some specific cell markers in the tissue cut is checked.

Table 1 below lists some examples of immune cell markers that are suitable for use in the present invention, and the cells that express them:

T l 1: M r r l l r r i n IH l l n r hi l i

When the presence of a cell marker in a tissue section according to the present invention is determined, the tissue section is exposed to a molecular detection medium that specifically binds to said cell marker.

The first part of a molecular detection medium could be a natural ligand associated with the cell marker to which it specifically binds. Alternatively, and preferably, the first part is an antibody, and often a monoclonal antibody or a fragment thereof.

The second part of a molecular detection means normally comprises an enzyme capable of generating a detectable polymer in the presence of a suitable substrate. Non-limiting examples of such enzymes are alkaline phosphatase and peroxidases, such as horseradish peroxidase. Peroxidases, for example, generate a detectable brown polymer in the presence of the 3,3'-diaminobenzidine peroxidase substrate (abbreviated DAB), and alkaline phosphatases generate detectable polymers in the presence of Ferangi Blue and Vulcan Fast Red. Another non-limiting example of a substrate that generates an at least partially soluble polymer is Vina Green. The resulting polymer of Vina Green is soluble in water under certain conditions. Another non-limiting example of a substrate that generates at least partially soluble polymer is aminoethyl carbazole (a Ec). The resulting aminoethylcarbazole polymer is soluble in lower alcohols, such as ethanol, under certain conditions. Other substrates can generate polymers that are soluble in other fluids.

The skilled person can find other suitable enzymes and substrates, or he will also know that, instead of enzymes, fluorochromes can be used to visualize marker molecules by immunofluorescence microscopy.

Molecular detection means could be provided as a single conjugate, which typically comprises an antibody that binds to a cell marker and an enzyme such as an alkaline peroxidase or phosphatase, where the antibody and the enzyme are linked by a chemical bonding group. Alternatively, and more preferably, the molecular detection means are provided as a molecular aggregate comprising a first monoclonal or polyclonal antibody or a fragment thereof and a second antibody or fragment thereof chemically linked to an enzyme such as alkaline phosphatase or a peroxidase The second molecular antibody or fragment thereof specifically binds to the first primary antibody or fragment thereof, thus forming the molecular aggregate.

The method of the invention provides a way to differentiate elements and structures in a tissue cut and, consequently, a series of such molecular detection means is used. Depending on the type of organ or tissue from which the tissue cut was taken, as well as the basic knowledge of cell markers of different types of cells and tissues, the expert can design a suitable series of molecular detection means that will be used in differentiating types of cells and tissues in tissue cutting.

It is understood that different types of molecular detection means can be combined within a single embodiment of the method of the invention. This can be advantageous since the elements and structures can be more easily differentiated from each other and their surroundings during the image analysis of the primary digital images.

According to the present invention, the following detection process for tissue cutting is executed one by one with a molecular detection means of said series of molecular detection means:

1) contacting said tissue section of step a) with said particular molecular detection means resulting in a specific binding to a particular element of said predetermined series of cell markers;

2) washing said tissue cut to remove molecular detection means that have not bound to any cell marker;

3) add a suitable substrate that results in the generation of a detectable polymer;

4) wash the tissue cut to remove the rest of the substrate; Y

5) scan / obtain images of the tissue cut to generate a primary digital image that may contain one or more undetermined marker areas of the image associated with the generation of a detectable polymer. The detection process is illustrated as a method in Figure 8, where the method comprises the steps of:

- provide (811) a tissue cut;

- provide (812) molecular detection means;

- contacting (813) the tissue cut with the molecular detection means, as described in step 1) above;

- wash (814) the tissue cut, as described in step 2) above;

- add (815) a suitable substrate, as described in step 3) above;

- wash (816) the tissue cut, as described in step 4) above; Y

- obtain images (817) of the tissue cut, as described in step 5) above.

By repeating steps 812-817, as indicated by 821, for different molecular detection media and substrates suitable for different types of primary cells, a series of images are provided comprising the images generated in step 817. An example of such a series of images is illustrated in Figure 4, which will be described in more detail below.

The scanning / viewing step included as step 5 in the detection process could be carried out with any commercially available slide scanning equipment designed for tissue cutting, a microscope equipped with a digital camera or a complete slide scanner robot.

Repeated steps 812-817 can be carried out in an automated manner. As a non-limiting example, the so-called sliding camera technique can be used for repeated steps. In the sliding chamber technique, tissue cutting is arranged in a microcompartment through which molecular detection means, washing liquids, etc. can pass. The different steps 812-817 can thus be carried out without the need to move the tissue sample to and from the imaging means. Therefore, primary images can be provided with the same image characteristics, such as revealing exactly the same part of the tissue sample and having the same depth of focus. In addition, the method can be carried out more efficiently over time and without the need for manual manipulation or interaction. The person skilled in the art realizes that the method can also be carried out by means of other automated techniques.

When the detection process has been carried out for all the molecular detection means of the series, a series of images is obtained where there is a greater amount of color spots. The first images corresponding to the treatment with a small amount of molecular detection means may comprise only a few spots. The last image, on the other hand, must comprise a multitude of spots and it is also possible that some spots have fused and expanded in large colored areas.

In one embodiment, the series of images can be provided by imaging a tissue cut according to the following detection process:

1) contacting the tissue cut with a particular molecular detection means resulting in a specific binding to a particular element of said predetermined series of cell markers;

2) wash the tissue cut to remove molecular detection media that have not bound to any cell surface marker;

3) add suitable detection reagents, such as a secondary antibody that recognizes the primary detection antibody. Normally, the secondary antibody is labeled with an enzyme, for example, peroxidase. 4) wash the tissue cut to remove molecular detection media that have not bound to any cell surface marker;

5) add an enzyme substrate that results in the generation of a detectable polymer;

6) wash the tissue cut to remove the remaining substrate; Y

7) scan / obtain tissue cut images to generate a primary digital image that may contain one or more undetermined marker areas associated with the generation of a detectable polymer.

As indicated in Table 1 above, some cell surface markers are associated with a restricted range of cells, while other markers appear in a wider range. In Table 1, it can be seen, for example, that CD20 is primarily associated with B lymphocytes. In contrast, CD123 is associated with a substantially larger range of cell types. When configuring said series of molecular detection means, at the beginning of the series of molecular detection means it is advantageous to include specific molecular detection means for only a small number of cell types, and preferably for only one cell type. Then, the confusion of cell types and structures associated and detected with the first molecular detection means can be ruled out by determining cell types and structures using more general molecular detection means later and the accuracy of cell identification increases. . Based on this information, experts find it easy to devise adequate series of molecular detection markers.

It turns out that the following examples of series of molecular detection media that bind to cell surface markers provide good results:

A: D i n im l n 1 l i n l i n i nifi i n l l r m r

continuation

B: Simultaneous identification of multiple tissue histological compartments.

C: Analysis of leukocyte infiltration patterns in relation to the areas of tissue damage / repair and the principles

E: I nifi i n m r l i n l l n ríi D i l r

The central part of the present invention relates to how the series of images obtained are analyzed and transformed into new edited images and three-dimensional representations comprising additional information to visualize multiple cell populations and tissue structures and their spatial relationship within the same space. bi or three dimensional.

The image analysis portion of the present invention will now be described in more detail below with reference to the accompanying drawings, in which certain embodiments of the invention are shown. This invention may, however, be materialized in many different ways, and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided by way of example so that this description is thorough and complete, and fully conveys the scope of the invention to those skilled in the art. Similar numbers refer to similar elements throughout the document.

Figure 1 is a block diagram illustrating a device, generally given by 1, to differentiate areas in a series of images according to the present invention. The device comprises an apparatus 10 comprising a processor 11 and a memory 12. The apparatus 10 could be part of a computer. The processor 11 may be programmed to record the shape and position of the areas in an image. The shape and position can be stored associated with or in connection with the image, for example in a database in memory 12. The processor 11 may also be programmed to evaluate an image in order to identify image marker areas according to default selection criteria. In addition, the processor 11 may be programmed to compare two images and identify image marker areas present in one of the images but not in the other. The processor 11 may also be programmed to insert new image marker areas that have the same shape and position as the image marker areas identified in another image, where the inserted markers are identifiable in the other image by a unique feature.

In one embodiment, an imaging unit 13 is connected to the apparatus 10. The imaging unit 13 is, for example, a CCD digital camera or a digital scanner such as a slide scanner. Alternatively, instead of having an imaging unit 13 connected to the apparatus 10, images can be provided to the apparatus 10 by connecting a storage medium, such as a USB memory, containing the images. The images provided can be stored in memory 12.

An output unit 14 can be connected to the apparatus 10 to provide output from the apparatus 10 to a user. The output unit 14 is, for example, a screen such as a computer screen or a mobile phone screen. The output preferably has the form of a software interface, that is, a graphical user interface for displaying an image. The apparatus 10 preferably further comprises an input unit 15 for receiving user input. Typical examples of an input unit 15 are a keyboard or data connection means.

Figure 2a generally illustrates a method according to the present invention for differentiating areas of markers in a series of images that can be executed in the apparatus 10. The method comprises the following steps:

- A first step 211 of evaluating a series of primary digital images to define areas of image markers according to predetermined selection criteria.

- A second step 212 to create a new image, based on the series images, by inserting new markers corresponding to the image marker areas defined in the previous step 211, so that the markers are identifiable.

The method will now be described in detail with reference to Figures 2b and 1.

Step 221 comprises providing a series of N primary digital images h, I ² , ..., IN where N is an integer equal to or greater than 2. The images are provided by the imaging unit 13 or by a unit storage (not shown) connected to the device 10.

The series of N primary digital images is provided by obtaining images of a tissue cut according to the following detection process (also described above and as illustrated in Figure 8):

1) contacting the tissue cut with a particular molecular detection means resulting in a specific binding to a particular element of said predetermined series of cell markers;

2) wash the tissue cut to remove molecular detection media that have not bound to any cell marker;

3) add a suitable substrate that results in the generation of a detectable polymer;

4) wash the tissue cut to remove the rest of the substrate; Y

5) scan / obtain tissue cut images to generate a primary digital image that may contain one or more undetermined marker areas associated with the generation of a detectable polymer.

The detection process is repeated a desired number of times. Step 5 (corresponding to step 817 in Figure 8) generates a primary digital image in each process cycle. Therefore, N cycles generate a series of N images.

The process generates images where the In ⁺¹ image comprises at least the same number of undefined image marker areas as the In image for 2 <n <N, where n is an integer. Therefore, the series of images comprises an increasing amount of indeterminate marker areas where the image I ⁿ comprises the largest amount of indeterminate marker areas and Ii comprises the least amount of indeterminate marker areas.

Next, step 222 comprises evaluating each In image for 1 <n <N according to the predetermined selection criteria, and defining image marker areas. In particular, undetermined marker areas are evaluated. Areas of the image that meet the default selection criteria are defined as image marker areas. As already mentioned in the definitions section above, there could normally be size criteria, shape criteria and color criteria. The selection criteria greatly influence the outcome of the evaluation process. For example, a comparatively high threshold level in relation to size may lead to clearer images that are easy to evaluate, but there is always a risk that relevant structures that are smaller in size are not detected. In deciding the selection criteria, therefore, it is preferred to consider data relating to cells and cell structures normally present in a cut of the type of tissue that is about to be studied. The expert has this knowledge.

An area consists of one or more pixels. In addition, an area can be defined as a plurality of adjacent pixels in the image. How an area is defined could be part of the default selection criteria. The criteria could, for example, include a criterion that only areas of more than twenty pixels should be defined as image marker areas. Another criterion could be that the pixels that make up the area should resemble a particular shape. Step 222 corresponds to step 211 in Figure 2a.

Step 223 comprises providing a new Inueva secondary digital image. The new image can represent the same motif as the series of images. In particular, the new image may be a copy of one of the primary digital images in the series of images. In such an embodiment, Inueva can be created by copying one of the images into the series of images.

Advantageously, the new image represents the tissue sample as similar as possible to the original tissue sample, that is, before any detection process cycle. This can be achieved by obtaining images of the tissue cut, thus creating an I0 image, before the first detection process cycle is performed. In such an embodiment, the image I0 can be provided in a step before step 221 of providing the series of images. Alternatively, the new image can be a blank image, that is, without any content. The processor 11 can create a blank image. A blank image means, for example, an image where all pixel values, for example, the RGB value, are set to zero.

In another alternative embodiment, the new image is provided by capturing an image after an initial staining of the tissue sample with a standard counter staining such as hematoxylin, or any other staining that provides valuable information on the background of the tissue and does not interfere with subsequent immunohistochemistry and detection steps.

It should be borne in mind that the new image can be provided from, for example, an imaging unit or from a memory unit and that the method is not limited to any of these alternatives. The new image marker areas can be inserted in New in any order.

It should be noted that step 223 of providing the new Inueva image can be executed before step 222 of evaluating each image or before step 221 of providing the series of images, that is, the step of providing the new image does not depend on its previous steps. .

Step 224 comprises inserting new image marker areas, in the new Inueva image. For each In image for 2 <n <N, the new image marker areas are inserted into New with the same shape and position as the image marker areas present in the In image but not in the In ^-1 image. This is achieved by comparing each In image with its subsequent image In ^-1 and identifying the areas of image markers present in In but not In ^-1 . The evaluation does not need to be done for n in a certain order and, in fact, can be performed in the order deemed appropriate. However, the important thing is that the order of the images within the series is maintained and that only the image marker areas present in the In image but not in the In-1 image are identified.

The way to compare and insert areas of image markers with the same shape and position in the new image can be done in many ways, all well known to those skilled in the art. The examples are the storage of shape and / or position parameters in a database and; using a copy and paste function in image editor software; etc. Step 224 will be explained in more detail in relation to Figures 4 5.

The image marker areas inserted in Inueva are also identified in Inueva by a unique characteristic, in particular the unique value of a unique characteristic. The identifiable characteristic is a characteristic that was not originally present in Inueva and could be, for example, a color. In this case, a single value of a color could be a particular and unique nuance. What is important is that the value is unique for the particular molecular detection means used in the detection process cycle that generates the In image and, consequently, the corresponding element or structure. One purpose of the unique characteristic / value is that they differentiate different image markers that originate in different cycles of detection processes and, therefore, different molecular detection means for different elements / primary cells and tissue cutting structures. In one embodiment, the unique feature is visual markers in the new image. In one embodiment, the unique characteristic is a general color and the unique value of the unique characteristic is a specific color associated with a particular cell marker.

In another embodiment, the unique feature for image marker areas originating in In is a digital association / connection between image marker areas and the corresponding elements and / or structures that are the target to be marked in the cycle n of the detection process. The association / connection is stored in association / connection with the image In, such as in an associated database in memory 12.

Step 211 of evaluating a series of images to define image marker areas according to predetermined selection criteria will now be described with reference to Figure 3.

Figure 3 illustrates a primary digital image representing a tissue sample. The image originates from a cycle of the detection process described above and as illustrated in Figure 8. has different elements and structures corresponding to elements and structures of the tissue cut. For pedagogical reasons, the elements and structures of <t0 /> the tissue sample are represented as simplified geometric shapes in Figure 3. In reality, the image usually comprises thousands of elements and structures.

In this example, the detection process has been chosen in such a way that the generation of detectable polymer leads to a color change in the area where the polymer is generated. The color change is such that the areas become darker. When affected by the detection process (step 815 in Figure 8), the areas where a detectable polymer generation has occurred are called undetermined marker areas. In Figure 3, indeterminate marker areas are indicated by 31a, 32a, 33a and 34a.

Since the areas are generated by a detectable polymer, they can be identified by detecting the polymer.

In the present example, a larger generation of polymer detectable in one area produces a darker area. In an ideal detection process, the generation of detectable polymer is only present in relation to the elements that are directed towards the choice of particular molecular detection means. However, the frequently detectable polymer will also be generated in other areas also due to cross-reactivity and non-specific binding of the detection antibody (or components used for molecular detection). The image may comprise additional indeterminate marker areas, which have not been affected by the detection process. It is desirable to separate the areas that most likely originate from "true" elements and structures, that is, the elements and structures that are intended to be marked in a particular detection process cycle. Therefore, each image in the series of images is evaluated, as illustrated in step 310 in Figure 3, according to the predetermined selection criteria. The selection criteria are selected to adapt to the particular detection process.

The selection criteria may comprise a plurality of subcriteria, such as:

- Color threshold

- Color range

- Geometric properties (shape and size parameters)

- Nature of staining patterns within a marked area (for example, texture parameters; granularity, thickening, smooth uniform staining, dotted staining, etc.).

- Position (for example, in relation to tissue structures that, for example, can be identified in an unstained cut or after staining of background tissue).

The person skilled in the art knows that evaluating an image using these criteria or similar ones can be done using software such as ImageJ, provided by the National Institute of Health (NIH), USA. UU.; Image-Pro Plus from Media Cybernetica Inc, USA UU .; Visiomorph of Visiopharm A / S Denmark; Definiens Tissue Studio of Definiens AG, Germany; Genie from Aperio Technologies, USA UU .; MATLAB of Mathworks Inc, USA UU .; Adobe Photoshop, etc. A color threshold means a threshold in a color scale, such as an HLS (tonoluminosity-saturation) color scale, where a pixel with a color value above or below the threshold of the particular color scale meets the selection subcriterium.

Color range means a range within a color scale, such as an R value for an RGB color scale image within a particular range, such as 200-230. Pixel areas that have pixel values within the range comply with the selection subcriteria.

The color threshold and color range criteria are also applicable with other color scales, such as an HSB color scale or HIS color scales. Many other color scales also exist, as the skilled person knows.

Geometric properties means the parameters associated with the shape and / or size of the area. Examples are roundness, circularity, length, irregularity parameters, etc. Form value criteria can be used to define true elements / structures from false elements / structures by their form. For example, the nerves have an elongated shape, so the indeterminate marker areas in an image that result from a sensing process for the nerves, which have different shapes than the elongated ones, can be excluded from being defined as marker areas of Image for the particular image.

As understood by one skilled in the art, other suitable types of selection criteria may also be used in the present invention. By appropriate it is understood that the selection criteria are adapted to classify the indeterminate marker areas that are most likely to originate from the elements and structures that are addressed in the corresponding detection process cycle in particular.

In one embodiment, the selection criteria comprise a threshold for a visual property, such as a color, a texture, a size or a roundness. Visual property means some type of characteristic aspect of a marker area. Note that it is not necessary to display the property in, for example, an output unit to be a visual property.

The selection criteria may comprise one or more of the types of criteria mentioned above. A combination of different types of criteria may also be included, such as a combination of a color threshold criterion and a value criterion in which both criteria must be met in an undetermined marker area in order to define it as a marker area. of image.

In this example, the selection criteria comprise a visual property, more particularly, a color threshold. Only areas with sufficient dark color are defined as image marker areas and, therefore, are said to correlate with "true" elements and structures. Since this example only includes grayscale colors, the color threshold can be set on all pixels with a higher intensity of a certain value on the grayscale. On a gray scale of 0-1, where 0 corresponds to black and 1 corresponds to white, a threshold value of 0.75 can be set. In other embodiments comprising images of another color scale, for example in an RGB color scale, a threshold can be set in the corresponding manner as understood by those skilled in the art.

Different selection criteria produce different evaluation results and, therefore, different secondary images. In the present example of Figure 3, the selection criteria comprise that the pixels of an indeterminate marker area must be above a grayscale color threshold. Through evaluation, a secondary digital image hb with an image marker area 31b is obtained. The image marker area 31b is defined by evaluating the undetermined marker area 31a that meets the selection criteria. Other areas 32a, 33a, 34a of undetermined image marker do not meet the selection criteria and, therefore, are not defined as image marker areas. In a further step 311, the evaluation information, comprising, for example, the geometric spatial parameters (coordinates) or the shape index of the evaluated marker areas, is stored in or in connection / association with the secondary image l1b , such as in an associated database. Said database can be updated later with information on the areas of indeterminate markers that are defined as areas of image markers. The information can be used to insert new image marker areas into the new image in step 224. The image hb can be exactly the same image as it has or can be copied and / or edited digitally, for example when inserting a visual mark in image marker areas. However, the secondary digital image is not limited to being the same image as the primary digital image.

The difference between a secondary digital image and a primary digital image is that the secondary digital image has been evaluated to define image marker areas, while a primary digital image has unprocessed, unedited and unassessed data. Any digital image edited or created in the method is known as a secondary image, since it is not an unprocessed image obtained directly from digital imaging or scanning. Therefore, also the new images created in, for example, steps 212 and 224 of the present invention are called secondary digital images.

In one embodiment, a user can evaluate a primary digital image by using image software and define areas of image markers according to a predetermined selection criterion, such as a certain color intensity, surroundings of the position and / or shape Examples of useful imaging software for this type of evaluation are Image-Pro Plus, from Media Cybernetica Inc, USA. UU .; Visiomorph, from Visiopharm A / S Denmark; Definiens Tissue Studio, from Definiens AG, Germany; and Matlab, of Mathworks Inc, USA.

Step 212 of creating a new image by inserting new identifiable and categorized markers corresponding to the defined image marker areas will now be described with reference to Figure 4. Figure 4 describes a series of images of an embodiment of the present invention. The images h, I ² , I ³ , I ⁴ form a series of N images, where N = 4. The object represented in each image is a tissue cut (or a part of a tissue cut). Each image comprises (the same part of) the same tissue cut. Each of the images has been evaluated according to step 211 and step 222.

Each image comprises image marker areas: I ¹ comprises an image marker area 411a; I ² comprises image marker areas 411b and 421a; I ³ comprises areas 411c, 421b, 431a and 432a of image marker; and I ⁴ comprises areas 411d, 421c, 431b, 432b, 441a and 442a of image marker. Information related to image marker areas, such as position parameters, shape parameters, color values, intensity values, etc., are preferably stored in or in connection / association with the image itself. , such as in a database associated with the image. Said database can be arranged in memory 12. The parameters are stored, for example, in accordance with step 311 of Figure 3 (described above).

As described above, the images in the series of images comprise at least the same or usually an increasing amount of image marker areas. I ⁴ comprises the largest amount of image marker areas and I ¹ comprises the least amount of image marker areas. In comprises at least the same amount of image marker areas as In ^-1 for n = 2, 3 or 4.

When comparing I ⁴ , I ³ and / or related information in the associated database, it is found that areas 441a and 442a of image markers are present in I ⁴ but not in I ³ . Therefore, the new image marker areas that have the same shape and position as the image marker areas 441a and 442a are inserted in New according to step 224. The new image marker areas are further identified by a feature. unique, in particular a unique value of a unique characteristic.

In figures 5a and 5b, examples of new images are given. These can be obtained in the present example. The new Inueva image in Figure 5a is provided by copying one of the images I ¹ -I ⁴ , such as I ¹ . In Figure 5b, the new Inueva image is provided by creating an empty image, that is, without any information.

As described above, new image marker areas are inserted that have the same shape and position as image marker areas 441a and 442a. In Figures 5a-5b, these are called 541 and 542. In Figures 5a-5b, the unique features are visual markers and comprise unique patterns for the areas of inserted image markers of a specific group. Image marker areas 541 and 542 belong to a group of marked areas in the detection process cycle between obtaining image I ³ and obtaining image I ⁴ . For areas 541 and 542 of image marker, the visual marker of the unique feature is a dot pattern.

To insert the image marker areas 541 and 542 with the same shape and position as the image marker areas 441a and 442a, a database may comprise information that refers to the image marker areas 441a and 442a. As described above, said information can be stored in a database in the additional step 311 in Figure 3. In addition, the method according to Figure 2b may comprise an additional step to record the shape and position of each marker area of image identified according to the default selection criteria. By knowing the shape and position of the image marker areas 441a and 442b, the new image marker areas 541 and 542 can be inserted. As one skilled in the art can understand, new image marker areas can be inserted into the new image by other methods commonly known in the art.

When comparing I ³ with I ² , the image marker areas 431a and 432a are identified as present in I ³ but not in I ² . The new image marker areas 531 and 532 with the same shape and position as the image marker areas 431a and 432a are inserted into New. In Figures 5a-5b, the visual marker of the unique feature of 531 and 532 is a fill pattern.

When comparing I ² with Ii, the area 421a of the image marker is identified as present in I ² but not in Ii. The new image marker area 521 with the same shape and position as the image marker area 521a is inserted in New. In Figures 5a and 5b, the visual marker of the unique characteristic of 521 is a pattern of lines.

Since I ¹ is the result of the first detection process cycle, I ¹ does not need to be compared with any other image.

In this example, the image marker area 411a is identified as present in I ¹ and, therefore, originates in the detection process cycle that generates I ¹ . A new marker area 511 is inserted into Inueva with the same shape and position as 411a. In addition, the image marker area 511 can be identified according to a unique feature. In Figures 5a and 5b, the visual marker of the unique feature of area 511 of the image marker is a checkered pattern.

Therefore, a new Inueva image has been created comprising areas of image markers categorized according to the detection cycle and, consequently, which cell marker and primary cell each image marker area represents. By the method of the present invention, multiple cells or tissue structures can be identified and categorized using an iteration of the same type of detection process. The markers used, such as detectable polymers or fluorochromes, to mark tissue elements or structures need not be unique in themselves. You can use a single type of detectable polymer or a fluorochrome of a specific wavelength in each cycle of the detection process, and generate areas of markers that have the same intensity, color or wavelength of light emitting. This advantage is illustrated by image I ⁴ in figure 4 and with Inueva in, for example, figure 5a. With the known techniques, the result of multiple detection processes generates the image I ⁴ . Here, the marker areas are impossible to differentiate from each other, that is, it is not possible to see what type of primary cell or tissue structure represents an image area in I ⁴ . However, by means of the novel concept of the present invention in which the obtaining of multiple images is combined with the described image analysis method, an image comprising image marker areas with a categorization can be achieved in a simple and efficient manner. . In this way, many problems can be solved with known techniques, such as the limitation of detecting multiple markers within the same cut.

Figures 6a-6b illustrate examples of unique visual characteristics for image marker areas categorized in the New Image according to the present invention. The images represent a sample of tissue with elements and structures. The images have been provided through the method of the present invention.

In Figure 6a, the unique feature is different patterns.

In Figure 6b, the unique feature is the gray color and the unique values are intensities of the gray color and, in particular, different grayscale intensities. A new image could comprise a plurality of features, such as different colors, and said colors could be further subdivided into unique values, whose values could be different intensities of said different colors.

In Figure 6c, the unique feature is different symbols. In this embodiment, the symbols are characters, but they could also be digits, geometric symbols, etc. or a combination thereof. The symbols are arranged in the digital image. New near the area of the image marker they represent, so that it is clear which image marker they are associated with.

As mentioned earlier, it should be borne in mind that the unique characteristic does not necessarily have to be a visual characteristic arranged in the new image. Instead of, or in combination with, a visual marker can be digital information, such as a single digital value, where the image marker areas of a group are associated with each other. Said information may be stored in a database, and preferably in the database mentioned above, which includes information regarding the position, shape and other staining characteristics of the image marker areas.

Figure 7 illustrates an embodiment of the present invention in which a graphical interface, generally given by 7, is used to provide a user with information related to the method of the present invention. A computer program that provides the graphical interface 7 can be stored in memory 12 and executed by processor 11. The computer program can alternatively be stored in any suitable storage unit, such as a USB memory or a CD-ROM. The computer program can also execute the method of differentiating marker areas in a series of images according to the present invention. The graphic interface 7 comprises a graphic window 71 that is provided to a user through an output unit 14, such as a computer screen, in connection with the apparatus 10. The user can provide the computer program with an input, such as a selection, by means of an input unit 15, such as a computer mouse or keyboard, in connection with the apparatus 10.

The graphic window 71 comprises an image 72 corresponding to a new Inueva image provided by the method according to the present invention. In this embodiment, image 72 represents the same tissue section as Figures 4-5.

The graphic window 71 further comprises at least one frame 75 to adapt how to view the image 72. The frame 75 may comprise a selection frame 76 to provide a choice between different alternatives. In this embodiment, table 75 offers a choice on which group of primary cell populations to stand out in image 72. By choosing the "B lymphocyte" alternative in selection table 76, image marker areas 751 and 752 are highlighted. associated with the primary cells of B lymphocytes. The association between an image marker area and a primary cell can be stored in a database, where information related to the association can be obtained through the database computer program. The database can be obtained by the method according to the present invention as described above, in particular by step 311. The alternatives provided by Table 75 are not limited to comprising only one type of cell types, but could also comprise a group of a plurality of cell types.

The graphic window 71 may comprise other multiple selection boxes or other suitable adaptation means, such as check boxes, multiple selection windows, etc.

The graphic window 71 further comprises a table 77 associated with a table 79 of information. The information provided in the information box is associated with the image marker areas of the image 72. The information can be obtained through a database comprising information relating to the image marker areas of the image 72. Table 77 Selection includes a selection table 78. The user can choose between different types of information to display said specific information about areas of image markers in the information box 79. In this illustrated example, the user has chosen to show the coordinates by selecting the alternative "X, Y coordinates" in the selection box 78. Information regarding the x and y coordinates of areas 751 and 752 of highlighted image markers, chosen in selection box 76, is provided in information table 79.

Of course, the graphical interface 7 can take many different forms and comprise many different functions. Therefore, the type of information relating to the areas of image marker that can be provided to the user by the graphical interface 7 is not limited by this example. By the claimed method, a user can be provided with information such as:

- the area, expressed in, for example, 4 | im2, for any marker area.

- shape values for an image marker area, for example perimeter and roundness

- intensity values, for example average pixel intensity in an image marker area

- distances within the image, for example distances between two image marker areas

- spatial correlations between image marker areas, or groups of marker areas; for example, by using statistical algorithms in the software, a user can obtain information about the spatial relationship between, for example, marker areas corresponding to B lymphocytes and those marker areas in the same tissue area that correspond to the population cell of, for example, T lymphocytes.

It should be noted that when requesting the present invention to serially cut tissue cuts (for example, 30 consecutive cuts, each with a fixed thickness of, for example, 4 | im2), a similar graphical interface such as that Shown in Figure 7 can also show the three-dimensional distribution of image marker areas in a 3D image. Similarly, a user can obtain 3D information about image marker areas, such as their x, y and z coordinates. By using commonly known algorithms for 3D rendering, a user can obtain calculations (or graphical representations) on the volume of a marked cell or a tissue structure.

The computer program and the graphic interface 7 should be considered as means to provide the user with any information that can be extracted from the images of the method of the present invention. The way to extract and store such information, for example, in a database is of general knowledge to a person skilled in the art. How to develop a computer program to meet the characteristics described in relation to the above can also be carried out by a person with general knowledge in the art. It should be noted that the computer program is not limited to the features described above and, for example, could include other well-known features, such as editing and reviewing digital images. The graphical interface 7 could also comprise features to perform the evaluation of primary digital images.

In summary, the present application describes a method for differentiating areas in a series of digital images, the method comprising the steps of: providing a series of images comprising marker areas undetermined; evaluate each In image for 1 <n <N according to the predetermined selection criteria and define image marker areas as indeterminate marker areas that meet the predetermined selection criteria; provide a new new image; and inserting new image marker areas in the new Inueva image, said new image marker areas having the same shape and position as the image marker areas present in the In image but not in the In ^-1 image, and being these new image marker areas identifiable in Inueva for a unique feature.

In addition, the application describes a method for visualizing cell populations in tissue section of a histological sample.

In addition, the application describes a method to visualize the three-dimensional distribution of multiple cell populations in a histological sample.

Examples

The present invention will now be described further with reference to the attached examples.

Example 1: Tissue handling, sample preparation and cut generation.

The following human tissues were included:

- Human distal colon: surgical removal due to chronic inflammation and suspicion of nonspecific colitis.

- Human lung tissue of patients with chronic obstructive pulmonary disease, COPD and cystic fibrosis: pulmonary excision due to suspected lung cancer, the analyzed tissue was not affected by cancer and was obtained as far as possible from the tumor.

- Human lymph nodes: large draining lymph nodes removed in association with a lung transplant due to severe COPD or cystic fibrosis.

- Human tonsils: they are removed as part of a routine tonsillectomy due to repeated episodes of tonsillitis.

Samples (ie, pieces of tissue) of all types of tissue were submerged when subjected to routine fixation by immersion in a routine fixative (4% buffered formaldehyde, pH 7.6). After fixing overnight, the samples were dehydrated in a series of solutions with increasing concentration of alcohol (EtOH) and final immersion in xylene. Dehydration was carried out in an automatic dehydration machine (Shandon Hypercenter XP Tissue Processor, Shandon / ThermoFisher Sceintific, Waltham, MA, USA). Dehydrated samples were subsequently embedded in paraffin at 60 ° C using a paraffin embedded machine. The paraffin sections (4 mm) were generated with a routine paraffin cutting microtome (Microm HM360 paraffin microtome, Microm, Germany) and mounted on standard microscope glass slides. The cuts were then stored at 4 ° C until use.

Example 2: Multiple immunohistochemical staining and generation of digital images in series.

Immunohistochemical staining was performed with an automated immunohistochemical robot (Autostainer CL-classic; Dako Cytomation, Glostrup, Denmark) with the DAKO REAL EnVision detection system, a sensitive standard method for the detection of primary mouse or rabbit antibodies (kit IHC Code K5007, Dako Cytomation, Denmark; for more details, see www.dako.com). The primary antibodies used to detect cell-specific antigens (formerly called "cell markers") are listed in Table 2 and applied in the cuts at the dilution recommended by commercial producers for immunohistochemical staining of human tissues prepared for Routine pathological examination (i.e., sample cuts fixed with formalin and embedded in paraffin). Table 3 lists examples of series of markers that were used in the ESMS evaluation.

T l 2. E m l ni r iliz r l v li i n x rim n l l ni E M

continuation

Before the passage of the immunohistochemistry itself, the paraffin sections were dewaxed and subjected to heat-induced antigen recovery. This procedure was carried out using a commercial and programmable <t0 /> antigen recovery machine (PT Link from Dako Cytomation, Denmark) with a maximum temperature of 95 ° C and the Envision FLEX objective recovery solution, pH 6.1 ( Dako Cytomation).

After antigen recovery, the slides were placed in the Autostainer robot. The programmed immunohistochemical protocol was as follows:

1) Rinse step with Envision FLEX wash regulator (pH 7.6) for 5 min.

2) Block endogenous peroxidase in 0.3% H ² O ² in dH ² O (10 min).

3) Incubation with properly diluted primary antibodies (see table 2) for 60 min. The antibodies were diluted in a PBS regulator supplemented with a 0.1% Tween detergent.

4) Rinse step with Envision FLEX wash regulator (pH 7.6) for 5 min.

5) Incubation for 30 min. with secondary reagent (anti-mouse and anti-rabbit antibodies bound to a dextran polymer with bound detection enzyme, HR peroxidase (HRP).

6) Repeated rinse steps with the Envision FLEX wash regulator (pH 7.6) for 5 min.

7) Incubation with HRP enzyme substrate solution (diaminobenzidine, DAB) for 10 min.

8) Repeated rinse steps with the Envision FLEX wash regulator (pH 7.6) for 5 min.

9) The developed cuts were carefully mounted with standard coverslips using PBS regulator supplemented with 0.1% Tween as a mounting medium.

10) Next, the staining pattern information in each cut was digitized. The brown insoluble precipitation formed by the HRP enzyme at the site of immunoreactivity was captured throughout the cut using a commercial slide scanner robot (Aperio Scanscope CS, Aperio Technology, USA). The scanning was performed using an x20 microscope lens and the ultra-high resolution image size generated for each cut was normally 2 to 5 GB in size (and originally in an SVS image file format; parts of the image Large SVS were also exported as TIFF images using the export functions provided by the ImageScope software supplied by Aperio, see below).

11) Alternatively, or as a complement to the digitization of complete slides, the selected regions of the cuts were also captured with a larger magnification (x400 or x600; TIFF or JPEG images) using a light-field microscope (Nikon 80i Research Microscope, Nikon, Japan) equipped with a digital color camera (Olympus DP-50, Olympus, Japan) and image capture software (Viewfinder Lite, v1.0, 2000, Pixera Co).

12) After scanning, the coverslips were carefully removed and the slides were rinsed in a regulator before entering a new immunohistochemical cycle (beginning in step 2 of the protocol). In some cases, before the cuts entered the next staining cycle, they were immersed in a solution of blocking that makes the primary primary antibodies not recognizable to the secondary detection antibodies in the subsequent staining cycle. The interruption of the antigen recognition site was carried out by chemical modification. Said chemical modification was carried out in two different ways. The antigen recognition site was destroyed by protein denaturation using the denaturing blocking solution DNS001H from BioCARE, Concord, CA, USA. UU. Alternatively, the antibodies could be enzymatically cleaved.

13) Next, the cut entered a new staining cycle that begins with step 2 (block of H ² O ² in the previous protocol).

14) After n numbers of cycles and the development of the immunoreactivity of the final marker, the cuts are rinsed in ddH20, stained countercurrently with hematoxylin (Htx, Merck, Darmstadt, Germany), dehydrated through a series of solutions alcoholic and xylene, and finally mounted with Pertex mounting media. (HistoLab, Gothenburg, Sweden) before being digitized as described above.

T l. E m l ri m r r iliz n l v li i n l ni E M

Example 3: Analysis of computerized images and decoding of specific staining patterns of markers. The digitized sections of the Aperio slide scanner, corresponding to a series of primary images according to the described invention, were manually inspected using a visualization software (Aperio ImageScope, version 10.0.35.1798, Aperio Technologies Inc). In the initial evaluation, the regions of interest in each cut were selected for a more detailed analysis. Using the image extraction and export function in the ImageScope software, unprocessed images, that is, primary digital images, were exported as TIFF or JPEG files; one image for each staining cycle. Together, the images formed a series of images by region of interest.

For some images, the brown DAB precipitation distribution pattern was already delineated before exporting the image using the color segmentation features ("positive pixel" algorithm) included in the ImageScope software after selecting RGB and tone values characteristic of brown DAB precipitation.

In the event that the brown immune staining, that is, the areas of the image marker, had not already been delineated in ImageScope, this was done using easily available software with color recognition functions (for example, ImageJ, version 1.44o , National Institute of Health (NIH), USA or Adobe Photoshop® CS4 Extended, version 11.0.2, Adobe Systems Incorporated, USA). Through the visual feedback of the spots detected, a person with knowledge about the typical staining pattern and the appearance of staining of each cell type adjusted the threshold values until the optimal detection occurred.

Next, the series of images was evaluated, with the accumulated indeterminate image marker areas produced after each molecular detection cycle. Image marker areas were cut digitally and they were given a unique pseudocolor for the corresponding cycle of the detection process. Using the last image, that is, the image N in the series of N images, as a template, the stained color-coded staining points, that is, the colored image marker areas, were copied and pasted into the template in order Inverse (if necessary, this procedure was preceded by an alignment step that, using the contour of the tissue as reference points, corrected minor occasional differences in physical orientation between the images within the same series).

For example, in the case of a series of images with seven images, that is, resulting from seven cycles of the detection process, a copy of the last image (the seventh) was used, with all the staining accumulated as the new template of image. In some embodiments it is advantageous to copy the latter, since it has the best morphology because the slide with the tissue cut was optimally and definitively mounted on a non-aqueous mounting medium before generating the last primary digital image. The image marker areas in the sixth image were copied and pasted into the new image. Therefore, the image marker areas that were already present in the new image were masked, that is, the marker areas that were not generated in the seventh round of detection were masked. Similarly, the image marker areas in the fifth image masked all image marker areas that were not generated in the sixth round of detection process.

The new composite image generated finally showed seven different colors: one for each group of image marker areas that originated from different staining cycles, that is, detection process cycles.

The new image information could be extracted by automatically selecting a marker color (for example, using ImageJ software or MATLAB®) and then using the "particle analysis algorithm", or similar operation, to generate detailed information about each stained stain (perimeter, area, shape index, x coordinates, and for the centroid of the stain, etc.).

This function can also be included in the graphical user interface described (illustrated in Figure 7).

Another treatment for generating composite images by evaluating the series of images was to use the color segmentation tools and particle analysis tools provided by Image J (v 1.44) and the add-ons available for free for Image J. In summary, each image of the series was subjected to the following procedure:

1) stains stained with brown DAB, that is, image marker areas, were segmented by color-based segmentation by evaluating the images with selection criteria comprising HSB threshold values (hue, saturation, brightness) appropriate;

2) the images were transformed into a binary image in black / white (that is, they were made binary)

3) the transformed images from step 2) were evaluated using the Image J particle analysis tool (version 1.4.4) with the appropriate size restrictions and a list of data was produced and stored, that is, a database of all image marker areas, that is, all image marker areas. The data list contained the x, y coordinates, area, perimeter, circularity, roundness, average staining intensity, etc. for all individual areas of image marker. The program automatically produced a marked image where the marker areas are delineated along with the number of the marker area that corresponds to the same spot in the data list.

Then, by comparing the numerical values of the distribution of the marker area (i.e., the x, y coordinates) it was possible to calculate which areas of the image were present, for example, in the image n but not in n-1. This treatment was performed on all pairs of consecutive images, which generated information about which image marker appeared after each new molecular detection cycle. Finally, using this information, together with the list of data of all the accumulated marker areas of the last image, it was possible to create a new composite image. After activating the Region of Interest (ROI) administrator, the marker areas belonging to a specific molecular detection cycle were assigned a specific color in the corresponding marked image.

To illustrate an advantage of the information that can be extracted, consider the scenario of an inflamed tissue and the staining of multiple populations of tissue-infiltrating immune cells (leukocytes). In a conventional routine cut, there are usually tens of thousands of cells in each population. The extraction of the x, y coordinates for individual cells within multiple leukocyte populations makes it possible to perform a new type of powerful cell pattern analysis. For example, the relatively new and emerging field of spatial analysis (spatial statistics) and group analysis can be performed to obtain information about potentially disease-specific cellular constellations (infiltration patterns), which cells attract each other or are attracted by certain micolocation in the tissue, or certain combinations of cells etc.

Claims (13)

1. A method for differentiating areas in a series of N primary digital images of a tissue cut, where N is an integer> 1, thus creating a new image, said method comprising the steps of: a) provide a series of N primary digital images (221) comprising first marker areas, wherein an In ⁺¹ image comprises at least the same first marker areas as a primary In digital image at for 1 <n <N in where n is an integer; b) evaluate (211, 222, 310) each primary In digital image for 1 <n <N according to the predetermined selection criteria and identify as image marker areas those first marker areas that meet the predetermined selection criteria , and storing information (311) about any of those image marker areas in or in connection / association with a corresponding resulting secondary digital image, thereby obtaining a series of N secondary digital images; c) provide (223) a new new image of tissue cutting without marker areas; d) inserting (212, 224) new image marker areas into the new Inueva image, said new image marker areas having the same shape and position as the image marker areas present in image I ¹ , and said images being Identifiable image marker areas in Inueva, for a unique feature; e) for each n for 2 <n <N of the series of secondary digital images obtained in step b), insert new image marker areas in the new New image, said new image marker areas having the same shape and position that the image marker areas present in the In image but not in the In ^-1 image, and said new image marker areas being identifiable in Inueva by a unique feature. 2. A method according to claim 1, characterized in that the new Inueva image is a copy of one of the images in said series of primary images. 3. A method according to claim 1 or 2, characterized in that said unique feature in steps d) and e) is a feature that has a unique value for each n, 1 <n <N. 4. A method according to any one of claims 3, characterized in that said unique feature is a general color and said unique value of said unique feature is a specific color associated with a particular cell marker. 5. A method according to any one of claims 1 to 4, characterized in that the predetermined selection criteria comprise a threshold for a visual property of a first marker area. 6. A method for visualizing cell populations within a histological section of tissue, said method comprising the steps of: a) providing (811) a tissue cut that has been prepared for molecular staining in a previously known manner; b) providing (812) a series of K particular molecular detection means to specifically bind and detect elements of a predetermined series of K cell markers that may be present in the tissue section of step a), said molecular detection means being capable of generating the formation of an initiable and detectable response, where K is an integer> 2; c) for each particular molecular detection means k = 1,2, ..., K of step b) carry out the following procedure: 1) contacting (813) said tissue section of step a) with said particular molecular detection means resulting in specific binding to a particular element of said predetermined series of cell markers; 2) washing (814) said tissue cut to remove molecular detection means that have not bound to any cell marker; 3) initiate the response (815) from the molecular detection means that may have been attached to cellular markers of the tissue cut thus allowing the detection of said molecular detection means; Y 4) when said molecular detection means can be detected, scan / obtain images (817) of the tissue cut to generate a primary digital image Ik that may contain one or more areas of first marker associated with the generation of a detectable polymer; whereby a series of K primary digital images Ik is obtained for k = 1, ..., K containing an increasing number of areas of the first marker; d) carrying out the method of claim 1 in the series of K primary digital images Ik for k = 1, ..., K obtained in step c), thus generating a New image in which said cellular structures are visualized . 7. A method according to claim 6, characterized in that said molecular detection means are a set of antibodies, preferably monoclonal antibodies or antibody fragments, wherein each antibody binds to a specific cell marker and where it has been conjugated an enzyme to each antibody, said enzyme being capable of generating, in the presence of one or more suitable substrates, the formation of a detectable polymer, where points 1) and 2) of step c) are carried out in such a way that: i) the tissue cut of step a) is contacted with an antibody that specifically binds to a particular element of said predetermined series of cell markers; said antibody being conjugated to an enzyme, said enzyme being capable of generating, in the presence of one or more suitable substrates, the formation of a detectable polymer; ii) after step i) above, the tissue cut is washed to remove unbound antibodies; and where point 3) of step c) is carried out in such a way that: iii) after point 2) tissue cutting is exposed to one or more suitable substrates for said enzyme, which leads to the formation of detectable polymers in case said particular element of said predetermined series of cell markers is present in said tissue cutting 8. A method according to claim 6, characterized in that said molecular detection means are a set of molecular complexes, wherein each complex comprises a first antibody, preferably a monoclonal antibody, that binds to a specific cell marker, a second antibody or an antibody fragment, preferably a monoclonal antibody specifically bound to said first antibody and an enzyme conjugated to said second antibody, said enzyme being capable of generating, in the presence of one or more suitable substrates, the formation of a detectable polymer, where points 1) and 2) of step c) are carried out in such a way that: i) the tissue cut of step a) is contacted with a first antibody that specifically binds to a particular element of said predetermined series of cell markers; ii) after step i) above, the tissue cut is washed to remove unbound antibodies; iii) after step ii) above, the tissue cut is contacted with a second antibody that specifically binds to said first antibody, said second antibody being conjugated to an enzyme, said enzyme being capable of generating in the presence of one or more suitable substrates the formation of a detectable polymer; Y iv) after step iii) above, the tissue cut is washed to remove unbound antibodies; Y wherein point 3) of step c) is carried out in such a way that: v) after point 2) tissue cutting is exposed to one or more suitable substrates for said enzyme, which leads to the formation of detectable polymers in case said particular element of said predetermined series of cell markers is present in said tissue cutting 9. A method according to claims 7 or 8, characterized in that said enzyme is chosen from the group of alkaline phosphatase and peroxidase, such as horseradish peroxidase. 10. A method according to claim 9, characterized in that said substrate is selected from the group of 3,3-diaminobenzidine, Ferangi Blue, Vulcan Fast Red, aminoethyl carbazole (AEC) and Vina Green. 11. A method according to claim 6, characterized in that said molecular detection means are a set of molecular conjugates comprising a recognition part attached to a detection part, wherein said recognition part is capable of specifically binding to a particular element of said predetermined series of cell markers, said recognition portion being selected from the group of: an antibody, such as a polyclonal antibody, a monoclonal antibody and fragments thereof, and a nucleic acid molecule, such as a RNA molecule and a DNA molecule, said detection part being a fluorochrome, said fluorochrome being capable of emitting radiation of a particular wavelength after exposure to an initiation radiation different from said emitted radiation wherein point 3) of step c) is carried out in such a way that the tissue cutting and any molecular detection means that has been attached thereto are exposed to the initiation radiation that leads to the emission of radiation from a particular wavelength in case said particular element of said predetermined series of cell markers is present in said tissue cut; and wherein point 4) of step c) is carried out when said radiation of a particular wavelength is emitted. 12. A method according to any one of claims 7-11, wherein a substrate that generates at least partially soluble polymers, such as Vina Green or aminoethyl carbazole (AEC), is used as a substrate to produce a detectable polymer, the method also comprising the steps of e) washing said tissue cut to remove said detectable polymer; Y f) repeat steps b-d with a new series of molecular detection means. 13. A method for visualizing the three-dimensional distribution of multiple cell populations and cell structures within the same three-dimensional space in a histological sample, comprising the steps of: A) providing a tissue sample, and cutting said sample into a plurality of originally superimposed tissue cuts in a previously known manner; B) carrying out the method according to any one of claims 6-12 for all tissue cuts obtained in step A); Y C) superimpose the images obtained in step B) according to known principles, thus obtaining a three-dimensional visualization of the three-dimensional distribution of multiple cell populations and cell structures within the same three-dimensional space in a histological sample.