KR20210059973A - System for co-location tracking of correlative microscopy and opeation method thereof - Google Patents

Abstract

An ultra-high-resolution linked microscope system and a method of operation thereof according to various embodiments may comprise processes of: detecting first vertex coordinates for rest of vertices from a plate that is made of a cube having a rectangular plane containing four vertices and is chamfered adjacent to any one of the vertices through a first microscope module; detecting, through a second microscope module, second vertex coordinates for the rest of the vertices from the plate; and detecting, based on the first vertex coordinates and the second vertex coordinates, at least one target coordinate for imaging through the second microscope module on the plate.

Description

A system for tracking the same position of a linked microscope and its operation method {SYSTEM FOR CO-LOCATION TRACKING OF CORRELATIVE MICROSCOPY AND OPEATION METHOD THEREOF}

Various embodiments relate to a system for co-position tracking of an associated microscope and a method of operation thereof.

Recently, research on a linked microscope system for observing the same sample by linking heterogeneous microscope modules has been conducted. At this time, the linked microscope system acquires an image of a sample through one microscope module, and acquires an image of the same sample through another microscope module. Because of this, it is important for the linked microscope system to accurately detect the same location of the sample, for the microscope modules. However, the linked microscope system as described above has difficulty in accurately detecting the same location of a sample.

Various embodiments provide an integrated microscope system that can quickly and accurately capture the same sample through heterogeneous microscope modules.

Various embodiments provide, for heterogeneous microscope modules, a linked microscope system capable of quickly and accurately detecting the same location of a sample.

The operating method of the linked microscope system according to various embodiments is a plate made of a hexahedron having a rectangular plane including four vertices through a first microscope module, and chamfered adjacent to any one of the vertices. The operation of detecting first vertex coordinates for the rest of the vertices from, through a second microscope module, detecting second vertex coordinates for the rest of the vertices from the plate, and the first vertex coordinates And detecting at least one target coordinate for photographing through the second microscope module on the plate, based on the second vertex coordinates.

The linked microscope system according to various embodiments is manufactured as a hexahedron having a rectangular plane including four vertices, and is configured to photograph a sample inside a plate chamfered adjacent to any one of the vertices. It may include a processor configured to detect the position so that the first microscope module and the second microscope module, and the first microscope module and the second microscope module photograph the same position inside the plate.

According to various embodiments, the processor, through the first microscope module, detects first vertex coordinates for the rest of the vertices from the plate, and through the second microscope module, the vertex from the plate To detect second vertex coordinates for the remainder of them, and to detect at least one target coordinate for photographing through the second microscope module on the plate based on the first vertex coordinates and the second vertex coordinates. Can be configured.

According to various embodiments, the linked microscope system may capture the same location of cells through the first microscope module and the second microscope module. That is, the linked microscope system can perform effective linked imaging through the first microscope module and the second microscope module. At this time, since the linked microscope system detects the target coordinates for the second microscope module from the reference coordinates detected through the first microscope module through linear coordinate transformation, a plate placed on each of the first microscope module and the second microscope module Regardless of the location or direction of, it is possible to quickly and accurately detect the target coordinates.

In addition, for the plate, at least one of a common material such as a slide glass or a cover glass may be used. In addition, a simple manufacturing process such as chamfer can be applied for the plate. In other words, no special treatment is required for the plate for use in the linked microscope system. For this reason, expensive material costs or process costs may not be required to implement the linked microscope system.

1A is a diagram illustrating an associated microscope system according to various embodiments.
1B is a diagram illustrating a plate for an associated microscope system according to various embodiments.
2 is a diagram illustrating a method of operating an associated microscope system according to various embodiments.
3 is a view for explaining the operating characteristics of the linked microscope system according to various embodiments.
4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, and 16 are diagrams of an associated microscope system according to various embodiments. These are drawings for explaining the operation effect.

Hereinafter, various embodiments of the present document will be described with reference to the accompanying drawings.

1A is a diagram illustrating an associated microscope system 100 in accordance with various embodiments. 1B is a diagram illustrating a plate 110 for an associated microscope system 100 according to various embodiments.

1A and 1B, the linked microscope system 100 according to various embodiments is configured to take a sample inside the plate 110, and the first microscope module 120 and the second microscope It may include a microscope module 130 and a processor 140.

The plate 110 may be provided so that a sample is disposed inside. The plate 110 may include one chamfering 111 and three vertices 113, 115, and 117. To this end, the plate 110 has a rectangular plane including four vertices, and may be manufactured as a hexahedron having a predetermined thickness. And the plate 110 may be chamfered adjacent to any one of the vertices. Through this, the sample may be disposed inside the plane of the plate 110. For example, the plate 110 may include at least one of a slide glass on which a sample is placed or a cover glass that covers the sample.

The first microscope module 120 and the second microscope module 130 may be configured to take a sample inside the plate 110. The first microscope module 120 and the second microscope module 130 may include at least one of an optical microscope and an electron microscope. According to an embodiment, the first microscope module 120 may be an optical microscope, and the second microscope module 130 may be an electron microscope. For example, the first microscope module 120 may be a stochastic optical reconstruction microscopy (STORM), and the second microscope module 130 may be a scanning electron microscopy (SEM).

The processor 140 may be configured to detect the corresponding position so that the first microscope module 120 and the second microscope module 130 photograph the same position inside the plate 110. In this case, the corresponding position may be defined as at least one reference coordinate T in the first microscope module 120 and at least one target coordinate T′ in the second microscope module 120. To this end, the processor 140 may interface with the first microscope module 120 and the second microscope module 130. According to an embodiment, the processor 140 may be coupled to either the first microscope module 120 or the second microscope module 130. For example, the processor 140 may be coupled to the second microscope module 130.

The processor 140 may detect first vertex coordinates (A, B, C) with respect to the vertices 113, 115, and 117 from the plate 110 through the first microscope module 120. In this case, the processor 140 may identify the vertices 113, 115, and 117 based on the chamfered area 111 in the plate 110 through the first microscope module 120. In addition, the processor 140 may detect the reference coordinate T from the plate 110 based on the first vertex coordinates A, B, and C through the first microscope module 120. In this case, the processor 140 may detect the reference coordinate T while photographing a sample through the first microscope module 120. The processor 140 may detect second vertex coordinates (A', B', C') for the vertices 113, 115, and 117 from the plate 110 through the second microscope module 120. . In this case, the processor 140 may identify the vertices 113, 115, and 117 based on the chamfered area 111 in the plate 110 through the second microscope module 130.

Through this, the processor 140 is based on the first vertex coordinates (A, B, C) and the second vertex coordinates (A', B', C'), the plate ( 110) can detect the target coordinate (T'). At this time, the processor 140 performs linear coordinate transformation based on the first vertex coordinates (A, B, C) and the second vertex coordinates (A', B', C'), and The target coordinate T'can be detected. Here, the processor 140 includes two first corner vectors (i, j) and second vertex coordinates (A', B', C') detected from the first vertex coordinates (A, B, C). Linear coordinate transformation may be performed using two second corner vectors (i', j') detected from. Accordingly, the processor 140 may take a sample from the target coordinate T′ on the plate 110 through the second microscope module 130.

The linked microscope system 100 according to various embodiments is manufactured as a hexahedron having a rectangular plane including four vertices, and samples inside the plate 110 chamfered adjacent to any one of the vertices are respectively The first microscope module 120 and the second microscope module 130 configured to be photographed, and the first microscope module 120 and the second microscope module 130 are positioned so that the same position is captured from the inside of the plate 110. It may include a processor 140 configured to detect.

According to various embodiments, the processor 140, through the first microscope module 120, for the remaining (113, 115, 117) of the vertices from the plate 110, the first vertex coordinates (A, B, C), and detect second vertex coordinates (A', B', C') for the rest (113, 115, 117) of the vertices from the plate 110 through the second microscope module 130 And, based on the first vertex coordinates (A, B, C) and the second vertex coordinates (A', B', C'), for photographing through the second microscope module 130 on the plate 110 It may be configured to detect at least one target coordinate T'.

According to various embodiments, the processor 140, while photographing a sample through the first microscope module 120, is at least one related to the sample inside the plane based on the first vertex coordinates (A, B, C). It may be configured to detect the reference coordinate (T) of.

According to various embodiments, the processor 140 performs linear coordinate transformation based on first vertex coordinates (A, B, C) and second vertex coordinates (A', B', C'), It may be configured to detect the target coordinate T'from the reference coordinate T.

According to various embodiments, the processor 140 may be configured to take a sample from the target coordinate T′ on the plate 110 through the second microscope module 130.

According to various embodiments, the processor 140 includes two first corner vectors (i, j) detected from the first vertex coordinates (A, B, C) and the second vertex coordinates (A', B). It may be configured to perform linear coordinate transformation using two second corner vectors (i', j') detected from', C').

According to various embodiments, the first microscope module 120 may be an optical microscope, and the second microscope module 130 may be an electron microscope.

According to various embodiments, the plate 110 may include at least one of a slide glass or a cover glass.

2 is a diagram illustrating a method of operating the linked microscope system 100 according to various embodiments. 3 is a diagram for explaining the operating characteristics of the linked microscope system 100 according to various embodiments.

2, the linked microscope system 100 is through the first microscope module 120 in operation 210, for the vertices 113, 115, 117 from the plate 110, the first vertex coordinates (A, B, C) can be detected. In this case, the processor 140 may identify the vertices 113, 115, and 117 based on the chamfered area 111 in the plate 110 through the first microscope module 120. According to an embodiment, the processor 140 identifies the first vertex 113 on the opposite side of the chamfer area 111, and identifies the second vertex 115 in a counterclockwise direction from the first vertex 113, and , It is possible to identify the third vertex 117 in a clockwise direction from the first vertex 113. According to another embodiment, the processor 140 may sequentially identify the first vertex 113, the second vertex 115 and the third vertex 117 from the chamfer region 111 in a counterclockwise or clockwise direction. May be. Through this, the processor 140 may detect the first vertex coordinates A, B, and C from the vertices 113, 115, and 117, respectively, as shown in FIG. 3. According to an embodiment, the first microscope module 120 may be an optical microscope. For example, the first microscope module 120 may be an ultra-high resolution fluorescence microscope (STORM).

The linked microscope system 100 may detect the reference coordinate T from the plate 110 based on the first vertex coordinates (A, B, C) through the first microscope module 120 in operation 220. . In this case, the processor 140 may detect the reference coordinate T from the sample as shown in FIG. 3 while photographing the sample through the first microscope module 120.

Linked microscope system 100, through the second microscope module 120 in operation 230, the second vertex coordinates (A', B', C'for the vertices 113, 115, 117 from the plate 110). ) Can be detected. In this case, the processor 140 may identify the vertices 113, 115, and 117 based on the chamfered area 111 in the plate 110 through the second microscope module 130. According to an embodiment, the processor 140 identifies the first vertex 113 on the opposite side of the chamfer area 111, and identifies the second vertex 115 in a counterclockwise direction from the first vertex 113, and , It is possible to identify the third vertex 117 in a clockwise direction from the first vertex 113. According to another embodiment, the processor 140 may sequentially identify the first vertex 113, the second vertex 115 and the third vertex 117 from the chamfer region 111 in a counterclockwise or clockwise direction. May be. Through this, the processor 140 may detect the second vertex coordinates (A', B', C') from the vertices 113, 115, and 117, respectively, as shown in FIG. 3. According to an embodiment, the second microscope module 130 may be an electron microscope. For example, the second microscope module 130 may be a scanning electron microscope (SEM).

The linked microscope system 100 uses the second microscope module 130 based on the first vertex coordinates (A, B, C) and the second vertex coordinates (A', B', C') in operation 240. It is possible to detect the target coordinate (T') on the hazard plate 110. At this time, the processor 140 performs linear coordinate transformation based on the first vertex coordinates (A, B, C) and the second vertex coordinates (A', B', C'), and The target coordinate T'can be detected. The processor 140 detects from two first corner vectors (i, j) and second vertex coordinates (A', B', C') detected from the first vertex coordinates (A, B, C). Linear coordinate transformation may be performed using the two second corner vectors i'and j'. According to an embodiment, the first corner vectors (i, j) are first vertex coordinates (A, B) corresponding to the first vertex 113 and the second vertex 115 as shown in [Equation 1] below. It consists of one vector detected from and another vector detected from the first vertex coordinates (A, C) corresponding to the first vertex 113 and the third vertex 117, and the second corner vectors (i', j ') are the work vector and the first vertex 113 detected from the second vertex coordinates (A', B') corresponding to the first vertex 113 and the second vertex 115 as shown in [Equation 1] below. ) And the second vertex coordinates (A', C') corresponding to the third vertex 117.

For example, the reference vector (T) and the target vector (T') may be defined as shown in [Equation 2] below. Based on this, the processor 140 may calculate a feature vector for detecting the reference vector T from the first corner vectors i and j as shown in Equation 3 below. Through this, the processor 140 detects the target coordinate (T') as shown in FIG. 3 by using the second corner vectors (i', j') and the feature vector as shown in [Equation 4] below. can do.

The linked microscope system 100 may take a sample from the target coordinate T′ on the plate 110 through the second microscope module 130 in operation 250. Through this, the linked microscope system 100 may capture a location of a sample photographed through the first microscope module 120 through the second microscope module 130. In other words, the linked microscope system 100 may capture the same position of the sample through the first microscope module 120 and the second microscope module 130.

The operating method of the linked microscope system 100 according to various embodiments is manufactured as a hexahedron having a quadrangular plane including four vertices through the first microscope module 120, and any of the vertices The operation of detecting the first vertex coordinates (A, B, C) for the remaining (113, 115, 117) of the vertices from the chamfered plate 110 adjacent to one, the second microscope module 130 Through, the operation of detecting second vertex coordinates (A', B', C') for the rest (113, 115, 117) of the vertices from the plate 110, and the first vertex coordinates (A, Based on B, C) and the second vertex coordinates (A', B', C'), at least one target coordinate for photographing through the second microscope module 130 on the plate 110 ( It may include an operation of detecting T').

According to various embodiments, the operating method of the linked microscope system 100 is, through the first microscope module 120, a sample inside the plane based on the first vertex coordinates (A, B, C). It may further include an operation of detecting at least one reference coordinate T related to.

According to various embodiments, the operation of detecting the target coordinate T'is based on the first vertex coordinates (A, B, C) and the second vertex coordinates (A', B', C'). It may include an operation of detecting the target coordinate T'from the reference coordinate T by performing linear coordinate transformation.

According to various embodiments, the operation of detecting the reference coordinate T may include an operation of detecting the reference coordinate T while photographing the sample through the first microscope module 120.

According to various embodiments, the operation method of the linked microscope system 100 is an operation of photographing the sample from the target coordinate T'on the plate 110 through the second microscope module 130 It may further include.

According to various embodiments, the detection operation of the target coordinate T'includes two first corner vectors (i, j) detected from the first vertex coordinates (A, B, C) and the second The linear coordinate transformation may be performed using two second corner vectors (I', j') detected from the vertex coordinates (A', B', C').

According to various embodiments, the first microscope module 120 may be an optical microscope, and the second microscope module 130 may be an electron microscope.

According to various embodiments, the plate 110 may include at least one of a slide glass or a cover glass.

4, 5, 6, 7, 8, 9, 10, 11, 12, 13, and 14 illustrate the operation effect of the linked microscope system 100 according to various embodiments. These are drawings for explanation.

4, 5, 6, 7, 8, 9, 10, 11, 12 and 13, the linked microscope system 100 is through the first microscope module 120 As shown in (a), an image of the cell may be acquired, and an image of the cell may be acquired through the second microscope module 130 as shown in (b). Here, the first microscope module 120 may be an ultra-high resolution fluorescence microscope (STORM), and the second microscope module 130 may be a scanning electron microscope (SEM). At this time, the linked microscope system 100 detects the target coordinates (T') for the second microscope module 130 from the reference coordinates (T) of cells detected through the first microscope module 120, and the second An image of a cell may be acquired at the target coordinate (T') through the microscope module 130. Here, the linked microscope system 100 may detect the target coordinate T'at the micrometer level. The target coordinate T'detected by the linked microscope system 100 may represent an actual target coordinate for the cell and an average error value of 7 μm, as shown in FIG. 14. That is, the target coordinate T'estimated by the linked microscope system 100 may substantially coincide with the actual target coordinate for the cell. In addition, when the planar area of the plate 110 is 24 mm × 24 mm, the linked microscope system 100 is approximately 10 ⁷ times as much as compared to the previous one that cannot estimate the location and direction of the cells on the plate 110. It can quickly detect the location of cells.

15 and 16 are diagrams for explaining the operation effect of the linked microscope system 100 according to various embodiments.

15 and 16, the linked microscope system 100 acquires an image of intracellular actin as shown in (a) through the first microscope module 120, and the second microscope module As shown in (b) through (130), an image of intracellular actin can be obtained. Here, the first microscope module 120 may be an ultra-high resolution fluorescence microscope (STORM), and the second microscope module 130 may be a scanning electron microscope (SEM). At this time, the linked microscope system 100 calculates the target coordinate (T') for the second microscope module 130 from the reference coordinate (T) of the intracellular actin detected through the first microscope module 120 within a few seconds. After detection, an image of intracellular actin may be obtained at the target coordinate T'through the second microscope module 130. Here, the linked microscope system 100 may detect the target coordinate T'at the nanometer level. Through this, not only the first microscope module 120 can photograph intracellular actin from the plate 110 at a very high resolution, but the second microscope module 130 can also photograph the intracellular actin from the plate 110. .

According to various embodiments, the linked microscope system 100 may capture the same location of cells through the first microscope module 120 and the second microscope module 130. That is, the linked microscope system 100 may perform effective linked imaging through the first microscope module 120 and the second microscope module 130. At this time, the linked microscope system 100 is based on the first vertex coordinates (A, B, C) and the second vertex coordinates (A', B', C') through linear coordinate transformation, the reference coordinate (T ) From the target coordinate (T'), regardless of the position or direction of the plate 110 placed on each of the first microscope module 120 and the second microscope module 130, the target coordinate T' ) Can be detected. Here, the linked microscope system 100 can quickly and accurately detect the target coordinate T'at the micrometer level, further at the nanometer level. Accordingly, the linked microscope system 100 may perform ultra-high resolution linked imaging.

In addition, for the plate 110, at least one of a general material such as a slide glass or a cover glass may be used. In addition, a simple manufacturing process such as a chamfer may be applied for the plate 110. That is, no special treatment is required for the plate 110 for use in the linked microscope system 100. For this reason, expensive material cost or process cost may not be required to implement the linked microscope system 100.

Various embodiments of the present document and terms used therein are not intended to limit the technology described in this document to a specific embodiment, and should be understood to include various modifications, equivalents, and/or substitutes of the corresponding embodiment. In connection with the description of the drawings, similar reference numerals may be used for similar components. Singular expressions may include plural expressions unless the context clearly indicates otherwise. In this document, expressions such as "A or B", "at least one of A and/or B", "A, B or C" or "at least one of A, B and/or C" are all of the items listed together. It can include possible combinations. Expressions such as "first", "second", "first" or "second" can modify the corresponding elements regardless of their order or importance, and are only used to distinguish one element from another. It does not limit the components. When any (eg, first) component is referred to as being “connected (functionally or communicatively)” or “connected” to another (eg, second) component, the component is It may be directly connected to the component, or may be connected through another component (eg, a third component).

The term "module" used in this document includes a unit composed of hardware, software, or firmware, and may be used interchangeably with terms such as, for example, logic, logic blocks, parts, or circuits. The module may be an integrally configured component or a minimum unit that performs one or more functions, or a part thereof. For example, the module may be configured as an application-specific integrated circuit (ASIC).

According to various embodiments, each component (eg, module or program) of the described components may include a singular number or a plurality of entities. According to various embodiments, one or more components or operations among the above-described corresponding components may be omitted, or one or more other components or operations may be added. Alternatively or additionally, a plurality of constituent elements (eg, a module or program) may be integrated into one constituent element. In this case, the integrated component may perform one or more functions of each component of the plurality of components in the same or similar to that performed by the corresponding component among the plurality of components prior to integration. According to various embodiments, operations performed by a module, program, or other component may be sequentially, parallel, repeatedly, or heuristically executed, or one or more of the operations may be executed in a different order or omitted. , Or one or more other actions may be added.

Claims (15)

In the operating method of the linked microscope system having a first microscope module and a second microscope module,
Through the first microscope module, the first vertex coordinates are detected for the rest of the vertices from a plate made of a hexahedron having a rectangular plane including four vertices and chamfered adjacent to any one of the vertices. Action;
Detecting second vertex coordinates for the rest of the vertices from the plate through the second microscope module; And
And detecting at least one target coordinate for photographing through the second microscope module on the plate based on the first vertex coordinates and the second vertex coordinates.
The method of claim 1,
The method further comprising detecting, through the first microscope module, at least one reference coordinate related to the sample inside the plane based on the first vertex coordinates.
The method of claim 2, wherein the target coordinate detection operation,
And detecting the target coordinate from the reference coordinate by performing linear coordinate transformation based on the first vertex coordinates and the second vertex coordinates.
The method of claim 2, wherein the reference coordinate detection operation comprises:
And detecting the reference coordinate while photographing the sample through the first microscope module.
The method of claim 2,
The method further comprising taking the sample from the target coordinate on the plate through the second microscope module.
The method of claim 1, wherein the target coordinate detection operation comprises:
And performing the linear coordinate transformation using two first corner vectors detected from the first vertex coordinates and two second corner vectors detected from the second vertex coordinates.
The method of claim 1,
The first microscope module is an optical microscope,
The second microscope module is an electron microscope method.
The method of claim 1,
The plate is a method comprising at least one of a slide glass or a cover glass.
In the linked microscope system,
A first microscope module and a second microscope module made of a hexahedron having a quadrangular plane including four vertices and configured to photograph a sample inside the plate chamfered adjacent to any one of the vertices, respectively; And
And a processor configured to detect the position so that the first microscope module and the second microscope module photograph the same position inside the plate,
The processor,
Through the first microscope module, detecting first vertex coordinates for the rest of the vertices from the plate,
Through the second microscope module, detecting second vertex coordinates for the rest of the vertices from the plate,
A system configured to detect at least one target coordinate for imaging through the second microscope module on the plate, based on the first vertex coordinates and the second vertex coordinates.
The method of claim 9, wherein the processor,
A system configured to detect at least one reference coordinate associated with the sample inside the plane based on the first vertex coordinates while photographing the sample through the first microscope module.
The method of claim 10, wherein the processor,
A system configured to detect the target coordinate from the reference coordinate by performing a linear coordinate transformation based on the first vertex coordinates and the second vertex coordinates.
The method of claim 10, wherein the processor,
A system configured to photograph the sample from the target coordinates on the plate, via the second microscope module.
The method of claim 9, wherein the processor,
The system configured to perform the linear coordinate transformation using two first corner vectors detected from the first vertex coordinates and two second corner vectors detected from the second vertex coordinates.
The method of claim 9,
The first microscope module is an optical microscope,
The second microscope module is an electron microscope system.
The method of claim 9,
The plate is a system comprising at least one of a slide glass or a cover glass.

Images