Composite Structures 288 (2022) 115383

Contents lists available at ScienceDirect

Composite Structures
journal homepage: www.elsevier.com/locate/compstruct

Automatic void content assessment of composite laminates using a

machine-learning approach
João M. Machado a, *, João Manuel R.S. Tavares a, b, Pedro P. Camanho a, b, Nuno Correia a
a
Instituto de Ciência e Inovação em Engenharia Mecânica e Engenharia Industrial, Porto, Portugal
b
Departamento de Engenharia Mecânica, Faculdade de Engenharia, Universidade do Porto, Porto, Portugal

A R T I C L E I N F O A B S T R A C T

Keywords: Voids have a substantial impact on the mechanical properties of composite laminates and can lead to premature
Microscopy failure of composite parts. Optical microscopy is a commonly employed imaging technique to assess the void
Deep Learning content of composite parts, as it is reliable and less expensive than alternative options. Usually, image thresh­
Convolutional neural network
olding techniques are used to parse the void content of the acquired microscopy images automatically; however,
Materials characterization
these techniques are very sensitive to the imaging acquisition conditions and type of composite material used.
Additionally, these algorithms have to be calibrated before each analysis, in order to provide accurate results.
This work proposes a machine-learning approach, based on a convolutional neural network architecture, with
the objective of providing a robust tool capable of automatically parsing the void content of optical microscopy
images, without the need of parameter tuning.
Results from training and testing datasets composed of microscopy images extracted from three distinct types
of laminates confirm that the proposed approach parses void content from microscopy images more accurately
than a traditional thresholding algorithm, without the need of a previous calibration step. This work shows that
the proposed approach is promising, despite sometimes lower than expected precision in individual void
statistics.

1. Introduction that can be achieved, this methodology can be very sensitive to the
illumination conditions during the acquisition of the images as well as
Voids are created during manufacturing and can display different the type of material being analysed. Therefore, the calibration of the
morphologies. Research suggests that these characteristics are depen­ algorithm parameters (including the threshold value), is a necessary step
dent to the manufacturing process used, as well as the optimization before the analysis of a given set of images.
degree of the process parameters [1-3]. Voids have a negative impact on Several methods exist to enable the assessment of void content in
the mechanical properties of composite laminates, especially those that composite parts, namely density-based methods (acid digestion, matrix
are matrix dominated [4-7], as fatigue resistance and compression burn-off), optical or electron microscopy, ultrasonic testing, thermog­
strength [7-11]. Therefore, void content assessment is an essential step raphy, and X-Ray micro-CT.
to monitor the quality of manufactured parts, guaranteeing the reli­ Non-destructive methods such as ultrasonic testing and thermog­
ability of the composite structure. raphy have the added advantage of preserving the part, while allowing
Optical microscopy is the most commonly employed imaging tech­ to estimate void content. On the other hand, although X-ray micro-CT is
nique to evaluate void content [1,4,12-17], since it provides reasonable not a destructive technique by nature, microscopy, X-ray micro-CT and
accuracy and detail, and is simple. In order to avoid the time consuming density techniques usually require the partial or total destruction of the
task of evaluating the relative void content of micrography images composite part in other to assess void content in smaller samples [20].
manually [4,17], a commonly employed technique is automatic image Another relevant issue in void analysis is the extraction of void
segmentation by pixel intensity thresholding [1,13,18,19]. This tech­ characteristics, such as dimensions, shape, and number count. Such
nique relies on the different pixel intensities of the composite and the analysis requires a high level of detail, which not all analysis techniques
void and is usually user calibrated. However, despite the good results can provide, especially when the intended voids are small enough to be

* Corresponding author.
E-mail address: jmmachado@inegi.up.pt (J.M. Machado).

https://doi.org/10.1016/j.compstruct.2022.115383
Received 23 April 2021; Received in revised form 18 January 2022; Accepted 15 February 2022
Available online 19 February 2022
0263-8223/© 2022 Elsevier Ltd. All rights reserved.
J.M. Machado et al. Composite Structures 288 (2022) 115383

located inside or in between the fibre tows. It is known that density- order to overcome the shortcomings of common thresholding ap­
based techniques are not able to provide such data, whereas despite proaches, which reliability is greatly affected by the pixel intensity
the advancements in ultrasound testing techniques, still ultrasound and variability.
thermography usually do not provide the ideal level of detail for such For that matter, a machine learning approach based on convolutional
analysis [20,21]. neural networks was used to analyse microscopy images and obtain the
Usually, microscopy and X-ray micro-CT techniques are reported to corresponding void contents.
provide a good level of detail, which enables the accurate measurement This article is organized as follows: Section one presents a brief
and parametrization of void characteristics on the smaller length scales introduction to convolutional neural networks is given. Section two
[20-22]. Due to its simplicity, lower cost and reasonable accuracy and describes the methodology used to create the machine learning frame­
detail, optical microscopy is still a commonly employed imaging tech­ work. Section three shows the results obtained with the proposed model.
nique to conduct void content analyses [1,4,12-17]. Finally, section four presents the conclusions taken from this study.
In order to avoid the time-consuming task of evaluating the relative
void content of micrography images manually [4,17], a commonly
employed technique is automatic image segmentation by pixel intensity 1.1. Convolutional neural networks
thresholding [1,13,18,19]. This technique relies on the different pixel
intensities of the composite and the void and is usually user calibrated. In an artificial neural network, a set of inputs is mapped to an output,
However, despite the good results that can be achieved, this method­ by means of a mathematical function [28]. If the inputs are mapped
ology can be very sensitive to the illumination conditions during the directly to an output, it is denominated as a single-layer neural network.
acquisition of the images as well as the type of material being analysed. On the contrary, if the inputs are mapped to an output through a suc­
Therefore, the calibration of the algorithm parameters (including the cession of subsequent (hidden) layers, the neural network is denomi­
threshold value), is a necessary step before the analysis of a given set of nated as of the multi-layer type [28,29]. The universal approximation
images. theorem states that a neural network with at least one hidden layer can
Another problem that can undermine thresholding approaches is the be used to approximate any function well, provided that the network has
appearance of large voids in laminates. As void size increases, light enough hidden units [28,29].
coming from the microscope illumination can be reflected from the in­ Similarly to traditional artificial neural networks, the architecture of
side of the void cavities, which in turn originates lighter areas inside the convolutional neural networks is built upon layers, which are connected
dark ones. This translates to high pixel intensities, which should be in a logical sequence. In analogy to neural networks and the universal
classified as voids, that are mistakenly classified as matrix, due to its approximation theorem, a convolutional neural network can be used to
naturally higher pixel intensity. In turn, this renders the common approximate any continuous function to a desired non-zero amount of
thresholding approaches ineffective, as these techniques are not able to error, provided that the depth of the convolutional neural network is
detect the void areas entirely (Fig. 1). large enough [30].
The adoption of machine-learning algorithms to do automatic However, unlike traditional artificial neural networks, convolutional
detection of voids has been reported in the literature for several void neural networks can possess different types of layers: fully connected
assessment techniques, such as X-ray micro-CT [23-25], thermography layers, convolutional layers and pooling layers.
[26] and ultrasound testing [27]. Luo et al. used a deep learning Fully connected layers are a type of layer in which every neuron is
framework based on DeepLabV3+, which achieved good void segmen­ connected each neuron of the previous layer by a distinct set of weights,
tation results in optical microscopy images [22]. However, their results which are the layer trainable parameters:
show that the segmentation accuracy of a thresholding algorithm is very ∑
n

close to the one obtained by the deep learning algorithm. In turn, it is zl = wl−ij 1 xl−i 1 + bl− 1
(1)
plausible to infer that the images present in their dataset might not have
j=1

the complexity that is added when large pixel intensity scattering exists where z is the vector containing the input node values to layer l, wij is
due to the presence of large voids and reflections. This increased the connection weight between neurons, xl− 1 is the activated neuron
complexity could produce larger differences between thresholding and value of the previous layer, and b is the bias vector (omitted in Fig. 2 for
machine-learning results than the ones Luo reported. conciseness).
In this work, a machine vision algorithm based on machine-learning Fully connected layers are the staple of traditional artificial neural
was developed, to analyse microscopy images for void detection, in networks, which are only comprised by a succession of this type of

Fig. 1. Microscopy image with light reflecting voids (on the left) and poor performance of thresholding based segmentation method (on the right).

2
J.M. Machado et al. Composite Structures 288 (2022) 115383

the feature map before the convolution operation. This is done by adding
an outer layer of values, usually in the form of zeros, an operation
commonly designated as zero-padding (Fig. 4). Moreover, if enough
padding is added to the input feature map, one can obtain a bigger
output than the original input, leading to a transposed convolution (also
known as up-convolutions or deconvolutions). This increase of the size
of the feature map, commonly designated as up-sampling, can be useful
in certain network architectures, such as autoencoders [28].
At the end of each convolutional layer or fully connected layer, a
non-linear activation function can be commonly found. These functions
have been found to allow the network to learn more complex features in
data, compared to linear activation functions [28]. An activation non-
linearity commonly used in convolution neural networks is the Recti­
fied Linear Unit (ReLU), which is a piece-wise linear function that will
output the received input, in case it is positive, otherwise the output is
zero. This activation function is particularly relevant for deep learning
(neural network architectures with several layers), as it better preserves
the gradient information across several layers deep, compared to the
logistic, or commonly designated sigmoid activation function, which can
suffer from saturation for large activation values (Fig. 5) [29]. The ReLU
activation can be written as:
{
x∀x > 0
f (x) = (3)
0∀x ≤ 0

The logistic, or sigmoid, activation function (Fig. 6) can be written

as:
Fig. 2. Example of a fully connected layer. 1
f (x) = x
(4)
1 + e−
layers. However, in convolutional neural networks, this type of layers The last staple in a convolutional neural network structure is the
can be commonly found in the ending layers of the network [31-33]. pooling layer. Pooling layers can either extract the maximum, minimum
Convolutional layers implement the convolution operation, which or average value inside a sliding window with a predetermined size and
for two-dimensional tensors can be written as: stride (Fig. 7). No trainable parameters exist in this type of layers, as
∑∑ their objective mostly relies in reducing the size of the feature map, an
S(i, j) = (I*K)(i, j) = I(m, n)K(i − m, j − n) (2)
m n operation commonly designated as down-sampling. However, alike the
convolutional layers, pooling layers are not constrained to an input of a
where I is the two-dimensional tensor being convolved, K is a two-
predetermined size, as the pooling window slides throughout the entire
dimensional kernel and S is the resulting tensor.
tensor with a defined stride, independently of the tensor size.
Unlike fully connected layers, which can only accept an input of a
In the field of machine vision, images are interpreted as a collection
predetermined size, convolutional layers do not have this restriction, as
of pixel intensity values, which is represented as a tensor with varying
the learnable parameters are embedded in the kernel, which size is in­
depth, depending on the encoding of the image colours. A greyscale
dependent of the input tensor (feature map).
image can be therefore represented as a two-dimensional tensor, which
One characteristic outcome of convolutions (observable in Fig. 3) is
dimensions match the resolution of the image, whereas an RGB encoded
the reduction of the size of the feature map, whose extent depends on the
image can be represented as a three-dimensional tensor, for example,
size of the kernel. In case this behaviour is not desirable, and if one
with a depth equal to three, representing the red, green and blue
intends to maintain the size of the feature map, padding can be added to
channels.

Fig. 3. Example of a convolution operation.

3
J.M. Machado et al. Composite Structures 288 (2022) 115383

Fig. 4. Example of a padded convolution.

research branches were created to solve problems such as image clas­

sification (assignment of a single class per image) [33], or problems
which require a pixel-level type of inference, such as semantic seg­
mentation (segmentation based on the classes existing) [35], as well as
of instance segmentation (segmentation based on instances of each class
present in the image) [36]. In the scope of this work, the automatic
segmentation of microscopy images, for subsequent determination of
relative void content, is a semantic segmentation problem.

1.2. U-net architecture

The U-net is a semantic segmentation model architecture proposed

by Ronneberger et al. [37], which is built upon the previous fully con­
Fig. 5. ReLU activation function. volutional network model for semantic segmentation [35], making the
U-net architecture a fully convolutional network, itself (Fig. 8). As
mentioned in the former section, from the image segmentation point of
view, as fully convolutional networks do not contain fully connected
layers, they present the added advantage of being able to process images
with a variable size, while reducing the computational overhead, due to
connection sparsity and parameter sharing [29].
The U-net architecture possesses a contracting path (encoder), where
the initial feature-map (input image) is reduced in size, while the
number of feature channels increases. This is done through a series of
convolution layers followed by pooling layers, as in a regular CNN. The
relevant features of the image (context) are intended to be captured in
this portion of the network. The second portion of the network is an
expansive path (decoder), where the size of feature-map is up-sampled
and the number of feature channels is reduced. During the up-
Fig. 6. Sigmoid activation function. sampling procedure, the feature-map is concatenated with its corre­
sponding pair of the contracting path. This strategy ensures that the
When the neural network processes an image, the number of inputs network captures with more refinement the locations of the relevant
in the network will match the resolution of the image, multiplied by the features in the image. This forces the network to have an approximately
number of channels it possesses. The number of inputs can be therefore symmetric architecture. The up-sampling of the feature map is achieved
substantially large. As convolutional neural networks consist mainly of through transposed convolutions (or up-convolutions), followed by
convolutional and pooling layers, in which trainable parameters are not regular convolution layers. This portion of the network is thus intended
dependent on the size of the input tensor, this type of networks allow the to capture the location of the relevant features, to make a refined
design of deeper network architectures allied with a faster training, reconstruction of the image. Due to this network architecture strategy,
without recurring into memory and computational overloads, compared the model can be trained with smaller datasets and higher learning rates
to traditional artificial neural networks [29]. Consequently, the field of than other convolutional neural network models [37]. Because of this,
machine learning applied to computational vision has seen important several follow-through model architectures have been proposed with the
performance gains with the intensification of research around con­ objective of enhancing the accuracy of the segmentation results mainly
volutional neural networks (CNNs) [28,34]. As a result, different for specific biomedical imaging problems [38-41], as well as segmen­
tation of aerial and satellite imagery [42,43].

4
J.M. Machado et al. Composite Structures 288 (2022) 115383

Fig. 7. Example of a maximum pooling operation.

Fig. 8. Original U-Net architecture model [37].

Fig. 9. Proposed modified U-Net architecture.

5
J.M. Machado et al. Composite Structures 288 (2022) 115383

2. Methodology were generated by running a thresholding-based in-house software,

while further segmentation corrections were made manually, using
2.1. Network architecture GIMP open-source image processing software. From a total of sixty im­
ages in the dataset, thirty were used for training and the other half for
The implemented network architecture follows much of the original validation. The selection of the images was random.
U-Net architecture proposed by Ronneberger et al. [37], with a con­ In order to obtain a characterization of the dataset, for each type of
tracting path composed by successive 3x3 convolution layers, each fol­ laminate, void size and frequency were measured automatically, using
lowed by a rectified linear unit (ReLU) (Fig. 9). The down-sampling is the contours of the voids present in the ground truth masks. Contouring
achieved by 2x2 pooling layers after each pair of convolution layers. A is a well-established method for the representation of the geometry of
feature that can increase both the accuracy and reliability of the results, voids in binarized images, allowing sub-pixel accuracy [20,45]. In turn,
is adapting the pooling layers to the natural appearance of voids in the accuracy of the binarization process dictates the accuracy of the
micrographs, which translates to generally lower pixel intensity values. extracted void related data. Python methods available in the OpenCV
Hence, voids, which are the relevant features of the image under anal­ library were used for automatic contour extraction and respective area
ysis, can be better captured during the down-sampling operation. This calculation. The computation of the frequency measures was also carried
means that one can substitute the original maximum pooling layers by by a Python script written for this purpose, while further statistical
minimum pooling layers or, instead, invert the pixel intensities of the analysis was done in R statistical language.
image (voids become lighter and matrix darker) and maintain the In turn the frequency measures were grouped into five bins with
maximum pooling layers. In this work, the latter option was chosen, due equal width. Mean pixel intensity and standard deviation were also
to the lack of a minimum pooling layer implementation in the frame­ determined, encoding the original image in greyscale format. The
work used. Batch normalization layers are added before each ReLU different backgrounds of the microscopy samples were expected to
activation layer, as batch normalization is reported to increase both the foment an increase in the reliability of the network inference results, due
stability and speed of the learning process [44]. to a higher capacity for generalization.
As in the original U-net, the expansive path is constituted by four As it can be observed in Tables 2, 3 and 4, the size and frequency of
similar blocks, containing a set of different layers. These blocks start voids varies significantly depending on the manufacturing process.
with 2x2 transposed convolution layers, which are responsible for up- Laminates manufactured by vacuum infusion (type A and type C) seem
sampling the feature map. In order to enhance the capacity of the to possess a higher number of voids, as well as a higher variance in void
network to capture more precisely the location of the relevant features sizes, compared to the laminates manufactured by resin transfer
(voids), after the transposed convolution layer, the resulting feature map moulding (type B). This can be due to the lack of a mould packing
is concatenated with its corresponding pair of the contracting path. The procedure in vacuum infusion, whereas in resin transfer moulding, it is
resulting concatenation is fed to a pair of 3x3 convolution layers, each possible to do so by increasing resin injection pressure after mould
followed by batch normalization and a ReLU activation. filling, which in turn compacts the existent voids, therefore minimizing
The eight components of the remaining feature vector are mapped to void content in the part [3]. However, conducting such type of quanti­
the desired number of classes, adding a 1x1 convolution layer to the end tative analysis is out of the scope of this study.
of the network. As the intended pixel labelling is binary (void or matrix), Nevertheless, the void sizes obtained for all laminate types in our
the 1x1 convolution layer maps to a single tensor, where a final sigmoid dataset are in agreement with the range of void sizes found in the
activation translates the values obtained into values in the range ]0, 1[. literature [20,22]. Moreover, plotting the frequency measures of void
These values can be seen as probabilities of the pixel belonging to the sizes into an histogram, for each laminate type, it can be observed that
void class. After applying a probability threshold to the obtained values, the distribution of void sizes follows a Weibull distribution (Figs. 10, 11,
pixels with a value of 1 (one) are considered voids, whereas pixels with a 12), which is also consistent with the reported literature [46]. For the
value of 0 (zero) are considered matrix. sake of easiness of visualization, Figs. 10 and 12 only plot the frequency
measures on the first bin of laminate types A and C, respectively, as the
remaining bins have only residual frequency values.
2.2. Dataset

The dataset used for this study is comprised by microscopy images 2.3. Training
captured at INEGI, under the polishing and image capturing conditions
described in Table 1. The samples come from three different types of To conduct the U-net training, each of the high-resolution training
composite laminates (Fig. 13): glass fibre and epoxy laminate processed images was partitioned into a set of twenty grayscale 256x256 pixel
by vacuum infusion (Type A laminates); carbon fibre and epoxy lami­ smaller images. The benefits of this strategy were twofold: Firstly, this
nate processed by resin transfer moulding (Type B laminates); carbon strategy allowed to increase the number of filters of the network without
fibre and epoxy laminate processed by vacuum infusion (Type C incurring in GPU memory overloads. Moreover, this strategy allowed
laminates). each training batch to contain images of all types of laminates. As the
For each image in the dataset, a corresponding ground-truth mask network training is based on gradient optimization with an update of the
was generated. The ground truth masks consist of binary 8-bit gray-scale network weights on a per-batch basis, this strategy allows a better
images, where the pixels representing voids have a value of 255, estimation of the gradient, and therefore, a more efficient training.
whereas pixels representing matrix or fibers have a value of 0. Therefore, The network was trained using the Adam optimization algorithm
these ground-truth masks allow to determine inequivocally which pixels [47] with an initial learning rate of 0.001, binary cross-entropy loss
are voids (the object of interest), and which pixels are either matrix or function and a batch size of 40 images, for a total of 400 epochs. The
fibres (no distinction is necessary in our study). The ground-truth masks number of batches per epoch was estimated to assure that theoretically
all 256x256 dataset images would be processed during a training epoch.
The model was implemented in Keras, using Tensorflow and an Nvidia
Table 1
Quadro RTX6000 with 24 GB of memory.
Polishing and image acquisition conditions.
Laminate type Sandpaper grit (last polishing) Optical microscope
3. Results
A 2000 Olympus PMG3 w/ CCD camera
B 1000 Olympus PMG3 w/ CCD camera Four different metrics were calculated for both the training dataset,
C 1000 Olympus PMG3 w/ CCD camera
as well as the validation dataset, using a probability threshold of 0.35:

6
J.M. Machado et al. Composite Structures 288 (2022) 115383

Table 2
Properties of type A laminates.
Void area bin [µm2] Number of Frequency Mean area Area standard Coefficient of Mean pixel Pixel intensity standard
voids deviation variation intensity deviation

22.72–104962.73 5932 99.48% 1303.43 5255.83 403% 62.34 35.93

104962.73–209902.75 18 0.30% 138140.45 26567.77 19% 75.55 46.38
209902.75–314842.76 9 0.15% 258181.69 22393.60 9% 98.34 52.08
314842.76–419782.77 3 0.05% 359158.30 26297.97 7% 128.60 43.88
419782.77–524722.79 1 0.02% 440786.31 0.00 0% 57.19 21.30

Table 3
Properties of type B laminates.
Void area bin [µm2] Number of Frequency Mean area Area standard Coefficient of Mean pixel Pixel intensity standard
voids deviation variation intensity deviation

74.62–24913.59 42 73.68% 10430.46 8569.67 82% 45.86 6.56

24913.59–49752.57 11 19.30% 32487.34 4695.19 14% 44.40 5.93
49752.56–74591.54 1 1.75% 62479.84 0.00 0% 40.88 1.98
74591.54–99430.52 0 0.00% – – – – –
99430.52–124269.50 3 5.26% 113349.23 7794.01 7% 47.57 10.07

Table 4
Properties of type C laminates.
Void area bin [µm2] Number of Frequency Mean area Area standard Coefficient of Mean pixel Pixel intensity standard
voids deviation variation intensity deviation

29.21–93240.56 774 97.60% 3862.02 10122.17 262% 85.94 26.74

93240.56–186451.90 5 0.63% 124782.00 28115.30 23% 104.63 12.34
186451.90–279663.25 5 0.63% 262437.56 8067.09 3% 93.58 10.41
279663.25–372874.60 5 0.63% 320011.96 25303.36 8% 83.79 10.31
372874.60–466085.94 4 0.50% 449319.15 14296.00 3% 81.16 7.92

Fig. 10. Void size frequencies from the first bin in type A laminates and fitted weibull distribution (in red). (For interpretation of the references to color in this figure
legend, the reader is referred to the web version of this article.)

accuracy, precision, recall and intersection on union (IoU), to evaluate

TP
the performance of the proposed deep learning network, Table 5. These Precision = (6)
TP + FP
metrics can be calculated from the confusion matrix, which stores the
frequency measures for each true positive (TP), false positive (FP), false TP
negative (FN) and true negative (TN) pixel classifications, as demon­ Recall = (7)
TP + FN
strated in Fig. 14.
Subsequently, accuracy, precision, recall and IoU can be calculated TP
IoU = (8)
as: TP + FP + FN
TP + TN The use of different metrics allows one to obtain answers to different
Accuracy = (5)
TP + TN + FP + FN questions. Accuracy reflects the number of correctly classified pixels by
the total number of pixels analysed. Precision allows one to assess out of

7
J.M. Machado et al. Composite Structures 288 (2022) 115383

Fig. 11. Void size frequencies from all bins in type B laminates and fitted weibull distribution (in red). (For interpretation of the references to color in this figure
legend, the reader is referred to the web version of this article.)

Fig. 12. Void size frequencies from the first bin in type C laminates and fitted weibull distribution (in red). (For interpretation of the references to color in this figure
legend, the reader is referred to the web version of this article.)

Fig. 13. Microscopy samples of the used dataset: glass fibre laminate processed by vacuum infusion – Type A laminate (a); carbon fibre laminate processed by resin
transfer moulding – Type B laminate (b); carbon fibre laminate processed by vacuum infusion – Type C laminate (c).

all pixels classified by the network as voids, how many are voids. Recall classified as voids by the network (intersection). One relevant matter for
allows one to assess out of all pixels which are voids, how many were assessing metrics, is the fact that accuracy can be sensitive to unbalanced
classified by the network as voids. Lastly, IoU evaluates out of the group datasets (datasets in which one class is more representative than the
composed by all the pixels classified as voids, as well as the pixels which others), possibly giving biased results. In the case of an unbalanced
are actually voids (union), how many pixels are actually correctly dataset, preference should be given to metrics such as IoU, as these are

8
J.M. Machado et al. Composite Structures 288 (2022) 115383

Table 5 Fig. 17). In turn, the overall detected area of the void is smaller than in
Network performance evaluation. reality, leading to a biased detection of voids in the presented statistics.
Metric Training Dataset Validation Dataset Additionally, the network performed worse in detecting smaller
voids (first bin of void sizes for all laminate types), probably due to noise
Accuracy (binary) 0.9970 0.9936
Precision 0.9491 0.9299 present in the images. This noise is composed by abrupt color changes,
Recall 0.9907 0.9114 which may be due to small scratches in the matrix, or darker matrix
Intersection on Union (IoU) 0.9650 0.9241 features. In turn, as the convolutional neural network may have not
learned entirely which set of features is characteristic to smaller voids, it
may be producing a slight difference in the predicted quantity of voids,
as obtained in the current analysis where laminates type B and C have an
overprediction of voids, whereas laminte A suffers from an

Table 6
Segmentation results for type A laminate samples.
Void area bin [µm2] Void n◦ Voids detected IoU

22.72–104962.73 3188 2349 73.68%

104962.73–209902.75 12 11 91.67%
209902.75–314842.76 4 4 100%
314842.76–419782.77 0 0 100%
419782.77–524722.79 2 2 100%

Table 7
Fig. 14. Confusion matrix.
Segmentation results for type B laminate samples.
Void area bin [µm2] Void n◦ Voids detected IoU
less sensitive to unbalances between the dataset classes.
A physical interpretation of the segmentation results achieved by the 381.92–20089.10 23 30 76.67%
20089.10–39796.28 8 7 87.5%
network, was produced by frequency measures, which were computed
39796.28–59503.46 1 0 0%
for the different void sizes present in the validation dataset. In turn, 59503.46–79210.65 0 1 0%
these measures were compared to the ones obtained by the segmentation 79210.65–98917.83 1 1 100%
results of the network. Using the same confusion matrix analogy for void
instance statistics, intersection on union was computed for each
computed bin of void sizes. Tables 6, 7 and 8 contain the obtained results Table 8
for each type of laminate under analysis. Segmentation results for type C laminate samples.
From the results presented in Tables 6, 7 and 8, it can be seen that the Void area bin [µm2] Void n◦ Voids detected IoU
network correctly identified the majority of voids present in the
29.21–93240.56 83 100 83%
micrography images, whereas for the type B laminate dataset, the
93240.56–186451.90 2 2 100%
network had its worst performance. This lack of performance may be 186451.90–279663.25 2 3 66.66%
due to the slightly decreased capacity of the neural network in delin­ 279663.25–372874.60 3 2 66.66%
eating edges of the voids, when these have fuzzy edges (as exemplified in 372874.60–466085.94 2 2 100%

Fig. 15. Comparison of void content absolute estimation error between the proposed machine learning algorithm and a thresholding based algorithm.

9
J.M. Machado et al. Composite Structures 288 (2022) 115383

Fig. 16. Comparison of void content relative estimation error between the proposed machine learning algorithm and a thresholding based algorithm.

effect is localized in very small regions of the voids, compared to their

global area, which in turn do not affect greatly their measured areas.
This effect can be seen in Fig. 19.
The inference metrics appear to be better than the reported void
detection results. A reasonable explanation is that inference metrics are
measured on a by-pixel basis, which has no relation to void instance
statistics. This means that although the pixel classification done by the
network is good enough globally, the segmentation may not be entirely
precise, due to the reasons delineated above. Nevertheless, this relative
segmentation error is low enough as the mean absolute void content
error is below 1%, independently of the laminate type (Table 9). This
error is the mean of the estimated errors for each validation dataset
(Equation (9)). As expected, no error was obtained between void con­
tents derived from the real images and the inference results of the
training dataset.
The global void content estimation error of the proposed algorithm
Fig. 17. Decreased capacity in void edge delineation, for voids containing was compared to the results obtained with a thresholding based algo­
fuzzy edges (laminate type B). rithm developed prior to this study by the same authors. The thresh­
olding parameters were optimized on a laminate type basis. Absolute
undeprediction. An example can be seen in Fig. 18. and relative errors were derived for each algorithm type:
Finally, in certain large bubbles, the network still fails to capture the Errorabsolute = |voidreal − voidalgorithm | (9)
entire area of the bubble. This happens probably due to the high scat­
tering of pixel intensities that can be found inside certain bubbles, which |voidreal − voidalgorithm |
intensity values can reach close to the maximum, 255. However, this Errorrelative = (10)
voidreal
From Fig. 15, it can be seen that the absolute void content estimation
errors associated with the proposed machine learning algorithm were
lower than the ones obtained using the manually thresholding-based
algorithm. No clear dependence between real void content and estima­
tion error was detected; however, it can be observed that with the void
content increased the standard deviation of the estimation errors also
increases. Nevertheless, a cause-effect study is out of the scope of this
study. Regarding relative estimation errors, plotted in Fig. 16, it can be
seen that the errors were proportionally higher for lower void contents.
This was expected, independently of the segmentation algorithm, as the
segmentation error does not reach an absolute null value. Therefore, as
the void content approaches zero, since the estimation error does not
drop accordingly, the relative error tends to rise. Nevertheless, the errors
related to the proposed machine learning algorithm were lower than the
Fig. 18. Overdetection of small voids (laminate type C).

10
J.M. Machado et al. Composite Structures 288 (2022) 115383

Fig. 19. Underdetection of void area, for big voids (laminate type A).

Convolutional neural networks, such as the U-Net, are designed to make

Table 9
inferences based on the interpolation of several features present on the
Error analysis of global void content detected in dataset images.
dataset provided. This means that the reliability of inference results
Mean void content error Error standard deviation outside the training and testing datasets may be greatly affected since
Type A laminate dataset 0.66% 0.29% these algorithms are not designed to make extrapolations outside the
Type B laminate dataset 0.34% 0.27% training data. Therefore, the generalization capability of such algo­
Type C laminate dataset 0.50% 0.20%
rithms is linked to how general the dataset is itself.
Fig. 20 depicts the segmentation results for different types of mi­
errors related to thresholding alternative. croscopy images.
It is important to emphasize that the results shown in this study
should only be interpreted in the context of the dataset used.

Fig. 20. Segmentation results (on the right) for different microscopy images (on the left).

11
J.M. Machado et al. Composite Structures 288 (2022) 115383

4. Conclusion [8] Hapke J, Gehrig F, Huber N, Schulte K, Lilleodden ET. Compressive failure of UD-
CFRP containing void defects: In situ SEM microanalysis. Compos Sci Technol
2011;71(9):1242–9. https://doi.org/10.1016/j.compscitech.2011.04.009.
It was successfully demonstrated that using machine learning tech­ [9] Maragoni L, Carraro PA, Peron M, Quaresimin M. Fatigue behaviour of glass/epoxy
niques applied to computational vision, common micrography samples laminates in the presence of voids. Int J Fatigue 2017;95:18–28. https://doi.org/
can be automatically segmented, in order to calculate their relative void 10.1016/j.ijfatigue.2016.10.004.
[10] Sisodia S, Gamstedt EK, Edgren F, Varna J. Effects of voids on quasi-static and
content. tension fatigue behaviour of carbon-fibre composite laminates. J Compos Mater
The u-net architecture is a rather convenient machine learning 2015;49(17):2137–48. https://doi.org/10.1177/0021998314541993.
approach for semantic segmentation, as it needed very few annotated [11] Talreja R. Studies on the failure analysis of composite materials with
manufacturing defects. Mech Compos Mater 2013;49(1):35–44. https://doi.org/
images and training time. Using a microscopy image dataset built for 10.1007/s11029-013-9318-6.
this study, the segmentation results suggest that the network performs [12] Jeong H. Effects of voids on the mechanical strength and ultrasonic attenuation of
worse in detecting smaller voids, while the appearance of fuzzy void laminated composites. J Compos Mater 1997;31(3):276–92. https://doi.org/
10.1177/002199839703100303.
edges may also affect the accuracy of the segmentation. At last, pixel [13] Guerdal Z, Tomasino AP, Biggers SB. Effects of processing induced defects on
intensity variability can also be a factor for incomplete segmentation of laminate response. Interlaminar tensile strength. SAMPE J 1991;27:39–49.
the void area. Nevertheless, the achieved inference results are very [14] Hamidi YK, Aktas L, Altan MC. Three-dimensional features of void morphology in
resin transfer molded composites. Compos Sci Technol 2005;65(7-8):1306–20.
promising as the obtained average void content error was below 1%, https://doi.org/10.1016/j.compscitech.2005.01.001.
regardless of the laminate type. These results have surpassed a thresh­ [15] Naganuma T, Naito K, Kyono J, Kagawa Y. Influence of prepreg conditions on the
olding based algorithm manually calibrated for each laminate type void occurrence and tensile properties of woven glass fiber-reinforced polyimide
composites. Compos Sci Technol 2009;69(14):2428–33. https://doi.org/10.1016/
dataset, thus proving the applicability of this methodology.
j.compscitech.2009.06.012.
[16] Grunenfelder LK, Nutt SR. Void formation in composite prepregs - Effect of
CRediT authorship contribution statement dissolved moisture. Compos Sci Technol 2010;70(16):2304–9. https://doi.org/
10.1016/j.compscitech.2010.09.009.
[17] Purslow D. On the optical assessment of the void content in composite materials.
João M. Machado: Conceptualization, Methodology, Software, Composites 1984;15(3):207–10. https://doi.org/10.1016/0010-4361(84)90276-3.
Validation, Formal analysis, Investigation, Writing – original draft, [18] Little JE, Yuan X, Jones MI. Characterisation of voids in fibre reinforced composite
materials. NDT E Int 2012;46:122–7. https://doi.org/10.1016/j.
Writing – review & editing. João Manuel R.S. Tavares: Conceptuali­
ndteint.2011.11.011.
zation, Writing – original draft, Writing – review & editing. Pedro P. [19] Yang P, El-Hajjar R. Porosity Defect Morphology Effects in Carbon Fiber - Epoxy
Camanho: Formal analysis, Writing – original draft, Writing – review & Composites. Polym - Plast Technol Eng 2012;51(11):1141–8. https://doi.org/
editing. Nuno Correia: Writing – original draft, Writing – review & 10.1080/03602559.2012.689050.
[20] Mehdikhani M, Gorbatikh L, Verpoest I, Lomov SV. Voids in fiber-reinforced
editing, Supervision. polymer composites: A review on their formation, characteristics, and effects on
mechanical performance. J Compos Mater 2019;53(12):1579–669. https://doi.
Declaration of Competing Interest org/10.1177/0021998318772152.
[21] Abdelal N, Donaldson SL. Comparison of methods for the characterization of voids
in glass fiber composites. J Compos Mater 2018;52(4):487–501. https://doi.org/
The authors declare that they have no known competing financial 10.1177/0021998317710083.
interests or personal relationships that could have appeared to influence [22] Luo L, Zhang B, Lei Y, Zhang G, Zhang Z, Meng B, et al. Identification of voids and
interlaminar shear strengths of polymer-matrix composites by optical microscopy
the work reported in this paper. experiment and deep learning methodology. Polym Adv Technol 2021;32(4):
1853–65. https://doi.org/10.1002/pat.5226.
Acknowledgements [23] Stamopoulos AG, Tserpes KI, Dentsoras AJ. Quality assessment of porous CFRP
specimens using X-ray Computed Tomography data and Artificial Neural Networks.
Compos Struct 2018;192:327–35. https://doi.org/10.1016/j.
The authors would like to acknowledge the support from the Asso­ compstruct.2018.02.096.
ciated Laboratory for Energy, Transports and Aeronautics (LAETA) [24] Madra A, El HN, Benzeggagh M. X-ray microtomography applications for
quantitative and qualitative analysis of porosity in woven glass fiber reinforced
under the Research Grant UIDB/50022/2020 and CompRTow (Com­ thermoplastic. Compos Sci Technol 2014;95:50–8. https://doi.org/10.1016/j.
posites from Recycled TOWs) NORTE-01-0247-FEDER-038349. compscitech.2014.02.009.
[25] Madra A, Van-Pham DT, Nguyen MT, Nguyen CN, Breitkopf P, Trochu F.
Automated identification of defect morphology and spatial distribution in woven
Data Availability composites. J Compos Sci 2020;4:1–17. https://doi.org/10.3390/jcs4040178.
[26] Manzano C de JG, Ngo ACY, Sivaraja VK. Intelligent infrared thermography
The raw/processed data required to reproduce these findings cannot inspection of subsurface defects. In: Oswald-Tranta B, Zalameda JN, editors.
Thermosense Therm. Infrared Appl. XLII, SPIE; 2020, p. 33. https://doi.org/
be shared at this time as the data also forms part of an ongoing study. 10.1117/12.2558958.
[27] Meng M, Chua YJ, Wouterson E, Ong CPK. Ultrasonic signal classification and
References imaging system for composite materials via deep convolutional neural networks.
Neurocomputing 2017;257:128–35. https://doi.org/10.1016/j.
neucom.2016.11.066.
[1] Bodaghi M, Cristóvão C, Gomes R, Correia NC. Experimental characterization of
[28] Aggarwal CC. Neural Networks and Deep Learning. Cham: Springer International
voids in high fibre volume fraction composites processed by high injection pressure
Publishing; 2018. https://doi.org/10.1007/978-3-319-94463-0.
RTM. Compos Part A Appl Sci Manuf 2016;82:88–99. https://doi.org/10.1016/j.
[29] Goodfellow I, Bengio Y, Courville A. Deep Learning 2016:785.
compositesa.2015.11.042.
[30] Zhou D-X. Universality of Deep Convolutional Neural Networks. Appl Comput
[2] van Oosterom S, Allen T, Battley M, Bickerton S. An objective comparison of
Harmon Anal 2020;48(2):787–94. https://doi.org/10.1016/j.acha.2019.06.004.
common vacuum assisted resin infusion processes. Compos Part A Appl Sci Manuf
[31] Krizhevsky A, Sutskever I, Hinton GE. ImageNet classification with deep
2019;125:105528.
convolutional neural networks. NIPS 2017;60(6):84–90. https://doi.org/10.1145/
[3] Bodaghi M, Costa R, Gomes R, Silva J, Correia N, Silva F. Experimental
3065386.
comparative study of the variants of high-temperature vacuum-assisted resin
[32] Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, et al. Going deeper with
transfer moulding. Compos Part A Appl Sci Manuf 2020;129:105708.
convolutions. Proc IEEE Comput Soc Conf Comput Vis. Pattern Recognit 2015;
[4] Olivier P, Cottu JP, Ferret B. Effects of cure cycle pressure and voids on some
07–12-June:1–9.. https://doi.org/10.1109/CVPR.2015.7298594.
mechanical properties of carbon/epoxy laminates. Composites 1995;26(7):509–15.
[33] He K, Zhang X, Ren S, Sun J. Deep Residual Learning for Image Recognition. Proc
https://doi.org/10.1016/0010-4361(95)96808-J.
IEEE Comput Soc Conf Comput Vis Pattern Recognit 2015;2016-Decem:770–8.
[5] Harper BD, Staab GH, Chen RS. A Note on the Effects of Voids Upon the Hygral and
https://doi.org/10.1109/CVPR.2016.90.
Mechanical Properties of AS4/3502 Graphite/Epoxy. J Compos Mater 1987;21(3):
[34] Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, et al. ImageNet Large
280–9. https://doi.org/10.1177/002199838702100306.
Scale Visual Recognition Challenge. Int J Comput Vis 2015;115(3):211–52.
[6] Liu L, Zhang B-M, Wang D-F, Wu Z-J. Effects of cure cycles on void content and
https://doi.org/10.1007/s11263-015-0816-y.
mechanical properties of composite laminates. Compos Struct 2006;73(3):303–9.
[35] Shelhamer E, Long J, Darrell T. Fully Convolutional Networks for Semantic
https://doi.org/10.1016/j.compstruct.2005.02.001.
Segmentation. IEEE Trans Pattern Anal Mach Intell 2017;39(4):640–51. https://
[7] de Almeida SFM, dos Neto Z, Sn.. Effect of void content on the strength of
doi.org/10.1109/TPAMI.2016.2572683.
composite laminates. Compos Struct 1994;28:139–48. https://doi.org/10.1016/
0263-8223(94)90044-2.

12
J.M. Machado et al. Composite Structures 288 (2022) 115383

[36] He K, Gkioxari G, Dollar P, Girshick R. Mask R-CNN. IEEE Trans Pattern Anal Mach [42] Zhang Z, Liu Q, Wang Y. Road Extraction by Deep Residual U-Net. IEEE Geosci
Intell 2020;42(2):386–97. https://doi.org/10.1109/TPAMI.2018.2844175. Remote Sens Lett 2018;15(5):749–53. https://doi.org/10.1109/
[37] Ronneberger O, Fischer P, Brox H. U-Net: Convolutional Networks for Biomedical LGRS.2018.2802944.
Image Segmentation. Med. Image Comput. Comput. Interv. – MICCAI 2015;9351 [43] Ji S, Wei S, Lu M. Fully Convolutional Networks for Multisource Building
(2015):234–41. https://doi.org/10.1007/978-3-319-24574-4_28. Extraction from an Open Aerial and Satellite Imagery Data Set. IEEE Trans Geosci
[38] Weng Yu, Zhou T, Li Y, Qiu X. NAS-Unet: Neural architecture search for medical Remote Sens 2019;57(1):574–86. https://doi.org/10.1109/TGRS.2018.2858817.
image segmentation. IEEE Access 2019;7:44247–57. https://doi.org/10.1109/ [44] Ioffe S, Szegedy C. Batch normalization: Accelerating deep network training by
ACCESS.2019.2908991. reducing internal covariate shift. 32nd Int Conf Mach Learn ICML 2015 2015;1:
[39] Jin Q, Meng Z, Sun C, Wei L, Su R. RA-UNet: A hybrid deep attention-aware 448–56.
network to extract liver and tumor in CT scans 2018:1–13. [45] Nikishkov Y, Airoldi L, Makeev A. Measurement of voids in composites by X-ray
[40] Li X, Chen H, Qi X, Dou Qi, Fu C-W, Heng P-A. H-DenseUNet: Hybrid Densely Computed Tomography. Compos Sci Technol 2013;89:89–97. https://doi.org/
Connected UNet for Liver and Tumor Segmentation from CT Volumes. IEEE Trans 10.1016/j.compscitech.2013.09.019.
Med Imaging 2018;37(12):2663–74. https://doi.org/10.1109/TMI.2018.2845918. [46] Melenka GW, Lepp E, Cheung BKO, Carey JP. Micro-computed tomography
[41] Zeng Z, Xie W, Zhang Y, Lu Y. RIC-Unet: An Improved Neural Network Based on analysis of tubular braided composites. Compos Struct 2015;131:384–96. https://
Unet for Nuclei Segmentation in Histology Images. IEEE Access 2019;7:21420–8. doi.org/10.1016/j.compstruct.2015.05.057.
https://doi.org/10.1109/ACCESS.2019.2896920. [47] Kingma DP, Adam BJL. A method for stochastic optimization. 3rd Int Conf Learn
Represent ICLR 2015 -. Conf Track Proc 2015:1–15.

13