Dtu09 PDF
Dtu09 PDF
Dtu09 PDF
Received 15 November 2005; received in revised form 25 May 2006; accepted 25 May 2006
Available online 25 July 2006
Abstract
This paper presents a systematic approach based on robust statistical techniques for development of a data-driven soft sensor, which is an
important component of the process analytical technology (PAT) and is essential for effective quality control. The data quality is obviously of
essential significance for a data-driven soft sensor. Therefore, preprocessing procedures for process measurements are described in detail. First, a
template is defined based on one or more key process variables to handle missing data related to severe operation interruptions. Second, a univariate,
followed by a multivariate principal component analysis (PCA) approach, is used to detect outlying observations. Then, robust regression techniques
are employed to derive an inferential model. A dynamic partial least squares (DPLS) model is implemented to address the issue of auto-correlation
in process data and thus to provide smoother estimation than using a static regression model. The proposed methodology is illustrated through
applications to a cement kiln system for estimation of variables related to product quality, i.e., free lime, and to emission quality, i.e., nitrogen
oxides (NOx) emission. The case studies reveal the effectiveness of the systematic framework in deriving data-driven soft sensors that provide
reasonably reliable one-step-ahead predictions.
© 2006 Elsevier Ltd. All rights reserved.
0098-1354/$ – see front matter © 2006 Elsevier Ltd. All rights reserved.
doi:10.1016/j.compchemeng.2006.05.030
420 B. Lin et al. / Computers and Chemical Engineering 31 (2007) 419–425
grinding circuit (Casali et al., 1998). Soft sensor development detect and handle missing data related to severe operating inter-
in batch/fed-batch processes has been review extensively in ruptions. Specifically, a template is defined by using the kiln
Dochain and Perrier (1997) and James, Legge, and Budman drive measurement to identify missing observations. In case a
(2000). The 1990 Clean Air Act requires continuous emission small block (i.e., less than 2 h) of data is missing, interpolated
monitoring devices equipped for NOx, SO2 and CO2 for certain values based on neighbouring observations will be inserted. If
large sources, such as industrial boilers and furnaces (Dong, larger segments of missing data are detected, these blocks will
McAvoy, & Chang, 1995; Qin, Yue, & Dunia, 1997). Although be marked and not used to build a soft sensor.
costly online analyzers have been installed at many plants, the Missing data do not always show a systematic pattern. A
emission measurement from a sensor may become unavailable missing segment might exist in only one of the process mea-
due to instrument failure, maintenance or repair. Consequently, surements. In this case, such blocks can be replaced by using
applications of multivariate soft sensors to emission monitor- model-based interpolation methods that fill the missing gap
ing have been increasingly reported (Dong et al., 1995; Qin et with a model derived from the data set (Gupta & Lam, 1996;
al., 1997). Recently, the Food and Drug Administration (FDA) Nelson, Taylor, & MacGregor, 1996). Missing data are one type
introduces process analytical technology (PAT) into the pharma- of outliers. The second type denotes abnormal operating condi-
ceutical industry to ensure high and consistent product quality. tions. For example, the malfunction of process equipment might
An essential component of PAT is the real-time information of cause a change in process measurements that may affect sev-
product properties. Soft sensors derived with multivariate sta- eral successive samples. For detection of these outlying process
tistical approaches can be powerful tools for pharmaceutical observations, both univariate and multivariate approaches have
industry to facilitate process understanding, to monitor pro- been developed.
cess operation and quality, to detect abnormal situations and The 3σ edit rule is a popular univariate approach to detect
to improve process reliability (Hinz, 2006; Kourti, 2006). outliers (Ratcliff, 1993),
Online process measurements are often contaminated with
data points that deviate significantly from the true values due |x(i) − x̄| > t · σ (1)
to instrument failure or changes of operating conditions. Since
where x̄ is the mean of the data sequence and t = 3 is the threshold.
outlying observations may deteriorate the regression model,
This method labels outliers when data points are three or more
robust statistical approaches have been developed to provide
standard deviations from the mean.
reliable results in the presence of abnormal observations. This
Unfortunately, this procedure often fails in practice because
paper presents a systematic approach using robust multivariate
the presence of outliers tends to inflate the variance estimation,
techniques to build a soft sensor from available process mea-
causing too few outliers to be detected. The Hampel identifier
surements. The application examples are the estimation of free
(Davies & Gather, 1981) replaces the outlier-sensitive mean and
lime and NOx emission for cement kilns.
standard deviation estimates with the outlier-resistant median
The paper is organized as follows. First, a generic proce-
and median absolute deviation from the median (MAD). The
dure is presented. Data preprocessing in Section 2 includes
MAD scale estimate is defined as:
both univariate and multivariate approaches for detecting out-
lying observations. Robustified PCR and PLS approaches are MAD = 1.4826 median{|xi − x∗ |} (2)
described in Section 3. Section 4 contains illustrative applica-
tions on development of free lime and NOx soft sensors for where x* is the median of the data sequence. The factor 1.4826
cement kilns, followed by conclusions in Section 5. is chosen such that the expected MAD is equal to the standard
deviation σ for normally distributed data.
2. Data preprocessing Fig. 1 shows 300 samples of SO2 measurement from a cement
plant during otherwise steady operating conditions. Due to harsh
Outliers are commonly defined as observations that are operating conditions, especially the flying dust within the kiln
not consistent with the majority of the data (Chiang, Pell, & system that may block the measurement probe, the data seg-
Seasholtz, 2003; Pearson, 2002a), including missing data points ment of the gas analyzer measurement contains many outlying
or blocks, and observations that deviate significantly from nor- observations. It should first be noted that the mean value of the
mal values. A data-driven soft sensor derived with PCR or PLS sequence is biased significantly from the nominal value, while
deteriorates in the presence of abnormal observations, resulting the median value is close. In addition, outliers inflate the stan-
in model misspecification. Therefore, outlier detection consti- dard deviation such that most of the outlying observations are
tutes an essential prerequisite step for design of a data-driven treated as normal data. With the threshold of xMed ± 3 · xMAD ,
soft sensor. the Hampel identifier identifies most outliers successfully.
Although missing data with regular patterns are not com- A moving window Hample filter can be implemented with
mon in data from well-designed experiments, they often exist two tuning parameters: the threshold, t, and the width of the time
in operating data. For example, in the cement kiln system near window, K. The following choices are recommended (Pearson,
zero drive current data simply correspond to a stop of cement 2002b): 2 ≤ t ≤ 5, 3 ≤ K ≤ 5, implying that 7–11 points are used
kiln operation. During such a period, other kiln measurements for calculating the median and MAD of moving data window.
obviously will not be reliable or meaningful. Therefore, a heuris- Since process measurements from chemical processes are
tic approach has been implemented in the proposed procedure to not independent, detecting outliers using univariate diagnos-
B. Lin et al. / Computers and Chemical Engineering 31 (2007) 419–425 421
The operating data from a cement kiln log system are used to
derive a soft sensor of free lime in the clinker. There are totally
19 process measurements available, including kiln drive current,
kiln feed, fuel flow rates to calciner and kiln, plus several tem-
perature measurements within the kiln system. Thirteen process Fig. 4. PRESS of PLS model for CaO during validation period.
B. Lin et al. / Computers and Chemical Engineering 31 (2007) 419–425 423
Fig. 8. PRESS of CaO soft sensor with a DPLS model (order from 0 to 6)
Fig. 5. PRESS of PCR model with six PCs for CaO with significance level evaluated on validation data with 6000 samples.
varying from 100 to 90%.
Fig. 7. Validation of robust PLS model (PRESS = 41.7) for CaO with two LVs Fig. 9. Validation of CaO soft sensor (PRESS = 38.0) with a DPLS with of order
((*) laboratory measurements; (solid line) PLS). 4 ((*) laboratory measurements; (solid line) DPLS).
424 B. Lin et al. / Computers and Chemical Engineering 31 (2007) 419–425
Fig. 10. PRESS of PCR model of NOx soft sensor during validation period. Fig. 13. PRESS of robust PLS model for NOx during validation period.
Fig. 14. Validation of robust PLS model (PRESS = 8.40 × 107 ) with two LVs
for NOx ((*) online measurements; (solid line) PLS).
Fig. 11. PRESS of PCR model with seven PCs for NOx with significance level
varying from 100 to 90%.
ples is selected: the last 10,000 samples for modelling and the
first 10,000 samples for validation.
Fig. 10 shows the relation of PRESS versus the number of
PC for a PCR soft sensor. Although PCR model with one PC has
the numerically smallest PRESS, the model hardly captures pro-
cess dynamics. The PCR model of seven PCs with the PRESS
of 8.58 × 107 is employed to determine optimal Q and T2 tests
Fig. 15. Validation of NOx soft sensor (PRESS = 8.39 × 107 ) with a 10th order
significance levels. As shown in Fig. 11, the minimum PRESS DPLS of two LVs ((*) online measurements; (solid line) DPLS).
is obtained with an optimal significance level 95.5%, which
detect 4155 outliers out of the 10,000 samples for modelling.
the NOx soft sensor with a standard PLS model. As shown in
It is observed that the optimal Q and T2 tests significance lev-
Fig. 14, the performance of the NOx soft sensor is significantly
els achieve the trade-off between rejecting outlying points and
improved by the univariate and multivariate outlier detection
essential process dynamics. In addition, optimal Q and T2 tests
steps.
significance levels also depend on the quality of the modelling
A dynamic PLS NOx soft sensor is also developed. The
data block.
study reveals that the PRESS curve levels off after introduc-
The PRESS value of the NOx soft sensor developed by
ing two time-lagged input blocks. As shown in Fig. 15, the 10th
the standard PLS procedure (see Fig. 12) is 9.89 × 107 . The
order DPLS soft sensor of two LVs (PRESS = 8.39 × 107 ) pro-
performance is slightly improved by incorporating the univari-
vides much smoother prediction than a static NOx soft sensor.
ate outlier detection procedure (PRESS = 9.48 × 107 ). Fig. 13
Compared to the PCR model with seven PCs, the PLS model
shows the relationship between the number of LVs and the
demonstrates the advantage of obtaining a similar PRESS with
PRESS of the NOx soft sensor from a PLS model following the
a much lower number of LVs.
RPCA outlier detection. The minimum PRESS of 8.40 × 107 is
obtained with three LVs, which is around 15% less than that of
5. Conclusions
analyzers and laboratory measurements. Smooth estimation is Jackson, J. E., & Mudholkar, G. S. (1979). Control procedures for residuals asso-
obtained with a dynamic model by introducing an appropriate ciated with principal component analysis. Technometrics, 21(3), 341–349.
number of time-lagged terms. More importantly, the real-time James, S. C., Legge, R. L., & Budman, H. (2000). On-line estimation in biore-
actors: A review. Reviews in Chemical Engineering, 16, 311–340.
estimation of free lime shows potential to be used for quality Joseph, B., & Brosilow, C. (1978). Inferential control of processes. AIChE Jour-
control of the product. nal, 24, 485–509.
The case studies demonstrate the effectiveness of outlier Kourti, T. (2006). The process analytical technology initiative and multivari-
detection with a robust PCR approach and downweighing out- ate process analysis, monitoring and control. Analytical and Bioanalytical
lying observations to enhance the predictability of a regression Chemistry, 384(5), 1043–1048.
Kresta, J. V., Marlin, T. E., & MacGregor, J. F. (1994). Development of inferential
model. The case studies also indicate the existence of an opti- process models using PLS. Computers and Chemical Engineering, 18(7),
mal significance level. However, the issues in finding the optimal 597–611.
Q and T2 significance levels for regression model development McAvoy, T. (2002). Intelligent “control” applications in the process industries.
and integrating the information from irregularly sampled off- Annual Reviews in Control, 26(1), 75–86.
line quality measurements into the weighting vector need further Nelson, P. R. C., Taylor, P. A., & MacGregor, J. F. (1996). Missing data methods
in PCA and PLS: Score calculations with incomplete observations. Chemo-
investigation. metrics and Intelligent Laboratory Systems, 35, 45–65.
Park, S., & Han, C. (2000). A nonlinear soft sensor based on multivariate smooth-
References ing procedure for quality estimation in distillation columns. Computers and
Chemical Engineering, 24, 871–877.
Casali, A., Gonzalez, G., Torres, F., Vallebuona, G., Castelli, L., & Gimenez, P. Pearson, R. K. (2002a). Outliers in process modeling and identification. IEEE
(1998). Particle size distribution soft-sensor for a grinding circuit. Powder Transactions on Control Systems Technology, 10(1), 55–63.
Technology, 99(1), 15–21. Pearson, R. K. (2002b). Scrub data with scale-invariant nonlinear digital filters.
Chiang, L. H., Pell, R. J., & Seasholtz, M. B. (2003). Exploring process data with EDN, 47(2), 71–78.
the use of robust outlier detection algorithms. Journal of Process Control, Prasad, V. V., Schley, M. M., Russo, L. P., & Wayne Bequette, B. (2002). Product
13(5), 437–449. property and production rate control of styrene polymerization. Journal of
Davies, L., & Gather, U. (1981). The identification of multiple outliers. Journal Process Control, 12(3), 353–372.
of the American Statistical Association, 88, 782–801. Qin, S. J., & McAvoy, T. J. (1992). Nonlinear PLS modeling using neural net-
Devlin, S. J., Gnanadesikan, R., & Kettenring, J. R. (1981). Robust estimation works. Computers and Chemical Engineering, 16(4), 379–391.
of dispersion matrices and principal components. Journal of the American Qin, S. J., Yue, H., & Dunia, R. (1997). Self-validating inferential sensors with
Statistical Association, 76, 354–362. application to air emission monitoring. Industrial & Engineering Chemistry
Dochain, D., & Perrier, M. (1997). Dynamical modelling, analysis, monitoring Research, 36, 1675–1685.
and control design for nonlinear bioprocesses. Advances in Biochemical Radhakrishnan, V. R., & Mohamed, A. R. (2000). Neural networks for the iden-
Engineering Biotechnology, 56, 147–198. tification and control of blast furnace hot metal quality. Journal of Process
Dong, D., McAvoy, T. J., & Chang, L. J. (1995). Emission monitoring using Control, 10(6), 509.
multivariate soft sensors. In Proceedings of the American control conference. Ratcliff, R. (1993). Methods for dealing with reaction time outliers. Psycholog-
Gupta, A., & Lam, M. S. (1996). Estimating missing values using neu- ical Bulletin, 114, 510–532.
ral networks. Journal of the Operational Research Society, 47(2), 229– Wise, B. M. (1991). Adapting multivariate analysis for monitoring and modeling
238. dynamic systems. Ph.D. thesis. University of Washington.
Hinz, D. C. (2006). Process analytical technologies in the pharmaceutical indus- Zamprogna, E., Barolo, M., & Seborg, D. E. (2005). Optimal selection of soft
try: The FDA’s PAT initiative. Analytical and Bioanalytical Chemistry, sensor inputs for batch distillation columns using principal component anal-
384(5), 1036–1042. ysis. Journal of Process Control, 15(1), 39–52.