Keywords: Identification for Control, Frequency Domain Identification, Mechanical and Aerospace
Abstract: Robust control allows for guaranteed performance for a range of candidate models. The aim of this paper is to investigate the role of model complexity in the identification of model sets for robust control. A key point is that model quality and model complexity should be evaluated with respect to the control goal. Regularization using a worst-case control criterion in conjunction with a specific model uncertainty structure allows robust control of multivariable systems using accurate models with low complexity. Simulations confirm that the model order should be selected in view of the control objectives. Overall, the framework allows for systematic identification of model sets for robust control.

Keywords: Identification for Control, Neural Networks, Data-driven Control
Abstract: This paper investigates the problem of controller identification given the data from a linear quadratic Model Predictive Controller (MPC) with constraints. We propose an approach for learning MPC that explicitly uses the gradient information in the training process. This is motivated by the observation that recent differentiable convex optimization MPC solvers can provide both the optimal feedback law from the state to control input as well as the corresponding gradient. As a proof of concept, we apply this approach to explicit MPC (eMPC), for which the feedback law is a piece-wise affine function of the state, but the number of pieces grows rapidly with the state dimension. Controller identification can here be used to find an approximate low complexity functional approximation of the controller. The eMPC is modelled using a Neural Network (NN) with Rectified Linear Units (ReLUs), since such NNs can represent any piece-wise affine function. A key motivation is to replace on-line solvers with neural networks to implement MPC and to simplify the evaluation of the function in larger input dimensions. We also study experimental design and model evaluation in this framework, and propose a hit-and-run sampling algorithm for input design. The proposed algorithms are illustrated and numerically evaluated on a second order MPC problem.

Keywords: Identification for Control, Nonparametric Methods, Mechanical and Aerospace
Abstract: Mobile robots become increasingly important for many industrial and logistics applications. Especially omnidirectional mobile robots are interesting due to their flexibility as they are able to move in any direction regardless of their orientation. In this context, we developed a robot platform for research on said types of robot. It turns out that this robot shows interesting behavior, which simple first principle models commonly used in the field fail to reproduce. Additionally, different robots of the same platform show different behavior although they should be identical, necessitating adaptive modeling techniques. As the sources of the mismatches are unknown, we opt for nonparametric Gaussian process regression to train a model of our robots based on carefully selected input-output data, which shows promising results. With an accurate prediction of the robot's future position and orientation, it is possible to plan the trajectory of the robot several time steps ahead allowing for a good open-loop performance. This can be useful, e.g., if in industrial applications not the entire warehouse is covered with optical position feedback. We then proceed to develop an optimization-based ancillary controller, that drives the initially unknown system dynamics to the nominal one.

Keywords: Parameter Estimation, Identification for Control, Bounded Error Identification
Abstract: We consider a cooperative system identification scenario in which an expert agent (teacher) knows a correct, or at least a good, model of the system and aims to assist a learner-agent (student), but cannot directly transfer its knowledge to the student. For example, the teacher’s knowledge of the system might be abstract or the teacher and student might be employing different model classes, which renders the teacher’s parameters uninformative to the student. In this paper, we propose correctional learning as an approach to the above problem: suppose that in order to assist the student, the teacher can intercept the observations collected from the system and modify them to maximize the amount of information the student receives about the system. We formulate a general solution as an optimization problem, which for a multinomial system instantiates itself as an integer program. Furthermore, we obtain finite-sample results on the improvement that the assistance from the teacher results in (as measured by the reduction in the variance of the estimator) for a binomial system. In numerical experiments, we illustrate the proposed algorithms and verify the theoretical results that have been derived in the paper.

Keywords: Process Control, Identification for Control, Nonlinear System Identification
Abstract: The increasing displacement of conventional power generation with renewable generation, typically non-programmable and endowed with very small or even no rotating inertia, is being accompanied by an increase of the amplitude and speed of grid frequency fluctuations. New control actions are therefore being introduced by grid operators, in the form of fast ancillary services for frequency regulation. An iterative procedure, based on the Gauss-Newton approach, is proposed here to compute the needed quantities of two fast innovative controls, namely fast primary frequency control and synthetic inertia support. In particular, by means of the identification of an output-error model, non-linear behaviours are effectively considered in the computation: dead bands on the frequency measures feeding the controllers; for renewable power plants, downward modulation schemes; for conventional ones, limits on the available control power and limitations on the maximum gradient (time derivative) of the power. The approach is tested by means of simulations in a 2030 predicted scenario for the Sardinian power system.

Keywords: Grey Box Modelling, Data-driven Control, Identification for Control
Abstract: This paper presents closed-loop intervention simulations for an identified, data-validated model of Social Cognitive Theory (SCT). A reduced-complexity SCT model structure consisting of dynamic operant conditioning and self-efficacy loops is considered for the prediction of physical activity behavior. Consistent with real-world requirements, including the need for hybrid decision rules policies, the proposed closed-loop intervention design follows a Hybrid Model Predictive Control formulation. The prime goal of this paper is to reinforce the viability of the system identification and control engineering frameworks in the design of optimized and perpetually adaptive behavioral health interventions.

Keywords: Dynamic Network Identification, Identifiability
Abstract: Identifiability conditions for networks of transfer functions require a sufficient number of external excitation signals, which are typically measured reference signals. In this abstract, we introduce an equivalent network model structure to address the contribution of unmeasured noises to identifiability analysis in the setting with partial excitation and partial measurement. With this model structure, unmeasured disturbance signals can be exploited as excitation sources, which leads to less conservative identifiability conditions.

Keywords: Dynamic Network Identification, Identifiability, Hybrid and Distributed System Identification
Abstract: Identifiability of linear dynamic networks typically depends on the presence and location of external (excitation or disturbance) signals, in relation to the topology of the parametrized network model set. For closed-loop identification, it is known that switching (non-parameterized) controllers can also provide excitation, thereby rendering the model set identifiable. In this paper, we derive verifiable conditions for network identifiability of the non-switching modules in presence of (non-parameterized) switching modules. These conditions generalize the classical result in closed-loop identification towards network identification. Furthermore, verifiable path-based conditions for identifiability in a generic sense are developed on the graph of the network model set.

Keywords: Dynamic Network Identification, Experiment Design, Closed Loop Identification
Abstract: The problem of choosing the best allocation of excitations and measurements for the identification of a dynamic network is formally stated and analyzed. The best choice will be one that achieves the most accurate identification with the least costly experiment. Accuracy is assessed by the trace of the asymptotic covariance matrix of the parameters estimates, whereas the cost criterion is the number of excitations and measurements. Analytical and numerical results are presented for two classes of dynamic networks in state space form: branches and cycles. From these results, a number of guidelines for the choice emerge, which are based either on the topology of the network or on the relative magnitude of the modules being identified.

Keywords: Time Series, Blind Estimation, Other Applications
Abstract: Identifying the interconnections among modules in a dynamic network from observed data poses a significant challenge in many scientific disciplines. Many methods for network reconstruction from observational data significantly limit the type of systems they are considering. For example, Granger causality considers only networks with strictly causal dynamics, and methods from the graphical models literature are focused on reconstructing networks with static relationships. In this article, we focus on a novel network reconstruction method, called Mixed-Delay (MD) that can consistently reconstruct a wide class of linear dynamic networks that do not contain any algebraic loops. However, the steps in the MD algorithm are of combinatorial complexity. In this article, we propose an optimization to the MD method that yields the method more informative and polynomial for sparse networks, while preserving the theoretical guarantees of the method. We demonstrate the optimized MD method on simulated and real data. The first real-data application aims to reconstruct networks that show the spread of COVID-19 in the US. Then we apply the method on monthly average temperature data and reconstruct temperature relationships among states in the US, as well as European and South-East Asian countries.

Keywords: Identifiability, Dynamic Network Identification
Abstract: In this paper we determine under which additional requirements a dynamic network is generically identifiable when some structures within it are known to be generically identifiable. We derive a set of necessary or sufficient conditions to determine generic identifiability of the whole network. The conditions take form as rank conditions of these specific structures and also in the topology of the network. Necessary and sufficient conditions for generic identifiability are given for classes of networks with parallel paths among the nodes. For the quite general case of networks whose graph is acyclic, we present necessary and sufficient conditions to determine whether a particular node needs to be excited and/or measured.

Keywords: Fault Detection and Diagnosis, Filtering and Smoothing, Automotive Systems
Abstract: An online model-based fault detection and isolation method for salient-pole permanent magnet synchronous motors over a finite horizon is proposed. The proposed approach combines parity-space-based residual generation and modulation function-based filtering. Given the polynomial model equations, the unknown variables (i.e. the states, unmeasured inputs) are eliminated resulting in analytic redundancy relations used for residual generation. Furthermore, in order to avoid needing the derivatives of measured signals required by such analytic redundancy relations, a modulation-function-based evaluation is proposed. This results in a finite-horizon filtered version of the original residual. The fault detection and isolation method is demonstrated using simulation of various fault scenarios for a speed controlled salient motor showing the effectiveness of the presented approach.

Keywords: Fault Detection and Diagnosis, Monitoring, Other Applications
Abstract: In this paper, the problem of quality control in the textile industrial field is addressed. Because of the general unavailability of labelled data from real production plants and the imbalanced nature of the problem, this task is faced with novelty detection methods that monitor the behaviour of the system and identify whether shifts from the nominal conditions arise. In particular, we utilize techniques from Elastic Shape Analysis to analyse the shapes created by the yarns intersections of the fabrics and to extract features used to define distance metrics that quantify the shapes variability. The proposed approach is applied to images of four different textiles, where only some defect free images are needed for the training phase. The results of this preliminary study confirm the effectiveness of the proposed approach.

Keywords: Fault Detection and Diagnosis, Subspace Methods, Vibration and Modal Analysis
Abstract: Statistical subspace-based change detection residuals have been developed to infer a change in the eigenstructure of linear systems. Their statistical properties have been properly evaluated in the case of a known reference and constant noise properties. Previous residuals have favored the family of null space-based approaches, whereas the possibility of using other metrics such as the Mahalanobis distance has been omitted. This paper investigates the development and study of such a norm under the premise of a varying noise covariance. Its statistical properties have been studied and tested on a numerical example of a mechanical system.

Keywords: Fault Detection and Diagnosis, Nonparametric Methods, Monitoring
Abstract: A data driven structural change detection method is described and evaluated where the data are acceleration and force measurements from a mechanical structure in the form of a vehicle. By grouping the measured signals as inputs and outputs an hypothesized MIMO linear dynamic relation between the inputs and outputs is assumed. It is assumed that baseline data are available to build statistical models for the estimated frequency function of the baseline system at selected frequencies. When new data is available, the monitoring algorithm re-estimates the non-parametric frequency function and uses a test statistic based on the statistical distance to detect possible change. To generate the frequency function estimates a non-parametric MIMO frequency function estimator based on the local rational model (LRM) method is developed. A statistical analysis of the proposed test statistic shows that it has an F-distribution for data from the baseline case. The method is evaluated on simulated data from a high fidelity full scale vehicle simulation generating both baseline data and data from a structurally changed vehicle. In the evaluation, the frequency response functions were estimated by the non-parametric LRM method, the parametric ARX estimate and the non-parametric ETFE. The results show that all three methods can detect the structural change while the LRM method is more robust with respect to the selection of the hyperparameters.

Keywords: Fault Detection and Diagnosis, Time Series, Monitoring
Abstract: In modern manufacturing, each stage of industrial processes is accurately measured via multiple sensors and, consequently, a large amount of data is made available for analytics, monitoring and control purposes. A possible use of such data is to detect anomalies in order to prevent potential damages and hazards. In this paper, we will consider a sensor setup returning distributed time series measurements that can be used for failure identification. In particular, an anomaly detection strategy based on Vector Autoregressive (VAR) modeling for multivariate time series will be presented and analyzed in detail. The effectiveness of the proposed methodology will be assessed on experimental data from a real industrial case study.

Keywords: Fault Detection and Diagnosis, Monitoring, Time Series

Keywords: Parameter Estimation, Regularization and Kernel Methods, Machine Learning and Data Mining
Abstract: System identification aims to build models of dynamical systems from data. Traditionally, choosing the model requires the designer to balance between two goals of conflicting nature; the model must be rich enough to capture the system dynamics, but not so flexible that it learns spurious random effects from the dataset. It is typically observed that the model validation performance follows a U-shaped curve as the model complexity increases. Recent developments in machine learning and statistics, however, have observed situations where a ``double-descent'' curve subsumes this U-shaped model-performance curve. With a second decrease in performance occurring beyond the point where the model has reached the capacity of interpolating---i.e., (near) perfectly fitting---the training data. To the best of our knowledge, such phenomena have not been studied within the context of dynamic systems. The present paper aims to answer the question: ``Can such a phenomenon also be observed when estimating parameters of dynamic systems?'' We show that the answer is yes, verifying such behavior experimentally both for artificially generated and real-world datasets.

Keywords: Parameter Estimation, Algorithms, Time Series
Abstract: We consider the problem of detecting the common dynamic among several observed signals. It has been shown in (Markovsky et al., 2019) that the problem is equivalent to a generalization of the classical Hankel low-rank approximation to the case of multiple rank constraints. We propose an optimization method based on the integration of ordinary differential equations describing a descent dynamic for a suitable functional to be minimized. We show how the proposed algorithm improves the numerical solutions computed by existing subspace methods which solve the same problem.

Keywords: Parameter Estimation, Bayesian Methods, Subspace Methods
Abstract: In some recent works, the authors have proposed and developed an Empirical Bayes framework for frequency estimation. The unknown frequencies in a noisy oscillatory signal are modeled as uniform random variables supported on narrow frequency bands. The bandwidth and the relative band centers are known as hyperparameters which can be efficiently estimated using techniques from subspace identification. In the current paper, we examine carefully how the estimated frequency prior can be used to produce a Bayesian estimate of the unknown frequencies based on the same data (for hyperparameter estimation). To this end, we formulate the Bayesian Maximum A Posteriori (MAP) optimization problem and propose an iterative algorithm to compute its solution. Then, we do extensive simulations under various parameter configurations, showing that the MAP estimate of the frequencies are asymptotically close to the band centers of the frequency priors. These results provide an attractive link between the conventional Bayesian method and the Empirical Bayes method for frequency estimation, and in retrospect justify the use of the latter.

Keywords: Recursive Identification, Parameter Estimation, Continuous Time System Estimation
Abstract: Fractional-order calculus has already proven to be effective in order to model diffusive phenomena, such as heat transfer or anomalous mass transfer. This has led to developments of fractional-order transfer function models and, as a consequence, identification algorithms have been developed to identify the parameters for this type of structures. However, it may be necessary to perform real-time system identification in which the parameters evolve over time while more and more data is acquired. Extensions to recursive identification algorithms are presented for fractional-order transfer functions. Two methods are proposed for the recursive estimation of the coefficients of continuous-time fractional order systems: the recursive least squares with state variable filters and the Prediction-error method. These two methods are compared with different signal to noise ratio levels through Monte Carlo simulations.

Keywords: Parameter Estimation, Continuous Time System Estimation
Abstract: We are considering the problem of sampled-data observer design for nonlinear time-varying delay systems that are state- and parameter-affine. The novelty is that the system is subject to both distributed delay and parameter uncertainty. A Kalman-like observer is developed to deal with both state and parameter estimation. Its main components are: (i) a time-varying-gain state-estimator involving both output and parameter rate injections; (ii) a distributed-nature adaptive output-predictor that compensate for delay and output sampling delay; (iii) an optimized parameter-estimator that copes with parameter uncertainty, the optimization feature is intended in the sense that we make use of all available information, unlike previous estimators. The resulting observer is shown to be exponentially convergent, for small delays and sampling intervals, provided the input signal is sufficiently exciting. The analysis is performed using a Lyapunov-Krasovskii functional, Halanay’s lemma, Wirtinger’s inequality and other tools.

Keywords: Parameter Estimation
Abstract: We study the distributed parameter estimation problem in large-scale sensor networks. The goal of the sensors is to find the global estimate of an unknown parameter minimizing the aggregate cost via only local communication with their neighbors. A large number of sensors can estimate the unknown parameter more accurately than small sensor networks. However, the larger the network, the more problems it possesses. We impose the communication constraints to avoid network overload since the sensors may flood the communication channels during message exchange. The optimization problem is solved using the weighted SPSA-based consensus algorithm. Usually, each sensor estimates the unknown parameter based on noisy measurements. Most existing methods, however, assume that some statistical characteristics of the noise are known. The SPSA-based method allows us to relax this assumption. We assume that the noise is unknown-but-bounded, which is more practically reasonable. In this paper, we provide the convergence analysis of the weighted SPSA-based consensus algorithm in stationary case and show its efficacy based on simulation.

Keywords: Biological Systems, Process Control
Abstract: The growth process of a Chlamydomonas reinhardtii cell population is modelled with experimental data obtained in a batch reactor. To describe the growth process of this culture, the Droop model, extended by cell population balance model, is considered. On the basis of available measurements and the mathematical model, an optimization problem is defined in order to determine the kinetic parameter values for the growth functions of the Droop model and the cell division parameters of the cell population balance model.

Keywords: Identifiability, Algorithms, Nonlinear System Identification
Abstract: Identifiability, observability, and controllability are important structural properties of a dynamic system model. Our interest lies in the detection of a lack of identifiability/observability and/or controllability through the computation and subsequent analysis of the exact nullspace of the gramian for non-linear systems. For this analysis we have developed a user-friendly application with the name StrucID which runs in Matlab. The StrucID App requires as input a model definition in (possibly non-linear) state space format. In addition, an output equation that may also be non-linear is required. Through a rank test (SVD) on an associated sensitivity matrix, so-called signature graphs are produced. These represent a model's singular values and nullspace vectors and provide a visual summary. The results can now be used in a substantially reduced symbolic computation (not included yet in the current version of StrucID) that computes a Fliess series expansion of the output signal to arrive at the nullspace of an associated Jacobi matrix. Solving an underlying partial differential equation then completes the structural analysis and generates a re-parametrisation and/or state transformation that allows for model reduction in an exact manner. A few examples will be presented.

Keywords: Biological Systems, Process Control, Parameter Estimation
Abstract: The parameter identification problem of cell population balance models in a yeast batch process is addressed. The model is described by a partial integro-differential equation coupled with a set of ordinary differential equations. Based on the solution equivalence of the mass balance model and the first moment of the cell population balance model, the identification process is carried out in two steps. The growth related parameters are identified by only taking mass balance-based measurements into account. The growth parameters are then utilized to identify the parameters related to cell division in the cell population balance model by means of optimization. The results are verified with an R2 analysis.

Keywords: Biological Systems, Parameter Estimation, Maximum Likelihood Methods
Abstract: In previous work, we have developed an autoregressive Mixed-Effects model of the evolution of the kinetic gene expression parameters along cell generations, and an identification method simultaneously exploiting single-cell gene expression profiles and known parental relationships among cells (lineage tree data). Here, we extend our modelling and identification approach to explicitly account for stochasticity of promoter activation, and demonstrate via simulation the performance of the method and the improvement relative to the original approach where this source of noise is not accounted for.

Keywords: Biomedical Systems, Grey Box Modelling, Parameter Estimation
Abstract: During invasive mechanical ventilation, knowledge of the patient's respiratory effort is valuable in guiding the clinical team to perform a personalized therapy. The same adjustment of the ventilator can produce excessive transpulmonary pressures for different patients, capable of generating or aggravating pre-existing lung injuries. The measurement of transpulmonary pressure (the difference between airway and pleural pressures) is not easily performed in practice. Although airway pressure measurement is available on most current mechanical ventilators, pleural pressure measurement is indirectly performed using an esophageal balloon. In many cases, esophageal pressure reading takes other phenomena into account and is not a reliable measure of pleural pressure. This work will study system identification techniques to obtain reliable pleural pressures based on esophageal pressure readings, aiming to provide the clinical team with information about the patient's ventilatory therapy's real status. By estimating transfer function models, autoregressive with external input (ARX) and output error (OE) polynomial models, waveforms were adjusted in two different patients' occlusion maneuver data. Different metrics were used to assess the quality of the models obtained. The analysis results showed that the estimation methods used provided plausible representations of the underlying dynamic system. Therefore, there are indications that the system identification techniques presented in this work can be used in general, in a clinical environment, to provide reliable estimates of pleural pressure based on esophageal pressure measurements.

Keywords: Machine Learning and Data Mining, Other Applications, Multivariable System Identification
Abstract: The paper addresses the identification of spatial-temporal mirror surface deformations as a result of laser-based heat load within the lithography process of integrated circuit production. The thermal diffusion and surface deformation are modeled by separation of the spatial-temporal effects using data-driven orthogonal decomposition. A novel tree adjoining grammar (TAG) and sparsity enhanced symbolic-regression-based learning methods are deployed to discover temporal dynamics that connect the spatial variation. The resulting data-driven procedure is applied to automatically synthetise a compact model representation of synthetic thermal effects induced mirror surface deformations.

Keywords: Grey Box Modelling, Bayesian Methods, Machine Learning and Data Mining
Abstract: This paper attempts to bridge the gap between standard engineering practice and machine learning when modelling stochastic processes. For a number of physical processes of interest, derivation of the (auto)covariance is achievable. This paper suggests their use as priors in a standard Gaussian process regression as a means of enhancing predictive capability in situations where they are reflective of the process of interest. A covariance function of a linear oscillator under random load is derived and used in a regression context to predict the displacements of a vibratory system. A simulation case study is used to demonstrate the enhancement over a standard Gaussian process regression model.

Keywords: Nonlinear System Identification, Neural Networks
Abstract: This work presents a new regularization scheme for identifying nonlinear finite impulse response (NFIR) models using artificial neural networks (ANN). Prior knowledge, such as the exponentially decaying nature of an impulse response, is included during the identification using a regularization approach inspired on the well-known regularized linear finite impulse response identification literature. More specifically the sensitivity of the modeled output with respect to the delayed input of the NFIR model is penalized to provide an exponentially decaying prior. The proposed method is illustrated and compared to other ANN regularization schemes on a simulation example.

Keywords: Bayesian Methods, Machine Learning and Data Mining, Uncertainty Quantification
Abstract: The increased availability of observation data from engineering systems in operation poses the question of how to integrate this data into digital twins. To this end, we propose a novel statistical construction of the finite element method, dubbed as statFEM, which provides the means of synthesising measurement data and finite element models. The Bayesian statistical framework is adopted to treat all the uncertainties present in the data, the mathematical model and its finite element discretisation. From the outset, we postulate a statistical generating model which additively decomposes data into a finite element, a model misspecification and a noise component. Each of the components may be uncertain and is considered as a random variable with a respective prior probability density. The prior of the finite element component is given by a conventional stochastic forward problem. The prior probabilities of the model misspecification and measurement noise, without loss of generality, are assumed to have zero-mean and known covariance structure. We use the Bayes rule to infer the posterior densities of the three random components and the hyperparameters from their known prior densities and a data-dependent likelihood function. The posterior densities of the finite element component and the true system response are determined using the prior finite element density given by the forward problem and the data likelihood. Approximating the prior finite element density with a multivariate Gaussian distribution allows us to obtain a closed-form expression for the posterior. We demonstrate the accuracy and versatility of statFEM with a one-dimensional Poisson-Dirichlet example and illustrate its use in creating a physics-informed digital twin of an operational railway bridge.

Keywords: Nonlinear System Identification, Grey Box Modelling, Mechanical and Aerospace
Abstract: A semi-parametric regression methodology is formulated to identify the unsteady lift characteristics of a small UAS undergoing dynamic stall. Based on the trailing edge separation model of Leishmann and Beddoes, the nonlinear evolution of the separation point is formulated so that it can be estimated by non-parametric Machine Learning methods. Validation of the methodology is presented with the identification of the lift coefficient based on quasi-steady wind tunnel tests.

Keywords: Nonlinear System Identification, Neural Networks, Grey Box Modelling
Abstract: We present a novel Deep Neural Network (DNN) architecture for non-linear system identification. We foster generalization by constraining DNN representational power. To do so, inspired by fading memory systems, we introduce inductive bias (on the architecture) and regularization (on the loss function). This architecture allows for automatic complexity selection based solely on available data, in this way the number of hyper-parameters that must be chosen by the user is reduced. Exploiting the highly parallelizable DNN framework (based on Stochastic optimization methods) we successfully apply our method to large scale datasets.